...
Capturing observations and data in Bahmni has been done primarily with the help of a keyboard and mouse. While this has worked until now, we wanted to explore faster methods for capturing patient data, which would enable to doctor to quickly capture the required data , thus providing increased consultation time with the patient. In the long run, the speech assistant could also be enhanced in other areas of Bahmni like providing faster navigation, quick view of dashboards, etc.
| Table of Contents | ||||
|---|---|---|---|---|
|
The solution :
...
After quick rounds of brainstorming activities, we converged on the idea to try out speech assistant for consultation notes and use it for initial user testing and general feedback. Some notable decisions were:
...
Setting up Vakyansh:
Clone Speech Assistant Package
Download models by running download_models.sh in the scripts folder.
Run docker-compose up -d. It will bring two containers: vakyansh-api and vakyansh-proxy.
Setting up Speech Assistant Frontend:
As Speech Assistant is a microfrontend, so to use it, corresponding bundle.js needs to be included in the script tag.
bundle.js can be availed either from this url, which contains the latest bundle, or while building the app present in the repository.
Demo of Speech Assistant
...
(Youtube Video):
https://www.youtube.com/watch?v=i2R_odYHAeA
| Widget Connector | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Directions:
Currently the language model been used by Vakyansh is normal English model. To understand medical words, model needs to be trained with the relevant vocabulary. Trained medical model then can be used by Vakyansh to get the proper text back.
Vakyansh api works better when deployed in a GPU machine. One instance of api can easily serve upto 10 concurrent audio connections. If it needs to be increased further, the api needs to be scaled.
If NLP library is applied to understand the meaning from the sentence, then the usecase could be extended to other consultation tasks like Medications, Symptoms, etc.