Bahmni Standard Performance Testing Reports
This page contains the performance reports that has been executed for testing Bahmni Standard. The tests are run against Bahmni Standard 1.0.0 Beta. The scenarios tested are same as here.
Configurations of the Bahmni environment under test:
Hardware Configuration:
AWS EC2 m5.xlarge
CPU: 4 vCPU
RAM: 16 GB
Storage: 100 GB
OS: Ubuntu 22.04.2 LTS
Bahmni Installation Details:
Bahmni Installation Method: Docker Compose
No.of Running Containers: 21 (EMR, OpenELIS, Odoo, PACS, Agents for scraping metrics(node-exporter,cadvisor))
Database Containers (all on the same node): OpenMRS DB (MySQL 8.0), OpenELIS DB (Postgres 9.6), Odoo DB (Postgres 16.1), PACS DB (Postgres 9.6)
Bahmni Metdata Details:
Concept Dictionary: CIEL (v2023-12-15)
Concept Count: 56884
Concept Reference Terms: 170167
Concept Reference Maps: 218243
Configurations of the agent which executes the test suite:
AWS EC2 c5.xlarge
CPU: 4 vCPU
RAM: 16 GB
Storage: 10 GB
OS: Ubuntu 22.04.2 LTS
Execution Tools:
Java 11 JDK
Performance Suite: https://github.com/Bahmni/performance-test/tree/bahmni-standard-ciel
Test Data Generator: https://github.com/Bahmni/bahmni-test-data-generator
Initial Data Generated using generator: 99995 patients, 70931 visits, 70931 encounters, 425586 observations
The initial backup taken after generating the above test data is available here.
📗 Test 1 - 70 Concurrent Users 4 Hours
Duration: 4 hours
Active Users: 70 users
Ramp Up: 5 mins
Database pre-state: 99995 patients, 70931 visits, 70931 encounters, 425586 observations
Report Link: https://bahmni.github.io/performance-test/bahmnistandard-70users-5hours-20240726/index.html
Observations:
No.of Patient Created: 1730, No.of Visits created: 4991, No.of Encounters Created: 2040, No.of Observations Created: 12970
Simulations | Scenario | Load share | Patient Count | Min Time (ms) | 95th Percentile (ms) | 99th Percentile (ms) | Max Time (ms) |
---|---|---|---|---|---|---|---|
Frontdesk 50% Traffic (35 Users) | New Patient Registration → Start OPD Visit | 40% | 1680 | 51 | 107 | 158 | 321 |
Existing Patient Search using ID → Start OPD Visit | 30% | 1200 | 84 | 143 | 256 | 464 | |
Existing Patient Search using Name → Start OPD Visit | 20% | 1200 | 176 | 310 | 397 | 716 | |
Upload Patient Document | 10% | 360 | 72 | 127 | 217 | 399 | |
Doctor 50% Traffic (35 Users) | Doctor Consultation
| 100% | 1680 | 389 | 701 | 876 | 2015 |
📗 Test 2 - 70 Concurrent Users 8 Hours
Duration: 8 hours
Active Users: 70 users
Ramp Up: 5 mins
Database pre-state: 101725 patients, 75922 visits, 72971 encounters, 438556 observations
Report Link: https://bahmni.github.io/performance-test/bahmnistandard-70users-8hours-20240726/index.html
Observations:
No.of Patient Created: 3410, No.of Visits created: 9912, No.of Encounters Created: 4060, No.of Observations Created: 25296
There were 20 encounter API call failure with 400 which due to the system trying to add the same drug order which is already active.
Simulations | Scenario | Load share | Patient Count | Min Time (ms) | 95th Percentile (ms) | 99th Percentile (ms) | Max Time (ms) |
---|---|---|---|---|---|---|---|
Frontdesk 50% Traffic (35 Users) | New Patient Registration → Start OPD Visit | 40% | 3360 | 46 | 122 | 175 | 465 |
Existing Patient Search using ID → Start OPD Visit | 30% | 2400 | 82 | 160 | 243 | 410 | |
Existing Patient Search using Name → Start OPD Visit | 20% | 2400 | 174 | 345 | 439 | 609 | |
Upload Patient Document | 10% | 720 | 70 | 102 | 160 | 569 | |
Doctor 50% Traffic (35 Users) | Doctor Consultation
| 100% | 3360 | 298 | 718 | 952 | 1684 |
📗 Test 3 - 120 Concurrent Users 8 Hours
Report Link: https://bahmni.github.io/performance-test/bahmnistandard-120users-8hours-20240727
Observations:
No.of Patient Created: 5810, No.of Visits created: 17449, No.of Encounters Created: 7049, No.of Observations Created: 42969
There were 151 encounter API call failure with 400 which due to the system trying to add the same drug order which is already active.
Note: The 99th percentile of post encounter went above expected to 1257
Simulations | Scenario | Load share | Patient Count | Min Time (ms) | 95th Percentile (ms) | 99th Percentile (ms) | Max Time (ms) |
---|---|---|---|---|---|---|---|
Frontdesk 50% Traffic (60 Users) | New Patient Registration → Start OPD Visit | 40% | 5760 | 48 | 154 | 254 | 701 |
Existing Patient Search using ID → Start OPD Visit | 30% | 4320 | 83 | 217 | 400 | 917 | |
Existing Patient Search using Name → Start OPD Visit | 20% | 4320 | 178 | 417 | 614 | 1128 | |
Upload Patient Document | 10% | 1440 | 70 | 102 | 160 | 569 | |
Doctor 50% Traffic (60 Users) | Doctor Consultation
| 100% | 5760 | 286 | 867 | 1257 | 2641 |
📗 Test 4 - 120 Concurrent Users 24 Hours
Report Link:
Observations:
No.of Patient Created: 17336, No.of Visits created: 51462, No.of Encounters Created: 20716, No.of Observations Created: 124238
There were 884 encounter API call failure with 400 which due to the system trying to add the same drug order which is already active.
There are 3(delete session, getAllObservation Templates, getActive visits) requests failed with a 502 status
Note:
The 99th percentile of post encounter went above expected(1000ms) to 1413
The max time of post encounter went above expected(3000ms) to 4015
The 95th percentile of emrapi.sqlSearch.activePatientsByLocation went above expected(1000ms) to 1056
The 99th percentile of emrapi.sqlSearch.activePatientsByLocation went above expected(1000ms) to 1545
The max of emrapi.sqlSearch.activePatientsByLocation went above expected(3000ms) to 4056
Simulations | Scenario | Load share | Patient Count | Min Time (ms) | 95th Percentile (ms) | 99th Percentile (ms) | Max Time (ms) |
---|---|---|---|---|---|---|---|
Frontdesk 50% Traffic (60 Users) | New Patient Registration → Start OPD Visit | 40% | 17280 | 48 | 179 | 271 | 1688 |
Existing Patient Search using ID → Start OPD Visit | 30% | 12960 | 83 | 272 | 452 | 1661 | |
Existing Patient Search using Name → Start OPD Visit | 20% | 12960 | 175 | 532 | 748 | 2257 | |
Upload Patient Document | 10% | 4320 | 69 | 226 | 366 | 1827 | |
Doctor 50% Traffic (60 Users) | Doctor Consultation
| 100% | 17280 | 275 | 948 | 1413 | 4015 |
📗 Test 5 - 200 Concurrent Users 8 Hours
Report Link:
Observations:
No.of Patient Created: 9110, No.of Visits created: 28142, No.of Encounters Created: 10738, No.of Observations Created: 64412
There were 1262 encounter API call failure with 400 which due to the system trying to add the same drug order which is already active.
There are 3(find drug, get login locations, find encounter, start visit, search by ID) requests failed with a 502 status
Note: APIs which exceeded expected results
Metric | Expected (ms) | Actual (ms) | |
---|---|---|---|
1 | 95th percentile of post encounter | 1000 | 2481 |
2 | 99th percentile of post encounter | 1000 | 3772 |
3 | Max time of post encounter | 3000 | 6005 |
4 | 95th percentile of emrapi.sqlSearch.activePatientsByLocation | 1000 | 1209 |
5 | 99th percentile of emrapi.sqlSearch.activePatientsByLocation | 1000 | 1921 |
6 | Max of emrapi.sqlSearch.activePatientsByLocation | 3000 | 3538 |
7 | 99th percentile of emrapi.sqlSearch.activePatientsByProvider | 1000 | 1032 |
8 | 99th percentile of emrapi.sqlSearch.activePatients | 3000 | 1098 |
9 | 95th percentile of Search Patient By Name | 1000 | 1166 |
10 | 99th percentile of Search Patient By Name | 1000 | 1686 |
11 | Max of Search Patient By Name | 3000 | 3107 |
12 | 99th percentile of Search Patient By Identifier | 1000 | 1078 |
Final Database State: 137931 patients, 182887 visits, 115534 encounters, 695471 observations. The backup of the all the subsystems is available here.
The Bahmni documentation is licensed under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)