Bahmni Standard Performance Testing Reports

Owned by Mohankumar Thangavel
Last updated: Jul 29, 2024
6 min read

This page contains the performance reports that has been executed for testing Bahmni Standard. The tests are run against Bahmni Standard 1.0.0 Beta. The scenarios tested are same as here.

Configurations of the Bahmni environment under test:

Hardware Configuration: AWS EC2 m5.xlarge CPU: 4 vCPU RAM: 16 GB Storage: 100 GB OS: Ubuntu 22.04.2 LTS Bahmni Installation Details: Bahmni Installation Method: Docker Compose No.of Running Containers: 21 (EMR, OpenELIS, Odoo, PACS, Agents for scraping metrics(node-exporter,cadvisor)) Database Containers (all on the same node): OpenMRS DB (MySQL 8.0), OpenELIS DB (Postgres 9.6), Odoo DB (Postgres 16.1), PACS DB (Postgres 9.6) Bahmni Metdata Details: Concept Dictionary: CIEL (v2023-12-15) Concept Count: 56884 Concept Reference Terms: 170167 Concept Reference Maps: 218243

Configurations of the agent which executes the test suite:

AWS EC2 c5.xlarge CPU: 4 vCPU RAM: 16 GB Storage: 10 GB OS: Ubuntu 22.04.2 LTS Execution Tools: Java 11 JDK Performance Suite: https://github.com/Bahmni/performance-test/tree/bahmni-standard-ciel Test Data Generator: https://github.com/Bahmni/bahmni-test-data-generator Initial Data Generated using generator: 99995 patients, 70931 visits, 70931 encounters, 425586 observations

The initial backup taken after generating the above test data is available here.

📗 Test 1 - 70 Concurrent Users 4 Hours

  • Duration: 4 hours

  • Active Users: 70 users

  • Ramp Up: 5 mins

  • Database pre-state: 99995 patients, 70931 visits, 70931 encounters, 425586 observations

Report Link: https://bahmni.github.io/performance-test/bahmnistandard-70users-5hours-20240726/index.html

Observations:

  • No.of Patient Created: 1730, No.of Visits created: 4991, No.of Encounters Created: 2040, No.of Observations Created: 12970

Simulations
Scenario
Load share
Patient Count
Min Time (ms)
95th Percentile (ms)
99th Percentile (ms)
Max Time (ms)
Simulations
Scenario
Load share
Patient Count
Min Time (ms)
95th Percentile (ms)
99th Percentile (ms)
Max Time (ms)

Frontdesk

50% Traffic

(35 Users)

New Patient Registration → Start OPD Visit

40%

1680

51

107

158

321

Existing Patient Search using ID → Start OPD Visit

30%

1200

84

143

256

464

Existing Patient Search using Name → Start OPD Visit

20%

1200

176

310

397

716

Upload Patient Document

10%

360

72

127

217

399

Doctor

50% Traffic

(35 Users)

Doctor Consultation

  • 1 Observation

  • 1 Diagnosis

  • 2 Lab Orders (1 Test, 1 Panel)

  • 1 Radiology Order

  • 1 Procedure Order

  • 1 Medication

100%

1680

389

701

876

2015

image-20240726-061927.png

CPU was at average of 35%, Network load was ~250KBps

 

📗 Test 2 - 70 Concurrent Users 8 Hours

  • Duration: 8 hours

  • Active Users: 70 users

  • Ramp Up: 5 mins

  • Database pre-state: 101725 patients, 75922 visits, 72971 encounters, 438556 observations

Report Link: https://bahmni.github.io/performance-test/bahmnistandard-70users-8hours-20240726/index.html

Observations:

  • No.of Patient Created: 3410, No.of Visits created: 9912, No.of Encounters Created: 4060, No.of Observations Created: 25296

  • There were 20 encounter API call failure with 400 which due to the system trying to add the same drug order which is already active.

Simulations
Scenario
Load share
Patient Count
Min Time (ms)
95th Percentile (ms)
99th Percentile (ms)
Max Time (ms)
Simulations
Scenario
Load share
Patient Count
Min Time (ms)
95th Percentile (ms)
99th Percentile (ms)
Max Time (ms)

Frontdesk

50% Traffic

(35 Users)

New Patient Registration → Start OPD Visit

40%

3360

46

122

175

465

Existing Patient Search using ID → Start OPD Visit

30%

2400

82

160

243

410

Existing Patient Search using Name → Start OPD Visit

20%

2400

174

345

439

609

Upload Patient Document

10%

720

70

102

160

569

Doctor

50% Traffic

(35 Users)

Doctor Consultation

  • 1 Observation

  • 1 Diagnosis

  • 2 Lab Orders (1 Test, 1 Panel)

  • 1 Radiology Order

  • 1 Procedure Order

  • 1 Medication

100%

3360

298

718

952

1684

image-20240726-161325.png

CPU was at average of 35%, Network load was ~250KBps

 

📗 Test 3 - 120 Concurrent Users 8 Hours

Report Link: https://bahmni.github.io/performance-test/bahmnistandard-120users-8hours-20240727

Observations:

  • No.of Patient Created: 5810, No.of Visits created: 17449, No.of Encounters Created: 7049, No.of Observations Created: 42969

  • There were 151 encounter API call failure with 400 which due to the system trying to add the same drug order which is already active.

  • Note: The 99th percentile of post encounter went above expected to 1257

Simulations
Scenario
Load share
Patient Count
Min Time (ms)
95th Percentile (ms)
99th Percentile (ms)
Max Time (ms)
Simulations
Scenario
Load share
Patient Count
Min Time (ms)
95th Percentile (ms)
99th Percentile (ms)
Max Time (ms)

Frontdesk

50% Traffic

(60 Users)

New Patient Registration → Start OPD Visit

40%

5760

48

154

254

701

Existing Patient Search using ID → Start OPD Visit

30%

4320

83

217

400

917

Existing Patient Search using Name → Start OPD Visit

20%

4320

178

417

614

1128

Upload Patient Document

10%

1440

70

102

160

569

Doctor

50% Traffic

(60 Users)

Doctor Consultation

  • 1 Observation

  • 1 Diagnosis

  • 2 Lab Orders (1 Test, 1 Panel)

  • 1 Radiology Order

  • 1 Procedure Order

  • 1 Medication

100%

5760

286

867

1257

2641

Note: The CPU usage load was high above ~60% compared to 70 users run which was at ~35%

CPU was at average of 63%, Network load was ~500KBps

📗 Test 4 - 120 Concurrent Users 24 Hours

Report Link:

Observations:

  • No.of Patient Created: 17336, No.of Visits created: 51462, No.of Encounters Created: 20716, No.of Observations Created: 124238

  • There were 884 encounter API call failure with 400 which due to the system trying to add the same drug order which is already active.

  • There are 3(delete session, getAllObservation Templates, getActive visits) requests failed with a 502 status

  • Note:

    • The 99th percentile of post encounter went above expected(1000ms) to 1413

    • The max time of post encounter went above expected(3000ms) to 4015

    • The 95th percentile of emrapi.sqlSearch.activePatientsByLocation went above expected(1000ms) to 1056

    • The 99th percentile of emrapi.sqlSearch.activePatientsByLocation went above expected(1000ms) to 1545

    • The max of emrapi.sqlSearch.activePatientsByLocation went above expected(3000ms) to 4056

Simulations
Scenario
Load share
Patient Count
Min Time (ms)
95th Percentile (ms)
99th Percentile (ms)
Max Time (ms)
Simulations
Scenario
Load share
Patient Count
Min Time (ms)
95th Percentile (ms)
99th Percentile (ms)
Max Time (ms)

Frontdesk

50% Traffic

(60 Users)

New Patient Registration → Start OPD Visit

40%

17280

48

179

271

1688

Existing Patient Search using ID → Start OPD Visit

30%

12960

83

272

452

1661

Existing Patient Search using Name → Start OPD Visit

20%

12960

175

532

748

2257

Upload Patient Document

10%

4320

69

226

366

1827

Doctor

50% Traffic

(60 Users)

Doctor Consultation

  • 1 Observation

  • 1 Diagnosis

  • 2 Lab Orders (1 Test, 1 Panel)

  • 1 Radiology Order

  • 1 Procedure Order

  • 1 Medication

100%

17280

275

948

1413

4015

CPU was at average of 63%, Network load was ~500KBps

📗 Test 5 - 200 Concurrent Users 8 Hours

Report Link:

Observations:

  • No.of Patient Created: 9110, No.of Visits created: 28142, No.of Encounters Created: 10738, No.of Observations Created: 64412

  • There were 1262 encounter API call failure with 400 which due to the system trying to add the same drug order which is already active.

  • There are 3(find drug, get login locations, find encounter, start visit, search by ID) requests failed with a 502 status

  • Note: APIs which exceeded expected results

Metric

Expected (ms)

Actual (ms)

Metric

Expected (ms)

Actual (ms)

1

95th percentile of post encounter

1000

2481

2

99th percentile of post encounter

1000

3772

3

Max time of post encounter

3000

6005

4

95th percentile of emrapi.sqlSearch.activePatientsByLocation

1000

1209

5

99th percentile of emrapi.sqlSearch.activePatientsByLocation

1000

1921

6

Max of emrapi.sqlSearch.activePatientsByLocation

3000

3538

7

99th percentile of emrapi.sqlSearch.activePatientsByProvider

1000

1032

8

99th percentile of emrapi.sqlSearch.activePatients

3000

1098

9

95th percentile of Search Patient By Name

1000

1166

10

99th percentile of Search Patient By Name

1000

1686

11

Max of Search Patient By Name

3000

3107

12

99th percentile of Search Patient By Identifier

1000

1078

CPU was at average of 100%, Network load was ~1.1MBps

 

Final Database State: 137931 patients, 182887 visits, 115534 encounters, 695471 observations. The backup of the all the subsystems is available here.

The Bahmni documentation is licensed under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)