1. PurposeWe are intending to baseline the performance of the Bahmni app for clinics. The test will be extended to check the maximum Scope
Develop performance test automation to validate total number of clinics that can be supported with current infrastructure without scaling up.
2. Introduction
As part of Bahmni NEXT, the solution should meet the acceptance criteria in both functional and cross functional areas. on a given infrastructure configuration.
Baseline Hardware:
Bahmni Lite: RAM 16GB, 4 vCPU and 100GB Secondary storage
Database: RAM 8GB, 2 vCPU and 100GB Secondary storage
The automation would simulate Varying Traffic Load conditions
Standard Traffic Load: 40-50 Active users
High Traffic Load: 50-70 Active users
Peak Traffic Load: 70-110 Active users
No existing data would be considered (It will start from Fresh DB)
Support 2-3 user flows such as registration, and consultation
Performance Baselining would be performed for BahmniLite (Clinic implementation) on AWS environment
2. Introduction
The purpose of this document is to provide an outline for the performance test where baselining & capacity of the system will be covered.Following will be covered in this document:
Entry & exit criteria
Load tool Selection & approach
Environment requirement
Baseline test approach
Baseline and Capacity test approach
Test execution activities
Risk Factors
...
Identification of critical flows & agreement
Patient registrationRegistration (New and Existing Patients)
Doctor consultation
Data Patient documents upload
[Stretch scope] Bahmni Report generation
Performance test types finalization
Baseline for single clinic operation
Capacity or max number of clinics can be successfully supported without auto cluster
Load generator setup
Load gen tool: (expected response time) to support clinical operation on a given hardware configuration for varying traffic load
Load generator tool
Gatling
Server performance monitoring
Prometheus (Metrics)
Pothemeus & Grafana (Dashboard)
Data set up (For one clinic)
Basic users setup (1 registration, 1 billing, 3 doctors)
No existing data, start from scratch
Users setup (2 front-desk, 2 doctors - per clinic)
[Draft] Data growth pattern for a clinic Click Here
setup
...
The testing activity will be completed when:
The baseline results are shared and discussed with the team.
Find the capacity of the app to support the maximum number of clinics without any error.
Finalise the costing infra to support above results.
Gatling Performance test reports are generated and published
Report summarizing number of clinics (by user categories) that can be supported for standard load (standard baseline)
5. Load Tool Selection & approach
...
The key challenge with Gatling would be Scala as the programming language.
6. Environment requirement
The performance test will be executed in an new environment which is exclusive for these tests. A stable version of the Bahmni build will be deployed for it. The build number will be the same for all the tests unless any major issue is uncovered.
Server Configuration:
RAM: 16 GB
Hard Drive: 100 GB
8 vCPUs
The server will be instrumented with Pothemeus & Grafana which will help to collect server metrics during the testing.
7. Baseline test approach
Baseline tests will be performed for a single clinicis the version upgrade e.g. minor version upgrade results into breaking changes.
6. Baseline and Capacity test approach
⭕️ Simulations
Below test simulations would be executed to depict various user flows. It won’t have any pre-existing data except for the users preconfigured. The script will perform the following operations:
The test script will include an one hour test of:
Search for patient name/id (User as front desk access)
Existing customer flow: Set appointment for OPD
New customer: Create new patient & Set appointment for OPD
Perform consultation (User as doctor access)
Add up to 8 observations
Add an order
Upload image of prescription
Access patient profile
Following are the numbers associated with a clinic’s operation. Below numbers are the maximum per clinic.
...
User Setup
1 user at registration counter
1 user at billing
3 users as 3 doctors
...
User journeys
Maximum number of patients in 1 hr: 20
Out of 20, 12 are existing patients and 8 are for new registration
1 document (jpeg files) gets uploaded per patient
Admin operations
Weekly/Monthly/Yearly reports on:
...
New patient/existing patients
...
Disease distribution
...
Revenue daily/weekly/monthly
...
Lab consulting / reference
...
Referred out
...
Front-desk / Reception - Patient registration and visit
New Patient: Registration and start OPD visit
Existing Patient: Search using Name/ID and start an OPD visit
Doctor’s Consultation
Access patient profile and documents
Average Patient health condition
Record 10 Observations
Prescribe 3 Medication
Order 1-2 lab test
Expected data presence for a clinic
For baseline, a clinic will start only with preset users only. There will be no patient data.
For subsequent tests to mimic growth of the clinic, we will retain test data.
The tests will be designed according to the above data. Different users will be created for each operation; i.e registration, doctors. SInce, billing is not part of Bahmni, billing will not be considered for performance test scenarios.
⭕️ Baselining
We need to baseline expected response profile for various API’s during user flows (get, query, update, list etc) for varying traffic load conditions (standard, high and peak)
⭕️ Capacity
The peak traffic load simulation would give us maximum load (clinical active users) the system on a given hardware configuration can take (acceptable UX performance) before its starts to either fail OR degrade UX.
⭕️ Key observations
Client side
50th, 75th, 80th, 90th, 95th percentile response time of each API call
Any error received
Server Side
CPU usage
Memory usage
Disk utilisation
Admin operation includes report pull during off-peak hours. For this we need to pre-populate the data as mentioned above.
As per the test, the admin user needs to perform the following operations at once:
Weekly/Monthly/Yearly reports on:
New patient/existing patients
Disease distribution
Revenue daily/weekly/monthly
Lab consulting / reference
Referred out
Follow ups
Key observations
Client side
Response time of each API call
Any error received
Download file integrity
Server Side
CPU usage
Memory usage
Disk utilisation
8. Capacity Test approach
As per this requirement, the BAU operation and admin test will be performed; however, the emulation of clinic numbers will be multiplied by multiplying the count of users and operations. We will take the odd multiplier approach. So it will be x, 3x, 5x and so on. Data accumulated during the tests will work as existing clinics. If needed, we can import synthetic data. This test will be done for 30 mins and both client & server side observations will be noted. The increment will continue until we notice errors on the client side which are usually timeouts or server response errors. Also we will stop incrementing when the CPU, memory, disk space are exhausted. Once we reach this breaking point, we need to reduce the number of clinics to the next below number where the system is stable. That number will be our capacity for the system. To confirm the same, we may need to perform a soak test with that number of clinics.
AWS costing will be based on this number.
...
utilization
7. Test execution activities
The following tasks need to be performed to complete the performance test.
Create performance test environment
Install Pothemeus Prometheus & Grafana on the server
Enable server log for debugging
Scripting of user flows in Gatling
Sequence the user profiling
Execute the tests
Analysis of each execution
Performance Report generation
Report sharing & feedback
...
and publish
8. Risk Factors
Following might affect the execution of the test plan/timelines.
Scala is relatively new for the team. So we need to refer to the documentation & existing Gatling code.
We don't have real time log monitoring. This will create hindrance to debug and get the real response time of the server. In this case, we have to be dependent on tool or client side response time.
Need a spike on best way to import or set up prerequisite data. This is needed to expedite the test relaunch.
...
.
...