1. Purpose
We are intending to baseline the performance of the Bahmni app for clinics. The test will be extended to check the maximum number of clinics can be supported with current infrastructure without scaling up.
2. Introduction
As part of Bahmni NEXT, the solution should meet the acceptance criteria in both functional and cross functional areas. The purpose of this document is to provide an outline for the performance test where baselining & capacity of the system will be covered.
Following will be covered in this document:
Entry & exit criteria
Load tool Selection & approach
Environment requirement
Baseline test approach
Capacity test approach
Test execution activities
Risk Factors
3. Entry criteria
Identification of critical flows & agreement
Patient registration
Doctor consultation
Data upload
Report generation
Performance test types finalization
Baseline for single clinic operation
Capacity or max number of clinics can be successfully supported without auto cluster
Load generator setup
Load gen tool: Gatling
Server performance monitoring
Pothemeus & Grafana
Data set up (For one clinic)
Basic users setup (1 registration, 1 billing, 3 doctors)
No existing data, start from scratch
Data growth pattern for a clinic Click Here
4. Exit criteria
The testing activity will be completed when:
The baseline results are shared and discussed with the team.
Find the capacity of the app to support the maximum number of clinics without any error.
Finalise the costing infra to support above results.
5. Load Tool Selection & approach
Gatling will be used as the load generation tool as the team has earlier experience of working with it. Gatling is good at creating and emulating a large number of users compared to Jmeter without adding more hardware. Comparatively Gatling is less resource intensive, hence it makes a good choice compared to Jmeter.
The key challenge with Gatling would be Scala as the programming language.
6. Environment requirement
The performance test will be executed in an new environment which is exclusive for these tests. A stable version of the Bahmni build will be deployed for it. The build number will be the same for all the tests unless any major issue is uncovered.
Server Configuration:
RAM: 16 GB
Hard Drive: 100 GB
8 vCPUs
The server will be instrumented with Pothemeus & Grafana which will help to collect server metrics during the testing.
7. Baseline test approach
Baseline tests will be performed for a single clinic. It won’t have any pre-existing data except for the users preconfigured. The script will perform the following operations:
The test script will include an one hour test of:
Search for patient name/id (User as front desk access)
Existing customer flow: Set appointment for OPD
New customer: Create new patient & Set appointment for OPD
Perform consultation (User as doctor access)
Add up to 8 observations
Add an order
Upload image of prescription
Access patient profile
Following are the numbers associated with a clinic’s operation. Below numbers are the maximum per clinic.
User Setup
1 user at registration counter
1 user at billing
3 users as 3 doctors
User journeys
Maximum number of patients in 1 hr: 20
Out of 20, 12 are existing patients and 8 are for new registration
1 document (jpeg files) gets uploaded per patient
Admin operations
Weekly/Monthly/Yearly reports on:
New patient/existing patients
Disease distribution
Revenue daily/weekly/monthly
Lab consulting / reference
Referred out
Follow ups
Expected data presence for a clinic
For baseline, a clinic will start only with preset users only. There will be no patient data.
For subsequent tests to mimic growth of the clinic, we will retain test data.
The tests will be designed according to the above data. Different users will be created for each operation; i.e registration, doctors. SInce, billing is not part of Bahmni, billing will not be considered for performance test scenarios.
Key observations
Client side
50th, 75th, 80th, 90th, 95th percentile response time of each API call
Any error received
Server Side
CPU usage
Memory usage
Disk utilisation
Admin operation includes report pull during off-peak hours. For this we need to pre-populate the data as mentioned above.
As per the test, the admin user needs to perform the following operations at once:
Weekly/Monthly/Yearly reports on:
New patient/existing patients
Disease distribution
Revenue daily/weekly/monthly
Lab consulting / reference
Referred out
Follow ups
Key observations
Client side
Response time of each API call
Any error received
Download file integrity
Server Side
CPU usage
Memory usage
Disk utilisation
8. Capacity Test approach
As per this requirement, the BAU operation and admin test will be performed; however, the emulation of clinic numbers will be multiplied by multiplying the count of users and operations. We will take the odd multiplier approach. So it will be x, 3x, 5x and so on. Data accumulated during the tests will work as existing clinics. If needed, we can import synthetic data. This test will be done for 30 mins and both client & server side observations will be noted. The increment will continue until we notice errors on the client side which are usually timeouts or server response errors. Also we will stop incrementing when the CPU, memory, disk space are exhausted. Once we reach this breaking point, we need to reduce the number of clinics to the next below number where the system is stable. That number will be our capacity for the system. To confirm the same, we may need to perform a soak test with that number of clinics.
AWS costing will be based on this number.
9. Test execution activities
The following tasks need to be performed to complete the performance test.
Create performance test environment
Install Pothemeus & Grafana on the server
Enable server log for debugging
Scripting of user flows in Gatling
Sequence the user profiling
Execute the tests
Analysis of each execution
Report generation
Report sharing & feedback
10. Risk Factors
Following might affect the execution of the test plan/timelines.
Scala is relatively new for the team. So we need to refer to the documentation & existing Gatling code.
We don't have real time log monitoring. This will create hindrance to debug and get the real response time of the server. In this case, we have to be dependent on tool or client side response time.
Need a spike on best way to import or set up prerequisite data. This is needed to expedite the test relaunch.
Presentation: https://docs.google.com/presentation/d/1lks897jA8kmihs_dS0apmDe5gSIlgwNFlL6m0Gz11js/edit?usp=sharing
Task details & other information: https://docs.google.com/spreadsheets/d/1WfgTU_rYxb7UWAwYEMJex4xOB6xQUngQH8LVOMexWzw/edit?usp=sharing