Clinic implementation - Performance Baseline & capacity test plan

 

1. Purpose

We are intending to baseline the performance of the Bahmni app for clinics. The test will be extended to check the maximum number of clinics can be supported with current infrastructure without scaling up.

 

2. Introduction

As part of Bahmni NEXT, the solution should meet the acceptance criteria in both functional and cross functional areas. The purpose of this document is to provide an outline for the performance test where baselining & capacity of the system will be covered.

Following will be covered in this document:

  • Entry & exit criteria

  • Load tool Selection & approach

  • Environment requirement

  • Baseline test approach

  • Capacity test approach

  • Test execution activities

  • Risk Factors

 

3. Entry criteria

  • Identification of critical flows & agreement

    • Patient registration

    • Doctor consultation

    • Data upload

    • Report generation

  • Performance test types finalization

    • Baseline for single clinic operation

    • Capacity or max number of clinics can be successfully supported without auto cluster

  • Load generator setup

    • Load gen tool: Gatling

  • Server performance monitoring

    • Pothemeus & Grafana

  • Data set up (For one clinic)

    • Basic users setup (1 registration, 1 billing, 3 doctors)

    • No existing data, start from scratch

    • Data growth pattern for a clinic Click Here

 

4. Exit criteria

The testing activity will be completed when:

  • The baseline results are shared and discussed with the team.

  • Find the capacity of the app to support the maximum number of clinics without any error.

  • Finalise the costing infra to support above results.

5. Load Tool Selection & approach

Gatling will be used as the load generation tool as the team has earlier experience of working with it. Gatling is good at creating and emulating a large number of users compared to Jmeter without adding more hardware. Comparatively Gatling is less resource intensive, hence it makes a good choice compared to Jmeter.

The key challenge with Gatling would be Scala as the programming language.

 

6. Environment requirement

The performance test will be executed in an new environment which is exclusive for these tests. A stable version of the Bahmni build will be deployed for it. The build number will be the same for all the tests unless any major issue is uncovered.

Server Configuration:

RAM: 16 GB

Hard Drive: 100 GB

8 vCPUs

The server will be instrumented with Pothemeus & Grafana which will help to collect server metrics during the testing.

 

7. Baseline test approach

Baseline tests will be performed for a single clinic. It won’t have any pre-existing data except for the users preconfigured. The script will perform the following operations:

The test script will include an one hour test of:

  • Search for patient name/id (User as front desk access)

    • Existing customer flow: Set appointment for OPD

    • New customer: Create new patient & Set appointment for OPD

  • Perform consultation (User as doctor access)

    • Add up to 8 observations

    • Add an order

  • Upload image of prescription

  • Access patient profile

Following are the numbers associated with a clinic’s operation. Below numbers are the maximum per clinic.

  • User Setup

    • 1 user at registration counter

    • 1 user at billing

    • 3 users as 3 doctors

  • User journeys

    • Maximum number of patients in 1 hr: 20

    • Out of 20, 12 are existing patients and 8 are for new registration

    • 1 document (jpeg files) gets uploaded per patient

  • Admin operations

    • Weekly/Monthly/Yearly reports on:

      • New patient/existing patients

      • Disease distribution

      • Revenue daily/weekly/monthly

      • Lab consulting / reference

      • Referred out

      • Follow ups

  • Expected data presence for a clinic

    • For baseline, a clinic will start only with preset users only. There will be no patient data.

    • For subsequent tests to mimic growth of the clinic, we will retain test data.

The tests will be designed according to the above data. Different users will be created for each operation; i.e registration, doctors. SInce, billing is not part of Bahmni, billing will not be considered for performance test scenarios.

 

Key observations

  • Client side

    • 50th, 75th, 80th, 90th, 95th percentile response time of each API call

    • Any error received

  • Server Side

    • CPU usage

    • Memory usage

    • Disk utilisation

 

Admin operation includes report pull during off-peak hours. For this we need to pre-populate the data as mentioned above.

As per the test, the admin user needs to perform the following operations at once:

  • Weekly/Monthly/Yearly reports on:

    • New patient/existing patients

    • Disease distribution

    • Revenue daily/weekly/monthly

    • Lab consulting / reference

    • Referred out

    • Follow ups

Key observations

  • Client side

    • Response time of each API call

    • Any error received

    • Download file integrity

  • Server Side

    • CPU usage

    • Memory usage

    • Disk utilisation

8. Capacity Test approach

As per this requirement, the BAU operation and admin test will be performed; however, the emulation of clinic numbers will be multiplied by multiplying the count of users and operations. We will take the odd multiplier approach. So it will be x, 3x, 5x and so on. Data accumulated during the tests will work as existing clinics. If needed, we can import synthetic data. This test will be done for 30 mins and both client & server side observations will be noted. The increment will continue until we notice errors on the client side which are usually timeouts or server response errors. Also we will stop incrementing when the CPU, memory, disk space are exhausted. Once we reach this breaking point, we need to reduce the number of clinics to the next below number where the system is stable. That number will be our capacity for the system. To confirm the same, we may need to perform a soak test with that number of clinics.

 AWS costing will be based on this number.

 

9. Test execution activities

The following tasks need to be performed to complete the performance test.

  • Create performance test environment

  • Install Pothemeus & Grafana on the server

  • Enable server log for debugging

  • Scripting of user flows in Gatling

  • Sequence the user profiling

  • Execute the tests

  • Analysis of each execution

  • Report generation

  • Report sharing & feedback

10. Risk Factors

Following might affect the execution of the test plan/timelines.

  • Scala is relatively new for the team. So we need to refer to the documentation & existing Gatling code.

  • We don't have real time log monitoring. This will create hindrance to debug and get the real response time of the server. In this case, we have to be dependent on tool or client side response time.

  • Need a spike on best way to import or set up prerequisite data. This is needed to expedite the test relaunch.

 

Presentation: https://docs.google.com/presentation/d/1lks897jA8kmihs_dS0apmDe5gSIlgwNFlL6m0Gz11js/edit?usp=sharing

Task details & other information: https://docs.google.com/spreadsheets/d/1WfgTU_rYxb7UWAwYEMJex4xOB6xQUngQH8LVOMexWzw/edit?usp=sharing