Incremental Update

Bahmni-mart supports incremental update during data flattening.This can be easily enabled by adding incremental update config in /var/www/bahmni_config/bahmni-mart/bahmni-mart.json for few specific jobs. Not all of them come under this category. Please check this section for more details.

Note: Due to some technical differences in Bahmni, incremental update config is not suggested for bacteriology job. Find more details in openmrs-talk

Check here for example.

Key Name	Description	Default Value	Required
updateOn	Column name in the analytics database depends on which incremental update should work	N/A	yes
eventCategory	Category name present in event_records table (openmrs database)	N/A	yes
openmrsTableName	Table name in OpenMRS for given eventCategory	N/A	yes

Enable/Disbale Incremental Config for Specific Jobs:

The below jobs have incremental config enabled by default. So every time we do the bahmni-mart load, the tables specific to the job run on incremental load. If somebody wants to run any of the job specific tables on full load with every bahmni-mart load, we can remove the incrementalUpdateConfig section under that table from the specific job json file.

Person
Patients
Programs
Appointments
Bed Management
Operation Theater
Visits and Encounters
Medication And Orders
Diagnoses And Conditions
Provider

But this is not a simple configuration change that we can do in the bahmni-mart.json file directly in the bahmni instance. Changing the configuration in the job specific json file would require building the bahmni-mart rpm and deploying the same on bahmni instance.

1. Person Job:

Flattened Mart Tables

person_details_default 
person_address_default
person_attribute_info_default
address_hierarchy_level_default

Both person_details_default and person_address_defaultmart tables have incrementalUpdateConfig enabled by default. And the rest of the tables run on full load.

person.json:

{
  "jobs": [
    {
      "name": "Person details",
      "type": "customSql",
      "readerSql": "SELECT p.person_id, pn.person_name_id, pn.preferred, pn.prefix, pn.given_name, pn.middle_name, pn.family_name_prefix, pn.family_name, pn.family_name2, pn.family_name_suffix, p.gender, p.birthdate, EXTRACT(YEAR FROM p.birthdate) AS birthyear, p.birthtime, p.birthdate_estimated, FORMAT((DATEDIFF(CURRENT_DATE, p.birthdate) / 365), 0) AS age, CASE WHEN ROUND(DATEDIFF(CURRENT_DATE(), p.birthdate) / 365, 0) < '5' THEN '0-5' WHEN ROUND(DATEDIFF(CURRENT_DATE(), p.birthdate) / 365, 0) BETWEEN '5' AND '14' THEN '05-14' WHEN ROUND(DATEDIFF(CURRENT_DATE(), p.birthdate) / 365, 0) BETWEEN '15' AND '24' THEN '15-24' WHEN ROUND(DATEDIFF(CURRENT_DATE(), p.birthdate) / 365, 0) BETWEEN '25' AND '34' THEN '25-34' WHEN ROUND(DATEDIFF(CURRENT_DATE(), p.birthdate) / 365, 0) BETWEEN '35' AND '44' THEN '35-44' WHEN ROUND(DATEDIFF(CURRENT_DATE(), p.birthdate) / 365, 0) BETWEEN '45' AND '54' THEN '45-54' WHEN ROUND(DATEDIFF(CURRENT_DATE(), p.birthdate) / 365, 0) BETWEEN '55' AND '64' THEN '55-64' WHEN ROUND(DATEDIFF(CURRENT_DATE(), p.birthdate) / 365, 0) >= '65' THEN '65+' END AS age_group, p.dead, p.death_date, p.deathdate_estimated, p.cause_of_death FROM person p INNER JOIN person_name pn ON p.person_id = pn.person_id AND p.voided = FALSE AND pn.voided = FALSE",
      "tableName": "person_details_default",
      "incrementalUpdateConfig": {
        "updateOn": "person_id",
        "eventCategory": "patient",
        "openmrsTableName": "person"
      }
    },
    {
      "name": "Person address",
      "type": "customSql",
      "readerSql": "SELECT person_id, preferred, address1, address2, address3, address4, address5, address6, address7, address8, address9, address10, address11, address12, address13, address14, address15, city_village, state_province, postal_code, country, county_district, latitude, longitude, start_date, end_date FROM person_address WHERE voided = FALSE",
      "tableName": "person_address_default",
      "incrementalUpdateConfig": {
        "updateOn": "person_id",
        "eventCategory": "patient",
        "openmrsTableName": "person"
      }
    },
    {
      "name": "Person Attribute Info",
      "type": "customSql",
      "readerSql": "SELECT person_attribute_type_id, name, description FROM person_attribute_type",
      "tableName": "person_attribute_info_default"
    },
    {
      "name": "Address hierarchy levels",
      "type": "customSql",
      "readerSql": "SELECT address_hierarchy_level_id, name, parent_level_id, address_field FROM address_hierarchy_level",
      "tableName": "address_hierarchy_level_default"
    }
  ]
}

2. Patients Job:

Flattened Mart Tables

patient_state_default
patient_allergy_status_default
patient_identifier

All the patient mart tables have incrementalUpdateConfig enabled by default.

patients.json:

3. Programs Job:

Flattened Mart Tables

Both patient_program_data_default and program_attributes tables have incrementalUpdateConfig enabled by default. And the rest of the mart tables run on full load.

Programs.json:

4. Appointments Job:

Flattened Mart Tables

Note : The below Mart tables are created by customSql job type.

patient_appointment_default , appointment_service_default and service_availability_default tables have incrementalUpdateConfig enabled by default. And appointment_speciality_default table run on full load.

appointments.json:

5. Bed Management Job:

Flattened Mart Tables

Note : The below Mart tables are created by customSql job type.

Both bed_patient_assignment_default and bed_tags_default tables have incrementalUpdateConfig enabled by default. And current_bed_details_default table run on full load.

bedManagement.json:

6. Operation Theater Job:

Flattened Mart Tables

surgical_block_default, surgical_appointment_default and surgical_appointment_attributes tables have incrementalUpdateConfig enabled by default.

And surgical_appointment_attribute_type_details_default table run on full load.

operationTheater.json:

7. Visits and Encounters Job:

Flattened Mart Tables

Only patient_encounter_details_default table has incrementalUpdateConfig enabled by default. Rest of the tables run on full load.

visitsAndEncounters.json:

8.Medication And Orders Job:

Both Medication and Orders are combined into one job, have one single json file for this.

Flattened mart tables

All the above mart tables specific Medication and Orders have incrementalUpdateConfig enabled by default.

medicationAndOrders.json:

9. Diagnoses And Conditions Job:

Both Diagnoses and Conditions are combined into one job, have one single json file for this.

Mart Tables

Only Diagnosis specific tables have incrementalUpdateConfig enabled by default. conditions_default run on full load by default.

diagnosesAndConditions.json:

10. Provider Job:

Flattened Mart Tables

Only provider_attributes table has incrementalUpdateConfig enabled by default. And rest of the tables run on full load by default.

provider.json: