Setup Centralised Logging

Bahmni supports centralised logging which helps to visualise logs from different containers at a unified UI. Centralised logging has been configured with PLG (Promtail, Loki, Grafana) stack. We primarily leverage Loki and Grafana components of the stack as Bahmni runs on docker containers. If system logs needs to be extracted then, Promtail can be used.

1. Install Loki Logging Driver

Loki stack uses Loki Docker Driver Client for getting logs of container and pushing to Loki. Run the following command to install the driver or take a look here.

docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions

Note: This plugin needs to be installed in both the Bahmni application server and monitoring server.

2. Update Logging Method in docker-compose.yml

The default configuration of docker-compose ships with default logging that is configured in docker daemon.

To enable loki logging for the services, in the docker-compose file under bahmni-lite or bahmni-standard folder, find the x-log-config section and update the value from *default to *loki.

Note: Remember to make the change in the docker-compose.yml file in both bahmni-docker repository and bahmni-observability repositories.

3. Update the URL of Loki

The .env file in bahmni-docker/bahmni-lite (or) bahmni-docker/bahmni-standard directories contain a variable LOKI_URL which determines where the loki instance is running and the logs should be pushed.

If you are running the loki stack on the same machine, then use http://localhost:3100/loki/api/v1/push as the value for the variable

If you are running the loki stack on an external server, then use http://<ip>:3100/loki/api/v1/push as the value for the variable.

4. Start Loki stack services

Step 1: Clone the bahmni-observability repository

git clone https://github.com/Bahmni/bahmni-observability

Step 2: Add logging profile to the COMPOSE_PROFILES in the .env file

Example: 
COMPOSE_PROFILES=logging
(or)
COMPOSE_PROFILES=monitoring,logging

Now loki stack services can be started by running from the bahmni-observability directory,

5. Viewing Logs

Now navigate to http://localhost:81 (or) https://localhost:445 and login with admin as username and password and reset your password.
Next click on Explore Icon in the Grafana Left Panel.
Use the log browser to choose label as compose_service and value with the container needed, then Click on Run Query on the top right of the screen. This will show the current logs.
You can also click on Live on the top right to see the live stream of the logs.

6. Loki Configuration

Loki supports different configuration in the form of YAML file. These configurations can be added in loki-config.yaml file.

i. Log Retention

The centralised logging stack is configured with a default retention period of 24 hours. But this can be configured as per requirements. To do so find the retention_period config in the loki-config.yaml and change the value (minimum: 24h) as per requirement.

Example:

After updating the configuration file, restart loki container by running docker compose restart loki