Install Components on VMs

This guide details the process of installing various components such as Kafka, Postgres, MongoDB, and different exporters on virtual machines using Ansible scripts.

Preparation Steps

1. Note Down VM Details

Record the IP addresses and names of the VMs created. Names will follow the format {ENV_NAME}vm{NUMBER}, such as demovm1, demovm2, and demovm3.

2. Prepare Ansible Environment Variables

Create and configure the following files:
host.ini
group_vars/all/kafka.yaml
group_vars/all/main.yaml

Example `host.ini` Configuration

[master]
local  ansible_host=10.0.16.4 ansible_ssh_common_args='-o StrictHostKeyChecking=no' ansible_user=ubuntu

[kafka]
demovm1 ansible_host=10.2.16.5 ansible_ssh_common_args='-o StrictHostKeyChecking=no'
demovm2 ansible_host=10.2.16.4 ansible_ssh_common_args='-o StrictHostKeyChecking=no'
demovm3 ansible_host=10.2.16.6 ansible_ssh_common_args='-o StrictHostKeyChecking=no'

[zookeeper]
demovm1 ansible_host=10.2.16.5 ansible_ssh_common_args='-o StrictHostKeyChecking=no'
demovm2 ansible_host=10.2.16.4 ansible_ssh_common_args='-o StrictHostKeyChecking=no'
demovm3 ansible_host=10.2.16.6 ansible_ssh_common_args='-o StrictHostKeyChecking=no'

[postgres]
demovm1 ansible_host=10.2.16.5 ansible_ssh_common_args='-o StrictHostKeyChecking=no'

[mongodb]
demovm3 ansible_host=10.2.16.6 ansible_ssh_common_args='-o StrictHostKeyChecking=no'

[all:vars]
ansible_connection=ssh
ansible_user=ubuntu

Example group_vars/all/kafka.yaml Configuration

kafka_cluster_info:
  demovm1: 1
  demovm2: 2
  demovm3: 3
zookeeper_cluster_info:
  demovm1: 1
  demovm2: 2
  demovm3: 3

Example group_vars/all/main.yaml Configuration

environ: demo

mongodb_ip: 10.2.16.6
mongodb_pass: skxmSnzh
mongodb_admin: admin

# Filebeat can be installed later once logging solution is implemented
# filebeat_output_logstash_hosts:
#  - 10.0.0.33:5044

3. Organize Ansible Files

Place the prepared files under the following directory structure:

envs/
  demo/
    inventory/
      host.ini
      group_vars/
        all/
          kafka.yaml
          main.yaml

Installation Steps

Install Kafka

Run the following commands from the bastion host to install Kafka on the VMs:

ansible-playbook roles/deploy_kafka.yaml -i envs/azure_demo/inventory/host.ini
ansible-playbook roles/create_topics.yaml -i envs/azure_demo/inventory/host.ini

Verification

Check the status of Zookeeper and Kafka services on each VM:

service zookeeper status
service kafka status

Install Postgres

Deploy the Postgres Database on demovm1:

ansible-playbook roles/deploy_patroni.yaml -i envs/azure_demo/inventory/host.ini

Postgres User and Database Setup

Log in to the Postgres host (demovm1) and create the necessary users and databases:

-- Connect to Postgres
psql -U postgres

-- Create Helix application user and database
CREATE USER helixapp WITH PASSWORD '<PASSWORD_FOR_HELIX>';
CREATE DATABASE helixdb OWNER helixapp;
GRANT ALL PRIVILEGES ON DATABASE helixdb TO helixapp;

-- Create FHIR server user and database
CREATE USER fhirserver WITH PASSWORD '<PASSWORD_FOR_FHIR_SERVER>';
CREATE DATABASE fhirserver OWNER fhirserver;
GRANT ALL PRIVILEGES ON DATABASE fhirserver TO fhirserver;

-- Create Keycloak server user and database
CREATE USER keycloak WITH PASSWORD '<PASSWORD_FOR_FHIR_SERVER>';
CREATE DATABASE keycloak OWNER keycloak;
GRANT ALL PRIVILEGES ON DATABASE keycloak TO keycloak;

Grant Privileges on helixdb: Connect to the helixdb database and execute the following queries:

\c helixdb
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO helixapp;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO helixapp;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON TABLES TO helixapp;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON SEQUENCES TO helixapp;

Grant Privileges on fhirserver: Connect to the fhirserver database and execute the following queries:

\c fhirserver
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO fhirserver;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO fhirserver;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON TABLES TO fhirserver;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON SEQUENCES TO fhirserver;

Grant Privileges on keycloak: Connect to the keycloak database and execute the following queries:

\c keycloak
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO keycloak;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO keycloak;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON TABLES TO keycloak;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON SEQUENCES TO keycloak;

Install MongoDB

Install MongoDB on the demovm3 host:

ansible-playbook roles/deploy_mongo.yaml -i envs/azure_demo/inventory/host.ini