Monitor Consul menggunakan Prometheus dan Grafana

Monitoring pada consul server terkadang diperlukan agar dapat mengetahui performa atau sekedar mengumpulkan data metric agar dapat dibaca melalui Grafana. Agar metric dapat dikirimkan ke Grafana, maka diperlukan prometheus sebagai data sumber atau data source.

Enable telemetry

Buat file server.json lalu edit seperti berikut.

{
   "datacenter": "dc1",
   "telemetry" : {
      "prometheus_retention_time" : "60s",
      "disable_hostname" : true
        },
   "enable_local_script_checks" : true,
   "enable_script_checks" : true,
}

Lalu jalankan consul sebagai server.

docker run -dit --name consul-server \
-v server.json:/consul/config/server.json:ro \
-p 8500:8500 -p 8600:8600 \
consul agent -server -ui -node=consul-server bootstrap-expect=1 -client=0.0.0.0

Selanjutnya test akses metrics

curl http://127.0.0.1:8500/v1/agent/metrics

{"Timestamp":"2023-06-05 00:15:30 +0000 UTC","Gauges":[{"Name":"consul.autopilot.failure_tolerance","Value":0,"Labels":{}},
{"Name":"consul.autopilot.healthy","Value":1,"Labels":{}},{"Name":"consul.consul.members.clients","Value":0,"Labels":{"datacenter":"dc1"}},
{"Name":"consul.consul.members.servers","Value":1,"Labels":{"datacenter":"dc1"}},{"Name":"consul.consul.state.config_entries","Value":0,"Labels":
...

Configure node-exporter

Untuk menjalankan node-exporter

docker run -dit --name node \
-p 9100:9100 \
-v /proc:/host/proc:ro \
-v /sys:/host/sys:ro \
-v /:/rootfs:ro \
prom/node-exporter '--path.procfs=/host/proc'  '--path.sysfs=/host/sys' --collector.filesystem.ignored-mount-points "^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)"

Configure prometheus

Buat file prometheus.yml lalu edit seperti berikut

global:
  scrape_interval: 30s
  scrape_timeout: 10s

scrape_configs:
  - job_name: services
    metrics_path: /metrics
    static_configs:
      - targets:
          - 'prom:9090'
  - job_name: 'consul'
    metrics_path: '/v1/agent/metrics'
    params:
      format: ['prometheus']
    static_configs:
      - targets: ['consul-server:8500']
  - job_name: node
    metrics_path: /metrics
    static_configs:
      - targets:
          - 'node-exporter:9100'

Jalankan prometheus

docker run -dit --name prom \
-v ./prometheus.yml:/etc/prometheus/prometheus.yml \
-p 9090:9090 \
prom/prometheus --web.enable-lifecycle --config.file=/etc/prometheus/prometheus.yml

Akses http://127.0.0.1:9090 melalui browser dan pastikan job service dan consul kondisi UP

Configure grafana

Buat file grafana.ini lalu edit seperti berikut.

# Sample file: https://github.com/grafana/grafana/blob/master/conf/sample.ini

instance_name = ${HOSTNAME}

#################################### Security ############################
[security]

# default admin user, created on startup
admin_user = admin

# default admin password, can be changed before first start of grafana, or in profile settings
admin_password = admin

admin_user dan admin_password dapat Anda sesuaikan.

Jalankan grafana

docker run -dit --name grafana \
-p 3000:3000 \
-v ./grafana.ini:/etc/grafana/grafana.ini \
grafana/grafana

Selanjutnya akses dashboard grafana melalui http://127.0.0.1:3000

Dari Home arahkan ke Connections > Your connections > Data sources dan klik Add data source

Pilih Prometheus dan masukan URL

Pastikan Data source is working

Kembali ke Home lalu arahkan ke menu Dashboard > New > Import

Masukan id 13396 lalu klik Load

Untuk bagian Name dan Folder dapat Anda sesuaikan lalu pilih prometheus dan klik Import

Tampilan dashboard

Lakukan hal yang sama untuk import dashboard node exporter.

Tampilan dashboard