Autonity with Docker Compose¶
Run an Autonity node using Docker Compose in containers with the following features:
- running telegraf for metrics
- fluentd for logging
- nginx to enable secure https traffic with the node
Docker Compose will start the containers at the same time with the correct arguments, and automatically restart them on failure. It will provide an internal network between the four containers, allowing them to securely communicate with each other. This guide explains how to run docker compose in a way that integrates with existing clearmatics infrastructure using telegraf for metrics and fluentd for logging. It can easily be modified to be used for your own metrics and logging solutions using telegraf and fluentd respectively.
Follow the instructions here to install Docker Compose.
Clone the 'autonity-compose' repo:
git clone firstname.lastname@example.org:clearmatics/autonity-compose.git cd autonity-compose
Edit the configuration file
docker-compose.ymlwith the following contents. Replace the tag for logs on line
59with one of your choosing:
version: "3.8" services: nginx: image: nginx ports: - "8080:8080" - "8080:8080/udp" - "8081:8081" - "8081:8081/udp" volumes: - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf - ./nginx/tls.crt:/etc/nginx/ssl/tls.crt - ./nginx/tls.key:/etc/nginx/ssl/tls.key - ./nginx/dhparam.pem:/etc/nginx/ssl/dhparam.pem depends_on: - autonity user: $UID:$GID fluentd: image: custom-fluentd:latest ports: - "24224:24224" - "24224:24224/udp" - "8888:8888" - "8888:8888/udp" user: $UID:$GID command: -v autonity: image: ghcr.io/clearmatics/services-testnet-agc:v0.7.1-bakerloo06 ports: - "8545:8545" - "8546:8546" - "30303:30303" - "6060:6060" user: $UID:$GID volumes: - .:/autonity entrypoint: ["./secrets/wait-for-fluentd.sh", "http://fluentd:8888", "--", "/secrets/main.sh", "--datadir=/autonity"] restart: on-failure logging: driver: "fluentd" options: fluentd-async-connect: 'true' fluentd-address: localhost:24224 tag: "autonity2" telegraf: image: telegraf ports: - "8092:8092/udp" - "8094:8094" - "8125:8125/udp" user: $UID:$GID volumes: - ./Telegraf.conf:/etc/telegraf/telegraf.conf restart: on-failure # To Run: # export UID=$(id -u) export GID=$(id -g) # docker-compose up
This file defines the images, ports, and volumes that are used to run the
telegrafcontainer, and the
fluentdcontainer. Similar to running Autonity with a bare docker container, the
autonitycontainer created by Docker Compose uses a shared datadir folder, so the blockchain and identity will be persistent. As Autonity must be started after fluentd has already spun up (so all the logs are saved), a script in the Autonity image is used to test when
fluentdis up and running and will only start Autonity when it has confirmed this.
Create a custom fluentd image for logging.
Go to the directory for building the custom fluentd image
Edit the file
fluent.conf, replacing the elasticsearch
password, which you can find in Bitwarden vault:
<source> @type forward port 24224 bind 0.0.0.0 </source> <match **> @type relabel @label @NORMAL </match> <source> @type http_healthcheck port 8888 bind 0.0.0.0 </source> <label @NORMAL> <match **> @id elasticsearch @type elasticsearch @log_level "info" include_tag_key true host elasticsearch.bakerloo.autonity.network port 9200 path "" scheme "https" ssl_verify true ssl_version "TLSv1_2" type_name "_doc" user <GET-FROM-BITWARDEN> password <GET-FROM-BITWARDEN> logstash_format true logstash_prefix "logstash" reconnect_on_error true </match> </label>#
Build the image:
docker build -t custom-fluentd:latest ./
Note: If for testing purposes you decide to omit this step so you can view the Autonity logs directly, comment out the fluentd section in
docker-compose.yml, and modify the Autonity section to be the following:
autonity: image: ghcr.io/clearmatics/services-testnet-agc:v0.7.1-bakerloo06 ports: - "8545:8545" - "8546:8546" - "30303:30303" - "6060:6060" user: $UID:$GID volumes: - .:/autonity command: "--datadir=/autonity" restart: on-failure
Edit the telegraf configuration file
telegraf.conf, adding the
afnc-name, and add the InfluxDB V2 token from Bitwarden under
[global_tags] networkid = "444900" afnc_name = "<node-name>" [agent] round_interval = true metric_batch_size = 1000 metric_buffer_limit = 10000 collection_jitter = "0s" flush_interval = "10s" flush_jitter = "0s" precision = "" debug = false quiet = false logfile = "" hostname = "" omit_hostname = false [[inputs.prometheus]] urls = ["http://autontiy:6060/debug/metrics/prometheus"] [[outputs.influxdb_v2]] urls = [ "https://telemetry.bakerloo.autonity.network"] organization = "influxdata" bucket = "default" token = "<GET FROM BITWARDEN>"
Notice that the url for the input is to
autonity rather than
localhost. This is because it is using the internal network between the two containers, rather than pointing to a port forwarded to localhost.
Optional but recommended¶
If you choose not to use nginx, comment out the nginx section in
Setup certificates for Nginx.
Nginx can be used as a proxy server between users and the Autonity node. This allows users to submit requests to the node using the more secure
HTTPSprotocol rather that
Next you must get a tls certificate and private key from a certificate authority such as Let's Encrypt, these are required to use
HTTPS. It is recommended to use the free Certbot service provided by Let's Encrypt, by following the instructions for using certbox with a nginx server on ubuntu. In step 7, run the command to just get a certificate.
Go to the
Save the tls certificate and private key from the previous step:
sudo cp /etc/letsencrypt/live/composer.test.autonity.io/fullchain.pem ./tls.crt sudo cp /etc/letsencrypt/live/composer.test.autonity.io/privkey.pem ./tls.key
Generate the DH Parameters, which are also required for ssl:
openssl dhparam -out dhparam.pem 1024
nginx.conffile sets how nginx will listen for encrypted ssl traffic on ports 8080 (for https) and 8081 (for wss), then decrypt the traffic and pass it to the Autonity container on ports 8545 (for http) and 8546 (for ws) respectively. With nginx you can now send secure https traffic to the Autonity node by targeting port 8080, or use wss (secure websocket) by targeting port 8081.
Before you run you should run the following command which will permission the user to open the shared Autonity folder:
export UID=$(id -u) export GID=$(id -g)
Now start docker compose from the
You will now see the logs output from the fluentd, telegraf, and nginx containers, as they are started in unison. The Autonity logs are sent to fluentd which sends them to elasticsearch.