Skip to content

Autonity with Docker Compose

Run an Autonity node using Docker Compose in containers with the following features:

  • running telegraf for metrics
  • fluentd for logging
  • nginx to enable secure https traffic with the node

Docker Compose will start the containers at the same time with the correct arguments, and automatically restart them on failure. It will provide an internal network between the four containers, allowing them to securely communicate with each other. This guide explains how to run docker compose in a way that integrates with existing clearmatics infrastructure using telegraf for metrics and fluentd for logging. It can easily be modified to be used for your own metrics and logging solutions using telegraf and fluentd respectively.

Setup

  1. Follow the instructions here to install Docker Compose.

  2. Clone the 'autonity-compose' repo:

    git clone git@github.com:clearmatics/autonity-compose.git
    cd autonity-compose
    
  3. Edit the configuration file docker-compose.yml with the following contents. Replace the tag for logs on line 59 with one of your choosing:

    version: "3.8"
    services:
    
      nginx:
        image: nginx
        ports:
          - "8080:8080"
          - "8080:8080/udp"
          - "8081:8081"
          - "8081:8081/udp"
        volumes:
          - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf
          - ./nginx/tls.crt:/etc/nginx/ssl/tls.crt
          - ./nginx/tls.key:/etc/nginx/ssl/tls.key
          - ./nginx/dhparam.pem:/etc/nginx/ssl/dhparam.pem
        depends_on:
          - autonity
        user: $UID:$GID
    
      fluentd:
        image: custom-fluentd:latest
        ports:
          - "24224:24224"
          - "24224:24224/udp"
          - "8888:8888"
          - "8888:8888/udp"
    
        user: $UID:$GID
        command: -v
    
      autonity:
        image: ghcr.io/clearmatics/services-testnet-agc:v0.7.1-bakerloo06
        ports:
          - "8545:8545"
          - "8546:8546"
          - "30303:30303"
          - "6060:6060"
        user: $UID:$GID
        volumes: 
          - .:/autonity
        entrypoint: ["./secrets/wait-for-fluentd.sh", "http://fluentd:8888", "--", "/secrets/main.sh", "--datadir=/autonity"]
    
        restart:
          on-failure
    
        logging:
          driver: "fluentd"
          options:
            fluentd-async-connect: 'true'
            fluentd-address: localhost:24224
            tag: "autonity2"
    
      telegraf:
        image: telegraf
        ports:
          - "8092:8092/udp"
          - "8094:8094"
          - "8125:8125/udp"
        user: $UID:$GID
        volumes:
          - ./Telegraf.conf:/etc/telegraf/telegraf.conf
        restart:
          on-failure
    
    # To Run:
    
    # export UID=$(id -u) export GID=$(id -g)
    # docker-compose up
    

    This file defines the images, ports, and volumes that are used to run the autonity container, the telegraf container, and the fluentd container. Similar to running Autonity with a bare docker container, the autonity container created by Docker Compose uses a shared datadir folder, so the blockchain and identity will be persistent. As Autonity must be started after fluentd has already spun up (so all the logs are saved), a script in the Autonity image is used to test when fluentd is up and running and will only start Autonity when it has confirmed this.

  4. Create a custom fluentd image for logging.

    • Go to the directory for building the custom fluentd image

      cd custom-fluentd
      

    Edit the file fluent.conf, replacing the elasticsearch user and password, which you can find in Bitwarden vault:

    <source>
      @type forward
      port 24224
      bind 0.0.0.0
    </source>
    
    <match **>
      @type relabel
      @label @NORMAL
    </match>
    
    <source>
      @type http_healthcheck
      port 8888
      bind 0.0.0.0
    </source>
    
    <label @NORMAL>
    <match **>
      @id elasticsearch
      @type elasticsearch
      @log_level "info"
      include_tag_key true
      host elasticsearch.bakerloo.autonity.network
      port 9200
      path ""
      scheme "https"
      ssl_verify true
      ssl_version "TLSv1_2"
      type_name "_doc"
      user <GET-FROM-BITWARDEN>
      password <GET-FROM-BITWARDEN>
      logstash_format true
      logstash_prefix "logstash"
      reconnect_on_error true
    
    </match>
    </label># 
    
  • Build the image:

    docker build -t custom-fluentd:latest ./
    

Note: If for testing purposes you decide to omit this step so you can view the Autonity logs directly, comment out the fluentd section in docker-compose.yml, and modify the Autonity section to be the following:

autonity:
  image: ghcr.io/clearmatics/services-testnet-agc:v0.7.1-bakerloo06
    ports:
      - "8545:8545"
      - "8546:8546"
      - "30303:30303"
      - "6060:6060"
    user: $UID:$GID
      volumes:
        - .:/autonity
    command: "--datadir=/autonity"
    restart:
      on-failure
  1. Edit the telegraf configuration file telegraf.conf, adding the afnc-name, and add the InfluxDB V2 token from Bitwarden under Grafana metrics-v2 here.

    [global_tags]
    networkid = "444900"
    afnc_name = "<node-name>"
    [agent]
    round_interval = true
    metric_batch_size = 1000
    metric_buffer_limit = 10000
    collection_jitter = "0s"
    flush_interval = "10s"
    flush_jitter = "0s"
    precision = ""
    debug = false
    quiet = false
    logfile = ""
    hostname = ""
    omit_hostname = false
    [[inputs.prometheus]]
    urls = ["http://autontiy:6060/debug/metrics/prometheus"]
    
    [[outputs.influxdb_v2]]
    urls = [ "https://telemetry.bakerloo.autonity.network"]
    organization = "influxdata"
    bucket = "default"
    token = "<GET FROM BITWARDEN>"
    

Notice that the url for the input is to autonity rather than localhost. This is because it is using the internal network between the two containers, rather than pointing to a port forwarded to localhost.

If you choose not to use nginx, comment out the nginx section in docker-compose.yml.

  1. Setup certificates for Nginx.

    Nginx can be used as a proxy server between users and the Autonity node. This allows users to submit requests to the node using the more secure HTTPS protocol rather that HTTP.

    In order to setup nginx, the VM you are using to run the node must have a domain name licenced, this can be done with services like Namecheap and Godaddy.

    Next you must get a tls certificate and private key from a certificate authority such as Let's Encrypt, these are required to use HTTPS. It is recommended to use the free Certbot service provided by Let's Encrypt, by following the instructions for using certbox with a nginx server on ubuntu. In step 7, run the command to just get a certificate.

  2. Setup Nginx.

    • Go to the autonity-compose/nginx directory:

      cd autonity-compose/nginx
      
    • Save the tls certificate and private key from the previous step:

      sudo cp /etc/letsencrypt/live/composer.test.autonity.io/fullchain.pem ./tls.crt
      sudo cp /etc/letsencrypt/live/composer.test.autonity.io/privkey.pem ./tls.key
      
    • Generate the DH Parameters, which are also required for ssl:

      openssl dhparam -out dhparam.pem 1024
      

    The nginx.conf file sets how nginx will listen for encrypted ssl traffic on ports 8080 (for https) and 8081 (for wss), then decrypt the traffic and pass it to the Autonity container on ports 8545 (for http) and 8546 (for ws) respectively. With nginx you can now send secure https traffic to the Autonity node by targeting port 8080, or use wss (secure websocket) by targeting port 8081.

Use

  1. Before you run you should run the following command which will permission the user to open the shared Autonity folder:

    export UID=$(id -u) export GID=$(id -g)
    
  2. Now start docker compose from the autonity-compose directory:

    docker-compose up
    

You will now see the logs output from the fluentd, telegraf, and nginx containers, as they are started in unison. The Autonity logs are sent to fluentd which sends them to elasticsearch.