Orchestrating Nym Nodes with Ansible

ℹ️

Our documentation often refer to syntax annotated in <> brackets. We use this expression for variables that are unique to each user (like path, local moniker, versions etcetra). Any syntax in <> brackets needs to be substituted with your correct name or version, without the <> brackets. If you are unsure, please check our table of essential parameters and variables (opens in a new tab).

Ansible (opens in a new tab) is an open-source automation engine that can perform IT tasks and remove complexity from workflows. Ansible ensures that your environment is exactly as you describe it. You can automate any command with Ansible to make your system maintenance very efficient. For nym-node operators Ansible is particularly useful as it can scale infinitely the amount of nodes operators can setup, bond, upgrade, maintain and re-configure from their local shell, removing the complexity and required time when managing many nodes one by one.

⚠️

This setup should be used only by operators who understand nym-node administration and requirements

Ansible is more suitable for skilled power users managing multiple nodes at the same time!

If you are not familiar with Ansible, operating Nym nodes may be a good motivation to learn something new and improve your admin skills, it's worth the time.

Start by reading through Ansible documentation pages (opens in a new tab)

Installation

Ansible installation

For anything regarding the installation and management of Ansible itself, the best is to refer to their documentation. On this page (opens in a new tab) you can see the installation guide.

If you are confident and want to start right away, install Ansible on your machine using one of these two ways:

apt repository:

sudo apt-get update
sudo apt-get install ansible

pip or pipx - recommended by Ansible community:

pip install ansible
# or
pipx install ansible

Nym Node Playbook Installation

Nym Node Ansible playbook template is located in our monorepo nymtech/nym/ansible/nym-node (opens in a new tab)

1. Get `nym/ansible/nym-node` playbook:

The easiest way is to use git to clone or pull the repository:

git clone https://github.com/nymtech/nym.git
 
# or navigate where you already have the repo and run
 
git checkout develop
git pull origin develop

2. Save the template to your location:

You may want to create a directory outside of the repository and move the template there so it can be modified without risking that your configuration will be accidentally shared when working with the repository in the future.

Navigate to any location and create a directory for your Ansible nym-node playbook:

cd <PATH>
mkdir `ansible`
cd ansible

Copy the template to the newly created location:

cp -r <PATH>/nym/ansible/nym-node ./

Now you have the template of Ansible playbook for nym-node remote administration. To make it work, there are a few variables requiring your attention.

Configuration

After getting the ansible Nym node playbpook to your location, it's time to configure it for your own needs.

Mind that idempotency is an essential character when dealing with orchestration. A playbook, even when run many times should ensure that state of your targeted system will not change from what you intended. Therefore, it is important to make sure that all tasks in your playbook do not change the system in any way if the change you required has already been applied.

⚠️

Before starting Ansible, ensure that your A and AAAA records are pointed to your server IPs and propagated. Good test is to be able to ping them or use them for ssh into the server.

Open your local copy of the playbook in your favourite text editor and begin with these steps:

1. Configure global variables:

Open playbooks/group_vars/all.yml
Setup any variables which you want to have propagated on all your nodes globally
Note that in the next step we will be setting up a node inventory, where each of the variable can be configured per node, taking priority over the global ones.
Setup a correct path for your SSH kety to ansible_ssh_private_key_file:
Use these variables or comment them out with #:
- ansible_user
- email
- website
- description
Keep hostname="" as a fallback for nodes without a hostname

2. Create node inventory:

Open playbooks/inventory/all
Make an entry for each of your node:

node1 ansible_host=<YOUR_SERVER_IP> ansible_user=<USER> hostname=<HOSTNAME> location=<LOCATION> email=<EMAIL> mode=<MODE> wireguard_enabled=<true/false> moniker=<MONIKER> description=<DESCRIPTION>

These are mandatory values specific for each node - must be defined in the inventory:
- ansible_host: IPv4 host address
- hostname: node domain, otherwise fallbacks to "" for nodes without domain
- location: node server location
These are mandatory values which can be setup per node or in group_vars/all globally:
- ansible_user
- email
- website
- moniker
- description
- mode
- wireguard_enabled

3. Test your setup

Run this command to check if everything is configured correctly in your inventory:

cd playbooks
ansible-inventory --graph

4. Configure `nym-node run` command arguments

Open roles/nym/defaults/main.yml and have a look on the variables used:

If you agree with Terms and conditions uncomment the line: accept_operator_terms: true without which your node can never take part in Nym Network.
The rest is up to your configuration but generally these flags workflows

These variables are read by the main task for nym-node installation: roles/nym/tasks/config.yaml

Open that yaml and have a look on the flags
In case of not needing some of the, delete them (ie when running --mode mixnode you can delete everything from --hostname to --announce-wss-port)

5. Configure `deploy.yml` playbook

Open playbooks/deploy.yml and comment out tunnel and quic roles in case of running your playbook for nodes in a mode mixnode.

Save all the files and test with:

cd playbooks
ansible-inventory --graph

Right now you should be ready to go.

Flow & Usage

This chapter describes fundamental commands for using Ansible playbooks in relation to orchestrating multiple servers running a nym-node. For a full understanding of Ansible usage, read Ansible documentation pages (opens in a new tab).

Logic

The main logic of the playbook flow when running with a basic command and playbook like this:

ansible-playbook <PLAYBOOK>.yml

1. Read inventory

Ansible parses inventory/all and performs the playbook on all entries in it, unless specified otherwise

2. Read global vars

Ansible parses group_vars/all.yml and asigns global variables to all inventory entries, unless they were defined in the inventory.

Variables defined in the inventory per entry take highest priority!

3. Follow roles in the playbook

Ansible reads the roles defined in <PLAYBOOK>.yml passed with the command and executes the tasks defined under each role

Usage

The simplest way is to run ansible-playbook binary with a provided playbook as a command. That will do the defined roles on all entries in the inventory. In Nym we currently have these playbooks:

1. Deploy

A playbook to deploy server and nym-node from scratch, configuring networking, routing, firewall, systemd, bridges, reverse proxy, exit policy and all required tasks.

This playbook will run roles on all the inventory entries in parallel by default.

cd playbooks
ansible-playbook deploy.yml

2. Bond

A playbook to interactively register your node to Nym network by bonding it to Nyx blockchain account.

This playbook is intercative as it prompts user for data from Nym wallet to sign a message. It will run roles on one inventory entry at a time by default.

cd playbooks
ansible-playbook bond.yml

3. Upgrade

A playbook to upgrade nym-node binary to the Latest by default. Operators can hard code a specific binary version in roles/upgrade/defaults/main.yml by un-commenting the nym-version line and providing their desired version.

This playbook will run roles on all the inventory entries in parallel by default.

cd playbooks
ansible-playbook upgrade.yml

Useful Commands

Ansible (opens in a new tab) has many smart ways to manage your playbooks, roles or inventory entries.

Here are some useful tips:

One node at a time

To test new configuration, it's advised to try it on one server at first. Of course you can comment out any entries in the inventory, but there are easier ways to do it.

Provide flag -l followed by inventory entry and Ansible will change state only of that entry:
Some possibilities are (in example we use upgrade.yml, you can use any playbook):

# point to one entry
ansible-playbook upgrade.yml -l node1
 
# point to multiple entries
ansible-playbook upgrade.yml -l "node1,node2"
 
# use regex
ansible-playbook upgrade.yml -l "*exit*"

Role limit

Sometimes you may want to run just one role at a time, for that use -q, for example:

# in case of wanting to run only quic deployment role
ansible-playbook deploy.yml -t quic
 
# in case of running the same on only one node
ansible-playbook deploy.yml -l node2 -t quic

nocows

Yes, by default there is a cow printed under each task, you can turn it off by opening playbooks/ansible.cfg and un-commenting the nocows line:

nocows = 1

Orchestration Performance Measurement