Orchestrating Nym Nodes with Ansible
Our documentation often refers to syntax annotated in <> brackets. We use this expression for variables that are unique to each user (like path, local moniker, versions et cetera).
Any syntax in <> brackets needs to be substituted with your correct name or version, without the <> brackets. If you are unsure, please check our table of essential parameters and variables (opens in a new tab).
Ansible (opens in a new tab) is an open-source automation engine that can perform IT tasks and remove complexity from workflows. Ansible ensures that your environment is exactly as you describe it. You can automate any command with Ansible to make your system maintenance very efficient. For nym-node operators Ansible is particularly useful as it can scale infinitely the amount of nodes operators can setup, bond, upgrade, maintain and re-configure from their local shell, removing the complexity and required time when managing many nodes one by one.
This setup should be used only by operators who understand nym-node administration and requirements
Ansible is more suitable for skilled power users managing multiple nodes at the same time!
If you are not familiar with Ansible, operating Nym nodes may be a good motivation to learn something new and improve your admin skills, it's worth the time.
Start by reading through Ansible documentation pages (opens in a new tab)
Installation
Ansible installation
For anything regarding the installation and management of Ansible itself, the best is to refer to their documentation. On this page (opens in a new tab) you can see the installation guide.
If you are confident and want to start right away, install Ansible on your machine using one of these two ways:
aptrepository:
sudo apt-get update
sudo apt-get install ansiblepiporpipx- recommended by Ansible community:
pip install ansible
# or
pipx install ansibleInstall the full ansible package, not only ansible-core. The playbooks use the ufw and modprobe modules, which ship in the community.general collection. ansible-core does not bundle collections, so deploy.yml and the kernel CVE playbooks fail with couldn't resolve module/action 'ufw'.
If you already have ansible-core, add the collection:
ansible-galaxy collection install community.generalNym Node Playbook Installation
Nym Node Ansible playbook template is located in our monorepo nymtech/nym/ansible/nym-node (opens in a new tab)
1. Get nym/ansible/nym-node playbook:
The easiest way is to use git to clone or pull the repository:
git clone https://github.com/nymtech/nym.git
# or navigate where you already have the repo and run
git checkout develop
git pull origin develop2. Save the template to your location:
You may want to create a directory outside of the repository and move the template there so it can be modified without risking that your configuration will be accidentally shared when working with the repository in the future.
- Navigate to any location and create a directory for your Ansible
nym-nodeplaybook:
cd <PATH>
mkdir `ansible`
cd ansible- Copy the template to the newly created location:
cp -r <PATH>/nym/ansible/nym-node ./Now you have the template of Ansible playbook for nym-node remote administration. To make it work, there are a few variables requiring your attention.
Configuration
After getting the ansible Nym node playbook to your location, it's time to configure it for your own needs.
Mind that idempotency is an essential character when dealing with orchestration. A playbook, even when run many times should ensure that state of your targeted system will not change from what you intended. Therefore, it is important to make sure that all tasks in your playbook do not change the system in any way if the change you required has already been applied.
Before starting Ansible, ensure that your A and AAAA records are pointed to your server IPs and propagated. Good test is to be able to ping them or use them for ssh into the server.
Open your local copy of the playbook in your favourite text editor and begin with these steps:
1. Configure global variables:
- Open
playbooks/group_vars/all.yml - Define all values under the section labeled as
## MANDATORY - uncomment & define- these values will be propagated on all your nodes globally - Optionally define values of your choice under the section
## OPTIONAL - uncomment & define - Note that in the next step we will be setting up a node inventory, where each of the variable can be configured per node, taking priority over the global ones.
- Setup a correct path for your SSH key to
ansible_ssh_private_key_file:, alternatively export your SSH key as an env var and use this:
ansible_ssh_private_key_file: "{{ lookup('env', '<YOUR_ANSIBLE SSH_KEY_ENV_VAR>') }}"- Keep
hostname=""as a fallback for nodes without a hostname
2. Create node inventory:
- Open
playbooks/inventory/all - Make an entry for each of your node:
node1 ansible_host=<YOUR_SERVER_IP> ansible_user=<USER> hostname=<HOSTNAME> location=<LOCATION> email=<EMAIL> mode=<MODE> wireguard_enabled=<true/false> moniker=<MONIKER> description=<DESCRIPTION>- Mandatory values specific for each node - must be defined in the inventory:
ansible_host: IPv4 host addresshostname: node domain, otherwise fallbacks to""for nodes without domainlocation: node server location - must be an ISO 3166 country: alpha-2 (CH), alpha-3 (CHE), numeric (756) or the country name (Switzerland). A city or region name will makenym-nodefail to start at the end of the deploy.
- Mandatory values which can be setup per node or in
group_vars/allglobally:ansible_useremailwebsitemonikerdescriptionmodewireguard_enabledaccept_operator_terms- Make sure to read Nym Operators Terms & Conditions first!
3. Test your setup
Run this command to check if everything is configured correctly in your inventory:
cd playbooks
ansible-inventory --graph4. Optional: remove some of nym-node run command arguments
These variables are read by the main task for nym-node installation: roles/nym/tasks/config.yaml
Open roles/nym/defaults/main.yml and have a look on the variables used:
5. Optional: Configure deploy.yml playbook
Open playbooks/deploy.yml and comment out tunnel and quic roles in case of running your playbook for nodes in a mode mixnode.
Save all the files and test with:
cd playbooks
ansible-inventory --graphRight now you should be ready to go.
Flow & Usage
This chapter describes fundamental commands for using Ansible playbooks in relation to orchestrating multiple servers running a nym-node. For a full understanding of Ansible usage, read Ansible documentation pages (opens in a new tab).
Logic
The main logic of the playbook flow when running with a basic command and playbook like this:
ansible-playbook <PLAYBOOK>.yml1. Read inventory
Ansible parses inventory/all and performs the playbook on all entries in it, unless specified otherwise
2. Read global vars
Ansible parses group_vars/all.yml and asigns global variables to all inventory entries, unless they were defined in the inventory.
Variables defined in the inventory per entry take highest priority!
3. Follow roles in the playbook
Ansible reads the roles defined in <PLAYBOOK>.yml passed with the command and executes the tasks defined under each role
Usage
The simplest way is to run ansible-playbook binary with a provided playbook as a command. That will do the defined roles on all entries in the inventory. In Nym we currently have these playbooks:
1. Deploy
A playbook to deploy server and nym-node from scratch, configuring networking, routing, firewall, systemd, bridges, reverse proxy, exit policy and all required tasks.
This playbook runs on all inventory entries in parallel. Test on a single node first with -l <node> before letting it loose on the whole inventory.
cd playbooks
ansible-playbook deploy.yml2. Bond
Anyone having access to your account mnemonic can take all your funds and manage your node, be careful where you store it!
Bonding can be managed via two playbooks:
bond.yml: an interactive way, requiring operator to use own wallet (desktop or CLI)auto-bond.yml: automatic bonding flow requiring operator to preparenodes.csvand havenym-cliinstalled
3. Upgrade
A playbook to upgrade nym-node binary to the Latest by default. To pin a specific release, un-comment nym_version either in playbooks/group_vars/all.yml (fleet-wide) or in roles/upgrade/defaults/main.yml, and give the tag in full form, e.g. nym-binaries-v2026.7-tola.
This playbook runs one node at a time (serial: 1) so the fleet is never restarted all at once.
Note: If you want to run it all nodes together, and get done with your upgrade faster, comment out the line (
serial: 1) inansible/nym-node/playbooks/upgrade.yml.
cd playbooks
ansible-playbook upgrade.yml4. System Maintenance
A playbook to de-clog and cleanup the servers (VMs) hosting nym-node. Use it, if your servers fill up disk space too fast. For operators hosting many VMs on a hypervisor dedicated / bare metal server - run this playbook alongside the guide to clean up the hypervisor.
Maintaining the VMs using this playbook and reclaiming space on hypervisor will prevent your servers from:
- Sudden disk full crashes
journaldrunaway growth- Duplicate
rsysloglogs nym-nodelog growthqcow2like allocation growth inside VPS storage
The playbook is located at playbooks/system-maintenance.yml. Many values can be adjusted in playbooks/group_vars/all.yml under the header ## SYSTEM MAINTENANCE PLAYBOOK KNOBS. Values set per node in inventory/all take priority over group_vars, as everywhere else in this playbook.
Run system-maintenance playbook
cd playbooks
ansible-playbook system-maintenance.ymlThis playbook does not run on every host at once. It defaults to serial: 25%, because each batch restarts nym-node and runs fstrim - doing that simultaneously on 20+ VMs sharing one hypervisor will stall the array. Override on the command line:
ansible-playbook system-maintenance.yml -e maintenance_serial=<N>
# for example 4 VMs at the same time
# ansible-playbook system-maintenance.yml -e maintenance_serial=4Switching parts off
The playbook can be run with a provided tag, for example:
# only the nym-node logging drop-in
ansible-playbook system-maintenance.yml -t nymnode
# only the space reclaim parts
ansible-playbook system-maintenance.yml -t cleanupAvailable tags: journald, logging, nymnode, rsyslog, logs, cleanup,
apt, snap, fstrim, writeback, tuning, report.
Or by using role's toggle in group_vars/all.yml: enable_journald_limits, enable_nymnode_logging, disable_rsyslog, enable_classic_log_cleanup, enable_journal_vacuum, enable_apt_cleanup, enable_snap_cleanup, enable_fstrim, enable_writeback_tuning.
serial is a play keyword, resolved before group_vars is applied, so it must be
passed with -e or edited directly in the playbook. Setting maintenance_serial in
group_vars/all.yml has no effect. Every other knob works from group_vars.
Never cap nym-node logs with systemd LogLevelMax=.
nym-node writes to stderr, and journald stamps everything a service writes to
stdout/stderr with SyslogLevel= (default info). LogLevelMax=warning therefore
discards every line the binary emits - including its errors - and the node looks
completely silent while running normally.
Use nymnode_rust_log instead. It sets RUST_LOG, which filters inside the binary, so
warnings and errors still reach the journal.
Verifying the run. The report role prints disk and journal usage before/after and
checks that nym-node is still writing to the journal. If it warns that the service is
active but producing no output, look for a stale drop-in:
ansible all -i inventory/all -a "ls -la /etc/systemd/system/nym-node.service.d/"
ansible all -i inventory/all -a "journalctl -u nym-node --since '-2 min' --no-pager -q"A --check run cannot validate this playbook. Check mode skips every command and
shell task, so the discard probe, the trim, the journal vacuum and the logging health
check are all reported as skipped. Use --check --diff to preview the config files it
would write, but treat a clean check run as not a clean bill of health.
fstrim reclaims nothing unless the guest disk has discard enabled. On a hypervisor,
qcow2 disks default to discard=ignore: the guest reports gigabytes "trimmed" and exits
0, while the image on the host never shrinks. Check on the hypervisor with
virsh dumpxml <VM> | grep discard and see
VPS & ISP troubleshooting for the fix.
The fstrim role warns automatically when the guest's root device reports a discard
granularity of 0.
If fail2ban reads log files rather than the journal, the playbook refuses to stop
rsyslog and preserves /var/log/auth.log, because removing them would silently stop
SSH brute-force banning on a public-facing gateway. Set backend = systemd in
/etc/fail2ban/jail.local to allow the cleanup, or force_disable_rsyslog: true if you
accept the risk.
5. Mitigate kernel CVE
This playbook is to mitigate some of the Kernel issues found in April and May 2026.
This playbook runs on all inventory entries in parallel. remove_kernel_CVE_mitigations.yml
reverses it.
cd playbooks
ansible-playbook mitigate_kernel_CVE.ymlUseful Commands
Ansible (opens in a new tab) has many smart ways to manage your playbooks, roles or inventory entries.
Here are some useful tips:
Node limit
To test new configuration, it's advised to try it on one server at first. Of course you can comment out any entries in the inventory, but there are easier ways to do it.
-
Provide flag
-lfollowed by inventory entry and Ansible will change state only of that entry: -
Some possibilities are (in example we use upgrade.yml, you can use any playbook):
# point to one entry
ansible-playbook upgrade.yml -l node1
# point to multiple entries
ansible-playbook upgrade.yml -l "node1,node2"
# use regex - ie all exit nodes
ansible-playbook upgrade.yml -l "*exit*"
# use group in inventory labeled as [group]
ansible-playbook deploy.yml -l new_nodesTag selection
To update your exit policy by using the newest NTM (opens in a new tab) via Ansible, just run:
ansible-playbook deploy.yml -t network_tunnel_managerThis will download the script from develop, make executable and run it with the command complete_networking_configuration.
Sometimes you may want to run just one tag at a time, for that use -t flag, for example:
# in case of wanting to run only quic deployment role
ansible-playbook deploy.yml -t quic
# in case of running the same on only one node
ansible-playbook deploy.yml -l node2 -t quicTo list all tags, run:
ansible-playbook <PLAYBOOK>.yml --list-tags
# for example
ansible-playbook deploy.yml --list-tags
ansible-playbook system-maintenance.yml --list-tagsDry run
Preview what a playbook would change without touching anything:
ansible-playbook <PLAYBOOK>.yml -l node1 --check --diff--diff shows the exact file contents that would be written. Note that check mode
skips command and shell tasks, so anything that shells out is reported as
skipped rather than verified.
dpkg lock failures
If a run fails with Could not get lock /var/lib/dpkg/lock-frontend, Ubuntu's apt-daily.timer / unattended-upgrades is holding the lock in the background. The playbooks raise the apt wait to 300s, which covers most cases. If it still collides, let the background job finish rather than killing it - interrupting dpkg mid-transaction can leave packages half-configured.
Run the following on the affected node (not on the Ansible controller), as root. It stops the timers so no new run starts, then waits for the run already in progress to release the lock. Do not systemctl stop apt-daily-upgrade.service: that sends SIGTERM to the whole service cgroup, including the running dpkg, which causes exactly the half-configured state described above.
# prevent new runs from starting
sudo systemctl stop apt-daily.timer apt-daily-upgrade.timer
# wait for the run already in flight to release the lock
while sudo fuser /var/lib/dpkg/lock-frontend >/dev/null 2>&1; do sleep 5; doneRe-run the playbook, then put the timers back:
sudo systemctl start apt-daily.timer apt-daily-upgrade.timerArbitrary command output
You can use ansible to read a STDOUT from any command, using this logic:
ansible all -i inventory/all -a "<COMMAND>"
# for example to get all node ID keys
ansible all -i inventory/all -a "/root/nym-binaries/nym-node bonding-information"-
Note that the command gets also run, be mindful what you executing
-
This logic can be combined with the arguments above, for example to limit the range of nodes
nocows
Yes, by default there is a cow printed under each task, you can turn it off by opening playbooks/ansible.cfg and un-commenting the nocows line:
nocows = 1