New: a threat-model-first guide to choosing your network defence, plus the nym-smoldvpn dVPN package and nym-swizzle sender hygiene.
Operators
Ansible

Orchestrating Nym Nodes with Ansible

ℹ️

Our documentation often refers to syntax annotated in <> brackets. We use this expression for variables that are unique to each user (like path, local moniker, versions et cetera). Any syntax in <> brackets needs to be substituted with your correct name or version, without the <> brackets. If you are unsure, please check our table of essential parameters and variables (opens in a new tab).

Ansible (opens in a new tab) is an open-source automation engine that can perform IT tasks and remove complexity from workflows. Ansible ensures that your environment is exactly as you describe it. You can automate any command with Ansible to make your system maintenance very efficient. For nym-node operators Ansible is particularly useful as it can scale infinitely the amount of nodes operators can setup, bond, upgrade, maintain and re-configure from their local shell, removing the complexity and required time when managing many nodes one by one.

⚠️

This setup should be used only by operators who understand nym-node administration and requirements

Ansible is more suitable for skilled power users managing multiple nodes at the same time!

If you are not familiar with Ansible, operating Nym nodes may be a good motivation to learn something new and improve your admin skills, it's worth the time.

Start by reading through Ansible documentation pages (opens in a new tab)

Installation

Ansible installation

For anything regarding the installation and management of Ansible itself, the best is to refer to their documentation. On this page (opens in a new tab) you can see the installation guide.

If you are confident and want to start right away, install Ansible on your machine using one of these two ways:

  1. apt repository:
sudo apt-get update
sudo apt-get install ansible
  1. pip or pipx - recommended by Ansible community:
pip install ansible
# or
pipx install ansible
⚠️

Install the full ansible package, not only ansible-core. The playbooks use the ufw and modprobe modules, which ship in the community.general collection. ansible-core does not bundle collections, so deploy.yml and the kernel CVE playbooks fail with couldn't resolve module/action 'ufw'.

If you already have ansible-core, add the collection:

ansible-galaxy collection install community.general

Nym Node Playbook Installation

Nym Node Ansible playbook template is located in our monorepo nymtech/nym/ansible/nym-node (opens in a new tab)

1. Get nym/ansible/nym-node playbook:

The easiest way is to use git to clone or pull the repository:

git clone https://github.com/nymtech/nym.git
 
# or navigate where you already have the repo and run
 
git checkout develop
git pull origin develop
2. Save the template to your location:

You may want to create a directory outside of the repository and move the template there so it can be modified without risking that your configuration will be accidentally shared when working with the repository in the future.

  • Navigate to any location and create a directory for your Ansible nym-node playbook:
cd <PATH>
mkdir `ansible`
cd ansible
  • Copy the template to the newly created location:
cp -r <PATH>/nym/ansible/nym-node ./

Now you have the template of Ansible playbook for nym-node remote administration. To make it work, there are a few variables requiring your attention.

Configuration

After getting the ansible Nym node playbook to your location, it's time to configure it for your own needs.

Mind that idempotency is an essential character when dealing with orchestration. A playbook, even when run many times should ensure that state of your targeted system will not change from what you intended. Therefore, it is important to make sure that all tasks in your playbook do not change the system in any way if the change you required has already been applied.

⚠️

Before starting Ansible, ensure that your A and AAAA records are pointed to your server IPs and propagated. Good test is to be able to ping them or use them for ssh into the server.

Open your local copy of the playbook in your favourite text editor and begin with these steps:

1. Configure global variables:
  • Open playbooks/group_vars/all.yml
  • Define all values under the section labeled as ## MANDATORY - uncomment & define - these values will be propagated on all your nodes globally
  • Optionally define values of your choice under the section ## OPTIONAL - uncomment & define
  • Note that in the next step we will be setting up a node inventory, where each of the variable can be configured per node, taking priority over the global ones.
  • Setup a correct path for your SSH key to ansible_ssh_private_key_file:, alternatively export your SSH key as an env var and use this:
ansible_ssh_private_key_file: "{{ lookup('env', '<YOUR_ANSIBLE SSH_KEY_ENV_VAR>') }}"
  • Keep hostname="" as a fallback for nodes without a hostname
2. Create node inventory:
  • Open playbooks/inventory/all
  • Make an entry for each of your node:
node1 ansible_host=<YOUR_SERVER_IP> ansible_user=<USER> hostname=<HOSTNAME> location=<LOCATION> email=<EMAIL> mode=<MODE> wireguard_enabled=<true/false> moniker=<MONIKER> description=<DESCRIPTION>
  • Mandatory values specific for each node - must be defined in the inventory:
    • ansible_host: IPv4 host address
    • hostname: node domain, otherwise fallbacks to "" for nodes without domain
    • location: node server location - must be an ISO 3166 country: alpha-2 (CH), alpha-3 (CHE), numeric (756) or the country name (Switzerland). A city or region name will make nym-node fail to start at the end of the deploy.
  • Mandatory values which can be setup per node or in group_vars/all globally:
3. Test your setup

Run this command to check if everything is configured correctly in your inventory:

cd playbooks
ansible-inventory --graph
4. Optional: remove some of nym-node run command arguments

These variables are read by the main task for nym-node installation: roles/nym/tasks/config.yaml

Open roles/nym/defaults/main.yml and have a look on the variables used:

5. Optional: Configure deploy.yml playbook

Open playbooks/deploy.yml and comment out tunnel and quic roles in case of running your playbook for nodes in a mode mixnode.

Save all the files and test with:

cd playbooks
ansible-inventory --graph

Right now you should be ready to go.

Flow & Usage

This chapter describes fundamental commands for using Ansible playbooks in relation to orchestrating multiple servers running a nym-node. For a full understanding of Ansible usage, read Ansible documentation pages (opens in a new tab).

Logic

The main logic of the playbook flow when running with a basic command and playbook like this:

ansible-playbook <PLAYBOOK>.yml
1. Read inventory

Ansible parses inventory/all and performs the playbook on all entries in it, unless specified otherwise

2. Read global vars

Ansible parses group_vars/all.yml and asigns global variables to all inventory entries, unless they were defined in the inventory.

Variables defined in the inventory per entry take highest priority!

3. Follow roles in the playbook

Ansible reads the roles defined in <PLAYBOOK>.yml passed with the command and executes the tasks defined under each role

Usage

The simplest way is to run ansible-playbook binary with a provided playbook as a command. That will do the defined roles on all entries in the inventory. In Nym we currently have these playbooks:

1. Deploy

A playbook to deploy server and nym-node from scratch, configuring networking, routing, firewall, systemd, bridges, reverse proxy, exit policy and all required tasks.

This playbook runs on all inventory entries in parallel. Test on a single node first with -l <node> before letting it loose on the whole inventory.

cd playbooks
ansible-playbook deploy.yml
2. Bond
⚠️

Anyone having access to your account mnemonic can take all your funds and manage your node, be careful where you store it!

Bonding can be managed via two playbooks:

  1. bond.yml: an interactive way, requiring operator to use own wallet (desktop or CLI)
  2. auto-bond.yml: automatic bonding flow requiring operator to prepare nodes.csv and have nym-cli installed
3. Upgrade

A playbook to upgrade nym-node binary to the Latest by default. To pin a specific release, un-comment nym_version either in playbooks/group_vars/all.yml (fleet-wide) or in roles/upgrade/defaults/main.yml, and give the tag in full form, e.g. nym-binaries-v2026.7-tola.

This playbook runs one node at a time (serial: 1) so the fleet is never restarted all at once.

Note: If you want to run it all nodes together, and get done with your upgrade faster, comment out the line (serial: 1) in ansible/nym-node/playbooks/upgrade.yml.

cd playbooks
ansible-playbook upgrade.yml
4. System Maintenance

A playbook to de-clog and cleanup the servers (VMs) hosting nym-node. Use it, if your servers fill up disk space too fast. For operators hosting many VMs on a hypervisor dedicated / bare metal server - run this playbook alongside the guide to clean up the hypervisor.

Maintaining the VMs using this playbook and reclaiming space on hypervisor will prevent your servers from:

  • Sudden disk full crashes
  • journald runaway growth
  • Duplicate rsyslog logs
  • nym-node log growth
  • qcow2 like allocation growth inside VPS storage

The playbook is located at playbooks/system-maintenance.yml. Many values can be adjusted in playbooks/group_vars/all.yml under the header ## SYSTEM MAINTENANCE PLAYBOOK KNOBS. Values set per node in inventory/all take priority over group_vars, as everywhere else in this playbook.

Run system-maintenance playbook

cd playbooks
ansible-playbook system-maintenance.yml

This playbook does not run on every host at once. It defaults to serial: 25%, because each batch restarts nym-node and runs fstrim - doing that simultaneously on 20+ VMs sharing one hypervisor will stall the array. Override on the command line:

ansible-playbook system-maintenance.yml -e maintenance_serial=<N>
 
# for example 4 VMs at the same time
# ansible-playbook system-maintenance.yml -e maintenance_serial=4

Switching parts off

The playbook can be run with a provided tag, for example:

# only the nym-node logging drop-in
ansible-playbook system-maintenance.yml -t nymnode
 
# only the space reclaim parts
ansible-playbook system-maintenance.yml -t cleanup

Available tags: journald, logging, nymnode, rsyslog, logs, cleanup, apt, snap, fstrim, writeback, tuning, report.

Or by using role's toggle in group_vars/all.yml: enable_journald_limits, enable_nymnode_logging, disable_rsyslog, enable_classic_log_cleanup, enable_journal_vacuum, enable_apt_cleanup, enable_snap_cleanup, enable_fstrim, enable_writeback_tuning.

ℹ️

serial is a play keyword, resolved before group_vars is applied, so it must be passed with -e or edited directly in the playbook. Setting maintenance_serial in group_vars/all.yml has no effect. Every other knob works from group_vars.

⚠️

Never cap nym-node logs with systemd LogLevelMax=.

nym-node writes to stderr, and journald stamps everything a service writes to stdout/stderr with SyslogLevel= (default info). LogLevelMax=warning therefore discards every line the binary emits - including its errors - and the node looks completely silent while running normally.

Use nymnode_rust_log instead. It sets RUST_LOG, which filters inside the binary, so warnings and errors still reach the journal.

Verifying the run. The report role prints disk and journal usage before/after and checks that nym-node is still writing to the journal. If it warns that the service is active but producing no output, look for a stale drop-in:

ansible all -i inventory/all -a "ls -la /etc/systemd/system/nym-node.service.d/"
ansible all -i inventory/all -a "journalctl -u nym-node --since '-2 min' --no-pager -q"
ℹ️

A --check run cannot validate this playbook. Check mode skips every command and shell task, so the discard probe, the trim, the journal vacuum and the logging health check are all reported as skipped. Use --check --diff to preview the config files it would write, but treat a clean check run as not a clean bill of health.

⚠️

fstrim reclaims nothing unless the guest disk has discard enabled. On a hypervisor, qcow2 disks default to discard=ignore: the guest reports gigabytes "trimmed" and exits 0, while the image on the host never shrinks. Check on the hypervisor with virsh dumpxml <VM> | grep discard and see VPS & ISP troubleshooting for the fix. The fstrim role warns automatically when the guest's root device reports a discard granularity of 0.

⚠️

If fail2ban reads log files rather than the journal, the playbook refuses to stop rsyslog and preserves /var/log/auth.log, because removing them would silently stop SSH brute-force banning on a public-facing gateway. Set backend = systemd in /etc/fail2ban/jail.local to allow the cleanup, or force_disable_rsyslog: true if you accept the risk.

5. Mitigate kernel CVE

This playbook is to mitigate some of the Kernel issues found in April and May 2026.

This playbook runs on all inventory entries in parallel. remove_kernel_CVE_mitigations.yml reverses it.

cd playbooks
ansible-playbook mitigate_kernel_CVE.yml

Useful Commands

Ansible (opens in a new tab) has many smart ways to manage your playbooks, roles or inventory entries.

Here are some useful tips:

Node limit

To test new configuration, it's advised to try it on one server at first. Of course you can comment out any entries in the inventory, but there are easier ways to do it.

  • Provide flag -l followed by inventory entry and Ansible will change state only of that entry:

  • Some possibilities are (in example we use upgrade.yml, you can use any playbook):

# point to one entry
ansible-playbook upgrade.yml -l node1
 
# point to multiple entries
ansible-playbook upgrade.yml -l "node1,node2"
 
# use regex - ie all exit nodes
ansible-playbook upgrade.yml -l "*exit*"
 
# use group in inventory labeled as [group]
ansible-playbook deploy.yml -l new_nodes
Tag selection
💡

To update your exit policy by using the newest NTM (opens in a new tab) via Ansible, just run:

ansible-playbook deploy.yml -t network_tunnel_manager

This will download the script from develop, make executable and run it with the command complete_networking_configuration.

Sometimes you may want to run just one tag at a time, for that use -t flag, for example:

# in case of wanting to run only quic deployment role
ansible-playbook deploy.yml -t quic
 
# in case of running the same on only one node
ansible-playbook deploy.yml -l node2 -t quic

To list all tags, run:

ansible-playbook <PLAYBOOK>.yml --list-tags
 
# for example
ansible-playbook deploy.yml --list-tags
ansible-playbook system-maintenance.yml --list-tags
Dry run

Preview what a playbook would change without touching anything:

ansible-playbook <PLAYBOOK>.yml -l node1 --check --diff

--diff shows the exact file contents that would be written. Note that check mode skips command and shell tasks, so anything that shells out is reported as skipped rather than verified.

dpkg lock failures

If a run fails with Could not get lock /var/lib/dpkg/lock-frontend, Ubuntu's apt-daily.timer / unattended-upgrades is holding the lock in the background. The playbooks raise the apt wait to 300s, which covers most cases. If it still collides, let the background job finish rather than killing it - interrupting dpkg mid-transaction can leave packages half-configured.

Run the following on the affected node (not on the Ansible controller), as root. It stops the timers so no new run starts, then waits for the run already in progress to release the lock. Do not systemctl stop apt-daily-upgrade.service: that sends SIGTERM to the whole service cgroup, including the running dpkg, which causes exactly the half-configured state described above.

# prevent new runs from starting
sudo systemctl stop apt-daily.timer apt-daily-upgrade.timer
 
# wait for the run already in flight to release the lock
while sudo fuser /var/lib/dpkg/lock-frontend >/dev/null 2>&1; do sleep 5; done

Re-run the playbook, then put the timers back:

sudo systemctl start apt-daily.timer apt-daily-upgrade.timer
Arbitrary command output

You can use ansible to read a STDOUT from any command, using this logic:

ansible all -i inventory/all -a "<COMMAND>"
 
# for example to get all node ID keys
ansible all -i inventory/all -a "/root/nym-binaries/nym-node bonding-information"
  • Note that the command gets also run, be mindful what you executing

  • This logic can be combined with the arguments above, for example to limit the range of nodes

nocows

Yes, by default there is a cow printed under each task, you can turn it off by opening playbooks/ansible.cfg and un-commenting the nocows line:

nocows = 1