Incident report: NymVPN gateway unavailability on 11–12 July 2025

Summary

In the afternoon of 11 July (UTC), the Nym team noticed the unavailability of gateways on the NymVPN app and network. This temporary network outage was caused by an update to hosting infrastructure that affected the APIs which probe and display gateway availability. This resulted in a disconnection between the probe data determining node status and the Node Status API, leading all gateways to appear unavailable. The problem was quickly resolved by a refresh to the node probes and NymVPN API.

Timeline of events

• Afternoon (UTC): Gateways disappeared from the NymVPN selector
• 1:30 UTC: The Nym network team assembled to refresh the Node Status API
• 2:00 UTC: Gateways were confirmed to be refreshed and live on all NymVPN apps
• Saturday, 12 July 2025: The following morning, a resulting problem with gateway availability was caught on the Nym Explorer, which was quickly addressed to make the Explorer up to date after the incident on 11 July.

Root cause

• The Nym network is made up of distributed infrastructure run by our fantastic operator community. To make sure NymVPN gateways are up and running, we have to regularly test (or probe) them via our Node Status API. This method of probing has a decided rate to guarantee the “freshness” of results regarding node performance.
• Following an upgrade to infrastructure, a disconnect occurred between the probing tests of gateways and the NymVPN API which fetches available gateways. This resulted in all gateways appearing as unavailable for the NymVPN app for a short period of time.

Improvements & next steps

• Updates to the NymVPN APIs are being considered to balance the “freshness” of node probes vs. the benefits of using reliable and more longterm test results of nodes.