Railway paused all deployments after attackers may have read decrypted environment variables

On the morning of 21 October, Railway saw unusual activity in the systems that store customer environment variables. Its advice: rotate every variable stored there, API keys and registry credentials included.

In brief

Railway believes the attack is part of a wider campaign against platform companies and AI inference providers, and has asked upstream providers to revoke keys. Deployments are paused during the investigation.

In the Railway Discord, users say stolen keys were already being used, and one says usage climbed on keys created only for Railway, 10 hours into the incident.

What Railway says was exposed

Railway security disclosure graphic
Railway security disclosure graphic

Railway's disclosure says an unauthorized third party may have accessed encrypted environment variables and, for some, obtained the decrypted contents. These can include API keys and database or registry credentials. Impact may be limited to a subset of users, but the post gives no figures; affected customers will get an email. Railway strongly recommends rotating every variable stored on the platform and any private registry credentials, and reviewing service logs. Follow-up posts on checking for suspicious GCP and AWS access came on 22 October.

Discord reports and a late email

"Seen multiple folks in the Railway Discord confirming that their secrets stored in Railway have been used by third parties, so it definitely sounds like the attackers were able to read decrypted environment variables." — a Railway Discord member

The timing drew anger.

"People were writing on discord 3 hours ago, 10 hours after the incident started, that keys created just for railway were running up usage." — a customer who watched the Discord for updates

He adds that builds had been down for more than 12 hours before the email went out.

Until Railway publishes a scope, assume your variables are in the subset. Rotate first, then check usage on any key that exists only for Railway, since that is where the Discord reports saw abuse first.

Get breakdowns like this every day

The top Hacker News stories, unpacked — by email, Telegram, or RSS.

Next story · 8 of 9 Read more
🌍Other

Why New York's bolt-on fire escapes lasted a century before the ban

In 1860 New York required fireproof stairways or exterior escapes after a tenement fire killed ten people, even for existing buildings. Bolting steel to the facade was simply cheaper than rebuilding stairs inside.

★ 188 · 💬 154 · practical.engineering · 2 min