A ticket had been closed for months: migrate the last workflows off the old middleware server onto the new back end. It was a real migration, done by competent people, and the board said done.
Reading the code on a routine audit turned up nine live call sites still pointing at the old host. Not commented out, not behind a disabled feature flag. Nine places in the running application that would send a request to a server everyone believed had been retired.
We have now found some version of this at enough clients that it is worth writing up, because the mechanism is always the same and it is not carelessness.
Closed on the Board Is Not Gone From the Code
The migration was genuinely mostly done. Document ingest, the daily refresh, the main processing paths: all native, all off the old system. Those are the paths people use every day, so those are the paths that got migrated, tested, and confirmed.
What was left behind was a bulk refresh, a freight lookup, and two price-update writes. A monthly job, an admin button somebody presses at quarter end, and two paths that only run when a specific kind of record changes. Frequency of use decided what got migrated, not importance, and nothing about the tracking system captured the difference.
The Fallback Is the Trap
Here is the part that turns a cleanup into an outage. Every one of those nine call sites read the target host from an environment variable, and every one of them hardcoded the old hostname as the fallback if that variable was unset.
That pattern is written thousands of times a day and it is usually harmless. During a decommissioning it is the worst possible behaviour, because it inverts what unsetting the variable means. The obvious way to prove a dependency is gone is to remove the configuration and see what breaks. Here, removing the configuration does not disable the dependency. It silently aims every call at the old host, which is exactly the server you are about to switch off.
So the failure does not arrive during the test. It arrives weeks later, at quarter end, in a batch job nobody is watching, as a timeout or a silent no-op. Prices do not update. The refresh returns nothing and the code treats an empty response as "no changes". Everything looks fine until somebody reconciles.
How to Actually Retire One
Five steps, in this order. The order matters more than any individual step.
- Log the traffic before you touch anything. Turn on logging for every outbound call to the legacy host and let it run for a full business cycle. Traffic is evidence. An architecture diagram is a guess, and a memory of what was migrated is a worse guess.
- Inventory by search, not by recall. Grep the entire codebase for the hostname, the IP address, the credential name, the queue name, the database name. Include configuration, scripts, scheduled tasks, and anything in the reporting layer. The number of hits is your actual scope, and it is routinely several times what the team expected.
- Remove the fallbacks first, before anything else. Make every call site fail loudly and immediately when the configuration is absent. This is the single highest-value change in the whole exercise, because it converts a silent dependency into a visible one, and it makes every later test trustworthy.
- Turn it off before you delete it, and leave it off. Power the old host down and leave it down through a month end and a quarter end. Whatever screams during that window is the list you missed. Deleting a machine you have not first left switched off for a full cycle throws away the only cheap way to find out.
- Close the ticket when the host is gone, not when the plan is agreed. The definition of done for a decommissioning is a machine that no longer exists and a codebase with zero hits for its name. Anything short of that is a plan, and plans get marked complete.
What Leaving It Up Actually Costs
The honest answer is usually not much per month, which is why it survives. But the costs are real and they compound. It is a second system to patch, with credentials that stay valid and an attack surface nobody reviews. It is a server that cannot be powered down during a network cutover, because nobody is quite sure what still depends on it. It is a licence or a subscription renewing for a system in name only.
And it is a dependency whose shape lives in one person's head. When that person leaves, the organisation inherits a running server that nobody can explain and nobody dares switch off, and the cost of finding out goes up every year.
The good news is that the audit is cheap. Grepping a codebase for a hostname takes an afternoon, and it either confirms the migration is finished or it hands you a precise, finite list. Either result is worth more than the assumption you are running on now.
Uptimize Solutions builds integrations and, when the time comes, retires them properly. If you have a server nobody is willing to switch off, see our system integration services or book a free workflow audit and we will find out what is still calling it.