Skip to content
fastnode_
Portal
menu

guide

Moving a live service to a new instance without an outage

Most migrations fail at the edges: a forgotten cron job, a DNS record with a one-day TTL, uploads written to the old box after the final copy. This runbook moves a web app and its database in five phases, and every phase can be undone.

Phase 0: inventory the old machine

Write down everything the old server does before you touch the new one. Missing items here become incidents later.

sudo ss -tulpn                          # what listens
systemctl list-units --type=service --state=running
systemctl list-timers
crontab -l; sudo ls /etc/cron.d /var/spool/cron/crontabs 2>/dev/null
df -h; du -sh /var/www /srv /home 2>/dev/null
ls /etc/nginx/sites-enabled /etc/nginx/conf.d

Note software versions too: php -v, node -v, psql --version. Moving between major versions during a migration doubles what can go wrong. Match versions now and upgrade next month.

Phase 1: one day before, lower the TTL

Look up the current TTL from a public resolver:

dig +noall +answer app.example.com @1.1.1.1

The second column is the remaining TTL in seconds. Set the record to 300 at your DNS provider and wait for at least the old value to pass. Do not change the address yet.

Phase 2: build and fill the new instance

Create the instance in London, run the first-hour checklist, and install the same stack. Then copy files in repeated passes. The first pass moves the bulk; later passes only move what changed.

# run on the NEW instance
rsync -aHAX --numeric-ids --delete --info=progress2 \
  -e "ssh -i ~/.ssh/migrate" \
  [email protected]:/var/www/ /var/www/

-H, -A and -X preserve hard links, ACLs and extended attributes. --numeric-ids keeps ownership correct even if user IDs differ between machines. Run it again an hour later and note how long the second pass takes; that is roughly your final sync time.

Phase 3: get the database across

Option A: dump and restore (simple, short freeze)

# on the old server
pg_dump -Fc -Z 3 appdb > /tmp/appdb.dump
# on the new server
scp old:/tmp/appdb.dump /tmp/ && pg_restore -d appdb -j 4 --no-owner /tmp/appdb.dump

Time a full rehearsal now. If it takes four minutes, your cut-over has a four-minute read-only window. -j 4 restores in parallel and makes good use of four vCPUs and NVMe-backed storage.

Option B: replication (near-zero freeze)

For PostgreSQL, set the new instance up as a streaming replica of the old one with pg_basebackup -R over a WireGuard tunnel, so the replication port is never public. For MySQL or MariaDB, use GTID-based replication. At cut-over, you stop writes, wait for zero replica lag, and promote. Seconds, not minutes.

Phase 4: test before anyone else sees it

Point only your own machine at the new server by editing your hosts file:

203.0.113.10  app.example.com

Issue the TLS certificate on the new server using DNS validation, or copy /etc/letsencrypt across from the old one, because HTTP validation will still reach the old box. Then click through every path that matters: login, checkout, file upload, an email-sending action, an API call with a real token. Check the new server's logs, not the old one's, to prove where requests are landing.

Phase 5: cut over

  1. Announce a short maintenance window if you are using dump and restore.
  2. Put the old app into read-only or maintenance mode so no new writes land there.
  3. Run the final rsync pass. It should be quick.
  4. Final database step: last dump and restore, or promote the replica.
  5. Change the A record to the new address. TTL is 300, so most traffic moves within five minutes.
  6. Enable scheduled jobs on the new server and disable them on the old one. Running a billing job on both is the classic double-charge.
  7. Watch logs on both machines. Traffic on the old one should drain to near zero.

As an extra safety net, configure the old server's nginx to proxy to the new address once the database has moved. Clients with stale DNS then still reach the right backend instead of a frozen copy.

Rollback

Until you write new data on the new server, rollback is just reverting the DNS record and lifting read-only mode on the old one. After that, you need to copy new writes back, which is why the first hour after cut-over deserves your full attention. Decide in advance what failure looks like and when you will call it.

After the move

  • Raise the TTL back to an hour or a day once things are stable.
  • Update reverse DNS for the new address in the portal if you send mail or need tidy PTR lookups.
  • Set up off-box backups before you decommission the old server.
  • Cancel the old server last, after a quiet week.

Questions

How far ahead should I lower the DNS TTL?

At least as long as the current TTL. If the record says 86400 seconds, lower it to 300 a full day before cut-over, otherwise resolvers keep serving the old address from cache after you switch.

Can I avoid a write freeze entirely?

Only with replication. With PostgreSQL or MySQL replicas running on the new instance, the freeze shrinks to the seconds it takes to promote the replica. With a dump and restore, expect a short read-only window sized to the database.

What about email and other DNS records?

Leave MX, SPF and anything not served by the old machine alone. Change only the A records for the services you are moving, and check whether any other hostname is a CNAME to them.

How long should I keep the old server?

Keep it running, untouched, for at least a week. It is your rollback and your reference when someone reports that one obscure feature behaves differently.