Zero-Downtime Cutover Plans
Context
Zero-downtime cutovers require strict environment parity, precise cache expiration controls, and atomic data synchronisation. They target webmasters, SEO engineers, site architects, and technical project managers who must move a live site between hosts without a visible outage or a ranking dip. Infrastructure drift or misaligned resolver caches fracture traffic routing and trigger immediate volatility, so the plan enforces deterministic failover paths and preserves crawl budget through the transition window. This work orchestrates the other phases of the DNS Configuration & Hosting Cutover sequence into a single rehearsed switch with a rehearsed reversal.
Zero downtime is not the absence of risk; it is the presence of a reversible, gradual switch. Two mechanisms make that possible. Blue-green keeps the legacy origin (blue) fully warm while the new origin (green) takes a slice of live traffic, so you can swing back instantly by shifting weight rather than waiting for DNS to re-propagate. Weighted DNS does the same at the resolver layer, advancing traffic from 10% to 100% in stages and watching error rates at each increment. Both depend on the same precondition: an atomic data switch, because you cannot have writes landing on two origins at once. The execution sequence therefore freezes writes first, proves parity, then moves traffic in controlled steps, each one cheap to reverse.
It is worth being precise about the promise. “Zero downtime” does not mean nothing can go wrong; it means no single step in the plan requires an outage to perform, and every step can be undone without one. A cutover that achieves an unbroken uptime graph while silently dropping the orders placed during the sync window has not delivered zero downtime — it has delivered zero visible downtime, which is a different and much weaker property. The measure that matters is whether a user’s session, cart, and writes survive the transition intact, and that is decided by the data sequencing rather than by the routing mechanism.
The two mechanisms also fail differently, which is worth knowing before you pick one. Weighted DNS fails gradually and opaquely: a bad new origin degrades a slice of users, and because resolvers honour the weighting on their own schedule you cannot say precisely which slice or move them back promptly. Blue-green fails sharply and controllably: the balancer knows exactly which requests went where and can move all of them back in one action. If your failure budget is expressed in affected users, blue-green gives you a lever proportional to that; if it is expressed in cost, weighted DNS is markedly cheaper.
Pre-flight Checks
Establish identical staging and production baselines before modifying any authoritative record. Configuration drift guarantees split-brain states and unpredictable routing.
- Mirror OS kernels, runtime versions (PHP/Node/Python), and database schemas using infrastructure-as-code templates.
- Validate network ACLs, firewall rules, and WAF policies across both environments.
- Reduce authoritative A/AAAA/CNAME TTLs to 300 seconds 48–72 hours before execution using TTL Optimization Strategies.
- Pre-stage new DNS records with identical TTLs pointing at staging IPs for dry-run validation.
- Confirm data and configuration parity through Staging to Production Sync before any traffic shifts.
Execution Steps
Execute the switch in strict sequence. Do not parallelise database locks with DNS updates; maintain write-block windows to guarantee atomic consistency.
1. Lock Source Writes
Enforce application-level write locks or run SET GLOBAL read_only = ON; on the source database 15 minutes pre-cutover (MySQL 5.7+/8.0). This freezes the transactional state so the final sync is a clean snapshot with no in-flight writes.
Announce the write lock rather than simply applying it. Fifteen minutes of read-only behaviour is a non-event if users see a clear maintenance notice on write actions and a support team knows it is coming; it generates a wave of duplicate tickets if a checkout button silently fails. Implement the lock at the application layer where you can control the message, and use the database-level read_only flag as the backstop that catches anything bypassing the application rather than as the primary mechanism.
2. Finalise Data and Asset Sync
Run incremental replication with mysqldump --single-transaction --routines --triggers or an AWS DMS continuous task, then rsync -avz --checksum --delete static assets for byte-perfect parity. Capture the final delta exactly as described in Staging to Production Sync to avoid primary-key collisions on merge.
3. Reconfigure the Edge and Bust Caches
Update CDN origin pull endpoints to the new hosting IP and configure origin shield routing to absorb backend load spikes. Inject Cache-Control: no-store, max-age=0 and Pragma: no-cache on critical HTML/CSS/JS so the edge does not serve stale assets during the swap.
no-store on HTML during the window is deliberate and temporary, and it has a cost worth acknowledging: it removes edge caching from exactly the pages that benefit from it most, so origin load rises at the moment you are least keen on surprises. Size the origin for that. Just as importantly, put the restoration of normal cache headers on the post-cutover checklist with an owner — a migration that permanently disables HTML caching is a migration that permanently slowed the site down, and it is the kind of regression that takes months to notice.
4. Swap Records and Shift Traffic Gradually
Update A/AAAA records to production IPs, monitor SOA serial increments on secondaries, then shift load balancer weight in stages using Using Weighted DNS for Gradual Traffic Migration and the session-persistence approach in Implementing Blue-Green Deployments for Site Migrations.
5. Invalidate Edge Caches and Verify
Trigger a full edge cache invalidation via the provider API immediately after propagation completes, then run DNS Propagation Tracking across 10+ global resolver nodes to confirm adoption before retiring the legacy origin.
Laid out against the clock, the reason the sequence cannot be parallelised becomes obvious: the write lock has to close before the final sync starts, and the record swap has to land after the edge is already pointing at the new origin. Every overlap in that grid is a window where two systems disagree about which origin is authoritative.
Rehearse the whole sequence end to end against staging before the real window, including the reversal. The rehearsal is where you discover the things no plan captures: that the write lock takes four minutes to drain rather than thirty seconds, that the sync command needs a flag nobody documented, that the balancer’s health check marks the new origin unhealthy for ninety seconds after every deploy. Record the elapsed time of each phase during the rehearsal and use those numbers — not estimates — to size the maintenance window you announce.
Configs / Commands
Zone file excerpt — use the exact production IP:
@ 300 IN A 203.0.113.10 ; low TTL set during the pre-flight reduction
www 300 IN CNAME origin.newhost.com.
Database sync pipeline (MySQL):
# --single-transaction takes a consistent snapshot without locking InnoDB tables
mysqldump -u root -p \
--single-transaction \
--set-gtid-purged=OFF \
--routines --triggers \
production_db \
| mysql -h new_host_ip -u root -p production_db
CDN origin headers (Nginx):
# Force fresh fetches during the transition; relax after adoption is global
add_header Cache-Control "no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0" always;
add_header X-Frame-Options "SAMEORIGIN" always;
Validation command chain (dig / openssl / curl):
# Verify DNS resolution, NS delegation, TLS chain, and HTTP status in one pass
dig @8.8.8.8 yourdomain.com A +short
nslookup -type=NS yourdomain.com 8.8.8.8
openssl s_client -connect yourdomain.com:443 -servername yourdomain.com < /dev/null 2>/dev/null | grep -E "subject|issuer|Verify"
curl -s -o /dev/null -w '%{http_code}\n' https://yourdomain.com
The four-command chain above is not redundancy — each command proves a different link, and a green result from one tells you nothing about the others. Read it as a ladder: if an early rung fails there is no point running the rest.
Retire the legacy origin in stages rather than in one action. Stop the application first but leave the host reachable, so anything still routed there fails visibly and traceably instead of hitting a black hole you cannot diagnose. Keep the address itself allocated for considerably longer than feels necessary — cloud providers recycle released IPs quickly, and a resolver still holding your old record will hand your users, and any traffic carrying your session cookies, to whoever holds that address next. Releasing an IP is the one step in this playbook with no rollback at all.
Validation
Verify technical integrity immediately after the switch; confirm global resolver adoption and protect organic search equity.
- Validate redirect maps from URL Mapping & Redirect Architecture:
curl -I -L -s -o /dev/null -w '%{http_code} %{url_effective}'. - Verify SSL certificate chain validity, HSTS enforcement, and canonical tag consistency across migrated URLs.
- Monitor real-time error rates, TLS handshake failures, and connection pool exhaustion via APM dashboards.
- Confirm global resolver adoption with DNS Propagation Tracking across 10+ nodes.
- Begin the Search Console Handover so the new property starts collecting crawl and coverage data.
Critical Failure Modes:
- TTL left at the default 86400 s causes 24–48 hour propagation delays and split traffic.
- Failing to lock database writes during the final sync results in data loss or duplicate entries.
- CDN origin shields pointed at decommissioned IPs trigger persistent 502/504 errors.
- Omitting HSTS preload updates or SSL chain validation causes browser security warnings.
- Skipping 301 mapping for legacy URL parameters causes immediate ranking decay and crawl waste.
Session persistence is the detail that turns a clean weighted shift into a support queue. Two origins that both work perfectly will still break a user who is bounced between them mid-journey if sessions are stored locally rather than in a shared store, or if the two environments were issued different session-signing secrets. Confirm before the shift that a session created on one origin is honoured by the other — the fastest check is to authenticate against the new origin, then replay the cookie against the legacy one and confirm you are still logged in. Where a shared store is not achievable, pin sessions at the balancer so a user stays on whichever origin greeted them.
Write the abort criteria down before the window and put them somewhere the whole team can see during it. Thresholds agreed in advance are read as facts; the same thresholds recalled from memory at 02:00, with a migration half-complete and everyone invested in finishing, are read as suggestions. The specific failure this guards against is the slow slide — error rates sitting just under the trigger for long enough that the trigger starts to feel unreasonable, and the shift advancing anyway. A printed number and a named owner are the cheapest defence available against that particular form of optimism.
Rollback Triggers
Abort the cutover and revert to the previous authoritative configuration if any threshold is breached. Do not patch a failing live switch — follow Migration Rollback Playbooks.
- Propagation Stagnation: global resolver adoption falls below 85% after 60 minutes.
- Database Divergence: checksum mismatch or replication lag exceeds 300 seconds during final sync.
- CDN Origin Failure: 5xx error rate exceeds 2% or origin timeout spikes above 5 seconds.
- TLS/SSL Breakage: certificate chain validation fails or HSTS headers drop unexpectedly.
- Traffic Drop: organic session volume falls more than 15% within the first 30 minutes post-switch.
FAQ
What is the minimum safe TTL value before initiating a cutover? Set TTL to 300 seconds at least 48 hours in advance so global ISP and recursive resolver caches expire, allowing near-instant propagation when authoritative records are updated.
How do I prevent database inconsistencies during the final sync window?
Enable read-only mode on the source (SET GLOBAL read_only = ON;), run a final incremental sync, then atomically switch application connection strings, using maintenance flags to block writes during the transition.
Should I purge the CDN cache before or after updating DNS? Purge immediately after records propagate globally — purging beforehand forces edges to re-fetch from the old origin, while purging post-cutover ensures fresh assets from the new infrastructure.
How do I verify SSL/TLS integrity post-cutover?
Run openssl s_client -connect domain.com:443 -servername domain.com < /dev/null to validate the chain, expiry, and SNI, then cross-check with curl -I -v https://domain.com for HTTP/2 and HSTS delivery. Check the intermediate chain specifically, not just the leaf certificate — a new origin frequently serves a valid certificate without the intermediates, which browsers paper over using cached issuers while non-browser clients, payment integrations, and mobile apps reject outright. The failure therefore looks like “the site is fine but the app is broken”.
How do I decide when a weighted stage is safe to advance? Advance on evidence from the current stage, never on elapsed time. Each increment should hold for long enough to accumulate a statistically meaningful sample at the new weight — for most sites a few thousand requests rather than a fixed number of minutes — and the error rate, p95 latency, and conversion rate at that weight should all sit inside the tolerance agreed beforehand. Advancing on a timer means a 5% stage that was quietly failing becomes a 50% stage that is loudly failing.
Is HSTS a problem during a cutover?
It can be, in one direction. If the legacy origin issued a long Strict-Transport-Security header and the new origin’s TLS is not ready, browsers that have cached the policy will refuse to fall back to HTTP rather than showing a bypassable warning — which is the correct behaviour and also means a botched certificate is a hard outage for returning visitors. Have TLS fully working on the new origin before any traffic is shifted, and avoid enabling preload in the same window as a migration.
What should the load balancer health check actually test?
Something that exercises the dependencies, not just the web server. A health endpoint returning a static 200 confirms only that the process is listening, which means the balancer will happily route full traffic to an origin whose database connection pool is exhausted or whose cache backend is unreachable. Have /healthz touch the database and any critical downstream service and fail if either is unavailable — while keeping it cheap enough to be polled every ten seconds without becoming load in its own right.
Can I run a zero-downtime cutover without a load balancer at all? Yes, using weighted DNS alone, and for many sites that is the pragmatic choice. What you give up is the instant, precise reversal: without a balancer, moving traffic back means changing weights and waiting for resolver caches to honour them, so your reversal time is bounded by the TTL rather than by a single API call. That trade is reasonable for a content site with a well-lowered TTL and unwise for anything where a few minutes of split traffic means inconsistent writes.
How do I keep analytics and conversion tracking comparable across the switch? Decide before the window whether the new origin reports into the same property or a new one, and prefer the same one. Splitting into a fresh property makes the before-and-after comparison you most need — did conversion hold? — impossible without manually stitching two datasets, and it usually happens by accident because the new environment shipped with a placeholder tracking id. Verify the id on the new origin as part of the parity check, annotate the cutover timestamp in the analytics tool itself so later readers can see it, and expect a visible discontinuity in session counts on the day regardless: users mid-session when traffic shifts are frequently counted twice.
Related
- Implementing Blue-Green Deployments for Site Migrations
- Using Weighted DNS for Gradual Traffic Migration
- Staging to Production Sync
- DNS Propagation Tracking
- Migration Rollback Playbooks
← Back to DNS Configuration & Hosting Cutover