Rollback Cannot Restore a Contract You Already Removed

Wed, 01 Oct 2025 00:00:00 +0000

Release A stores a catalogue quote in amount_cents. Release B renames the column to amount_minor and deploys successfully. Every new B process is healthy, so the deployment dashboard looks reassuring.

One A process is still serving traffic during the rolling update. Its next query asks PostgreSQL for amount_cents and fails because the column no longer exists. Restoring the A image produces the same error. The old code returned; the contract it needs did not.

Observability Must Answer the Next Operator Question

Mon, 01 Sep 2025 00:00:00 +0000

A catalogue quote request returns 503 after its price dependency times out. One telemetry bundle produces five uncorrelated log lines, a metric labelled with request ID and raw URL, and three pages. None of those pages names an owner, the affected scope, the hold duration, or even the first query to run. There is plenty of telemetry and very little help.

The other bundle records less and tells the operator more. One page identifies the affected route, region, and release. Its first query measures the failure users can see. From there, an exemplar selects a failed trace, the trace locates the dependency wait, and a correlated event names the application failure class. The runbook then asks the question which actually matters: is it safe to bypass that dependency?

Observability and Deployment on Brian Faust

Rollback Cannot Restore a Contract You Already Removed

Observability Must Answer the Next Operator Question