Cloud Platform · production · shared release

DeployAngel checks each production deploy and says when it's safe to stop watching.

Cleared

run-99 cleared for normal traffic at 19:48 EDT (17 min after rollout)

Code release · 68e474e8271b · released Oct 6, 19:31 EDT · rolled out Oct 6, 19:31 EDT · deployed by GitHub Actions · database migration

Cleared for normal traffic, which doesn't cover everything:

Not assessed: HTTP p95 latency (needs 200 requests in baseline and observation; had 140)
Not run since the deploy: critical flows notifications and sign_in
Changed within limits: job duration p95 218 ms → 1002 ms (4.6×; it warns at 2.0× and +1000 ms)

Confidence: High
Coverage: 100%; of normally active routes and jobs, run at least once
Cleared in: 17 min; after rollout
Baseline: 5cf2a01, 2d40abb and 3 more; spans several releases, to reach enough samples

Compared with baseline

Before is the baseline (5cf2a01, 2d40abb and 3 more), after is run-99 since rollout.

13 signals

✓ No failing or warning signals. No new exceptions.

All signals: 13 (10 passing, 3 without enough data)

Signal	Before	After	Samples	Status
Job failure rate Job failure rate · EvaluateVerificationsJob Rule: needs 20 attempts EvaluateVerificationsJob 0.00% → 0.00% · 14 samples	0.00%	0.00%	14	Not enough data needs 20 attempts
p95 latency p95 latency Rule: needs 200 requests in baseline and observation 218 ms → 264 ms · 140 samples	218 ms	264 ms	140	Not enough data needs 200 requests in baseline and observation
p95 latency p95 latency · POST /api/v1/telemetry Rule: needs 50 requests in baseline and observation POST /api/v1/telemetry 52 ms → 57 ms · 48 samples	52 ms	57 ms	48	Not enough data needs 50 requests in baseline and observation
Checkpoint Checkpoint · telemetry.ingested Rule: expected from the baseline, per request or per job as recorded; failing: p < 0.0001 and <= 20% of expected; warning: p < 0.001 and <= 50% p = 0.69 telemetry.ingested ~60 expected → 63	~60 expected	63	–	Pass
HTTP 5xx rate HTTP 5xx rate Rule: failing: p < 0.001 and >= 2x and >= +0.5 pp; warning: p < 0.01 and >= 1.5x p = 1 0.00% → 0.00% · 156 samples	0.00%	0.00%	156	Pass
Job duration p95 Job duration p95 Rule: warning only: >= 2.0x and >= +1000 ms 218 ms → 1002 ms · 194 samples	218 ms	1002 ms	194	Pass
Job failure rate Job failure rate Rule: failing: p < 0.001 and >= 2x and >= +1 pp; warning: p < 0.01 and >= 1.5x p = 1 0.00% → 0.00% · 194 samples	0.00%	0.00%	194	Pass
Job failure rate Job failure rate · IngestTelemetryJob Rule: failing: rate >= 20% and baseline < 5% and p < 0.0001; warning: p < 0.001 and >= 3x p = 1 IngestTelemetryJob 0.00% → 0.00% · 176 samples	0.00%	0.00%	176	Pass
Recurring job Recurring job · evaluate_verifications Rule: expected by 19:47 EDT (declared schedule) evaluate_verifications every minute → ran	every minute	ran	–	Pass
Recurring job Recurring job · ingest_telemetry Rule: expected by 19:46 EDT (declared schedule) ingest_telemetry every 5 seconds → ran	every 5 seconds	ran	–	Pass
Recurring job Recurring job · update_path_health Rule: expected by 19:50 EDT (declared schedule) update_path_health every 5 minutes → ran	every 5 minutes	ran	–	Pass
Queue latency p95 Queue latency p95 Rule: warning only: >= 3.0x and >= +30000 ms 1002 ms → 1002 ms · 194 samples	1002 ms	1002 ms	194	Pass
Route 5xx rate Route 5xx rate · POST /api/v1/telemetry Rule: failing: rate >= 20% and baseline < 5% and p < 0.0001; warning: p < 0.001 and >= 3x p = 1 POST /api/v1/telemetry 0.00% → 0.00% · 59 samples	0.00%	0.00%	59	Pass

What production exercised

What normally runs during this window, from the last 14 days. Each item counts once however busy it is, and one run is enough. Clearing needs 50% coverage, and high confidence needs 80%.

Normally active routes 1 of 1 · 100%

Regular job classes 2 of 2 · 100%

Recurring jobs due 3 of 3 · 100%

Not observable: none.

Critical flows

Flows that matter more than their traffic suggests, counted since this release rolled out.

Deploy registration

POST /api/v1/deploymentsPOST /integrations/heroku/webhooks/:token

Ran once
Notifications

job:DeliverNotificationJob

Not run yet
Sign in

POST /session

Not run yet
Telemetry ingestion

POST /api/v1/telemetrycheckpoint:telemetry.ingested

Ran 122 times
Verification

job:EvaluateVerificationsJob

Ran 14 times

History

First telemetry from this release

Oct 6, 19:32 EDT
Confidence Low → Medium

Oct 6, 19:38 EDT
Confidence Medium → High

Oct 6, 19:43 EDT
Cleared, still watching recurring jobs

Oct 6, 19:48 EDT
Stopped watching: a newer release rolled out

Oct 6, 20:05 EDT