all systems nominal

tellr is watching itself.

This page is our own production instance of tellr, watching tellr. It's the same binary we ship. Secrets, tokens, and anything customer-adjacent are redacted. Everything else is live. Refreshes every 30 seconds.

uptime · 90d
99.982% ↑ 0.014pp
7m 46s down vs 17m 22s prev
silent streak
3d 14h 27m ↑ 2d 08h
no alerts pinged vs 1d 06h prev
checks / min
1,284 ↑ 9.2%
42 targets vs 1,176 prev
llm cost · 30d
$4.12 ↑ 18%
openai key sk-proj-7f2a91bc vs $3.49 prev

Components 90-day uptime, newest on right

component last check 18s ago

Recent incidents last 90 days

resolved Postgres replica lag spiked after migration
apr 18, 04:12 UTC · 23 min
A long-running migration on events table held an exclusive lock; replica lag climbed to 47s before the migration finished. Alerts routed to #ops-alerts and paged amr@tellr.dev.
04:12 · detected by db.replica-lag check · threshold 10s
04:14 · llm explanation posted to slack
04:35 · migration completed, lag back to 120ms · auto-resolved
resolved Outbound SMTP to alerter → gmail throttled
mar 29, 22:08 UTC · 6 min
Gmail started returning 421-4.7.28. We were sending too many digests from a fresh IP on fly-node-hel1-4. Switched senders and alerts resumed.
resolved Scheduler paused during hetzner maintenance
feb 04, 01:02 UTC · 14 min
Notified in advance. Scheduler drained, checks buffered in redis (host=10.0.2.4:6379), resumed cleanly with no lost events.
that's it. no other incidents in the last 90 days.

Live check feed 30s window · redacted where sensitive

streaming · ● live 0 events

Active monitors 42 targets · 113 checks

target config · redacted
api.tellr.dev
http · /healthz · every 30s
secrethmac_a3f2c91b4dhmac
100%
db.prod · postgres
tcp · 5432 · disk 80% warn
dsnpostgres://user@10.0.1.7url
100%
workers · sidekiq
queue · max_depth 500
redisredis://10.0.2.4:6379url
99.97%
alerter · outbound
http · slack/discord/tg
webhookshooks.slack/T0291/B…×3
99.94%
llm · composer
openai · gpt-4o · budget 5$/d
api_keysk-proj-7f2a91bc4de8key
100%
signup_funnel · llm check
hourly · conversion ±10%
queryselect date, count(*)sql
100%
page generated just now · region hel1 · tellr 0.9.2 · commit a3f21c · cpu 8% · mem 142MB · goroutines 47