fin1teds
v1.2.0
  1. ds
  2. Operations
  3. Incident history

Incident history

Updates from investigating to resolved, by day.

  1. Delayed webhook deliveriesMonitoring

    1. MonitoringThe backlog is draining at about 40,000 events a minute and should clear by 15:10. New events are delivered within 30 seconds.
    2. IdentifiedOne of three relay workers stopped acknowledging after a config push. It has been rolled back.
    3. InvestigatingWebhooks are arriving up to 12 minutes late. We are looking into it.

  2. No incidents reported.

  3. API errors in ap-south-1Postmortem

    1. ResolvedError rates are back to normal. 3.1% of requests in ap-south-1 failed between 09:12 and 09:41 IST.
    2. IdentifiedA database failover took longer than expected; connections are recovering.

    Database maintenanceCompleted

    1. CompletedPostgres 17.4 upgrade on the primary cluster. No downtime.
Show code
<ol class="incidents">
  <li>
    <time datetime="2026-10-11">11 Oct 2026</time>
    <article class="incident minor">
      <h3 class="incident-h"><a href="#">Delayed webhook deliveries</a><span class="badge warn">Monitoring</span></h3>
      <ol class="updates">
        <li class="monitoring"><b>Monitoring</b>The backlog is draining at about 40,000 events a minute and should clear by 15:10. New events are delivered within 30 seconds.<time>14:32 IST</time></li>
        <li class="identified"><b>Identified</b>One of three relay workers stopped acknowledging after a config push. It has been rolled back.<time>14:06 IST</time></li>
        <li class="investigating"><b>Investigating</b>Webhooks are arriving up to 12 minutes late. We are looking into it.<time>13:51 IST</time></li>
      </ol>
    </article>
  </li>
  <li>
    <time datetime="2026-10-10">10 Oct 2026</time>
    <p class="none">No incidents reported.</p>
  </li>
  <li>
    <time datetime="2026-10-09">9 Oct 2026</time>
    <article class="incident resolved">
      <h3 class="incident-h"><a href="#">API errors in ap-south-1</a><span class="badge">Postmortem</span></h3>
      <ol class="updates">
        <li class="resolved"><b>Resolved</b>Error rates are back to normal. 3.1% of requests in ap-south-1 failed between 09:12 and 09:41 IST.<time>10:05 IST</time></li>
        <li class="identified"><b>Identified</b>A database failover took longer than expected; connections are recovering.<time>09:30 IST</time></li>
      </ol>
    </article>
    <article class="incident maint resolved">
      <h3 class="incident-h">Database maintenance<span class="badge">Completed</span></h3>
      <ol class="updates">
        <li class="resolved"><b>Completed</b>Postgres 17.4 upgrade on the primary cluster. No downtime.<time>03:40 IST</time></li>
      </ol>
    </article>
  </li>
</ol>

Incident history by day, newest first. Days with nothing on them say so; a run of quiet days is the record you want people to see. Each incident has a title coloured by impact, a badge with where it stands, and its updates on a hairline rail with the newest on top.

Colour

An open incident's title takes its impact colour: red for an outage (the default), amber for .minor, the cool blue for .maint. Once .resolved, the title goes back to ink, because a fixed problem is history, not an alarm. The dot of the newest update shows the stage: red investigating, amber identified, blue monitoring, green resolved.

ClassOnWhat it does
.incidentsolDays, separated by rules. Each <li> opens with a <time> in the label style.
.nonepThe quiet-day line.
.incidentarticleOne incident. .minor, .maint; .resolved returns the title to ink.
.incident-hh3Title (a link to the incident page) and a .badge for the stage or "Postmortem".
.updatesolThe rail. Each <li>: <b> stage, text, <time>.
.investigating / .identified / .monitoring / .resolved.updates > liColour of the dot on the newest update.
Do
  • Say what users saw, how many and for how long: "3.1% of requests in ap-south-1 failed for 29 minutes."
  • Post the first update within ten minutes, even if it only says you are looking.
  • Link the postmortem from the resolved incident.
Don't
  • "Some users may be experiencing issues." Say which users and which issues.
  • Editing old updates. Add a new one.
  • Root cause guesses in the investigating update.

Accessibility

  • Use real <time datetime> elements with the timezone in the text.
  • Stage is in the bold word and the badge, never only in the dot colour.