Your automations report success.
That is not the same as working.

Monitoring watches for errors. The expensive failures do not throw one — the job runs, the log gets written, the exit code is zero, and nothing actually happened. Flatline is a free open-source tool that looks for that specific shape of failure.

The one that started it. A scheduled job on our own machine ran every weekday for four days, wrote a full log each time, and reported success. It had been starting in the wrong directory the whole time, so every file it wrote went nowhere. Four days of data, gone, with four green checkmarks on top of it.

How you would use this

Five shapes of the same problem. If any of these sounds familiar, it is worth twenty minutes.

The backup that backs up nothing

It has succeeded every night for eight months. Nobody has opened the file.

jobs — checks whether the output a job is supposed to write actually changed, instead of trusting the job's own exit code. A 0-byte file that keeps its timestamp fresh is the classic.

The client automation that stopped firing

A Zap, Make scenario or n8n flow whose trigger has returned zero rows since a field got renamed upstream. No error was raised, because finding nothing is not an error.

jobs + scan — a run history where the "records processed" column is 0 forever reads as a flat signal, not as activity.

The dashboard column that froze

One field has shown the same value since a schema change three months ago. The chart still draws, so nobody noticed.

scan — measures how much a column actually varies. A value that never changes carries no information; it is a constant wearing a signal's name.

The alert that has never once fired

You have a threshold alert. It has been quiet for a year. You believe that means everything is fine.

code — finds unreachable branches and errors caught and thrown away. On our own stack it found a desktop notification that could never fire, because the line above it ended the script.

The report that is quietly stale

A weekly summary still arrives on time. The numbers in it stopped moving a while ago.

verify — takes a claim ("this refreshed today", "these files are tracked") and checks it against the system instead of against another report.

What it checks

scanSignals that never vary. Measures information content, so a column of identical values is flagged even when it looks populated.

jobsScheduled tasks that ran but produced nothing. Compares intent against the output that actually changed on disk.

verifyStated claims against the underlying system, because a status field is a claim, not evidence.

codeUnreachable branches and swallowed exceptions — the guard wrapped in a catch that discards the thing it was guarding against.

What it has actually caught

A job's first-ever scheduled run, before it ran — three file writes that would have landed somewhere unrecoverable and reported success.
A diagnostic flag that fired on 100% of rows for a week. A warning that is always on is not a warning.
A desktop notification that had never fired once in the life of the script, because everything after one line was unreachable.
Three false positives in itself. Each was fixed by measuring real code rather than arguing about it — which is the standard we would hold your stack to.

Run it yourself

Free, MIT licensed, 108 tests. It reads; it does not change anything it looks at.

pip install flatline
flatline scan your-export.csv
flatline jobs

Source and documentation on GitHub →

Or have us point it at your stack

The tool finds the dead signals. Deciding which ones cost you money, and fixing them, is the part that takes judgment. We do that as a fixed-scope audit — you get the findings and the evidence for each one, including the ones that turn out to be fine.

Ask about an automation audit See everything we have built →

Your automations report success.That is not the same as working.