Ask most teams to evidence a control and you will be shown a dashboard.
It is a fair answer and I understand why it gets given. The panel is green. The service has held for ninety days. Nobody has been called out about it since the spring. Every one of those statements is true, checkable and easy to screenshot, and not one of them answers the question that was asked.
A control that works and a control nobody has ever exercised produce the same telemetry on a quiet day. Both look like nothing.
The difference surfaces on the day the control is needed, which is the one day you do not get to choose. So the question to put to your own estate, before somebody external puts it to you, is narrow and slightly uncomfortable. Can you prove the control worked? Or can you only prove that nothing has broken yet?
Your regulator stopped asking for logs some time ago
If you operate inside the EU financial perimeter, the obligation is already written down and already binding. Commission Delegated Regulation (EU) 2024/1774, the regulatory technical standard sitting underneath DORA, carries an article titled Logging. It is Article 12. It has been in force since July 2024, and DORA itself has applied since January 2025.
Read the verbs and not the noun. Financial entities shall "develop, document, and implement logging procedures, protocols and tools". Not hold logs. Build, write down and operate the machinery that produces them.
Then read the two clauses that are easy to skim past. Article 12 requires "measures to detect a failure of logging systems". It also requires the clocks of every ICT system to be synchronised "upon a documented reliable reference time source".
Those two are unusually well informed for a legal text. The law wants you to notice when your own evidence stops being produced. And it wants your timestamps to agree with each other, because evidence that cannot be ordered will not carry an argument. Anyone who has tried to reconstruct an incident across three systems with drifting clocks knows exactly what that second clause is protecting against. Somebody in Brussels has clearly had that afternoon too.
|
A control that works and a control nobody has ever exercised produce the same telemetry on a quiet day. |
Your auditors settled this argument decades ago
None of this is new thinking, only new to infrastructure.
Financial audit has a name for a report your own systems produced: Information Produced by the Entity. The doctrine around it is old, settled and entirely unsentimental. A system-generated report only becomes audit evidence once somebody has established that it is complete and accurate.
The US Public Company Accounting Oversight Board sets out the two acceptable routes in AS 1105. Test the information itself, or test the controls over its accuracy and completeness. BDO's guidance puts the consequence in a single clause: conclusions built on incomplete or inaccurate information may be invalid "even if the audit procedures themselves were well designed".
That should stop an observability leader mid-sentence. It says the quality of your investigation will not rescue you from the quality of your inputs. You can run a careful, well-documented incident review on a dashboard that has never been validated, and arrive at a confident wrong answer.
Your finance colleagues have lived inside that constraint for decades. Telemetry has mostly escaped it, because the question was never put to it.
The UK has now put a date on the question
PS26/2 from the FCA and PS7/26 from the PRA, both published on 18 March 2026, set new rules for operational incident and third-party reporting. Firms will have to notify the regulators of new or significantly changed material third-party arrangements, and keep a register of those arrangements, submitted every year. The rules take effect on 18 March 2027, five months after you read this.
The ground underneath moved this summer too. The Critical Third Parties (Designation) Regulations 2026 came into force on 13 July, designating four cloud providers: Amazon Web Services EMEA SARL, Google Cloud EMEA Limited, Microsoft Ireland Operations Limited and Oracle Corporation UK Limited. The Bank of England, the PRA and the FCA now oversee them directly.
The register lands on you, not on them. It gets filed every year, and every entry on it is a dependency your firm should be able to evidence. In a UK regulated firm, accountability like that tends to end at a job title and a surname. Not a department.
Most organisations are not generating the evidence
Catchpoint's SRE Report 2026, drawn from 418 responses, puts a number on the gap. Only 17 per cent of organisations run chaos experiments regularly. Thirty-four per cent have never tested failure in production at all.
Set that against what these rules are reaching for and the arithmetic is unkind. They want evidence that the machinery works, rather than a record that nothing broke. Fewer than one organisation in five runs chaos experiments on any regular basis.
I do not think this is negligence, and I would rather not see the other 83 per cent treated as though it were. Testing failure in production needs organisational permission that many engineering leaders have never actually been granted. The report's own cut points the same way: among the organisations that do not run chaos experiments regularly, tolerance for planned failure is lower. I read that as a permission problem. The teams are mostly not refusing to test. They are being told no, politely, usually by somebody who has never had to file a register.
The one-minute test
The test I would apply is deliberately blunter than a maturity model.
Pick a control. Ask whether a competent person who was not in the room can confirm it did what it claims, in under a minute, without rebuilding the reasoning from scratch.
Under a minute is the part doing the work. Given a week, a good engineer can evidence almost anything: pull the raw telemetry, reconstruct the sequence, write the narrative, produce something that satisfies everyone present. It is a research project that happens to land on the right answer, and it will not survive being run forty times in an audit cycle, or once at two in the afternoon during a live incident.
Controls tend to fail this test in one of three ways, and the way they fail tells you what to fix. Sometimes the evidence exists but is scattered across systems that do not share a clock, so the assembly is the work. Sometimes it exists in one place but needs an expert to interpret it, which means the control is really a person, and that person takes holidays. And sometimes it was never produced at all, and the green panel had been reporting an absence of alarms, not the presence of control.

The one-minute test: could someone who was not in the room confirm a control did what it claims, in under a minute? Three ways a control fails it, and the third is the common one.
That third failure is the most common, and the one people are least willing to say out loud.
What I would do before the next attestation
None of this needs a programme. It needs an afternoon and a tolerance for being embarrassed.
Pick three controls that would appear on a register. Not the ones you are proud of. The failover you have never invoked, the redaction step in your telemetry pipeline, the alerting rule that has not fired in a year.
Try to evidence them cold. No warning to the team.
Write down two numbers for each: how long it took, and what you could not produce at all. The second number is the finding. The first is only context.
Then work out whether the gap is a tooling problem or a permission problem. The fix is completely different, and the first is far easier to buy. An organisation that will not authorise a failover test in production does not have an observability gap. It has a decision nobody has made, sitting in the space where a control should be.
Run the exercise once a quarter and you end up with something more useful than a maturity score. You end up with a shrinking list of things you cannot prove.
Years ago, as head of monitoring, I worked an outage that ran past seventy hours, with several incident teams on it and the regulator involved. The blind spot was a vendor-managed API gateway, a black box with no metrics coming out of it. I had been asking the vendor for those metrics for months. In the incident room we finally agreed to put an agent from a tier-one observability product on it. Data was flowing about forty minutes after the approval, and we saw the problem straight away. Seventy hours blind. Forty minutes sighted. That is what sold observability to senior management there.
|
An organisation that will not authorise a failover test in production does not have an observability gap. |
Probably fine is a private position
Perhaps nothing is going wrong. Your controls may well be fine, and most of them probably are.
But "probably fine" is a position you hold privately, and privately is no longer where these questions get asked. From next March, the arrangements you depend on most have to be written down on a register and filed every year.
Most firms already own more telemetry than they can defend, so buying more will not settle it. What settles it is deciding, in advance and in daylight, which of your controls you would be content to be asked about.
Make that list before somebody hands you theirs.
|
Work with me An honest read on what your observability is actually doing. | |
|
If you lead observability in a regulated enterprise, I run a fixed-scope Observability Assessment for senior IT and engineering leaders. It ends in a written roadmap and a readout, not a sales deck.
Not ready to talk? Start with a free chapter of Metrics & Mayhem. |
