The Agent That Did the Right Thing Ten Thousand Times
TL;DR: Most governance thinking assumes the failure is a wrong decision. In utilities the characteristic failure is a correct decision applied at the wrong scale, which no amount of model quality prevents. Blast radius needs its own control, checked before the action, separate from whether the action was permitted at all.
Two different questions
Ask an engineer what could go wrong with an agent that adjusts setpoints and you will get answers about accuracy. It might misread the telemetry. It might act on stale data. It might have been trained on a period that does not resemble today’s grid.
All real. None of them is the failure that shows up in the incident reports.
A setpoint change is unremarkable. It happens constantly, it is exactly what the agent is for, and any individual instance of it would pass review. Ten thousand of them inside a minute is an event with a name.
The distinction worth building around is between two questions that sound similar and are not:
Is this action permitted? A policy question. Answered by looking at the action and the authority.
How much of this action may there be? A scale question. It cannot be answered by looking at the action, because every individual instance is fine.
Most systems only ask the first. The second usually lives in a rate limiter somewhere downstream, tuned by whoever was on call the last time something got noisy.
Why the rate limiter is not the control
Rate limiting is a good operational tool and a poor governance one, for three reasons.
It is set on the wrong axis. A limiter caps requests per second against an endpoint. The thing you need to bound is the number of assets a single decision may affect, which is a property of the decision rather than of the traffic.
It is not attributable. When a limiter trips, the record says a service exceeded a threshold. It does not say which authority was being exercised, or by whom, or against what mandate. In an incident review that gap is the whole conversation.
And it is invisible to the agent. A limiter drops or delays; it does not tell the agent it has hit a bound, so the agent retries, which is the correct behaviour for congestion and exactly the wrong behaviour for a limit.
An agent that hits a governance bound needs to be told, in terms it can act on, that this is not a transient condition and retrying will not help.
The order matters more than people expect
If checks run in a fixed order, and they should, where scale sits in that order changes what gets recorded.
Put the scale check late, after policy evaluation and budget, and every refusal at scale has already consumed the expensive checks. Worse, if budget is debited before the scale bound is tested, a refused action costs allowance it never used.
Put it earlier, after the structural checks that establish there is an authority at all but before the expensive policy work, and a refusal at scale is cheap, fast and does not spend anything.
That ordering has a practical consequence for a control room. The refusals you see most often are the cheapest ones to produce, so the check can sit on the path of every dispatch decision without becoming the reason dispatch is slow.
Four agents, four different bounds
The pattern generalises across the estate, and the bound is different every time.
Dispatch and balancing. The bound is on assets affected per decision. The agent should be able to move a fleet when the situation calls for it, and there should be a number above which a human is in the loop. That number is a business decision, not an engineering one.
Demand response. The bound is on aggregate curtailment, not per site. Curtailing one industrial customer is routine. Curtailing enough of them to breach a set of contracts in a single afternoon is a different event, and the per-site limit does not see it coming.
Outage switching. The bound is not really on scale at all, it is on authority. Switching is what puts field crews at risk, and the only acceptable answer to “who authorised this switching operation” is a named engineer with a scope and an expiry. Ambient permission to switch is not a governance gap. It is a safety incident with a date yet to be assigned.
Field service scheduling. The bound is on context. The agent reads work orders, asset histories and contractor notes, and contractor notes are free text written by people outside your organisation. An agent that treats them as instructions can be steered by whoever writes them, which matters more when the output is a crew arriving at a live asset.
What an incident review will ask for
The reason to build this now rather than after is that the artefact you need is the one you have to have been collecting.
A review asks what happened, what was supposed to happen, who authorised the difference, and what stopped it getting worse. The fourth question is the one utilities are least ready for and the one with the most upside, because a control that demonstrably operated is worth more in that room than any amount of documentation about a control that exists.
An agent that was stopped at a bound is that demonstration. An agent that has never been stopped is either perfectly bounded or entirely unbounded, and from the outside, including from the outside of a committee room, those are indistinguishable.
The second question deserves a warning too. Reviewers will ask what the limits were at the time, and configuration drifts. A limit raised for a maintenance window in March and never lowered is the most ordinary finding in this sector. The intended state has to be sealed into the record at the moment of the decision, not looked up in a repository afterwards, or the answer is a reconstruction and everyone in the room knows it.
On NIS2, honestly
It would be convenient to present all of this as NIS2 readiness. It is not, and a directive transposed differently in twenty-seven member states does not have a single clause map that could be honest.
What it does contribute is the evidence an incident report needs: what acted, under whose authority, within what bound, and what was refused, available as a query rather than as a fortnight of correlation across three systems. Whether that satisfies your competent authority is regulatory interpretation and it rests with you and your counsel.
Where to start
Pick the agent whose actions are hardest to reverse. In most utilities that is switching, not dispatch, because dispatch is continuous and switching is discrete and consequential.
Write the sentence for it. This agent may operate up to N devices, in this area, during this window, on the authority of this named engineer.
Then run it in a mode that records without refusing, for a fortnight, against real operations. You are not looking for bad decisions. You are looking for the gap between the number in that sentence and the number the agent actually attempts on an ordinary Tuesday.
That gap is the whole finding, and it is usually available within a week.