Compliant and still blind: What Article 72 monitoring doesn't see

Although the EU AI Act's post-market monitoring requirements are not yet applicable to most high-risk AI systems, many organizations are already preparing Article 72 monitoring plans. From what I have read, most are responding exactly as they always have when a new regulation arrives: writing the monitoring plan, drafting the incident process, gathering the logs, compiling the documentation required under Annex IV and closing the file. Another piece of legislation, another set of papers created, signed off and stored away without seeing daylight again. Once the paperwork exists, the instinct is to move on. The job feels done.

That is usually the moment the real problem begins.

A finished plan has a peculiar effect. People start checking the plan rather than the system. I have watched this for years across banks, telecommunications and consulting operations. It is not malicious. It is simply how governance behaves when it becomes paperwork. 

Organizations fall back on the habits they have always had: produce the document, file it and assume its existence is proof the work is happening. They treat it as red tape, something to get through, not something to think about. What is missing is any pause to consider why the requirement exists, what the consequences might be or how the system itself is changing underneath them. 

Companies are not taking the time to think strategically about the impact or the long-term implications. The logic has not evolved anywhere near as fast as the systems they are now meant to govern. That mismatch between the pace of AI and the pace of organizational habit is one of the biggest challenges they face, now and in the years ahead. It is what I call the "neglect trap."

The trouble is that many organizations risk designing Article 72 monitoring primarily around performance failure: accuracy dropping, error rates rising, complaints piling up. These plans are not designed to catch behavioral drift. Drift does not need to breach any threshold to be real. A model can hit every accuracy target and still be behaving differently from the version that was assessed. Nothing in the dashboard will show it. Nothing in the logs will look odd. But the system has changed shape.

What moves in a deployed AI system is not the architecture or the data pipeline. It is the behavior, in the same way a team or a department develops tendencies over time. Monitoring built only for performance will never catch that.

This is where the AI Act itself contains a quiet tension. Article 72 asks for continuous compliance with what the system was assessed to do. That is not the same question as whether the system still behaves the way it did at assessment. The act does not separate the two. Many monitoring plans do not either. And that is how drift becomes invisible.

Take a credit scoring model. At launch, it is assessed against a clean set of inputs, with clear weighting and a tidy rationale. Six months later, the lending team has been through a backlog, a shift in appetite and a period of increased manual overrides on borderline cases. Those overrides feed back into the operational data the model relies on. The distribution of accepted and rejected applications changes. 

The model is periodically recalibrated on this new data, as most production models are. Nothing in the technical performance metrics looks unusual. But the recalibration has quietly shifted the model's behavior: fewer escalations, more automated approvals, a slightly different reading of the same applicant profile. Performance looks fine. Behavior has moved.

That is drift. And drift is exactly what many post‑market monitoring plans will not catch.

Catching drift needs three changes, none of which sit comfortably inside a compliance file.

Someone needs to own behavioral change specifically. Not just uptime or incident handling, and not only model performance, but the question of what the system is becoming over time. In most organizations, nobody holds that responsibility. It is assumed to sit with whoever happens to be closest to the model. That is how drift becomes normal.

Monitoring needs a live comparison baseline. Not a tolerance band, not a performance threshold. A simple question: Is the system doing the same thing it did at the last review? Without that reference point, drift is invisible by design.

And every monitoring plan needs a second sensor, something that does not rely on the system or the team closest to it to notice when behavior has quietly shifted. Whoever holds this responsibility needs the authority to flag drift and trigger review without waiting for a threshold to be breached. Behavioral change rarely arrives as an incident. It accumulates quietly, one interaction at a time, until the system is doing something nobody ever signed off on.

The real consequence of drift is not regulatory. It is strategic. A system that behaves differently from the one leadership thinks it has is a governance problem, but it is also a business problem.

Article 72 is deliberately open on method, proportionate to risk, no single template. That flexibility is sensible. But it also means the quality of monitoring will depend entirely on whether an organization treats it as a deliverable or a discipline. No deadline will settle that. Only practice will.

The absence of an incident is not evidence of stability. It is usually just evidence that nobody has looked closely enough, recently enough, to notice.