The FRIA is coming: Assess AI connectors, not just AI systems

Over the past few months, I have spent a considerable amount of time discussing artificial intelligence governance programs with privacy, security and compliance teams. One pattern appears repeatedly.

Teams can usually describe the AI model, the vendor, the intended use case and even the regulatory obligations that apply to the deployment. Yet when the conversation shifts to what the AI system can actually access once connected to enterprise systems, the answers often become much less certain.

Can it retrieve information from email? Does it have access to human resources repositories? Can it search legal documents? Does it combine information from multiple systems? Are permissions enforced at the user level or through broader application access?

These questions often receive less attention than model risk, explainability or governance documentation. That is understandable. Most organizations are preparing to assess the AI system. Very few are assessing the permissions that determine what the system can see.

As organizations prepare for the EU AI Act's fundamental rights impact assessment requirements, that distinction may become increasingly important.

Privacy and AI governance teams are investing significant effort into understanding how AI systems influence individuals, decisions and outcomes. But many assessments remain focused on the AI system itself rather than the connector and retrieval layer that determines what the system can access, combine and expose.

In practice, the permissions assigned to the connector often matter more than the model architecture itself.

The most sophisticated AI model in the world cannot retrieve information it cannot reach. Conversely, a relatively ordinary model with broad access across enterprise repositories can create privacy, confidentiality and governance challenges that few assessment processes currently examine in detail.

The assessment gap