How shadow AI and hidden subprocessors are challenging governance and compliance efforts

With increased global emphasis on scaling and maturing governance programs for artificial intelligence, a report from privacy program management vendor DataGrail suggests companies may be inadvertently sliding into noncompliance with their vendor due-diligence requirements across a number of jurisdictions.

The latest Privacy and Trends Report, released at the end of May, reviewed the data protection assessments of 2,400 popular business software providers that advertise their AI capabilities. 

One of the major findings was 63.6% of third-party technology vendors are failing to disclose sub-processing activity conducted by another AI provider in their data protection assessments. The report also found 32.8% of AI systems self-disclosed that they engage in high-risk processing of sensitive data or enable automated decision-making.

DataGrail CEO and co-founder Daniel Barber told the IAPP that businesses have traditionally viewed information about processing activity disclosed in DPAs as "reliable agreements." The lack of disclosure of subprocessing activity in more than half of DataGrail's sample is "highly concerning," he said, due to disclosure requirements in emerging AI regulations, such as the EU AI Act, and risk assessment requirements, like those in the California Consumer Privacy Act. 

"This is primary research, it's data from either our investigation and analysis, or from DataGrail sampling," Barber said. "Our research shows that the subprocessor a business may disclose, and maybe they disclosed one of them, but they may be using two, three or four subprocessors. In our conversations with other companies, this brings up a question for general counsels and privacy leaders on how much can I trust the DPA from a vendor we're working with?"

Insufficient disclosure reflected in DataGrail's report currently presents an unknown with respect to how businesses approach procuring technology vendors in the short term, while weighing the risk they face from a compliance standpoint if they do not have adequate visibility on how a vendor may be using personal data further along their processing chain. Barber said it is "quite likely" vendors relying on third-party solutions of their own may not know the full extent of what additional subprocessing their partners may engage in as well. 

"Obviously vendors and technology providers are looking to be as transparent as they can, but also the reality is technology is moving very quickly and may be moving faster than the legal documents can keep up," Barber said. "Businesses are moving much faster than they were even 12 months ago and companies are now able to ship new features, new functionality faster than they were before. This is very challenging for product security teams, legal teams, to keep up with the speed at which engineering can advance individual products."

Navigating AI's 'one-to-many' relationship

Despite the growing maturity of AI governance programs, the "hundreds of different technologies" that comprise AI make it difficult for organizations to evaluate the full extent of shadow AI whether through unchecked employee use or if an onboarded vendor's solution uses an undisclosed subprocessor, according to Randstad Global Legal Director and Global Responsible AI Officer Martin Woodward. 

Shadow AI, according to Woodward, is the next logical evolution of "shadow IT," which he defines as the use of any unapproved tools within an organization. Outside of "highly classified environments" or a highly regulated industry, such as financial services, "most companies have not closed up every potential opportunity for shadow IT to enter."

"Most enterprises operate an open IT ecosystem, to some degree, so that has traditionally always invited, non-officially procured tools that are entered into the organization's ecosystem," he added. "With shadow AI, there are a few additional risks that are being introduced that make this challenge even more persistent. The key one being in most IT relationships, it's essentially a one-to-one relationship, whereas with shadow AI it can be a one-to-many relationship."

Auditing the full scope of how a company's IT stack interacts with any number of AI applications is much more complex than reviewing cloud relationships, which can be accomplished through monitoring web traffic. Woodward indicated the lack of disclosure of AI subprocessing activity DataGrail notes in its report may create situations where both providers and deployers of AI systems are engaging in high-risk processing without necessarily being aware.

"In the context of AI discovery, you can't simply send out a crawler into people's services and it'll report back if there's AI or not, because AI is this container term for potentially hundreds of different systems and it's challenging to detect," Woodward said. "Under the EU AI Act, for example, you have to know whether you're a provider or a deployer because each role comes with distinct requirements in the context of transparency obligations. Hopefully, if you're a provider you're aware of the obligations, but especially for that deployer role, you can much more easily be in breach of the legislation that you didn't know applied to your activities, because you weren't aware of the AI use-cases you had inadvertently onboarded."

Subprocessors as controllers

Bird & Bird Partner Vincent Rezzouk-Hammachi, CIPP/US, said under the EU General Data Protection Regulation, the processing activities a third-party technology vendor may engage in would likely qualify it as a data controller under the law. 

The failure to disclose a complete list of subprocessors has the potential to expose a vendor to GDPR enforcement before the main client faces penalties so long as they can demonstrate they undertook proper due diligence when they onboarded the vendor originally. Additionally, Rezzouk-Hammachi said an inadequate level of disclosure could make it more difficult for companies to fully respect consumer privacy rights, such deletion and right of access requests.

Although undisclosed AI subprocessing is a risk organizations face when they consider partnering with new technology vendors, Rezzouk-Hammachi is primarily concerned with raising awareness about the risks posed by employees using shadow AI when advising clients. 

"At the end of the day, if you don't capture the existence of a subprocessor for AI, it's because oftentimes you have not conducted your assessment properly," Rezzouk-Hammachi said in an interview. "Organizations are struggling to allocate AI governance ownership to one business unit because it touches a lot of areas and it becomes very political. Therefore, it's very difficult to develop and deploy a robust AI governance framework that you can review processes and double check everything before engaging with a vendor and signing an agreement, but it's getting better for companies."

Access controls vs. controlled experimentation 

JB-Nomo Independent Advisor John Bowman, AIGP, CIPP/E, CIPM, FIP, said depending on the level of AI literacy, access controls and training related to AI an organization engages in, shadow AI use by employees can be fairly straightforward to mitigate. Other organizations with mature AI governance have also found success empowering their employees to use any number of publicly available AI tools, as long as their use is contained within a secure environment, which was the case when Bowman previously worked at IBM as a senior program manager in its Risk, Compliance and Integrity unit.

"Access controls, training and improving AI literacy all could fall within business conduct rules or acceptable use policies, so potentially employees can be bound contractually in terms saying they can't use unauthorized tools," Bowman said. "What organizations can do is guide employees to use the tools within a controlled environment with the appropriate safeguards in place."

One strategy for identifying the business use cases where AI tools may realize the greatest efficiencies is allowing employees to experiment with different solutions and report up the chain of command what processes are being improved through using AI. 

However, Alethesis AI founder Elena Maran, AIGP, said organizations that are more likely to encourage such employee experimentation will be smaller and more nimble companies, instead of more established organizations within specific industries that have "established procedures and long vendor due diligence processes."

Screening tools becoming must-have for due-diligence

To account for shadow AI's potential impacts, Maran recommended screening tools that can detect AI that is unauthorized by the company. By accomplishing this, companies can best protect themselves from a due-diligence perspective should an undisclosed subprocessor violate the law. 

"The only way to sort all of this out is to conduct due diligence on a toll and ask the vendor to explicitly disclose whether there is some subprocessing and what kind of technology is used in this instance," Maran said. "In the company itself, it can deploy some tooling that can recognize processes that are not mainstream in the tool but can be analyzed directly by the company and be used to seek clarity from the vendor."

DataGrail's Barber echoed Maran's call for organizations seeking to proactively respond to any additional compliance risk they may be exposed to through their vendor's subprocessing activity to deploy additional tooling. He said it is a matter of "going back to basics" for organizations.

"You need a system inventory that actually captures both the processes: The companies you directly contract with, but also how those systems leverage subprocessors," he said. "The historical method of scanning the DPA, as an example, that is not sufficient in 2026 if we know 63.6% of DPAs are not actually accurate. We need another mechanism to actually populate your inventory of systems in a better way, which should be completed from a business process standpoint, and that is very important."