Probe finds ChatGPT's model training violated Canada's federal, provincial privacy laws

A joint investigation by the Office of the Privacy Commissioner of Canada and provincial privacy authorities found that training practices for OpenAI's ChatGPT violated Canadian privacy laws, including the Personal Information Protection and Electronic Documents Act. The regulators announced their findings 6 May, alleging OpenAI was in violation of over collection of data, nonconsensual data practices, data subject access request shortcomings and more.

The claims stem from a joint investigation launched in 2023 by the OPC alongside Quebec's data protection authority, the Commission d'accès à l'information du Québec, as well as the privacy commissioners of British Columbia and Alberta.

In a statement, Privacy Commissioner of Canada Philippe Dufresne said OpenAI "launched ChatGPT without having fully addressed known privacy issues," leaving Canadians exposed to "potential risks of harm such as breaches and discrimination on the basis of information about them."

According to the OPC, OpenAI has cooperated with the investigation and Dufresne's office is "encouraged by the privacy-protective measures that OpenAI has implemented or committed to implement."

Enforcement

The joint findings mark arguably the most significant Canadian enforcement action around AI to date. They arrive as other jurisdictions continue to grapple with the privacy and data protection implications of model building and the sources of data used to complete AI training.

The various claims were examined under relevant federal and provincial statutes. The OPC noted findings by each office "varied due to the differences in the laws that they enforce."

Dufresne noted organizations developing AI systems "must respect Canadians' privacy rights" and cannot rely on broad data-scraping practices without demonstrating compliance with privacy obligations. Regulators argued individuals are often unaware their personal information may be collected from online sources and later incorporated into generative AI systems.

The investigation report highlighted concerns that large language models trained on vast amounts of internet data can create significant privacy risks when organizations fail to implement appropriate safeguards or transparency measures. 

In addition to data collection claims, the regulators also determined OpenAI's models contained inaccuracies and did not enable appropriate data deletion and retention polices. 

"New technologies can bring us further and faster. That said, if we allow them to feed off of our private data, we will be paying the price, both collectively and individually," Québec Vice-President and Commissioner of the Oversight Division Naomi Ayotte said at the press conference. "Organizations need to adopt practices that respect privacy. Generative AI impacts all aspects of our lives, and so these technologies need to be created with an increased focus on fundamental rights."

To adapt to the rapid acceleration of technology, regulators recommended modernization of federal and provincial regulations in order to cover governance gaps. 

British Columbia Information and Privacy Commissioner Michael Harvey indicated he sent a letter to Canada's Minister of Citizen Services urging the government to modernize legislation to ensure digital risks are addressed. He said updating regulations "does not mean removing guardrails as this powerful technology races ahead. We need to make sure that AI works for us, not the other way around. This is how we can lay the groundwork for trustworthy innovation, and this report makes clear just how much work we have ahead to make that a reality."

OpenAI's efforts

The regulators released recommendations for OpenAI in place of issuing enforcement penalties due to OpenAI's collaborative efforts to comply with the investigation and implement changes to address concerns. 

Dufresne said OpenAI will be providing the OPC with updates on its data deletion policies outlined within the office's recommendations. "We're going to be monitoring this, but this is an example of a clear gap that we identified in this investigation, and that led to those recommendations," he said.

In addition to updated deletion measures, the company will improve transparency practices and accuracy standards while limiting the amount of sensitive data collected about consumers. 

"These measures will significantly limit the personal information that is used to train new ChatGPT models and will better protect the fundamental right to privacy of Canadians," Dufresne said. "They will also make Canadians more aware of the implications of using ChatGPT. I have concluded that the measures that have been and that will be implemented by OpenAI will address the concerns identified during the investigation with respect to PIPEDA, Canada's federal privacy law for the private sector."

Alberta's Information and Privacy Commissioner Diane McLeod echoed Dufresne's remarks, emphasizing the importance of accountability and proactive privacy governance across the AI sector. She hopes OpenAI "has learned from this investigation and that other technology companies that are developing and deploying AI or other novel technologies also learn from this report that privacy must be a top priority and cannot be an afterthought."