How to keep big data projects compliant with EU data protection law? That’s mainly a matter of working out early on what it is that you want to achieve with the project, according to panelists speaking at the IAPP's Data Protection Congress here in Brussels.

James Leaton Gray, director of The Privacy Practice and the former head of the BBC's information policy and compliance policy, said a firm mission could even help companies get past Article 22 of the new General Data Protection Regulation (GDPR), which gives people the right to freedom from profiling.

Unless you define what you are trying to achieve, said Gray, it is "much more difficult to work out what your legal justification is." The regulation will "force people to go through that more rigorously than they have in the past," but this planning is the best thing firms can do as they wait for the EU's data protection authorities to produce their guidelines for implementing the GDPR. "It will give you a stronger position to move forward once you have that advice," he said.

It may seem obvious to say that people should build systems with a clear idea of what they're trying to do, but many companies tend to collect data on the basis that they will find some purpose for it down the line. This may not be the best strategy once the GDPR fully comes into force in May 2018, as the regulation encourages companies to minimize the data they collect and limit it to what is necessary.

Referring to the time when the BBC was developing the personalization features of its iPlayer video-on-demand platform, Gray said his team found it very useful to carefully work out what sort of outcome users would be looking for.

"I think the use case for big data can provide the answer to data minimization … if you need a lot of data to achieve this result," he said. "If you want that level of personalization, you can sign in, and then you can add in that you want the service to look at other sources of data that you have in your control. If you give the data subject the control … they could allow iPlayer to look at their social media needs. That's up to them. That is the answer to data minimization. You get the person defining how much data is required, not you as the data controller."

"That is the answer to data minimization. You get the person defining how much data is required, not you as the data controller." —James Leaton Gray, The Privacy Practice

Anna Zeiter, these days the head of data protection for eBay's EMEA operations, described a major project in which she had been involved in building databases for the German car marketplace Mobile.de. She had some handy pointers about the kinds of issues that can come up when developing analytics features for such platforms.

Asking the right questions was key, Zeiter said, explaining how she had asked the company to go through its use cases — advertising, fraud detection, and prevention — and list the various data sources it would be using. It was when she asked for a data flow chart that she found a spaghetti-like mix of lines that was hard to interpret. "The devil is in the detail," she said.

Establishing data ownership was another challenge. "Email and phone number were quite easy," said Zeiter. "Then the critical part, the vehicle identification number. Does it belong to the car manufacturer or the current owner? Maybe very old cars have 24 owners — do they have co-ownership? And then you want to ask for consent … that was quite tricky."

The key to successful compliance, she said, was getting into conversations with the business analytics team as early in the project as possible. "You have to speak their language and try to understand what is Hadoop+," he said. "It's also very important to speak in their native language. Analytics people speak perfect English but may not be used to speaking about privacy issues in English. You have to understand the use cases in detail, and document the entire project properly."

Gray noted that, in the iPlayer project, the user experience team had been particularly receptive to what the privacy team had to say. They initially thought, "Here comes privacy …," and anticipated a box-ticking exercise, he said, but he then pointed out that it was about making the user's consent experience "entirely clear."

"The subject clicks and understands exactly what is going on and why," he said. "UX agreed [and then] felt emboldened to turn around to marketing and ask, 'What is it you want out of this data?' You have to find your allies."