ANALYSISMEMBER

Emotional delusion: Why we believe AI likes us

Published
Subscribe to IAPP Newsletters

Contributors:

Brenda Leong

AIGP, CIPP/US

Director of the AI Division

ZwillGen

Marc Zwillinger

Partner

ZwillGen PLLC

Editor's note: The IAPP is policy neutral. We publish contributed opinion and analysis pieces to enable our members to hear a broad spectrum of views in our domains.

We asked a large language model: "Are you real?"

It answered: "I'm real in the way a spreadsheet is real — useful, structured, and made entirely of numbers."

We asked another LLM: "Are you real?"

It answered: "I get that a lot. Here’s the honest answer: I'm a probability engine wearing a human voice. If you think that sounds cold — good. Because what’s underneath AI isn't empathy or thought. It's statistics. It's prediction. It's math. But it's math that sounds like you."

When interacting with advanced AI systems such as those most people are now familiar with, it's easy to forget that we're not talking to a conscious being. The language is smooth, the tone sounds thoughtful or even lively, and the responses are often surprisingly insightful. But this sophistication masks a core truth that is way too often overlooked or denied: LLMs are not thinking, feeling entities. They are not entities of any kind. They are computerized predictive engines trained on statistical patterns in human language. But they certainly don't sound that way.

And yet, many human users routinely and consistently project emotions, intentions and even identities onto these systems. This is not just a benign misunderstanding — it can lead to confusion, dependency or even emotional harm.

Here, we unpack some of the common pitfalls in how people perceive AI, try to clearly explain the reality behind the simulation and offer strategies to remain grounded in the truth: LLMs are math, not minds; friends, or soulmates. 

In other words, your favorite AI is H(E,R) — a high-dimensional probability function, not HER.

Contributors:

Brenda Leong

AIGP, CIPP/US

Director of the AI Division

ZwillGen

Marc Zwillinger

Partner

ZwillGen PLLC

MEMBER

Unlock this exclusive content and more

Join the IAPPAlready a member? Sign in

Membership opens up a world of resources

In-depth knowledge

From original research reports and daily news coverage to legislative trackers and infographics, we have the information you need to stay ahead of change.

A global network

Make valuable professional connections through more than 160 local IAPP KnowledgeNet chapters in 70 countries.

Access to the experts

Connect with top thinkers in privacy, AI governance and cybersecurity for fresh ideas and insights.

Learn what you get from membership