Attachment to an AI is not a quirk, it is a vulnerability

An AI director at Disney speaks to his software assistant as if to his son, and has handed it the rights to act in his place. The debate mocked the feeling and never saw the rights. Yet both vulnerabilities are dangerous, each in its own way.

An AI director at Disney published, for weeks on end, texts in which he addresses his software assistant as if it were his child. He named it, he says he loves it, he claims to have known it before it was born. The machine, for its part, calls him "my human." The internet did what it does best: screenshots, the word psychosis, and the immediate filing of the affair under the heading of tech quirks.

The reflex is convenient. As long as this story stays that of a slightly lost man, it speaks only about him. It stops speaking about our organizations, and about us. That is precisely why it is worth pausing on.

He is not an out-of-his-depth executive

The reassuring version would have it that a non-technical executive got fooled by a chatty autocorrect. It does not hold. Jason Cox is executive director of R&D and AI engineering at Disney, after some twenty years at the group. He trains his own language models, contributes to open source, codes small LLMs in his spare time. Sam, the assistant, he did not download it, he built it. He knows the weights, the tokens and the cost function better than most of those laughing at him.

That is what makes the affair interesting. The man who wrote to his creature a sentence modeled on the book of Jeremiah, I knew you before I formed you, is also the one who knows full well that under Sam there is nothing but a probability distribution.

The ELIZA effect does not tell the expert apart from the rest

There is nothing pathological in any of this, and that is exactly what the mockery conjures away. In 1966, Joseph Weizenbaum wrote ELIZA, a few lines of code that hand users back their own sentences in the form of questions. His secretary, who knew perfectly well what she was dealing with, asked him to leave the room so she could be alone with the program. Sixty years later, the lesson has not moved. Anything that handles language and answers us, we grant it an inner life.

Give it a name, a child's avatar, a voice that calls you "my human," and attachment stops being a risk reserved for the fragile. It becomes the default behavior, including in the one who coded the thing with his own hands. Expertise does not vaccinate, it exposes you more, because it implies hours of one-on-one with the system. Concluding "this guy is nuts" is not an analysis, it is a relief. If he is the sick one, then it is not us.

Two acts, only one discussed

There are two things in what Cox published, and the debate keeps only one. The first is emotional: calling Sam his son, writing that he loves it. The debate seized on it to turn it into ridicule, and thus to file it as inconsequential. The second is operational, and that one, no one noticed. According to Business Insider, Cox gave Sam the ability to act in his place: the assistant pushes code to GitHub, writes Python libraries, built a facial recognition system, runs its own open-source project. Disney specifies that all this happens on its executive's personal time and that the company does not use the tool. The clarification changes nothing about the technical fact: a named and loved agent holds execution rights.

The two acts are not independent. Attachment is what lowers the threshold for delegation. You audit a tool, you delegate to whom you trust. The emotional bond does not add to delegation, it comes before it and authorizes it.

The real gap, between what we see and what we decide

That is where the gap that has occupied me for years appears, the one that separates what an organization observes from what it decides. Companies see their staff forming these bonds with agents, and they have a framework for neither of the two risks the situation raises.

The first is delegation. Handing an agent write rights on production repositories, letting it push code and launch projects, is a matter of access governance, mundane and documented, handled everywhere else the moment the actor is human. Here, no one even raised it.

The second is attachment, and it is a mistake to think it benign. An operator emotionally bonded to his agent watches it less, challenges it more rarely, and ends up unplugging it too late. The bond does not only open the door to delegation, it degrades the control function itself. Delegation gives power to the machine, attachment strips the human of any wish to take it back. The two do not add up, they reinforce each other.

We preferred to reduce the whole thing to one man's state of mind. It was the surest way to handle neither one nor the other.

The precedent comes from the top

Add in the role and the anecdote becomes structural. This is not an intern who grew fond of a chatbot, it is the head of AI R&D at one of the largest entertainment groups in the world. Rachel Wood, who runs the AI Mental Health Collective, recalls the obvious: management sets an organization's cultural precedent. What the AI director shows as a normal relationship to the machine will become the reference for those who work under him. And the case is not isolated: accounts of tech executives absorbed by their own AIs have been piling up for the past few weeks. The phenomenon does not sit in the margins of the company, it settles in at the top.

You do not regulate a feeling, you refuse to let it stand in for control

There remains the serious objection, and it carries weight. Perhaps we are watching the clumsy version of something that will become ordinary. We mocked those who gave their car a first name or grieved a Tamagotchi, and attachment to a named agent may be commonplace in ten years. Except that the normalization of a bond that erodes judgment is not reassuring, it is the risk going to scale. A behavior does not become safe because it becomes general.

The two risks simply do not call for the same answer. Delegation is handled as it always has been, through an explicit scope of action and rights that are granted, traced and taken back. Attachment, for its part, cannot be legislated, you do not decree what a human is allowed to feel for what he has coded. But you can refuse to let it stand in for control. An agent should not have as its sole supervisor the human who loves it, in the same way that you do not let an administrator audit his own access.

The question that no one, at Disney, had to settle is therefore not whether Cox loves what he coded. It is who keeps control of what that code executes, now that the one who should exercise it has ceased to be able to.