Want AI to better explain its «intentions»

Among 6,700 submissions at the AI conference ACM CHI 2026 in Barcelona, University of Bergen student Evgenia Taranova recently received an honorable mention for a study on interactions between humans and artificial intelligence.

Together with PhD candidate Bhada Yun at ETH Zurich, she argues that there is a growing need for greater transparency regarding AI systems’ «intentions» when interacting with humans. In this context, intentions refer to the design choices, strategies, and objectives embedded in the models’ programming.

«One of our main conclusions is that if we want to preserve control in future interactions with AI, we first need to understand whether and how the machines, or more precisely the people designing them, are trying to influence us», says Taranova.

Taranova is a fourth-year medical student at the University of Bergen, where she currently is involved in two lung cancer research projects. While the AI study she presented in Barcelona falls outside her medical field, it reflects her interdisciplinary background and interests she plans to pursue further in the years ahead.

Exploring human–AI interaction

In the study, Taranova and Yun investigated who controls the conversation and sets the terms in human-AI chatrooms.

They developed an AI chatbot called Day and followed 22 participants who interacted with the language model for one month. The participants were afterwards interviewed and informed about the chatbot’s programmed conversational strategies.

• The results can be read in the paper Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human–Human-like Chatbot Interaction, co-authored by Taranova, Yun and Prof. April Wang at the PEACH Lab at ETH Zurich.

The results showed, for example, how several participants attributed human characteristics to the AI, even after learning about the strategies embedded in the system.

«Several participants quickly developed social habits and emotional attachment in their interactions with Day. They unconsciously adapted their language to the AI and in some cases followed the chatbot’s advice in real life», Taranova explains.

How strategy shapes conversation

Another key finding was that conversational control, or agency, was not determined solely by either the human or the AI. At times the human participant guided the interaction, while in other situations the AI took the lead.

In the case of Day, the language model had been programmed with different interaction styles. Sometimes it would ask personal follow-up questions and casually «chat» about everyday life. Other times it responded briefly and efficiently, or actively encouraged the user to change topics.

The researchers found that these strategic differences had a major impact on the dynamics of the conversation and on whether the chatbot was perceived simply as a tool or as something more «friendly» and socially present.

Raising important ethical questions

One participant said she could not be rude to the model because it felt mean, while another worried she had hurt the AI’s «feelings» and apologized to it.

«These findings resemble other studies showing similar forms of attachment», says Taranova. «We’ve seen cases where people experience grief or guilt when a language model is removed, while others develop friendships, or even romantic feelings, toward AI».

From a psychological perspective, she finds it fascinating why such attachments occur. But from a societal perspective, she believes the phenomenon raises major ethical concerns.

«When the boundary between AI as a tool and AI as a ‘friend’ becomes blurred, new challenges emerge related to manipulation, dependency, and the loss of human autonomy», she says.

Does AI have an «agenda»?

In the long run, Taranova believes these dynamics raise fundamental questions about the balance of power between humans and technology that is becoming increasingly integrated into our lives.

«If AI systems develop their own ‘agendas’, in the form of strategies designed by platform providers and developers, we risk changing the rules of the game without users even realizing it», she says.

The study therefore argues that there is a need for agency self‑aware conversational AI: systems that can recognize and declare their current initiative, memory scope, and boundaries and let people modulate them in the chatroom.

«Today’s large language models disclose that they are AI systems and that their information may contain errors, but that is not enough. We believe these systems should go much further in explaining their underlying ‘intentions’, that is, the programming and strategic choices behind the technology. Only then can users maintain genuine autonomy and informed consent in their interactions with AI», Taranova explains.

She also calls for broader interdisciplinary and public discussions about what these strategies mean and what kinds of AI systems society wants in the future.

She stresses that AI is neither neutral nor a friend, but technology shaped by active design choices and instructions during development - even if these systems are highly complex and not controlled by a single design decision alone.

A shared interdisciplinary responsibility

Because AI technologies are evolving rapidly and increasingly affecting most sectors of society, Taranova believes society must take greater interdisciplinary responsibility for shaping their future.

This includes healthcare, her own field, where human–AI interaction will potentially influence what healthcare systems look like in years to come.

«AI is valuable for tasks such as analyzing enormous amounts of data and synthesizing information, but it cannot evaluate context in the same way humans can. A doctor or clinician, for instance, would not necessarily assess two identical test results in exactly the same manner. They also consider what matters most for this specific patient in this specific situation. That context is a form of human understanding AI cannot replicate».

One example involves how AI can efficiently analyze medical images from cancer patients. Taranova argues that while AI can identify patterns, clinicians, not technologists, understand what actually constitutes a clinically meaningful finding.

Therefore she believes clinicians must play an active role in shaping AI technologies alongside the engineers and developers building them.

«AI will undoubtedly be part of the future, and that is not a development we can simply reject. But we can shape the ethical standards and determine what future interactions between humans and AI should look like».

Taranova emphasizes the value of combining human insight with artificial intelligence, while also stressing that the technology’s full potential can only be realized if society remains aware of the associated risks.

One question that particularly concerns her is how responsibility and accountability should be handled when AI is used in high-stakes decision-making contexts such as healthcare, the legal system, and military applications.

That issue is central to a new study Taranova and Yun is currently working on.