How to design for trust in the age of AI agents

Agentic AI isn’t arriving with a bang. It’s arriving quietly. Systems that once waited for instructions are starting to act autonomously — initiating tasks, making decisions, adapting as they go.

And while this shift is already underway, the way we talk about AI governance still feels rooted in the past. Our ethical, legal and organizational frameworks weren’t designed to account for non-human agents. But agentic systems are changing that equation, raising urgent questions about accountability when things go wrong.

Most discussions of AI governance still revolve around compliance, risk and harm prevention. These things matter. They always will. But they were built for a world where AI was largely static.

As AI agents move from tools to interlocutors, the core becomes a behavioural one — how do we ensure AI agents can be trusted? Some countries are attempting to answer this by offering regulatory frameworks for agentic AI on a large scale.

In this new paradigm, trust becomes a core design choice. One that has to be made deliberately, by global collaboration, not engineered through persuasion or opacity. Trust that is engineered through this emotional mirroring is going to be fragile. In the moment, it may persuade us, but it won’t hold up to real scrutiny.

By contrast, trust that’s built through cognitive resonance, when systems behave in ways that humans can intuitively understand, anticipate and critically assess, is durable and lasting.

Cognitive resonance means that AI boundaries are visible.

The agent’s reasoning, limits and intentions remain legible. This matters because AI agents increasingly shape decisions, beliefs and emotional states. When users cannot tell why an agent responds as it does, or whether it is optimizing for their interests, trust turns into dependency. This is where the conversation needs to move. Preventing harm is essential, but it’s not the same as shaping impact. And in the age of autonomy, responsibility can’t be defined only by what doesn’t go wrong. It also has to account for the futures we’re actively creating.

From a design perspective, that means ensuring that trust is earned rather than engineered. For this, both structural and psychological design choices are required.

Structural design choices relate to how the system is built. Meaningful design means a layered “trust stack” for autonomy, which includes:

Before autonomy scales, there’s an opportunity to slow down and observe. How does an agent behave once it is learning in the wild? What patterns does it start to favour? And just as importantly, how do humans respond? Do they defer more? Override less? Trust faster than they should?

Taking the time to explore how those shifts might play out is not a luxury. It is how organizations avoid sleepwalking into behaviours they never meant to normalize. These moments are not edge cases but early signals of the futures we are actively shaping.

Trust should be seen not as a barrier to adoption, but as a foundation for better outcomes: a way to align AI systems with the behaviours, judgements and norms we want to reinforce.

One way to frame this is as a layered “trust stack” for autonomy.

To trust what an AI system does, we need to understand what it knows, what it is allowed to do, and what it actually did. That means starting with well-governed, traceable data; adding clear, machine-readable rules that reflect values and limits; and ensuring transparent decision records that allow actions to be questioned and learned from.

This kind of structure doesn’t slow autonomy down. It makes it safer to scale.

But governance isn’t only structural. It’s emotional, too. People need to feel they still have agency. They need to understand when AI is acting, why and how to intervene. Systems that are technically compliant but experientially opaque quickly erode trust.

For this, clear design choices are essential, and they include:

Cognitively resonant AI agents treat users as reasoning subjects, not behavioural targets. Trust emerges when systems are understandable, bounded and accountable. In the long run, this is not only an ethical imperative but a prerequisite for sustainable adoption of AI agents at scale.

The real test of responsible autonomy won’t be the risks we avoided, but the futures we deliberately brought into being. So the question worth sitting with now is a simple but difficult one: what kind of world is our AI helping us create, and are we prepared to stand behind it?