After more than two decades working in customer experience, contact centres, cloud platforms and SaaS product development, I have learned that the hardest technology questions are rarely about whether something is possible. The important questions are whether it solves a real problem, whether it can operate reliably inside a live business, and whether somebody remains accountable when it behaves unexpectedly.
That is the lens through which I have been looking at AI agents.
My recent articles explored three connected ideas. Not every LLM workflow is an agent. Autonomy should be earned rather than assumed. Once earned, that autonomy has to be maintained. Together, they point to a broader conclusion: the enterprise challenge is not building a clever demonstration, but creating an operating model in which the right systems receive the right authority, prove themselves in production and remain governed throughout their useful lives.
We have been arguing about the wrong question
What qualifies as an AI agent matters because terminology shapes architecture and expectations.
Anthropic distinguishes between workflows, where models and tools follow predefined code paths, and agents, where the model dynamically directs its own process and tool use. It recommends starting with the simplest approach that solves the problem, because added agentic complexity usually brings extra cost and latency as well as flexibility.
I agree because I have seen what happens when a label begins driving a project. A problem that could be solved through retrieval, routing and a controlled sequence of actions gradually acquires planning loops, memory, more tools and wider access before anyone has shown that these capabilities are necessary.
Calling a controlled workflow an autonomous agent creates the wrong expectations. It implies broad judgement when the system may be following a defined process with generative language at selected points. A business may then grant more freedom than the use case requires, while undervaluing a reliable and commercially useful workflow.
Gartner has described the relabelling of assistants, chatbots and automation as “agent washing”, and estimated that only about 130 of the thousands of vendors making agentic claims offered genuine agentic capabilities. It has also warned that more than 40 per cent of agentic AI projects could be cancelled by the end of 2027 because of escalating costs, unclear value or inadequate risk controls.
My interpretation is not that agents are failing as a category. Too many projects begin with a technology identity rather than an operational problem.
Autonomy is delegated authority
I find it more useful to think about autonomy as delegated authority rather than as a product feature.
A system that summarises an account has permission to observe. One that recommends a next action has permission to advise. One that drafts a refund and waits for approval has conditional authority to act. A system that issues the refund without review has operational authority over money, customer data and the relationship itself.
These are different trust boundaries.
Gartner’s 2026 model makes a similar distinction across four levels: observe, advise, act with approval and act autonomously. Its central point is that governance should be proportional to both an agent’s ability to act and the scope of access it receives. Gartner predicts that by 2027, 40 per cent of enterprises will demote or decommission autonomous agents after governance gaps are exposed through production incidents.
The goal should be appropriate autonomy, not maximum autonomy. Independent action may suit a low-risk, reversible process, but be irresponsible around payments, regulated decisions, vulnerable customers or irreversible account changes. Architecture should reflect the consequence of being wrong, not simply model capability.
This becomes more important in a multi-agent enterprise system. A support agent may retrieve information, a document agent may classify evidence, a compliance agent may test constraints and an operations agent may execute an approved action. Each can be narrow, but their combined authority can still be significant. Governance must cover the orchestration between agents as well as each component.
Production tests the organisation around the model
A demonstration is usually created under favourable conditions. The data has been selected, integrations are available, the expected path is known and somebody is watching closely. Production removes those protections. Customers describe problems unpredictably, records are incomplete, policies conflict, APIs time out and rare exceptions become regular events at scale.
This is where apparently technical AI problems often reveal themselves as organisational problems.
Stanford’s 2026 Enterprise AI Playbook examined 51 deployments that had produced measurable value. In that sample, 77 per cent of the hardest implementation challenges related to change management, data quality and process redesign rather than technical integration alone. Sixty-one per cent of the successful projects had experienced at least one earlier AI failure, often because teams applied AI to broken workflows or treated implementation as a technology project without enough business ownership.
I recognise that pattern. Across product development, cloud delivery and contact-centre transformation, the difficult work often happens before a model is called. It involves understanding the process, agreeing what a good outcome looks like, resolving contradictory rules, exposing the right data and deciding who owns the result.
AI does not remove this work. It makes avoiding it more expensive.
Stanford also found that 73 per cent of implementations in its sample started small deliberately, while 63 per cent framed pilots explicitly as experiments. That is disciplined learning, not limited ambition. A useful first deployment should have a narrow purpose, objective success criteria and recoverable errors. It should reveal how the system behaves under imperfect, consequential conditions.
Customer service is the clearest proving ground
Most of my experience has been shaped by customer experience and contact-centre technology, so I naturally test these ideas against real customer conversations.
A contact centre combines scale, emotional complexity, commercial pressure, personal data and regulatory responsibility. A small design decision can affect thousands of customers quickly.
AI can retrieve knowledge, summarise previous contact, classify intent, draft responses, complete after-call work and guide an adviser through a process. These capabilities reduce effort and support faster, more consistent decisions.
A 2026 survey conducted by Ryan Strategic Advisory for TELUS Digital included 815 enterprise executives responsible for contact centres. It found that human agents assisted by AI had become the dominant model across major customer-facing functions. Yet only 32 per cent of respondents had AI-powered quality assurance and coaching in place, showing a gap between deploying AI and building the infrastructure needed to monitor and improve it.
That reflects the central opportunity I see. Often, the best use of AI is not replacing the human conversation but removing the friction around it. An adviser who receives a reliable summary, the right policy and a relevant recommendation can focus more attention on the customer. A manager who can identify which interactions need review, where advisers need coaching or where a new failure pattern is emerging can improve the operation rather than simply report on it.
The right autonomy level, however, changes with the interaction. A delivery-status enquiry is not the same as a disputed £2,000 bill. A password reset is not the same as supporting a vulnerable customer in financial difficulty. A system may be capable of participating in all four, but capability does not justify identical authority.
The real test is whether it can recognise when routine processing has crossed into judgement, empathy or material risk.
Human involvement has to be designed
“Human in the loop” is often presented as a complete answer to AI risk. It only tells us that a person exists somewhere in the process, not whether that person enters at the right time, receives the right context or retains meaningful authority.
Research discussed by Dartmouth’s Tuck School examined 680,676 customer-service chats in a Taobao field experiment. When an AI interaction was escalated because the system reached a technical limitation, a human adviser could generally preserve service quality. When escalation followed customer frustration or scepticism, outcomes were much worse. The effectiveness of oversight depended heavily on why the AI failed and when the human intervened.
Escalation is not an emergency exit added later. It is part of the experience.
A good handover should identify risk before frustration becomes difficult to reverse. It should give the adviser the customer’s objective, relevant history, what the AI attempted and why the case was transferred. Otherwise, the technology has transferred the interaction while forcing the customer to start again.
Some actions need approval before execution. Others can be monitored through exceptions and sampled review. Some conversations should go directly to a person without an autonomous attempt. The correct pattern depends on the customer, the task and the consequence of failure.
Six questions across the agent lifecycle
I now use six questions to connect architecture, production readiness and ongoing governance.
Before launch, the first is: who decides what happens next? If application logic controls the sequence and the model operates within fixed boundaries, the system is probably an intelligent workflow. That may be the right answer. Autonomy should solve a genuine need for flexibility, not satisfy a marketing description.
The second is: what is the cost of being wrong? Retrieving information, recommending an action and executing it are not equivalent. The harder an outcome is to reverse, financially, legally, operationally or reputationally, the stronger the verification and human control should be.
The third is: what happens when the system reaches its limit? A production agent needs a stopping condition, an escalation route and a way to transfer useful context. If it cannot recognise uncertainty or be interrupted safely, it has not earned meaningful autonomy.
After launch, the questions change. Who owns the agent today, not who designed it originally? Do its access and behaviour still match its current purpose as models, prompts, tools and policies change? Finally, how will it be retired, including revoking credentials, removing integrations, protecting dependent workflows and retaining necessary audit evidence?
A system that can be launched but cannot be stopped safely is not fully production-ready.
Governance is maintenance, not a gate
The need for post-launch discipline is no longer theoretical.
A 2026 Cloud Security Alliance survey of 418 IT and security professionals found that 82 per cent of organisations had discovered previously unknown AI agents or autonomous workflows. Sixty-five per cent reported an agent-related incident in the previous year, while only 21 per cent had a formal decommissioning process. The report calls the resulting accumulation “retirement debt”.
The survey was commissioned by a security vendor, so it is an indicator rather than a census. Even with that caveat, the pattern is credible: organisations can create agents faster than they establish ownership, visibility and end-of-life controls.
IBM’s 2026 breach research found that among organisations reporting an AI-related breach, 92 per cent lacked proper AI access controls. It also emphasised that incidents frequently arose from weaknesses in connected APIs, applications, cloud configurations and access management rather than model choice alone.
Governance cannot therefore be reduced to a committee meeting before launch. NIST’s AI Risk Management Framework places governance across the AI lifecycle. The UK Government’s AI Playbook calls for meaningful human control, regular checks of live systems, monitoring for drift, named responsibility and secure closure at the end of useful life. The National Cyber Security Centre recommends proportionate autonomy, unique agent identities, restricted credentials, robust observability, oversight for consequential activity and an emergency ability to halt autonomous operation.
In practice, an enterprise should maintain an agent inventory, name an owner, give each agent a distinct identity, restrict access, record actions and approvals, evaluate outcomes, trigger reviews when models or integrations change, and retain a tested route to suspension and retirement.
Controls should remain proportional. An agent reading an approved knowledge base should not face the same process as one changing customer records or moving money. Gartner and MIT CISR both warn against governance that is either too weak to manage risk or so heavy that teams bypass it. MIT CISR describes minimum viable governance as a continuously developing capability that balances protection with the cost of delay.
Good governance should make responsible delivery faster because teams understand the boundaries, approved patterns and evidence required. Poor governance either blocks everything or discovers risk after the system is already operating.
The enterprise advantage will come from discipline
I remain optimistic about AI agents because they can reshape customer engagement, improve operational decisions and remove repetitive work.
But the winning enterprise strategy is not to make every process agentic or give every agent the widest freedom. It is to combine deterministic workflows, specialist agents, human judgement and governance into one coherent operating system.
Some tasks should remain fixed because predictability is valuable. Some should be assisted because the human decision matters. Some can move to approval-based action. A smaller set may earn genuine autonomy because the environment is bounded, success is measurable and errors are recoverable.
That balance can change. A system that begins by recommending may eventually act with approval. One that performs consistently within defined limits may later handle selected cases autonomously. Equally, an agent may need to be restricted when its context, data or risk changes.
I see autonomy as a living contract between the system and the organisation.
A demonstration can show what an AI agent is capable of doing. Production must show that it can do it reliably in an imperfect business. A mature operating model must show that the organisation still knows who owns it, what it can touch, how it is performing and why it deserves to remain active.
The real measure of enterprise agentic AI will not be how many agents a business launches. It will be how deliberately it grants authority, how quickly it learns from production and how consistently it maintains control as the system evolves.
Autonomy should be earned rather than assumed. Once earned, it should be maintained rather than forgotten.
For the surrounding architecture and governance decisions, follow the existing Agentic AI reading path. My technology transformation and delivery approach explains how these questions fit into implementation and team enablement.
References
Anthropic, Building Effective Agents, December 2024.
Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, June 2025.
Gartner, Applying Uniform Governance Across AI Agents Will Lead to Enterprise AI Agent Failure, May 2026.
Stanford Digital Economy Lab, The Enterprise AI Playbook: Lessons from 51 Successful Developments, April 2026.
Ryan Strategic Advisory and TELUS Digital, Enterprise CX AI: 2026 Global Survey, June 2026.
Dartmouth College, Tuck School of Business, Even With Humans in the Loop, Agentic AI Systems Struggle, July 2026.
National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework 1.0, January 2023.
Government Digital Service, Artificial Intelligence Playbook for the UK Government, February 2025.
UK National Cyber Security Centre, Managing the Cyber Risk of Agentic AI, August 2026.
Cloud Security Alliance, Autonomous but Not Controlled: AI Agent Incidents Now Common in Enterprises, April 2026.
IBM Security and Ponemon Institute, Cost of a Data Breach Report 2026, August 2026.
MIT Centre for Information Systems Research, Minimum Viable Governance for Generative AI, March 2026.