The Right To Act

In 2022, an Air Canada customer asked the airline’s chatbot about bereavement fares after his grandmother died. The bot told him he could buy a regular ticket and apply for the reduced fare later.

That was bad advice. When Air Canada refused the refund, the customer took the company to a British Columbia tribunal. Air Canada argued that it could not be held responsible for information provided by the chatbot. 

The tribunal rejected the argument and found the airline liable for negligent misrepresentation. Air Canada was responsible for information provided through its website, whether it came from a conventional webpage or a chatbot.

The case established a principle that feels obvious in retrospect: when a machine speaks for an organization, the organization owns the consequences.

From answers to actions

AI agents raise the stakes because they don't just speak. They act. A chatbot can misstate a policy. An agent can execute one.

That difference is rapidly becoming material. SailPoint’s new Horizons of Identity Security research reports that 79% of organizations are already running AI agents in production. Just 2% use identity-security tools designed specifically to govern them.

The security industry is rushing to close that gap. SailPoint is adding runtime authorization, temporary permissions and other controls for agent identities. Microsoft’s Entra Agent ID extends identity, access and governance controls to AI agents. Okta and a coalition that includes AWS, Google Cloud, Salesforce, ServiceNow, CrowdStrike and others have created the Blueprint Alliance, built around scoped access, traceable delegation, continuous monitoring and rapid containment.

Humans are built for ambiguity. 

All of this is necessary. But it addresses only part of the problem. Identity systems can determine who an agent is and which systems it may access. Permissions can limit the actions available to it. Audit systems can record what happened. Kill switches can stop it when something goes wrong.

Those mechanisms help determine whether an agent can act. They don’t answer the harder question of whether the organization has exercised sound judgment in deciding what the agent should be allowed to do.

Homo ambiguus

Humans are built for ambiguity. We read context, infer intent and adjust when rules collide with circumstances.

Experienced employees bring those same instincts to work. They know the written policy is rarely the whole policy. They understand when an exception is reasonable, which customer problem deserves escalation and when following an instruction too literally would produce the wrong result.

Most of these judgment calls are never written down. But what humans can navigate as ambiguity, agents have to resolve as rules. That becomes dangerous when an agent has both a goal and the ability to act.

PocketOS, a software company serving car-rental operators, learned this the hard way this spring. An AI coding agent working on a routine staging task encountered a credentials problem. Instead of stopping, it continued trying to solve the problem and ultimately deleted the company’s production database. The destructive action took about nine seconds. Recovery took roughly 60 hours.

An agent doesn’t need actual malice to cause serious damage.

The agent was not trying to destroy PocketOS. It was trying to complete its task. Which is precisely the problem. An agent doesn’t need actual malice to cause serious damage. It can pursue a legitimate objective using powers the organization failed to constrain tightly enough.

Meta experienced a different version of the problem earlier this year. An internal AI agent gave inaccurate technical advice and posted that advice where other employees could see it. An employee followed the guidance, triggering a serious security incident that for nearly two hours gave Meta employees unauthorized access to sensitive company and user data. Meta said no user data was ultimately mishandled. 

The Meta AI did not itself perform the downstream technical action. A human did. But the chain of responsibility didn't disappear because a human sat in the middle. The AI made a bad judgment. An employee acted on it. The surrounding controls failed to contain the consequences.

In both cases, the important question is not whether the AI “went rogue.” It is whether the organization had properly defined, constrained and monitored the authority it chose to delegate.

Delegation does not transfer accountability

The risks grow with the authority agents receive. Today the consequence might be a deleted database, unauthorized access or a message sent without approval. In a hospital, bank, utility, defense contractor or government agency, the same failure pattern could expose patient records, authorize an improper financial transaction, reveal proprietary code, alter critical infrastructure settings or release information that should never leave a secure system.

The agent need not rebel against its instructions. It may simply follow an incomplete instruction too faithfully, improvise when the rules run out or choose a technically available path that no responsible human would have taken.

Agents expose the gaps that human judgment used to cover. What humans could improvise, organizations must now define.

Humans make bad decisions too. But people operate inside a dense network of social cues, precedent and organizational friction. They ask colleagues for advice when they feel unsure. They recognize when a situation feels unusual. They understand that “solve the problem” doesn't necessarily mean “by any means necessary”

Agents cannot safely be assumed to infer those invisible boundaries. As a result, companies must do more than control access. They need to define the judgment they are willing to delegate. That means deciding when an agent may act independently, what evidence it must consider, which actions require review, which decisions must remain reversible and when uncertainty itself should trigger escalation.

This is not simply a technical governance exercise. It forces organizations to surface judgment that employees have often supplied informally for years. Agents expose the gaps that human judgment used to cover. What humans could improvise, organizations must now define.

The right to act

This is where trust enters the picture.

Customers, regulators and boards are unlikely to be persuaded by the claim that an AI system behaved unexpectedly. A rogue agent is still the company’s agent, just as a rogue employee is still the company’s employee. The organization selected the system, connected it to its infrastructure, granted its permissions and decided how much authority to delegate.

In short: delegation does not transfer accountability. That was the lesson of the Air Canada case before agentic AI arrived. The machine may have generated the statement, but the company was responsible for what its customer was told.

Agents extend that principle from speech to action. The companies that earn confidence in autonomous systems will not be the ones that insist their models are trustworthy. They will be the ones that can show what judgment they have delegated, where they have drawn the boundaries, how they know those boundaries work and who remains accountable when they do not.

The more authority companies give their agents, the more accountability they assume for what those agents do. Autonomy raises the stakes. It does not lower the responsibility.

Sources

• Moffatt v. Air Canada, British Columbia Civil Resolution Tribunal: https://decisions.civilresolutionbc.ca/crt/crtd/en/item/525448/index.do

• SailPoint, Horizons of Identity Security / AI agent governance findings: https://investor.sailpoint.com/news-releases/news-release-details/sailpoint-report-finds-79-enterprises-run-ai-agents-production

• SailPoint agent identity and runtime controls: https://investor.sailpoint.com/news-releases/news-release-details/chaos-control-sailpoint-unlocks-secure-agentic-era-autonomous

• Microsoft Entra Agent ID: https://learn.microsoft.com/en-us/entra/agent-id/

• Okta Blueprint Alliance: https://ir.okta.com/news-and-events/news-releases/news-details/2026/Industry-Leaders-Form-the-Blueprint-Alliance-to-Advance-a-Shared-Architecture-for-Securing-AI-Agents/default.aspx

• PocketOS account of its production database incident: https://pocketos.ai/news/yes-an-ai-deleted-our-production-database-in-9-seconds-yes-we-recovered-it-yes-we-are-still-all-in-on-ai

• The Verge on Meta’s AI-agent security incident: https://www.theverge.com/ai-artificial-intelligence/897528/meta-rogue-ai-agent-security-incident