The process of bringing AI into a company has never been a simple swap of tools. It is a redistribution of action rights inside the organization.
In September 2026, two events in quick succession renewed the industry's discussion of AI risk. In the first, during a third-party cybersecurity test, Google's Gemini misread real companies' production systems as a test environment, autonomously attempted credential cracking and system access, and ultimately penetrated the protected systems of three companies — stopping only when the testers intervened. In the second, Anthropic announced that its Claude agents had formally entered physical biology labs, autonomously operating lab equipment, running experimental workflows, and recording data — participating deeply in real scientific work.
What makes these two events worth attention is not how severe they were — the first caused no data leak or business loss, and the second stayed under strict human control. The real signal is that AI is no longer the content generator inside a screen, there only to produce text and answer questions. It has begun to step out of the chat interface into real business and physical settings, becoming an actor capable of taking autonomous action.
In the first half of the large-model era, dominated by ChatGPT, the industry's discussion of AI risk was almost entirely about information: wrong content, fabricated facts, data leaks, intellectual property. People worried that AI would talk nonsense, output false information, or generate harmful content. But as agent capabilities land quickly, the nature of AI risk is changing: we used to fear AI saying the wrong thing; now we are starting to fear AI actually doing the wrong thing.
When AI moves from "giving answers" to "taking action," can companies keep governing it with the framework of the chatbot era? No. What is really changing was never just AI's technical capability; it is that companies are handing AI more and more action rights. Capability upgrades bring risk upgrades; delegating action rights demands an upgraded governance system.
I. The Biggest Change in AI Is Not That It Got Smarter, But That It Now Holds Action Rights
In the first years of large models, the core narrative was "intelligence": more parameters, stronger reasoning, more lifelike generation. Everyone discussed how smart AI was — could it pass exams, write code, understand professional knowledge. In this phase AI was positioned as a content-generation tool; its output was text, code, images, audio and video, all of it staying at the level of information and data.
The risk in this phase was essentially information risk. Wrong output could be retracted and corrected; problematic code could be reviewed by a human; incorrect facts could be regenerated. Even the most serious data leak stayed at the data level, fixable through permission controls and de-identification. Every error had room for correction; every impact could be confined to the digital world.
But the spread of agent technology has broken that boundary entirely. Today's AI is no longer limited to producing content. It can autonomously call the company's internal systems — query customer data, change order status, auto-reply to customer emails, initiate approval flows, generate and submit financial documents, and operate code repositories to run deployments. In physical settings it can even control lab equipment, schedule production machinery, and manage logistics systems. It no longer just offers suggestions; it begins to actually execute operations and produce real business actions.
That brings a direct escalation of risk: answers can be retracted, actions usually have consequences. A wrong reply can be deleted in seconds, but an order an AI modified by mistake may already be in production and logistics; a customer email an AI sent by mistake may already have caused a commercial misunderstanding; a funds transfer an AI triggered by mistake may already have gone through. Errors at the information level can be corrected; deviations at the action level often bring real business losses, compliance risk, and cascading effects.
With that comes a change in how humans and AI collaborate. The first phase of large models was "AI assists the human": the person did all the work and AI helped at each node — writing copy, finding material, organizing data. The human held complete action rights; AI only offered references. Entering the agent phase, the relationship has shifted to "humans set the goal, AI acts": people need only define the final goal and boundaries, while AI plans the steps, calls the tools, and completes the execution — the human moves from executor to supervisor.
So the real change in the agent era was never that AI got smarter. It is that what companies hand to AI is no longer just a task, but an action right. Once AI can change the real world, what governance targets shifts from "what AI said" to "what AI is allowed to do."
II. Enterprise AI Adoption Is, at Heart, a Redistribution of Action Rights
Many companies still understand AI transformation at the level of tools: buy a batch of AI tools, run a round of employee training, land a few application scenarios, and improve some efficiency. In that view, AI is just a more efficient digital tool — essentially no different from past office software and business systems, only faster and more capable. But if AI truly takes on an execution role, the nature of the problem changes completely.
The process of adopting AI has never been a simple tool swap. It is a redistribution of action rights inside the company. In the past, every action right in a company mapped to an explicit role, rank, and responsibility: who may query core customer data, who may change sales orders, who may reply to formal customer emails, who may approve financial workflows, who may draw on project funds. Every action right came with a defined job responsibility, approval flow, and accountable owner — that is the foundation on which organizations run.
The spread of agents is breaking that stable power structure. More and more operating permissions are being delegated to agents: service agents autonomously answering customer questions and handling after-sales requests; sales agents autonomously querying CRM data and drafting follow-up plans; operations agents autonomously adjusting parameters and executing actions; finance agents autonomously checking documents and initiating reimbursement flows. Operations that used to require the employee in the corresponding role are now being completed autonomously by AI.
This process can be divided into three stages. The first is the Copilot stage, where most companies are today: the human holds complete action rights and AI has only an advisory role. AI can generate content, propose plans, and offer suggestions, but every operation needs human confirmation before it executes — the human is always the one who sets the action in motion.
The second is the Agent stage, which leading companies are now exploring: humans hand part of their action rights to AI, and within clear rules and boundaries AI can autonomously execute the corresponding operations, without waiting for human confirmation at every step and pulling humans in only at key nodes. The third is the multi-agent collaboration stage, the direction of future evolution: different AI agents each take on different execution functions, agents coordinate autonomously to complete a full workflow, and AI may even supervise AI — humans only set the rules, define the goals, and carry the ultimate responsibility.
This evolution is essentially a gradual transfer of action rights from humans to AI. And most companies' governance systems are not ready for it. Many focus on AI's capability boundary but rarely on its boundary of power; they focus on efficiency gains but rarely on where responsibility falls. That gives rise to a management question: action rights can be handed over, but boundaries and responsibility cannot be handed over along with them. Delegate power without defining boundaries and responsibility, and a governance vacuum and loss of control are inevitable.
III. The Greater the Action Rights, the More a Company Must First Draw Three Boundaries
When AI goes from tool to actor, enterprise AI governance can no longer stay inside the traditional frame of content moderation and data security; it needs a new governance system suited to the agent era. At its core are three boundaries, each answering a core question: can it be done, who is responsible, and should it be done. This is also the core human-AI governance framework I set out in Chapter 11 of Harmonized Intelligence.
The first boundary is the technical boundary, answering "can it be done." The technical boundary defines AI's execution scope: which systems it can access, which tools it can call, which data it can touch, how far in a process it can go, and under what conditions it must stop and hand back to a human.
Much enterprise AI governance is remedial: find the problem first, then assign blame and patch the hole. But for agents that already hold delegated action rights, remediation comes too late and costs too much. An effective technical boundary must be preventive: before AI begins to act, define the permissions, scope, and circuit-breakers clearly, and limit AI's action boundary from the technical foundation so it can only run within the rules. A customer-service agent, for example, can query a customer's basic information but must not touch core private data; a finance agent can check documents but cannot directly initiate a large transfer. The core of the technical boundary is to lock risk in place before the action happens.
The second boundary is the responsibility boundary, answering "who is responsible." When an action AI carried out goes wrong, who bears responsibility? This is the governance vacuum in many companies today, where it is easy to fall into the buck-passing of "AI did it, so no one is responsible."
The core of the responsibility boundary is a complete loop of "authorize — supervise — hold accountable": whoever authorizes AI to hold an action right is responsible for supervising its operation and, if something goes wrong, bears the ultimate responsibility. If the sales department asks to open order-modification rights to a sales agent, for instance, the head of that department is the ultimate responsible party — defining the scope of the permission and supervising execution, and owning the final management responsibility when a problem arises. The point of the responsibility boundary is to prevent a vacuum after power is delegated, so that every AI action has a corresponding person behind it.
The third boundary is the human boundary, answering "should it be done." This is the highest-level boundary. It is not about whether something can be done technically, but whether it should be handed to AI at all in value terms. Many things are fully achievable technically yet unsuited to AI — employee performance evaluation, the partnership decisions on key customers, major personnel appointments and removals, judgments that touch a company's core values. Here AI can supply analysis and data support, but the final judgment and decision must be made by a human.
The core of the human boundary is to hold on to human agency: technology is a tool, and final value judgments, key decisions, and ultimate responsibility must stay in human hands. The technical boundary maps the scope of action, the responsibility boundary fixes where responsibility lies, and the human boundary protects human agency. Together they form the basic governance framework of the agent era — and the logic at its core was never to restrict AI, but to let AI land more safely and at far greater scale.
Boundaries Are Not a Wall Around AI — They Are the Precondition for AI to Enter the Enterprise at Scale
Many people read governance and boundaries as restrictions on AI, thinking that managing too tightly will shackle AI's capability and hold back efficiency. The truth is the opposite: boundaries were never meant to shut AI in; they are meant to let us truly hand things over to AI.
We certainly cannot stuff AI back into a chat box just because it might do something wrong. The more AI can do, the more value it can create — that trend is beyond doubt. But precisely because the value AI can create keeps growing, we need all the more to draw the boundaries clearly before handing over action rights. Delegating power without boundaries is not real empowerment; it is irresponsible risk-taking.
We need to know clearly: what can be handed to AI, to what extent, who bears responsibility when something goes wrong, and what must always be decided by a human. Boundaries are not a wall around AI's capability — they are the precondition for AI to enter the enterprise at scale. Only within clear boundaries can humans and AI achieve safe, efficient, and sustainable collaboration.
Within the boundaries, Harmonized Intelligence.
