To Whom Does Responsibility Belong in Artificial Intelligence Agents?

*Abdullah Yasir Atalan

Agent-based artificial intelligence (agentic AI) is one of the most widely circulated concepts of recent times and is often packaged as the next stage of the artificial intelligence trend, as if the chatbot revolution has exhausted its useful life. Yet behind this new wave lies not merely a technological leap but corporate strategies that channel significant resources into the sector. For instance, a survey conducted by BCG across 116 countries reported that “35 percent of companies have begun exploring agent-based AI,” revealing that the concept responds to an industrial need for a new narrative. Thus, the popularity of this concept stems not from a new technological revolution but from the construction of an industrial sales language.

In fact, the boundaries of chatbots—the first widespread application of large language models (LLMs)—underlie this shift in communication strategy. Companies have evidently recognized the upper limits of the chatbot concept, which is why they have begun adopting the language of agent-based AI. Chatbots, at their core, represent an infrastructure that engages users in dialogue and keeps the user at the center, yet in most scenarios they stop at generating recommendations. These platforms have weak claims to act on behalf of users without user prompting, user approval, or user authorization. The term “agent,” by contrast, represents a more ambitious shift in direction. A goal is given to the system, the system breaks it down into subtasks, accesses external tools (APIs, databases, enterprise software) when needed, and produces output through a sequence of steps. More practically, a system that scans a user’s email inbox on their behalf, groups messages by specific subjects, drafts reply templates, or suggests meeting times can be marketed as an agent. Similarly, a system that compares prices to plan a trip, manages reservation steps, and completes the process after confirmation is also placed in this category. In essence, the system does not merely generate text; it interacts with other systems, reads data, sometimes writes, and therefore its errors no longer amount to giving incorrect answers but to performing incorrect actions.

From this point onward, limiting the debate to the question “What exactly is an agent?” is the most comfortable but also the most misleading form of the issue. The real question should be: What risks does the term “agent” obscure, and what governance gaps does it hide behind the more intelligent automation narrative?

A fundamental observation is that the use of the term “agent” in artificial intelligence is not new, nor has it ever had a single universally accepted definition. In the 1990s, classic textbooks defined an AI agent in its simplest form as “a system that perceives its environment and acts upon it.” Yet even then, Wooldridge noted that there was no universally accepted definition of the term “agent.” Today the situation is no different, but the confusion is now more visible because the concept circulates not only in academia but also in the marketplace.

Corporate definitions lay bare this confusion. Recently, many technology firms have embraced this trend and rushed to define agent-based AI. A simple online search reveals definitions from companies such as Anthropic, Amazon, Google, IBM, OpenAI, McKinsey, and Deloitte. Yet these definitions differ significantly from one another, each tailored to meet distinct needs. Some companies define an agent as long-running systems that call tools and manage workflows autonomously, while others define it primarily as a “natural language interface,” describing agents as “more talkative software.” Some exclude chatbots from the category of agents, while others extend the definition so broadly that even basic customer service bots are called primitive forms of agents.

From a more practical standpoint, most corporate definitions of agents rely on the cognitive burden transferred from humans. That is, an agent is described as a system that accomplishes maximum work with minimal guidance, reduces human cognitive load, and breaks down goals into subtasks. This framework is understandable for product development and marketing because it ties success to the intuitive metric of reduced human effort.

But the source of the confusion lies in the same word both narrowing and expanding within marketing language. It narrows because the label “agent” must be positioned as a premium product segment. It expands because, as a natural consequence of this marketing language, companies can market basic automation as something more advanced. Thus, “agent” has become an umbrella term used simultaneously for both automation and autonomy.

It is crucial to recognize that this issue is not merely a definitional debate and that this conceptual dispute carries tangible governance consequences. When artificial intelligence becomes an agent, it is no longer merely a model that generates text. It becomes an interface that interacts with other systems, reads data, sometimes writes, and connects to organizational structures. This integration transforms an error from an epistemic problem into an operational one. An incorrectly summarized email is annoying, but an incorrectly sent email can trigger a corporate crisis. A wrongly suggested meeting time is tolerable, but a canceled meeting that triggers a chain of disruptions can cause widespread delays.

The most significant side effect of this transition is the blurring of accountability. During the chatbot era, an error could be framed simply as a “wrong response.” The user sees it, dislikes it, and corrects it. With agent-based AI infrastructure, however, errors often become invisible within a workflow: a tool call, a data retrieval, an automated action, the use of an access permission. At this point, how we understand agency becomes decisive. The first approach locates agency primarily within the artificial intelligence model itself. In this view, the agent receives data from the environment, processes it internally through a representation/inference mechanism, and produces an action or recommendation. Thus, the source of agency lies in the system’s internal decision function. This framework aligns well with the industry’s current definitions: the more work a system accomplishes with less guidance, the more “agent-like” it is, and the risk is typically seen as the model making an error (wrong inference, wrong plan, wrong tool call). This reading is inadequate for two reasons. First, the success or failure of an agent depends not only on the model’s accuracy but also on what data it accesses, which systems it has write permissions to, and what boundaries it operates within. Second, from an organizational perspective, the primary question is not “Is the model correct?” but “To what extent is the model authorized, and who granted this authority?” Thus, the model-centric reading pushes questions of authority delegation and accountability into the background.

The second perspective treats agency relationally. According to this view, agency is not the exclusive property of a single model. Rather, it emerges from the interaction of humans, software, interfaces, data infrastructure, organizational culture, procedures, and incentives. Under this approach, an agent-based AI system generates risk not because it is inherently “intelligent,” but because it is embedded within a workflow and reshapes decision-making processes there. The resulting governance questions inevitably broaden: Which models will be integrated into the organization? What testing and evaluation methods will be applied? Does the test plan measure only model scores or actual team performance within real workflows? How are access permissions layered? At what threshold does human approval come into play? Are every tool call and data write by the system recorded in an auditable manner? And when an error occurs, where does responsibility lie?

The common thread among these questions is this: “Agency,” or the capacity for action, can no longer be sought solely within an algorithm. It resides within the organization as a whole—in its practices of authority delegation, audit trails, incentive structures, and accountability regimes. By whose authority was access granted to the AI system? What approval mechanisms were in place? What records were kept? What boundaries were established? What incentives normalized this usage? Until the debate on agents confronts these questions, it will remain merely a technical competition in capability, whereas the issue is not technical but political and institutional. What we call an “agent” is often not a new form of intelligence but a new regime of delegation. And delegation regimes become secure only when their boundaries are drawn, their traces are tracked, and their areas of responsibility are clearly defined.