The choice of a foundation model shapes everything built on top of it. If it effectively handles function calling, connecting external systems becomes much more reliable. For example, the agent might retrieve a vendor profile from one platform and log scored opportunities in a CRM or deal pipeline.
Teams building agentic systems often use multiple foundation models. A lightweight, faster model performs the first pass, filtering out job offers that are clearly irrelevant. A larger, more capable model then evaluates the remaining opportunities and scores them against fit criteria.
Goal-setting layer
This layer turns a high-level objective into a specific execution plan. Depending on the project, the planning can be handled either by the foundation model itself or by a separate planning model built on top of it. This layer determines which subtasks need to be done, in what order, and what “done” means for the overall goal.
A good example is an agentic AI system with video analysis capabilities used for flagging fleet safety incidents in logistics operations. A single instruction for such a solution may hide a dozen smaller steps, such as pulling the relevant video files, running each clip through an incident detection model, generating an incident summary report, etc. The goal-setting layer of an agentic AI system breaks that instruction into an ordered and executable sequence.
This layer also handles prioritization and dependency management. An agentic AI system can run multiple tasks in parallel, but some steps still have to happen in sequence. For example, an incident report can only be generated after the video analysis is complete.
A well-designed goal-setting layer recognizes these dependencies and sequences work accordingly. Before considering the task complete, this layer verifies that all required steps have been completed and the expected outcome has been achieved.
Orchestration layer
Once a goal has been broken into subtasks, something has to actually run that plan. An orchestration layer takes on this responsibility, serving as the runtime engine that executes the plan the goal-setting layer produces. It calls system components, passes outputs from one step into the next, and adapts when a step doesn’t go as expected.
Let’s take a look at an agentic AI system built for fraud detection at a fintech company. The goal-setting layer has already decided that a flagged transaction must go through identity verification, transaction pattern analysis, and a risk-scoring step, in that order. The agentic AI orchestration layer runs each component in sequence and passes the identity check result to the pattern analysis model.
It also keeps track of the workflow state, including the account history, detected anomalies, and the current risk score while the transaction is being evaluated. If the identity-verification service times out, the orchestration layer decides whether to retry the call, fall back to a secondary verification provider, or escalate the transaction for manual review.
Memory layer
Agents need to remember context, both within a single task and across sessions. The memory layer addresses both needs by storing prior interactions, intermediate results, and learned preferences, so the agents don’t have to start from scratch every time.
Short-term memory is what lets an agent keep track of its location within a single multi-step task. For example, if an agent is three steps into processing an insurance claim and needs to pause to fetch a missing document, it has to remember what it already verified and what’s still outstanding when it resumes instead of re-checking everything from scratch.
Long-term memory works differently. It persists across sessions and often across users entirely. For instance, an agentic AI sales assistant that remembers a specific client prefers phone calls over email is drawing on long-term memory. This is usually implemented using a vector database, which stores information as embeddings and retrieves the most relevant context on demand rather than loading the entire interaction history into the prompt each time.
Memory design also has real cost and privacy implications. First, storing every detail forever isn’t free. Secondly, in regulated industries such as healthcare, an agent’s memory isn’t just a technical decision. Deciding what it can remember about a person, and for how long, is a compliance requirement.
External tooling
Agentic AI systems rely on external tools to complete tasks. These tools can range from APIs and databases to search engines and internal business systems. The external tooling layer enables AI agents to perform actions and complete specific tasks. For example, in a customer support system, the AI model can reason that a refund is warranted, but issuing it requires contacting a separate payment processor that actually has the authority to move the money.
External services often connect to an agentic system through custom API integrations built for a specific tool. Another option is to use standardized protocols, such as the Model Context Protocol (MCP), which allow an agent to use new tools without custom code for each one. Agentic AI orchestration frameworks like LangChain also ship with built-in connectors for common tools, saving teams from crafting every integration themselves.
Tool integration also introduces one of the harder engineering problems in agentic systems: deciding how much autonomy to give an agent over which tools. Mature agentic architectures build in permission tiers, so an agent might be allowed to query a database freely but required to get human sign-off before executing an action with financial or legal consequences.
AI feedback and validation loops
The loop is what makes an agent self-correcting. Without it, an agentic AI system will only get one attempt at a task, with no way to recognize or fix a failure.
Here’s what that looks like in practice. For instance, a system tasked with generating a sales report might query a database, build a summary, and then check its own output. It verifies that the reported revenue matches the source data, confirms that all requested report categories are included, and checks whether the output meets the required formatting rules. If any of these checks fail, the feedback loop prompts the agent to revise its approach. It may retrieve data from a different table, adjust an overly restrictive filter, or make other changes before attempting the task again.
Validation is what decides if an output is actually good enough to hand off. It’s a final check against a clear standard, such as a business rule or a compliance requirement, before the result is delivered to a person or a downstream system.
This loop also allows an agent to handle situations its initial plan didn’t anticipate. If a tool call fails because an API is down, the loop is what decides whether to retry, fall back to an alternative method, or escalate to a human.
The maximum number of iterations an agent is allowed to perform is an important design consideration. If the limit is too low, the agent may abandon tasks that could have been completed successfully. If it is too high, the agent may waste time and computational resources repeatedly attempting an unsuccessful approach. Selecting an appropriate iteration limit, together with clearly defined conditions for escalating a task to a human operator, is essential for building a reliable and efficient agentic system.
Leobit
