Ask an AI agent, “What was revenue in the Northeast last quarter?” and imagine it returns $48.2 million. Ask again later, and the answer is $46.9 million. A colleague phrases the question differently and gets $51 million.
The company did not suddenly change its books. The agent may simply have taken a different route to the answer.
That is one of the harder problems businesses face as AI agents move from demos into real workflows. AWS guidance on evaluating reliable AI agents explains that large language models are nondeterministic, so the same query can lead to different tool selections, reasoning paths, and outputs.
In brainstorming, some variation is useful. In finance, sales, or executive reporting, three versions of the same KPI are harder to explain.
The problem is not always that the model made something up. Sometimes every number came from real company data. The disagreement started earlier.
AI agents can give different answers to the same business question because a natural-language question does not fully specify how the answer should be calculated.
The agent might choose a different source, metric definition, date field, filter, join, or tool each time it runs. It may also retrieve different context or follow a different reasoning path.
Better prompting can reduce some variation. It cannot fix inconsistent business definitions underneath the prompt.
For questions where the business expects one governed answer, agents need trusted data, shared definitions, controlled context, and repeated evaluation.
To the user, the process looks simple:
Question → Answer Underneath,
an enterprise agent may need to:
Every step creates another place where two runs can diverge.
AWS production guidance for evaluating AI agents makes this distinction clear. Traditional software testing assumes the same input should usually produce the same expected output. Agents complicate that assumption because they make context-dependent decisions and may take different routes on later runs.
That is why getting the right answer once is not enough.
In one AWS production example, an evaluation pipeline improved tool-selection accuracy from 87% to 98% and reduced incorrect results from roughly one in eight queries to one in 50.
The same prompt does not always mean the same execution. When the execution changes, the business answer can change too.
People inside a company use words such as revenue, customer, conversion, margin, and active account every day. That familiarity makes those terms feel precise.
The data estate may disagree.
When “Revenue” Has Four Meanings
Imagine an agent finds four measures:
recurring revenue.
Now someone asks:
“How much revenue did we generate last quarter?”
Which measure should the agent choose?
A finance analyst may know that executive reporting always uses recognized revenue. The AI does not inherit that institutional knowledge simply because it understands the word revenue.
SAP’s explanation of how a semantic layer creates consistent business definitions describes the same problem. Different systems can contain their own terminology and logic, creating conflicting versions of supposedly identical metrics.
The model may know what “revenue” means. It does not automatically know what revenue means inside your company.
The ambiguity gets worse when agents can access several systems. A sales agent might have access to a CRM, ERP, warehouse, spreadsheets, BI models, and internal documents.
Ask for quarterly sales, and one run may use the CRM. Another may query the warehouse. A third may retrieve an existing BI definition. All three could contain legitimate sales data while differing in refresh timing, exclusions, currency treatment, or business logic.
A MIT Technology Review Insights study on scaling AI agents with trustworthy enterprise data, produced with Google Cloud, found that AI systems have access to only about 45% of enterprise data on average. More than half of surveyed executives, 55%, said existing data systems prevent them from scaling agentic AI.
Incomplete access does not always make an agent say, “I don’t know.” It may simply answer from the part of the business it can see.
It is tempting to reduce all of this to garbage in, garbage out. But a company can have clean data and still get inconsistent AI answers.
Clean data does not tell an agent whether cancelled orders count toward bookings, whether “active customer” means a recent login or an active contract, or whether a region is based on billing address or sales ownership. Those are questions of business context, not simply data quality.
KPMG’s 2025 AI Quarterly Pulse Survey found that organizations piloting AI agents jumped from 37% to 65% in a single quarter, while only 11% were deploying them. At the same time, 64% of leaders identified organizational data quality as a major challenge to their generative AI strategies.
Agent capability is moving quickly. The data and context underneath it do not always move at the same speed.
This is where a governed semantic layer starts to matter. Its job is not to make the language model completely deterministic. It is to reduce the number of business decisions the model has to improvise.
Instead of allowing each dashboard, analyst, application, and AI agent to recreate “net revenue,” the business defines it once.
A governed semantic layer can establish:
fiscal calendars;
filters and exclusions;
business terminology and synonyms;
access rules.
Now consider the original question again:
“What was revenue in the Northeast last quarter?”
Without shared semantics, the agent has several decisions to make. What counts as revenue? Which geography defines Northeast? Which calendar defines last quarter? Which source is authoritative? With a governed semantic layer, much of that meaning has already been settled.
SAP explains that the semantic layer sits between raw data and the tools using it. A familiar business term such as revenue or customer can be connected to the appropriate source, calculation, relationship, and filter. The agent may still phrase its response differently.
The underlying business definition does not have to change with it. A semantic layer does not remove all AI variability. It makes business meaning less negotiable.
Suppose an agent correctly answers:
“What was revenue last quarter?”
Now try:
“How much did we make last quarter?”
Then:
“Show me Q2 revenue.”
If those phrases refer to the same governed metric, they should not produce different calculations.
The 2026 State of Agent Engineering from LangChain surveyed more than 1,300 professionals. It found that 32% identified quality as a top barrier to putting agents into production. Nearly 89% had adopted observability, while only 52% had implemented evaluations.
That gap matters. Companies are getting better at seeing what an agent did after it ran. Fewer are proving that it behaves reliably before its answers reach users.
Test the Logic, Not Just the Sentence
For business questions, teams should check:
whether the authoritative source stayed the same;
whether equivalent dates and filters were applied;
whether synonyms mapped to the same business concept; and
whether the calculation can be traced.
The wording can change. The business meaning should not.
Variation is not always a failure.
Ask:
“Why did customer churn increase?”
One run might surface price sensitivity. Another may identify onboarding problems. A third could uncover a regional pattern. That can be useful because the question is exploratory.
Now ask:
“What was our churn rate in Q2?”
That is different.
So is:
“What was gross margin last quarter?”
Those questions should point back to agreed business definitions.
Exploration can tolerate variation. Governed facts require consistency.
When teams notice inconsistent answers, the first response is often to make the system prompt longer.
Use the official revenue metric. Follow the fiscal calendar. Exclude cancelled accounts. Always check this table first.
That can help, but eventually the prompt becomes a fragile policy manual.
Business logic should live closer to the governed data layer, where dashboards, analysts, applications, and agents can reuse it.
A prompt should tell the agent what job to perform. It should not be the only place explaining how the company defines reality.
AI agents will always have some probabilistic behavior. The goal is not to force identical wording from every response. It is to decide where variation is useful and where the business needs one governed answer.
That means defining business metrics once, controlling which sources agents can use, and testing the same questions repeatedly before treating a successful demo as production reliability.
When two AI agents disagree about how to explain an insight, that may be useful. When they disagree about what revenue means, you do not have an AI problem yet. You have a business-definition problem.
The next step for enterprise AI is not simply making agents smarter. It is giving them the business context, governed definitions, and trusted data they need to reason consistently.
That is what makes the difference between an AI agent that can answer a question and one a business can actually rely on.
At Intuitive Data Analytics (IDA), this principle is central to how we think about enterprise AI: connecting trusted business data and governed meaning with AI-driven reasoning so organizations can move from questions and information to insights and decisions.
The future of enterprise AI will not be defined only by how intelligent the agent is, but by how confidently a business can trust the answer.
