Databricks Mosaic AI: AI agents that truly master your SAP business logic
- Databricks
- Databricks
- Reading time: nine minutes
Kenan Hodzic
This article explains why AI agents in the SAP environment fail not because of data access issues, but due to a lack of semantics—and how Databricks Mosaic AI solves this problem with governed context. The focus is on Metric Views and Unity Catalog as a “logic contract” for KPIs and access, supplemented by graph reasoning (Neo4j Virtual Graph) for document flow and hierarchy questions. The result: less improvisation based on raw data, more control, traceability, and auditability—as the foundation for robust decision intelligence.
Table of contents
1. The Problem: Data Access Is Not the Same as Mastering Logic
1.1 What this is about
Many SAP AI approaches follow a deceptively simple pattern: a large language model (LLM) is connected to SAP data via OData, SQL, or APIs and uses that data to generate responses. This may work for simple data queries—but as soon as SAP business logic comes into play, this model reaches its limits.
SAP financial and process logic does not exist in isolation within individual tables. It arises from the interaction of ledgers, fiscal year variants, currency rules, document flows, hierarchies, key figure definitions, and authorizations. An agent that does not understand this logic can provide answers that appear formally correct but are factually incorrect—and in a financial and compliance context, that is more dangerous than no answer at all.
Such agents—which function technically but are unfamiliar with SAP’s semantic layer—are referred to in this article as “naive agents.” The term is not meant to be derogatory; rather, it accurately describes the core problem: the agent is “naive” with regard to business logic because it has no access to it.
An example illustrates the problem: A user asks for the actual operating margin of a product line in the last quarter. An inexperienced agent directly accesses ACDOCA—the central journal in SAP S/4HANA—totals revenues and costs, and returns a result. The result is technically correct but factually incorrect. The agent is missing key information:
- Which fiscal year applies to the company in question?
- What currency translation policy is required within the group?
- Which profit center or product line hierarchy is valid from a business perspective?
- What data is the user who asked the question actually allowed to see?
The real risk here lies not in a single incorrect figure, but in the fact that the answer appears linguistically convincing. In the context of finance, compliance, and management, a response that is formally plausible but factually incorrect is far more dangerous than no response at all.
True decision intelligence only emerges when an agent does not act as an SQL translator on raw tables, but instead uses the business rules defined within the company as the definitive ground truth.
The following sections show exactly where naive agents fall short—and what levels an agent must understand in order to truly master SAP logic.
1.2 Why Naive Agents Fail
The typical error categories always arise when an LLM operates directly on raw SQL or OData:
SAP fiscal years rarely correspond to the calendar year. An agent who is unaware of the fiscal year variant will split periods incorrectly.
Key figures that exist at different levels of aggregation are combined or aggregated at a level that is not permissible from a business perspective.
Document flow, reversal, and reconciliation logic are missing; the agent simply adds up whatever they find.
Planned vs. actual, gross vs. net, the calculated SAC ratio vs. the raw data—fundamentally different in meaning, yet often similar in name.
Company code, controlling area, profit center hierarchy: The agent aggregates data across organizational boundaries that must not be crossed for business purposes.
Leading zeros in material numbers, prefix patterns, backslash handling in member IDs, dimensioning conventions—all of these result in silent failures without any error messages.
The most dangerous scenario: The agent views and discloses data that, according to the SAP authorization model, the user making the query should never be able to see.
What all these errors have in common is that they occur because the agent operates at the table level, while the business logic resides one level above.
1.3 The Most Important Levels for Agents
A naive agent therefore fails not because of a lack of computing power, but because of a lack of context—it does not know what the data means. To understand what an agent would actually need to be capable of, a taxonomy of the relevant levels is helpful:
Level | Examples | Where Logic Lives |
Master Data Semantics | General ledger accounts, cost centers, materials | CDS Views, Datasphere Models |
Organizational Structures & Hierarchies | Company code, profit center tree | Hierarchy tables, semantic models |
Document Flow / Process Chains | Order-to-Cash, Procure-to-Pay, Record-to-Report | Document chaining, status fields |
Calculation logic | Deviations, derived metrics | CDS Views, Metric Views, Analytic Models |
Authorization Policy | Who is authorized to view which organizational unit | SAP Authentication Objects |
The key insight: This logic already exists. It is codified in CDS views, Datasphere analytic models, and metric views. An agent does not need to reimplement it—it must tap into it as a reliable source (ground truth) and must not bypass it.
2. The Solution: Databricks Mosaic AI in the SAP Context
Databricks Mosaic AI is the AI and machine learning layer of the Databricks Data Intelligence Platform. While many AI platforms are primarily focused on model training or simple chat interfaces, Mosaic AI offers an integrated stack for building, evaluating, and operating AI agents—directly on the Lakehouse, where the data is already stored. This makes Mosaic AI particularly relevant in the SAP environment: AI agents do not operate on an external platform with replicated data, but rather on the governed data foundation that has already been established in the Lakehouse.
Mosaic AI consists of several building blocks that can be combined to address specific requirements. Rather than an abstract list of features, each component is directly applied to a specific SAP logic problem.
Mosaic AI module | SAP issue addressed |
Agent Bricks (automated agent builder with synthetic data and benchmarks) | A faster, proven path from proof of concept to production-ready accuracy without manual trial and error |
Mosaic AI Agent Framework | Orchestration, Tool Calling, Memory |
Unity Catalog Metric Views | A governed KPI definition as a tool |
Vector Search + Graph Retrieval | Retrieval based on semantic relationships rather than simple keyword searches |
MLflow 3.0 Tracing | Auditability of every single agent decision |
AI Gateway | Model Governance, Cost Control, Security |
MCP (Model Context Protocol) | Clean, standardized tool integration with SAP-related systems |
Agent Bricks was introduced at the Data+AI Summit and automatically generates domain-specific synthetic data and benchmarks to optimize agents for cost and quality—rather than through manual trial and error. For SAP, this means that edge cases in business logic can be systematically modeled as test cases before an agent goes into production.
Andreas & Yvonne's Databricks-Guide
Would you like all the important information at a glance?
Download the free guide to SAP Databricks now!
3. Graph Reasoning: The Missing Level
Most of the errors described in Section 2 are essentially relational errors. Business logic in SAP is primarily a matter of relationships:
GL Account → Profit Center → Company Code
Supplier → Purchase Order → Goods Receipt → Invoice → Document Line Item → General Ledger Entry
Cost Center → Materials
It is precisely these relationships that a knowledge graph represents—and which an agent must be able to infer “several hops away.” A typical question might be: “Which cost centers downstream are affected by this supplier disruption?” A simple table query and a vector search are simply not sufficient for this.
The problem up until now: To perform graph reasoning on SAP data, the data previously had to be copied into a separate graph database. This creates a second system of record, fragments governance, and requires a separate pipeline—a serious drawback given the volume and sensitivity of SAP data.
3.1 The New Feature: Neo4j Virtual Graph
In private preview since May 2026, with Databricks support at launch. Virtual Graph allows you to run Cypher queries and graph algorithms directly on the data already stored in Databricks—using zero-copy technology, without moving any data.
The architecture consists of three parts:
- AI-generated data model: The graph is automatically derived from the existing table schema; entities become nodes, foreign keys (including undeclared, inferred ones) become relationships, and columns become properties. The model remains editable and synchronizes with the source data.
- Query Translator: Cypher is deterministically translated into optimized SQL and passed to the Databricks engine as a pushdown. Deterministic means: the same SQL for every execution, predictable performance and costs—the translation is not LLM-driven.
- Graph Compute Layer: Handles graph-specific operations (pattern matching, traversals, algorithms) that SQL alone cannot express efficiently.
The data never leaves its governance perimeter. This is the key point for SAP: The semantic backbone becomes a virtual layer over the data that has already been ingested and governed via BDC/Datasphere—rather than a replicated parallel world.
3.2 When to use a virtual graph, and when to use a native graph?
Virtual Graph and a graph stored natively in Neo4j are not substitutes—they solve different problems:
- Virtual Graph is a good fit when warehouse latency (ranging from seconds to minutes) is acceptable: GraphRAG for master data/reference data, batch enrichment, analyst-driven exploration, and agent-driven workflows that operate in seconds. Rule of thumb: agents that think in seconds.
- Native Neo4j graph (AuraDB) is ideal for millisecond-level requirements: real-time decision-making, online scoring, ACID writes, and continuously updated memory and knowledge graphs. Rule of thumb: agents that act within milliseconds.
For SAP analytics agents—reporting, financial analysis, supply chain reasoning—the majority of requests are processed in a matter of seconds. This makes Virtual Graph the obvious, governance-compliant choice for getting started, with the native graph serving as a complement for latency-critical paths.
4. Architecture & Tool Design
4.1 Architecture Blueprint
How SAP Logic is implemented on the agents:
The guiding principles of this architecture:
- BDC as a clean approach. The semantic models from Datasphere are not replicated to the Lakehouse in their raw form; instead, they are governed via Delta Sharing.
- Metric Views as a KPI contract. The agent retrieves defined metrics, not custom aggregations.
- Virtual Graph as a relationship layer. Multi-hop reasoning based on document flow and organizational structures, with zero-copy access to the governed data.
- Unity Catalog as a common governance framework for tables, metric views, and the virtual graph.
These principles define the framework—the next section shows how they are reflected in the specific tool design of an agent.
4.2 Tool Design: The Real Key
If there is one key point to this article, it is this: The crucial question is not which LLM is used or how the prompt is worded. The crucial question is what tools are available to the agent —and what those tools encapsulate.
The principle is simple, but its implications are far-reaching: A naive agent is given unrestricted access to SQL tables. Technically, this allows it to answer any question—but it decides for itself how to aggregate data, which time periods to include, and which hierarchies to ignore. The logic lies in the model, not in the architecture. That is the problem.
A semantically governed agent, on the other hand, does not receive raw SQL. Instead, it receives a curated set of tools that already incorporate the business logic:
- Metric View Tools – parameterized queries for defined key performance indicators. A Metric View is not a free-form query interface, but a formally defined KPI: which fields are included, which aggregation logic applies, and which organizational level is intended. The agent selects which metric to query—and in which context (company code, period, currency). The agent never decides how the metric is calculated. This decision is made in the governed layer and is immutable.
- Graph Tools (via Virtual Graph) – Relationship and path queries: “Track the document flow,” “Find all downstream cost centers for this profit center group,” “Which suppliers are associated with this company code?” These questions cannot be solved cleanly with SQL alone—they require multi-hop reasoning based on relationships. The Graph Tools encapsulate Cypher patterns that are deterministically translated into SQL. The agent calls the tool; what happens behind the scenes remains controlled and reproducible.
- Retrieval tools – semantic search across business documentation, key metric definitions, process descriptions, and policy documents. These tools ensure that the agent operates not only with numbers but also within the relevant context. “What is the corporate policy on currency conversion in the quarterly report?” is a question that no SQL query can answer—but a retrieval tool based on governed documentation certainly can.
The interaction between these three types of tools constitutes the actual design pattern. A metric-view tool provides the metric. A graph tool provides the organizational relationships within which the metric must be interpreted. A retrieval tool provides the rule according to which exceptions are to be handled. The agent orchestrates—it combines, infers, formulates—but it does not invent logic that does not exist in the governed layer.
That is the key difference between “being able to retrieve SAP data” and “mastering SAP logic”: The logic is encapsulated and governed within the tools—not left to the model’s improvisation.
5. Operations: Evaluation, Governance, and Security
5.1 Evaluation & Trust
When it comes to financial figures, “mostly correct” is worthless. That’s why a structured evaluation is needed—and that’s exactly what Agent Bricks provides
Dimension | Meaning |
Correctness | Is the answer factually correct? |
Latency | Does the agent respond within a reasonable amount of time? |
Execution | Are the right tools being used? |
Adherence | Does the agent adhere to the specified constraints? |
Relevance | Is the answer relevant to the question? |
Safety | Are permissions and security policies being followed? |
When supplemented with LLM judges and synthetic test sets specifically designed to cover SAP edge cases (fiscal year-end dates, currency conversion, reversal scenarios), an evaluation loop is created—without which no production deployment can be justified.
The second pillar is MLflow 3.0 Tracing: Every decision made by an agent—which tool, which parameter, which result—is traceable and auditable. In the SAP context, where auditing and compliance are not optional, this is not a luxury but a requirement.
5.2 Governance & Security
The unresolved challenge, to put it bluntly: mapping SAP authorizations to Unity Catalog row- and column-level security. Users should only be able to view what their SAP authorization scheme permits—and this must be enforced by the agent, not circumvented.
The advantage of the architecture outlined here is that, because Virtual Graph operates on data governed by Unity Catalog using a zero-copy approach, the same governance perimeter applies to relational queries, metric views, and graph reasoning. This eliminates the need for a second dataset that would require separate security measures. As a result, the attack surface and governance complexity are significantly reduced.
6. Assessment & Outlook
6.1 Reality Check
A vision statement without clear boundaries comes across as unreliable. That is why we have explicitly stated the most important limitations:
- Virtual Graph provides warehouse latency. Real-time millisecond paths still require native graph storage.
- As of the publication of this article, Virtual Graph is in Private Preview. Production designs should take the maturity timeline and roadmap into account—additional sources, adaptive caching, and composite queries via Aura and Virtual Graph have been announced.
- Semantic quality. The agent is only as good as the underlying Datasphere models and Metric Views. No agent layer can compensate for poor semantics in the underlying infrastructure.
If you know where the limits lie, you can plan realistically—and that is exactly what the following step-by-step model helps with.
6.2 Maturity Levels
The target architecture described does not have to be implemented all at once—and in practice, it rarely is. A more sensible approach is a phased implementation that starts with a clearly defined, production-ready foundation and builds from there. The following phased model provides guidance on where a company stands today and what the next realistic step is:
- Read-only Q&A. The agent answers questions via a Metric View. One metric, one governed source.
- Relationship reasoning. Virtual Graph is added; multi-hop queries regarding document flow and organizational structures.
- Multi-agent across process chains. Supervisor architecture with specialists for each process.
- Agent-based Decision Intelligence. Agents that don’t just respond, but suggest courses of action along with justifications and an audit trail.
Most companies in the SAP ecosystem are currently at stages 1 or 2. This isn’t a sign of lagging behind—it’s the right starting point. By implementing Stage 1 properly, you lay the governance foundation upon which all subsequent stages can be built solidly. The leap to Stage 4 isn’t a technical quantum leap, but rather the result of consistently refined semantics and architecture.
7. Conclusion
The crew has made contact. Not with alien life—but with their own company. For the first time, the system isn’t just responding to signals; it’s understanding what lies behind them. Mission Control breathes a sigh of relief. This isn’t just data retrieval anymore. This is Decision Intelligence.
The central theme of this article: The business logic already exists—in SAP’s semantic layer. Mosaic AI is not where you build it from scratch, but where you make it governable, evaluable, and accessible to agents.
Neo4j Virtual Graph fills this critical gap: it applies the relationship reasoning that is at the heart of business logic—with zero-copy processing—to the data that is already stored in the governed Lakehouse.
This is a more nuanced concept than “AI agents for SAP”—and the logical next step from business intelligence to decision intelligence.
Your data strategy is unique—your consulting should be too.
The choice between these methods depends on countless factors: your existing system landscape, your business goals, and your data culture. There is no standard answer.
Let’s talk, with no obligation, about which path is right for you.
Please contact us to schedule a personal consultation.
Sources
- Databricks, Agent Bricks (Data+AI Summit) automated agent builder with synthetic data and benchmarks.
- Neo4j, Introducing Neo4j Virtual Graph: Graph reasoning on the data you already have (May 28, 2026): https://neo4j.com/blog/graph-database/introducing-neo4j-virtual-graph-graph-reasoning-on-the-data-you-already-have/
- Mosaic AI Agent Framework Components: Vector Search, MLflow Tracing, Unity Catalog Governance, MCP Integration
