Home AI Readiness Universal Governance and Lineage for AI

Universal Governance and Lineage for AI

Universal Governance and Lineage for AI

As soon as data leaves the SAP environment, governance often comes to an end in many architectures. Permissions are managed in Databricks and Fabric —often more liberally than in the source system—and no one can say with complete certainty who viewed which data record and when. For AI agents that access data autonomously, this is an unacceptable risk.

Governance must therefore not stop at the system boundary. This wiki shows how SAP authorization profiles and Datasphere access controls are translated into global Unity Catalog policies and how MS Purview provides end-to-end lineage.

SAP authorizations are defined in the source system using PFCG roles (see the glossary for a detailed explanation). SAP Datasphere supplements these rules with Data Access Controls (DAC), which allow for targeted restriction of access to rows and columns within a data model. A mapping service translates both sets of rules into row- and column-level policies in the Databricks Unity Catalog. This ensures that users outside of SAP see exactly the rows and columns they would be permitted to view in the source system. At the same time, the origin of the data is recorded in the Microsoft Purview lineage graph, while access is also traceable via audit logs.

For AI agents, this is a prerequisite for being allowed to operate autonomously at all: They inherit the company's permissions rather than circumventing them, and every instance of data use can be traced afterward.

The result is a governance model that follows the flow of data rather than ending at the system boundary: The same user sees in Databricks and Fabric exactly the same rows and columns they would see in S/4HANA, and every use case can be traced back to the original SAP field.

Table of contents

1. Why this matters: Governance is the ticket to the agentic phase

When it comes to cross-cloud governance, two points are of central importance:

Don't maintain permissions separately on every platform. Permissions maintained in multiple places inevitably drift apart—usually becoming more permissive than in the source system. Only a mapping from a single source keeps the rules consistent across Databricks and Fabric.

Lineage is more than just a compliance requirement. For autonomous AI agents, end-to-end lineage is the technical prerequisite for being allowed to act at all: Without a traceable data path, no agent decision can be reviewed retrospectively.

2. The Technical Core: Permission Mapping and Purview Lineage

The PFCG roles and Datasphere DAC define who is allowed to view which rows and columns. These rules are transferred to Unity Catalog policies instead of being maintained anew in Databricks. Lineage is initially tracked without any additional tools: Unity Catalog automatically captures table and column lineage for every query—notebooks, jobs, dashboards, ML—and makes it queryable via system tables. Only when lineage needs to be consolidated across Databricks into a single catalog—from the S/4HANA field through the BDC to the Power BI report—does Microsoft Purview come into play; this is particularly useful in a Microsoft-centric landscape, where SAP already provides its lineage metadata to Purview. It’s important to note that while Purview links lineage within each platform, it currently cannot fully and automatically reconstruct the end-to-end path from the Databricks table through the transformations all the way to Power BI.

component

Function

PFCG Reels

Define the authorizations in the SAP source system.

Datasphere Data Access Controls (DAC)

Add row- and column-specific access rules in BDC.

Unity Catalog Row/Column Security

Apply the SAP rules as policies in Databricks.

MS Purview Lineage

Logs every access and links it to the SAP source.

3. How the Rules Change: Authorization Mapping in Detail

PFCG roles cannot be copied 1:1 to Databricks —SAP authorization objects and Unity Catalog policies are different models. Propagation therefore involves translating the semantics (who is allowed to view which rows and columns) and takes place across two separate levels, supplemented by a scalable workaround:

  • Identity: The relevant groups are federated from the identity provider (Microsoft Entra ID or Okta) to Databricks at the account level via SCIM—the PFCG role provides the group membership, not the rule itself. Unity Catalog permissions apply exclusively to these account-wide groups.
  • Rule: The actual row filters and column masks are stored as SQL functions (UDFs) in the Unity Catalog and evaluated at runtime for each query—typically using `is_account_group_member()` or `current_user()`, which check federated group membership.
  • Scaling via ABAC: For many tables, the attribute-based approach is recommended over individual UDFs—centralized policies that are attached via governed tags at the catalog or schema level and take effect automatically.

To ensure that the SAP side remains the leading source, the derivation of DAC/PFCG semantics into these UDFs or ABAC policies must be automated—repeated with every role change, rather than copied once. It is important to note a limitation of delta sharing: row filters and column masks do not automatically travel with a shared table. Either the BDC enforces them at the source via DAC, or the recipient side recreates them in its own Unity Catalog; using ABAC policies, a protected table can only be shared if the share owner on the provider side is excluded from the policy.

At the same time, every read and write operation must be recorded in a lineage graph and linked to the SAP source. Within Databricks, this lineage is created automatically via the Unity Catalog; if it is to be consolidated with the SAP source in a unified catalog, Microsoft Purview handles this—in a Microsoft-centric landscape. What matters is not the individual tool, but the seamless chain: from the S/4HANA field through the BDC data products and Databricks transformations to the AI output. Only this end-to-end linkage allows us to answer the critical question—“Where does this value come from, and who was authorized to view it?”—without manual reconstruction.

4. From Role to Policy: A Playbook for Data Architects

Governance across system boundaries is not achieved through a tool, but through clear responsibilities. Every playbook should include three guiding principles:

  • Establish a single source of truth: Designate SAP as the leading system for authorizations and prohibit manual maintenance in Databricks or Fabric. Every locally maintained exception is a potential source of future drift.
  • Automate mapping, don't copy: Use a service to translate PFCG roles and DAC into Unity Catalog policies, rather than copying them manually. Only automated propagation ensures that the rules remain consistent with every role change.
  • Ensure Lineage Before Deploying the First Agent: Ensure end-to-end lineage before granting AI agents access to production data. An agent whose data path cannot be traced is not eligible for use within the company.
  • Implementing lineage: Within Databricks, it is established without any additional effort—Unity Catalog automatically logs table and column lineage. If it is to be consolidated in Microsoft Purview along with the SAP source and Power BI consumption, the integration is handled via the OpenLineage Spark listener on the Databricks clusters, whose events are transmitted to Purview by a connector (Solution Accelerator via Azure Function App and Event Hub, using the Apache Atlas API); Native Purview models for Databricks are increasingly replacing this approach. Plan for this integration early on—the automatic end-to-end linkage all the way to Power BI is not yet seamless.

5. Common Pitfalls

Three mistakes keep cropping up in governance projects:

  • Rethinking permissions: Setting more lenient rules in Databricks than in the source system because it’s faster. That’s exactly how silent data leaks occur—leaks that no one can explain later on.
  • Lineage as a point to consider: Do not implement traceability until an audit requires it. Information that was not recorded from the outset cannot be reconstructed retroactively.
  • Focusing solely on the technology: ignoring data stewardship. Permissions and classifications require subject-matter expertise to maintain; no mapping service can replace the decision of who is allowed to see what.
Universal Governance and Lineage for AI | s-peers Wiki 04
Opportunities and Risks

6. Universal governance builds trust—
, and operational tasks.

Universal governance builds trust in cross-cloud data, but it also entails operational tasks that architects should actively manage.

Risk What it's all about What Architects Should Do
Authorization Drift Rules maintained locally in Databricks or Fabric drift away from the SAP source system and inadvertently allow unauthorized access. Prevent manual maintenance on the target platforms and automatically report deviations from the SAP standard.
Lineage Gaps If the lineage breaks at a transformation, the data path can no longer be verified end-to-end. Verify traceability at every stage of processing and treat any gaps as operational disruptions.
Mapping Complexity Translating fine-grained SAP roles into Unity Catalog policies can become time-consuming as the set of roles grows. Clean up the role structure before mapping and reduce it to the minimum set of permissions actually required.
Conclusion

7. Governance does not end at the system boundary.
It is the ticket to the agentic phase.

Governance that ends at the system boundary is not governance in a multi-cloud environment.

Only when SAP authorizations apply globally and every instance of access remains traceable may AI agents independently access corporate data. Universal lineage is therefore not an optional feature, but a prerequisite for the agent-based phase.

For data architects, this means that the authorization and lineage layer is not a downstream compliance issue, but rather part of the core architecture. Those who build it properly create a space within a heterogeneous multi-cloud landscape where AI agents operate with the same permissions and the same traceability as a human in the source system. This wiki builds on the semantic layer from Wiki 01 and establishes the governance foundation for agent-based orchestration in Wiki 06.

The Next Step

Your data architecture is unique.
Let's start a conversation.

Whether it’s authorization mapping, DAC design, or end-to-end purview lineage: governance across system boundaries stems from the right architecture, not from a single tool. Together, we’ll consistently map your SAP authorizations to Databricks and Fabric and build the seamless lineage that makes your AI initiative auditable.

Glossary

9. The Most ImportantTerms in "
."

PFCG Role

The role concept in SAP, through which authorizations are defined in the source system and assigned to users.

Data Access Control (DAC)

A row- and column-specific access rule in SAP Datasphere that supplements the PFCG authorizations in the BDC.

Unity Catalog

The Databricks governance layer, which enforces row filters and column masks as policies across Lakehouse data.

Row/Column Security

Access control at the level of individual rows (row filters) and columns (column masks) rather than entire tables.

Attribute-Based Access Control (ABAC)

Attribute-based access control, in which central policies are automatically applied via governed tags at the catalog or schema level—the scalable alternative to individually maintained row filters and column masks.

Data Lineage

The end-to-end tracking of a data value from its source to the result through all processing stages.

Microsoft Purview

Microsoft's governance and catalog service, which tracks lineage across system boundaries.

Data Stewardship

The technical management of permissions, classifications, and data quality by designated personnel.

Published by:

Dr. Andreas Wagner

Customer Success Executive

author

How did you like the article?

How helpful was this post?

Click on a star to rate!

Average rating 5 / 5.
Number of ratings: 9

No votes so far! Be the first person to rate this post!

INFORMATION

More information

Fuzzy Search in the search bar

Fuzzy Search Widget for the SAC

Our SAC expert Joana shows you, step by step, how to...
Knowledge Graph

SAP Knowledge Graph Explained for Joule and AI Agents

In May 2026, SAP announced at Sapphire in Orlando...
SAP Acquires Relatio

SAP Acquires Reltio: The Missing Piece of Its AI Strategy

This wiki discusses why a system of record such as an ERP system is no longer sufficient in the age of AI, and what...
SAP BDC Zero-Copy and Data Engineering with Databricks

Hybrid Data Platform Combining SAP Business Data Cloud and Databricks: Zero-Copy Delta Sharing and Medallion Architecture

How can SAP Data Products be used in Databricks outside of the SAP Business Data Cloud (BDC)? This wiki explains...
S4 Data in Databricks, Including Semantics

Connecting SAP S/4HANA and Databricks: Integrating Data and Semantics Together

How does SAP S/4HANA data get into the SAP Business Data Cloud (BDC)? This wiki explains the interaction between CDS views, business...
wiki-ai-readiness-1 (1)

Three Engines, One Truth: A Semantic Model for SAP Datasphere, Databricks, and Power BI

Three engines, one metric: How do SAP Datasphere, Databricks, and Power BI find common ground? This wiki explains...

SAP Sapphire 2026 Glossary: Joule, AI Agents, and the Autonomous Enterprise Explained Simply

Joule, AI agents, Autonomous Enterprise, A2A protocol—all the new SAP terms from Sapphire 2026 at a glance. Look them up now in the s-peers glossary.