Mission Lakehouse – All Wikis at a Glance
- Databricks
- Databricks
- 5 min reading time
Emma Colla
Our Lakehouse Mission is complete. In a series of 10 wiki articles, we’ve documented the entire journey—from the basics to the intelligent operation of a modern Databricks Lakehouse with SAP data. This page provides a concise overview of each mission module.
Table of contents
Wiki 1 — Your Introduction to Databricks & SAP BDC
Databricks is the leading technology platform for data science and AI—and provides an optimal mapping of the SAP stack. While SAP delivers process-critical business data, Databricks serves as a state-of-the-art engine for innovation. This wiki explains the platform’s key benefits and components (Unity Catalog, Lakeflow, Mosaic AI), demonstrates how the SAP Business Data Cloud (BDC) goes beyond a simple software bundle, and highlights three specific use cases: Mosaic AI for forecasting, a 360° view of data through the combination of SAP and third-party data, and high-performance big data processing. The common thread is the complete data flow from activation to visualization in the SAP Analytics Cloud.
Wiki 2 — AI Meets BI: Modern Reporting in the Databricks Lakehouse
Business intelligence and artificial intelligence were long trapped in separate silos—traditional BI departments maintained data warehouses for historical analysis, while data scientists experimented in isolation in data lakes, resulting in inconsistent data and high ETL latency. Databricks’ Lakehouse concept breaks down this separation.
Wiki 3 — SAP Databricks vs. Native Databricks: Choosing the Right Rocket
Both solutions come from the same source—but they are built for different mission profiles. SAP Databricks is a deeply integrated OEM solution licensed exclusively through SAP that receives valuable business semantics (hierarchies, currency conversions, CDS views) directly via the BDC connector, without the need for complex logic replication. Native Databricks, on the other hand, offers maximum technological freedom across Azure, AWS, and GCP. The wiki provides a direct comparison of purpose, integration, ML development, licensing, and tech stack—and explains why SAP BDC without SAP Databricks does not offer a complete pro-code ML workbench for data scientists.
The End of “Data Plumbing”: Instead of laboriously pumping data through pipelines, Zero-Copy Delta Sharing enables live access to SAP data in Databricks—without creating a single physical copy. Technically, this works via an elegant REST-based handshake: A bearer token authenticates the client, the BDC server provides a manifest with pre-signed URLs, and the Databricks worker nodes load only the required data columns directly from the SAP File Object Store via byte-range requests.
Wiki 5 — SAP Data to Databricks: A Strategic Guide to Data Integration
How does the data get into the system? The wiki answers this fundamental question with a comprehensive overview of all six integration paths from SAP to Databricks.
Wiki 6 — The Right Fuel: The Evolution of Table Formats in the Lakehouse
Just as a rocket won't tolerate the wrong fuel, a data platform won't tolerate the wrong format choice. The wiki clears up the misconception that Apache Parquet and Delta Lake are competing technologies.
Wiki 7 — BDC Connect: A Direct Connection to Databricks, Snowflake, and Others
Complex ETL processes, costly data copies, lost business logic—until now, this has been the norm when integrating SAP data into modern cloud platforms. BDC Connect ushers in a new era. As a direct connector between the SAP Business Data Cloud and platforms such as Databricks or Snowflake, it enables live connectivity without replication: Data flows without being copied. Crucially, the valuable business logic is preserved directly and does not have to be painstakingly reconstructed in the target system. The wiki explains how BDC Connect works from a technical perspective, which use cases it is particularly well-suited for, and what specific added value it offers compared to traditional ETL pipelines.
Wiki 8 — Mission Control: The Architecture of the Databricks Unity Catalog
Who accesses which data—and how to ensure this is done consistently, securely, and across all platforms: that is the role of the Unity Catalog. As the central hub of the Databricks platform, it manages permissions, data lineage, and access controls within a three-tier hierarchy and enforces row-level security and column-level masking across all platforms. When combined with Microsoft Entra ID as the central identity provider, this creates a seamless identity pipeline in which role changes or deactivations take effect immediately and consistently across the entire hybrid SAP-Databricks stack. The wiki explains why Unity Catalog is the linchpin of universal governance—and how Delta Sharing via the Catalog ensures that governance doesn’t stop at system boundaries.
Wiki 9 — The Space Station: Medallion Architecture at the Heart of the Lakehouse Mission
Every space station needs a clear structure so that data, crew, and systems can work together efficiently. The Medallion architecture is at the heart of every modern lakehouse implementation: a proven, three-tiered model that systematically transforms raw data into business-relevant insights. This wiki shows how SAP data products flow seamlessly into this architecture via BDC Connect and Delta Sharing—and outlines the best practices for the long-term operation and further development of the space station.
Wiki 10 — Databricks Mosaic AI: AI agents that truly master your SAP business logic
Data access does not equate to mastery of logic. An AI agent only “masters” SAP when it operates at the semantic level—working with key metric definitions, organizational structures, company codes, and consolidation rules. Mosaic AI makes exactly that possible. Orchestrated via MLflow and trained directly on proprietary SAP data products, AI agents are not only equipped with data access but are also grounded in real business logic. The wiki highlights the difference between simple data access and true mastery of logic, using practical examples such as automated variance analyses, proactive recommendations for action, and demand forecasting to explain what AI agents can truly accomplish in an SAP context—and how Mosaic AI transforms the BDC from a data platform into a true AI factory.
Wiki 11 — FinOps: The Art of Maximizing Value from Cloud-Based Costs
A Lakehouse initiative can only be sustainable if cloud costs are kept under control. FinOps—Financial Operations—is the discipline that balances technical capabilities with economic rationality. This wiki explains what FinOps means and why it is indispensable in the cloud era, highlights the key cost drivers in Databricks environments (compute, storage, data transfer, governance), and demonstrates how zero-copy delta sharing structurally reduces infrastructure costs by eliminating redundant data copies. Specific FinOps practices for SAP BDC—from workload optimization and intelligent cluster sizing to consistent tagging strategies—help ensure that every cloud expenditure contributes to measurable value creation. The guiding principle: Don’t save just for the sake of saving; instead, maximize the value per invested franc.
Ready for take-off?
You've learned about all the components of the Lakehouse mission. Now is the time to launch your own data strategy. Get in touch now >
Andreas & Yvonne's Databricks-Guide
Would you like all the important information at a glance?
Download the free guide to SAP Databricks now!
Your data strategy is individual - your consulting should be too
The introduction of zero-copy delta sharing is more than just a technical implementation—it is a strategic step toward modernizing your data architecture. Whether and how you make the most of this technology depends on your specific system landscape, your security requirements, and your long-term goals for analytics and AI. There is no one-size-fits-all solution here.
Let's talk, with no obligation, about how Delta Sharing can revolutionize data exchange in your company and unlock the full potential of your SAP data. Contact us for a personal consultation.
