SAP to Kafka Integration: How Onibex One Connect Streams SAP Data Without ABAP
Mariam Estrada
Last updated: August 2026
SAP to Kafka integration fails on the same obstacle every time: SAP splits one business entity across several tables, and SAP does not permit a direct database connection, so log based change data capture is ruled out. Onibex One Connect, a real-time SAP data streaming platform, listens to SAP business events at the business object level and writes Avro records into Confluent with the schema enforced at the producer.
The 4 key points:
- A single sales order lives in at least two tables, VBAK for the header and VBAP for the line items. Financial postings split across BKPF and BSEG, billing documents across VBRK and VBRP, material masters across MARA, MARC and MARD
- One Connect ships 150+ pre-packaged SAP data products covering the most common business objects, with the tables, join keys and exposed fields already mapped (Onibex, 2026)
- Extraction is triggered by SAP business events, which fire at business object level the moment the action posts. No scheduler, no polling, no ABAP
- The connection is outbound only, so no inbound firewall rules are opened on the SAP system
Why is SAP data so hard to stream in real time?
SAP is the system of record for financials, procurement, inventory and sales. The problem is not that the data is missing. It is that SAP was designed for transactions, not for analytics, and not for streaming.
A single sales order does not live in one place. VBAK holds the order header (customer, document date, total value) and VBAP holds the line items (materials, quantities, prices). To get a complete sales order as a meaningful business object, you need to know that relationship, write the join, and keep it in sync with any schema change SAP rolls out.
Multiply that across a full ERP. Each of those relationships carries business rules, dependent fields, and the custom extensions your team built years ago as Z tables. Two traditional approaches exist, and both leave you in the same place:
| Approach | Mechanism | Data age at read time | Breaks when |
|---|---|---|---|
| Batch ETL | Scheduled bulk pull into a warehouse | Up to a full cycle, commonly 18 hours | A real-time decision depends on last night's numbers |
| Custom ABAP extraction | Bespoke code pulling on a schedule or trigger | Minutes to hours | The ABAP developer leaves, or SAP releases a patch |
| Onibex One Connect | SAP business events at business object level, streamed to Kafka | Seconds | Requires Confluent as the destination cluster |
Stale data and brittle pipelines are the same outcome reached two different ways, and the maintenance cost of the second one is what keeps most teams from building anything with the data.
What is Onibex One Connect?
Onibex One Connect is an SAP-verified, real-time SAP data streaming platform. It sits between the SAP system and a Confluent cluster and handles the full path from raw SAP tables to schema-governed Kafka topics.
It has two components, each handling a different part of the problem. The One Connect Data Modeler is the modeling layer, where you define what your data should look like as a business object. The One Connect Smart Gateway is the delivery layer, where that modeled data gets serialized and streamed into Kafka.
SAP ECC / S/4HANA
|
Data Modeler <- Onibex Marketplace (150+ pre-packaged data products)
|
Smart Gateway -> Avro serialization + Schema Registry
|
Confluent Kafka topic
|
Any destination: Databricks, Snowflake, watsonx, Power BI
How does the Data Modeler turn raw tables into data products?
The Data Modeler is where you declare what you want. Instead of writing SQL joins or ABAP function modules, you work in a drag and drop interface to build data products: governed, stream-ready versions of your SAP business entities.
Business object and data product are related but distinct. A business object is SAP's internal representation of a real-world entity, structured as nodes and behaviors, living inside SAP. A data product is what One Connect produces from it: a structured, consumable version of that same entity, ready for streaming. One Connect bridges the two.
Starting with the Marketplace. One Connect ships with 150+ pre-packaged SAP data products covering sales orders, billing documents, purchase orders, material masters, customers and vendors. Select the billing document entity and it already knows that VBRK is the header, VBRP the line items, BKPF the accounting document header and BSEG the accounting line items, along with the join keys and the fields to expose. That mapping was pre-built following SAP best practices, and it streams the moment you activate it.
Adding your own tables. SAP systems at scale carry years of customization: Z tables, non-standard CDS views, or additional standard tables to append to an existing entity. The Data Modeler handles all three through the same interface. You click Insert Table, search by name, and the tool brings in the structure. From there you configure the join to the existing entity, select which columns travel downstream, and define filters at the SAP application layer.
That last point matters more than it looks. Filtering at the source rather than at the destination means you never move data you do not need across the network. For a table like BSEG, which can hold billions of rows in a mature SAP system, that is the difference between a pipeline that performs and one that never finishes loading.
CDS views. Core Data Services views are SAP's own abstraction over raw tables: named, reusable structures built into the ABAP stack that surface business-friendly field names and pre-computed relationships. One Connect treats them exactly like standard tables. If your SAP team has already defined a clean sales order structure in CDS, you reference the view instead of rebuilding it in the Modeler.
How does One Connect know when something changed in SAP?
Real-time streaming requires knowing when a record changed. In SAP, every meaningful business action, a sales order posted, a billing document created, a payment cleared, fires a business event. One Connect listens for business events natively, without polling and without custom ABAP.
Business events are the category. The framework that delivers them depends on your SAP version, and One Connect supports the full set:
| Framework | Where it applies | What it fires on |
|---|---|---|
| BOR (Business Object Repository) | ECC and hybrid environments | Business object state changes, for example EventDAct on the BILLING change document object |
| RAP (RESTful Application Programming Model) | S/4HANA | Native S/4HANA eventing |
| BTE, PPF, Change Pointers | Both, depending on the scenario | Transaction and post-processing events |
| BW Extractors | Both | Initial historical load |
The moment a billing document is created, updated or deleted in SAP, the event fires and One Connect streams the delta downstream. No scheduler. No polling.
What does the Smart Gateway do between SAP and Kafka?
The Smart Gateway is what actually moves the data once the Data Modeler has defined the entities and their triggering events. It handles serialization, schema management and physical delivery into Confluent, in four steps.
It receives the delta from SAP. When a business event fires, One Connect captures the changed record and passes it to the Smart Gateway. This covers inserts and updates.
It serializes to Avro. Rather than raw JSON or CSV, every record is serialized into Avro, a compact binary format that enforces a strict schema at write time. Every field has a defined type and every record is validated before it reaches Kafka. JSON is flexible in a way that causes problems at scale: a field is a string in one record and null in the next, a column gets renamed in SAP and the consumer finds out the hard way. Avro enforces the contract at the producer side instead.
It registers the schema. Before writing to Kafka, the Smart Gateway registers the schema for each data product in the Confluent Schema Registry, the centralized repository that lets downstream consumers know exactly what they are receiving. One Connect creates the schema on activation. You do not write schema definitions by hand or configure the registry yourself.
It creates the topic and writes the records. The Smart Gateway creates the Kafka topic for each data product automatically, named consistently from the entity. From that point every change to that SAP object writes a new Avro record to the topic, with the schema enforced by the registry.
What happens when SAP changes the schema?
Schema evolution is the most common failure point in a maintained SAP extraction. SAP releases patches, your team adds fields to custom tables, a CDS view gets extended, and in a traditional pipeline any of those can break downstream consumers.
One Connect handles evolution through the Schema Registry's compatibility rules. When a column is added to an entity, the Smart Gateway registers the updated schema as a new version. Consumers that have not updated keep reading with the previous version, and upgrade when they are ready. The registry stays the single source of truth.
Does connecting SAP to Kafka require opening firewall ports?
No. One Connect runs as a SaaS deployment on Azure, and the Smart Gateway communicates outbound from the SAP system to the Onibex Cloud, then onward to Confluent Cloud or Confluent Platform.
No inbound firewall rules are required on the SAP system. The Smart Gateway initiates the connection, so your security team does not open ports into a production SAP environment.
How does this map to the medallion architecture?
One Connect operationalizes the Bronze, Silver and Gold pattern specifically for SAP data.
Bronze is the raw extraction. Every SAP table feeding your data products is captured as it exists in the source: BKPF, BSEG, VBAK, VBAP, MARA, EKKO, plus your custom tables. No transformation, no aggregation.
Silver is the data product. Raw tables are joined, validated and assembled into governed entities: the sales order, the billing document, the material master. These are semantically meaningful objects that systems can reason about without knowing SAP's internal table structure.
Gold is whatever you build on top. Kafka topics flowing into Databricks for feature engineering, Snowflake for financial reporting, IBM watsonx for an agent answering supply chain questions, Power BI for a dashboard that refreshes as SAP records change.
The reason the layering matters for AI specifically: an agentic system is only as useful as the data it can reach, and how much of the meaning has already been resolved before it arrives. That resolution question is the subject of the semantic layer we built over these data products.
What separates One Connect from a generic SAP connector?
Four differences, stated as mechanisms rather than claims.
No ABAP required. The Data Modeler's drag and drop interface means the data engineering team configures extractions without touching SAP code, which removes the dependency on ABAP developer availability.
Real-time and batch from the same configuration. You load historical batch data to backfill topics, then activate streaming to capture deltas going forward.
Data products rather than table access. Most SAP connectors expose tables. One Connect exposes business objects, so a sales order arrives already understanding the relationship between VBAK and VBAP.
Automatic schema management. Avro schemas and registry entries are created on the fly for every entity activated.
One Connect is verified by SAP and certified in Confluent's Connect with Confluent program, and creates schema registries, topics, partitions and segments on the fly from a single integration.
Frequently asked questions
Can you connect Kafka directly to SAP?
Not through the database. SAP does not permit a direct database connection, which rules out log based change data capture, the mechanism most Kafka connectors rely on. The supported path is SAP business events fired at business object level, which is what One Connect consumes.
Do you need ABAP to stream SAP data to Kafka?
No. Extraction is configured in the One Connect Data Modeler through a drag and drop interface, including joins, column selection and source-side filters. No custom ABAP code is written or deployed.
What is the difference between a business object and a data product?
A business object is SAP's internal representation of a real-world entity, structured as nodes and behaviors inside SAP. A data product is the structured, consumable version produced from it, ready for streaming. One Connect bridges the two.
Does One Connect work with SAP ECC as well as S/4HANA?
Both. On ECC and hybrid environments business events are delivered through BOR, and on S/4HANA through RAP. BTE, PPF and Change Pointers cover additional scenarios on either.
How do you load historical SAP data before switching on streaming?
Through BW Extractors for the initial load, which backfills the Kafka topics, followed by business event streaming for every change after that. Both run against the same data product definition.
Mariam Estrada is a Tech Evangelist at Onibex, based in Austin, TX. Onibex builds real-time data streaming and agentic AI solutions for SAP environments.
If you want to see the path from an SAP table to a live Kafka topic on your own system, the 15-Hour Business Value Challenge is a no-cost proof of concept that connects a live SAP system and activates a set of data products within a business day.