SAP to Kafka in Two Wizards: How the Smart Gateway Turns Connector Building Into Form-Filling
Ricardo
Last updated: August 2026
Getting SAP data into Kafka usually means an RFC integration project on one end and a hand-built Kafka Connect deployment on the other. The Smart Gateway replaces both with two configuration wizards: one to pull data out of SAP, one to push it anywhere else.
Ask a data engineer what it takes to stand up a new Kafka connector and the answer is usually a small project: pick a connector class, write the JSON config, figure out authentication, decide on partitions and replication, test it against a topic that doesn't exist yet, and repeat all of that again for the next destination. Now put SAP on one end of that pipe, where the "source connector" isn't a JDBC driver but an RFC call into a production ERP, and the project gets bigger, not smaller.
The Onibex OneConnect Smart Gateway collapses both ends of that pipe into two guided wizards: one that opens a channel from SAP into Kafka, and one that opens a channel from Kafka into a destination platform. Neither one asks you to write a connector. Both are form-filling, not engineering.
The two directions, and why they're separate problems
An SAP-to-Kafka pipeline is really two connectors wearing one name: something has to get data out of SAP and into Kafka, and something else has to get it out of Kafka and into wherever it's actually used: Snowflake, Databricks, ClickHouse, or a dozen other platforms. Treating those as one monolithic integration is exactly what makes SAP projects slow, because the skills, the credentials, and the failure modes on each side are completely different.
The Smart Gateway keeps them separate on purpose:
- SAP → Kafka is handled by an SAP Link (also called an SAP Connector in the platform), configured once per SAP environment.
- Kafka → destination is handled by Kafka Connect, configured once per SAP Link, with one or more destination connectors added underneath it.
Both live in the same web console. Neither requires touching a Kafka Connect JSON config file or an ABAP report.
Wizard 1: getting data out of SAP
Creating an SAP Link is a short, four-part form, not a project. Name the connector (that name doubles as the topic prefix for everything it produces), point it at the SAP environment it belongs to, and set the RFC username and password it will use. One detail worth knowing before you fill that in: once created, that password can't be retrieved again, by you or by Onibex, so the wizard offers a one-click download of it as a .txt file.
The rest of the form is where the defaults do the real work. Connection details are stored as a reusable Secret, tested live in the wizard before you commit to anything downstream, and reused across every future SAP Link or Kafka Connect deployment without re-entering a credential. Topic settings, partitions, replicas, retention, are set once and inherited by every topic this link will ever produce. Advanced settings size the connector's own Kubernetes container.
Finish the form and the one artifact it exists to produce shows up: a URL. That goes into SAP transaction SM59 as an outbound HTTP RFC destination, and from that point SAP has a path to send data through with no inbound port, VPN tunnel, or firewall exception required.
(The field-by-field walkthrough of this wizard, screen by screen, is already documented step by step in the OneConnect documentation. What's above is the shape of what you're actually deciding, not a repeat of it.)
Wizard 2: getting data into somewhere useful
An SAP Link on its own is a Kafka producer with nowhere downstream configured yet. The second half of the setup, Kafka Connect plus one or more connectors underneath it, is what actually lands the data in a destination platform, and it's configured from inside the same SAP Link view.
Configuring Kafka Connect underneath the SAP Link is a three-field wizard: the Kafka secret, a replica count, and a storage replication factor. Both default to 1, which is fine for a proof of concept; raise replicas to at least 2 and the replication factor to 3 on a cluster with three or more brokers once this stops being one.
Adding a connector is its own short wizard: pick a destination type (Databricks, Snowflake, ClickHouse, SAP HANA, SQL Server, PostgreSQL, DB2, BigQuery, Amazon S3, Microsoft Fabric), name the instance, and fill in the destination's own connection fields (a JDBC URL and credentials, for something like Snowflake), the Kafka secret reused from earlier, and one decision that determines how much you'll ever touch this connector again: subscribe to an explicit list of topics, or to a regex. A real one, from a link named TestOnibex, looks like this:
TestOnibex.*(?<!_RAWDATA)(?<!_PROCESSING_ERRORS)(?<!_OUTBOUND_ERRORS)
That pattern picks up every topic the link produces, including ones that don't exist yet because nobody has activated that data product in SAP, while skipping the platform's own auxiliary error and raw-data topics. Choose the regex, and a new entity activated upstream in SAP shows up here with zero reconfiguration. Choose an explicit list, and it means reopening this wizard every time.
The behavioral settings underneath decide whether the destination genuinely mirrors SAP or just accumulates it: Insert Mode: UPSERT so records update in place instead of duplicating, Auto Create and Auto Evolve so new tables and columns appear without anyone writing DDL, Delete Enabled so a record deleted in SAP actually disappears downstream instead of sitting there stale. Confirm the summary and the connector starts streaming immediately, visible under the SAP Link's Connectors tab.
(Again, the click-by-click version of this wizard, with a screenshot for every field, lives in the OneConnect documentation. What's above is why each of those defaults is worth knowing before you click through them.)
Want a second destination for the same SAP data (Snowflake for finance, ClickHouse for an operational dashboard)? Run the wizard again against the same Kafka Connect deployment. Nothing about the SAP side changes, because SAP was never talking to Snowflake or ClickHouse in the first place. It was only ever talking to Kafka.
What "easy" is actually buying you
None of this is easy in the sense of being a toy. The Databricks, Snowflake, and ClickHouse connectors carry Confluent Gold Verification, Confluent's own certification for functional correctness across CDC operations, performance under load, and reliability during failures, and every one of them writes idempotently, meaning a redelivered Kafka message (a retry, a rebalance, a restart) never produces a duplicate row in the destination. Authentication into the destination runs on OAuth 2.0 with automatic token refresh, so there's no static credential sitting in a connector config waiting to be rotated by hand or leaked in a diff.
What the two wizards remove isn't rigor. It's the requirement that a human hand-write and hand-test the plumbing that provides that rigor:
| What you'd normally build yourself | What the wizard already handles |
|---|---|
| An RFC/API integration project to extract SAP data | A short form producing a ready RFC endpoint |
| A hand-configured Kafka Connect cluster | A three-field wizard with production-ready HA settings |
| Per-connector JSON configs, one per destination | A guided form, re-run per destination, reusing the same secret |
| Manually re-subscribing connectors as new topics appear | A topic regex that picks up new topics automatically |
| Custom DDL for every new SAP field reaching the warehouse | Auto Create / Auto Evolve, on by default |
| Deduplication logic for retried messages | Idempotent writes, guaranteed by Confluent Gold Verification |
Add it up and the honest comparison isn't "faster integration." It's that an SAP-to-Kafka-to-warehouse pipeline stops being a project with a start date and an end date, and becomes a form you fill out again the next time you need one more destination.
Frequently asked questions
Do I need a separate SAP Link for every destination platform? No. One SAP Link produces the Kafka topics; any number of connectors can be added underneath its Kafka Connect deployment, each pointed at a different destination, all reading from the same topics.
What happens if I add a new data product in SAP after the connector is already running? If the connector subscribes via a topics regex rather than an explicit list, the new topic is picked up automatically: no change to the connector configuration.
Can two SAP Links share the same Kafka cluster credentials? Yes. Connection details are stored as secrets, and a secret created for one SAP Link can be reused across other SAP Links, Kafka Connect deployments, and connectors without re-entering anything.
What stops a retried message from creating a duplicate row downstream? The Premium Kafka Connectors (Databricks, Snowflake, ClickHouse) guarantee idempotent writes as part of their Confluent Gold Verification: a message delivered twice produces the same end state as delivered once.
If you want to see how quickly this actually goes against your own SAP system (an SAP Link, a Kafka Connect deployment, and a live connector into a warehouse of your choice), Onibex runs the 15-Hour Business Value Challenge as a no-cost proof of concept.