rmoff's random ramblings
about talks

goldengate

Dec 12, 2018
Dec 12, 2018

Streaming data from Oracle into Kafka

This is a short summary discussing what the options are for integrating Oracle RDBMS into Kafka, as of December 2018 (refreshed June 2020). For a more detailed background to why and how at a broader level for all databases (not just Oracle) see this blog and this talk. What techniques & tools are there? Franck Pachot has written up an excellent analysis of the options available here. As of June 2020, this is what the line-up looks like:
Feb 1, 2018
Feb 1, 2018

HOWTO: Oracle GoldenGate + Apache Kafka + Schema Registry + Swingbench

This is the detailed step-by-step if you want to recreate the process I describe in the Confluent blog here I used Oracle’s Oracle Developer Days VM, which comes preinstalled with Oracle 12cR2. You can see the notes on how to do this here. These notes take you through installing and configuring: Swingbench, to create a sample “Order Entry” schema and simulate events on the Oracle database Oracle GoldenGate (OGG, forthwith) and Oracle GoldenGate for Big Data (OGG-BD, forthwith) I’m using Oracle GoldenGate 12.
Nov 21, 2017
Nov 21, 2017

Installing Oracle GoldenGate for Big Data 12.3.1 with Kafka Connect and Confluent Platform

Some notes that I made on installing and configuring Oracle GoldenGate with Confluent Platform. Excuse the brevity, but hopefully useful to share! I used the Oracle Developer Days VM for this - it’s preinstalled with Oracle 12cR2. Big Data Lite is nice but currently has an older version of GoldenGate. Login to the VM (oracle/oracle) and then install some useful things: sudo rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm sudo yum install -y screen htop collectl rlwrap p7zip unzip sysstat perf iotop sudo su - cd /etc/yum.
Sep 12, 2017
Sep 12, 2017

Oracle GoldenGate / Kafka Connect Handler troubleshooting

The Replicat was kapput: GGSCI (localhost.localdomain) 3> info rkconnoe REPLICAT RKCONNOE Last Started 2017-09-12 17:06 Status ABENDED Checkpoint Lag 00:00:00 (updated 00:46:34 ago) Log Read Checkpoint File /u01/app/ogg/dirdat/oe000000 First Record RBA 0 So checking the OGG error log ggserr.log showed 2017-09-12T17:06:17.572-0400 ERROR OGG-15051 Oracle GoldenGate Delivery, rkconnoe.prm: Java or JNI exception: oracle.goldengate.util.GGException: Error detected handling operation added event. 2017-09-12T17:06:17.572-0400 ERROR OGG-01668 Oracle GoldenGate Delivery, rkconnoe.prm: PROCESS ABENDING. So checking the replicat log dirrpt/RKCONNOE_info_log4j.
Jul 29, 2016
Jul 29, 2016

OGG-15051 oracle.goldengate.util.GGException: Class not found: “kafkahandler”

Similar to the previous issue, the sample config in the docs causes another snafu: OGG-15051 Java or JNI exception: oracle.goldengate.util.GGException: Class not found: "kafkahandler". kafkahandler Class not found: "kafkahandler". kafkahandler This time it’s in the kafka.props file: gg.handler.kafkahandler.Type = kafka Should be gg.handler.kafkahandler.type = kafka No capital T in Type! (Image credit: https://unsplash.com/@vanschneider)
Jul 28, 2016
Jul 28, 2016

OGG - Class not found: “com.company.kafka.CustomProducerRecord”

In the documentation for the current release of Oracle GoldenGate for Big Data (12.2.0.1.1.011) there’s a helpful sample configuration, which isn’t so helpful … [...] gg.handler.kafkahandler.ProducerRecordClass = com.company.kafka.CustomProducerRecord [...] This value for gg.handler.kafkahandler.ProducerRecordClass will cause a failure when you start the replicat: [...] Class not found: "com.company.kafka.CustomProducerRecord" [...] If you comment this configuration item out, it’ll use the default (oracle.goldengate.handler.kafka.DefaultProducerRecord) and work swimingly! (Image credit: https://unsplash.com/@vanschneider)
Apr 14, 2016
Apr 14, 2016

Streaming Data through Oracle GoldenGate to Elasticsearch

Recently added to the oracledi project over at java.net is an adaptor enabling Oracle GoldenGate (OGG) to send data to Elasticsearch. This adds a powerful alternative to [micro-]batch extract via JDBC from Oracle to Elasticsearch, which I wrote about recently over at the Elastic blog. Elasticsearch is a ‘document store’ widely used for both search and analytics. It’s something I’ve written a lot about (here and here for archives), as well as spoken about - preaching the good word, as it were, since the Elastic stack as a whole is very very good at what it does and a pleasure to work with.
Mar 16, 2016
Mar 16, 2016

Fun and Games with Oracle GoldenGate, Kafka, and Logstash on BigDataLite 4.4

The Oracle by Example (ObE) here demonstrating how to use Goldengate to replicate transactions big data targets such as HDFS is written for the BigDataLite 4.2.1, and for me didn’t work on the current latest version, 4.4.0. The OBE (and similar Hands On Lab PDF) assume the presence of pmov.prm and pmov.properties in /u01/ogg/dirprm/. On BDL 4.4 there’s only the extract to from Oracle configuration, emov. Fortunately it’s still possible to run this setup out of the box in BDL 4.

Robin Moffatt

Robin Moffatt works on the DevRel team at Confluent. He likes writing about himself in the third person, eating good breakfasts, and drinking good beer.

Story logo

© 2025