The Replicat was kapput:

GGSCI (localhost.localdomain) 3> info rkconnoe

REPLICAT   RKCONNOE  Last Started 2017-09-12 17:06   Status ABENDED
Checkpoint Lag       00:00:00 (updated 00:46:34 ago)
Log Read Checkpoint  File /u01/app/ogg/dirdat/oe000000
                     First Record  RBA 0

So checking the OGG error log ggserr.log showed

2017-09-12T17:06:17.572-0400  ERROR   OGG-15051  Oracle GoldenGate Delivery, rkconnoe.prm:  Java or JNI exception:
                              oracle.goldengate.util.GGException: Error detected handling operation added event.
2017-09-12T17:06:17.572-0400  ERROR   OGG-01668  Oracle GoldenGate Delivery, rkconnoe.prm:  PROCESS ABENDING.

So checking the replicat log dirrpt/RKCONNOE_info_log4j.log showed:

tl;dr Make sure that key.converter.schema.registry.url and value.converter.schema.registry.url are specified, and that there are no trailing whitespaces.

---

I’ve been building on previous work I’ve done with Oracle GoldenGate and Kafka Connect, looking at how to have the change records from the Oracle database come through to Kafka in Avro format rather than the default JSON that the sample configuration gives.

Simply changing the Kafka Connect OGG configuration file (confluent.properties) from

value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter=org.apache.kafka.connect.json.JsonConverter

to

I’ve been doing some noodling around with Confluent’s Kafka Connect recently, as part of gaining a wider understanding into Kafka. If you’re not familiar with Kafka Connect this page gives a good idea of the thinking behind it.

One issue that I hit defeated my Google-fu so I’m recording it here to hopefully help out fellow n00bs.

The pipeline that I’d set up looked like this:

• Eneco’s Twitter Source streaming tweets to a Kafka topic
• Confluent’s HDFS Sink to stream tweets to HDFS and define Hive table automagically over them

It worked great, but only if I didn’t enable the Hive integration part. For me the integration with Hive to automatically define schemas was one of the key interests for this platform, so I wanted to see if I could get it to work. The error I got was

The Oracle by Example (ObE) here demonstrating how to use Goldengate to replicate transactions big data targets such as HDFS is written for the BigDataLite 4.2.1, and for me didn’t work on the current latest version, 4.4.0.

The OBE (and similar Hands On Lab PDF) assume the presence of pmov.prm and pmov.properties in /u01/ogg/dirprm/. On BDL 4.4 there’s only the extract to from Oracle configuration, emov.

Fortunately it’s still possible to run this setup out of the box in BDL 4.4, with bells on because it includes Kafka too. And, who doesn’t like a bit of Kafka nowadays?