Why Do We Need Streaming ETL?
(This is an expanded version of the intro to an article I posted over on the Confluent blog. Here I get to be as verbose as I like ;)) My first job …
HOWTO: Oracle GoldenGate + Apache Kafka + Schema Registry + Swingbench
This is the detailed step-by-step if you want to recreate the process I describe in the Confluent blog here I used Oracle’s Oracle Developer …
Kafka - AdminClient - Connection to node -1 could not be established. Broker may not be available
See also Kafka Listeners - Explained A short post to help Googlers. On a single-node sandbox Apache Kafka / Confluent Platform installation, I was …
Installing Oracle GoldenGate for Big Data 12.3.1 with Kafka Connect and Confluent Platform
Some notes that I made on installing and configuring Oracle GoldenGate with Confluent Platform. Excuse the brevity, but hopefully useful to share! I …
Apache Kafka™ talks at Oracle OpenWorld, JavaOne, and Oak Table World 2017
There’s an impressive 19 sessions that cover Apache Kafka™ at Oracle OpenWorld, JavaOne, and Oak Table World this year! You can find the full …
Configuring Kafka Connect to log REST HTTP messages to a separate file
Kafka’s Connect API is a wondrous way of easily bringing data in and out of Apache Kafka without having to write a line of code. By choosing a …
kafka.common.KafkaException: No key found on line 1
A very silly PEBCAK problem this one, but Google hits weren’t so helpful so here goes. Running a console producer, specifying keys: …
kafka-avro-console-producer - Error registering Avro schema / io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException
By default, the kafka-avro-console-producer will assume that the schema registry is on port 8081, and happily connect to it. Unfortunately, this can …
Kafka Connect - java.lang.IncompatibleClassChangeError
I hit this error running Kafka Connect HDFS connector from Confluent Platform v3.1.1 on BigDataLite 4.6: [oracle@bigdatalite ~]$ connect-standalone …
OGG-15051 oracle.goldengate.util.GGException: Class not found: “kafkahandler”
Similar to the previous issue, the sample config in the docs causes another snafu: OGG-15051 Java or JNI exception: …
OGG - Class not found: “com.company.kafka.CustomProducerRecord”
In the documentation for the current release of Oracle GoldenGate for Big Data (12.2.0.1.1.011) there’s a helpful sample configuration, which …
Kafka Connect JDBC - Oracle - Number of groups must be positive
There are various reasons for this error, but the one I hit was that the table name is case sensitive, and returned from Oracle by the JDBC driver in …
Kafka Connect - HDFS with Hive Integration - SchemaProjectorException - Schema version required
I’ve been doing some noodling around with Confluent’s Kafka Connect recently, as part of gaining a wider understanding into Kafka. If …
Streaming Data through Oracle GoldenGate to Elasticsearch
Recently added to the oracledi project over at java.net is an adaptor enabling Oracle GoldenGate (OGG) to send data to Elasticsearch. This adds a …
Decoupling the Data Pipeline with Kafka - A (Very) Simple Real Life Example
I’ve recently been playing around with the ELK stack (now officially known as the Elastic stack) collecting data from an IRC channel with …
Fun and Games with Oracle GoldenGate, Kafka, and Logstash on BigDataLite 4.4
The Oracle by Example (ObE) here demonstrating how to use Goldengate to replicate transactions big data targets such as HDFS is written for the …