Streaming Data from MySQL into Kafka with Kafka Connect and Debezium
Debezium is a CDC tool that can stream changes from MySQL, MongoDB, and PostgreSQL into Kafka, using Kafka Connect. In this article we’ll see …
Streaming data from Kafka into Elasticsearch
This article is part of a series exploring Streaming ETL in practice. You can read about setting up the ingest of realtime events from a standard …
Oracle GoldenGate / Kafka Connect Handler troubleshooting
The Replicat was kapput: GGSCI (localhost.localdomain) 3> info rkconnoe REPLICAT RKCONNOE Last Started 2017-09-12 17:06 Status ABENDED Checkpoint …
Kafka Connect - JsonDeserializer with schemas.enable requires “schema” and “payload” fields
An error that I see coming up frequently in the Kafka Connect community (e.g. mailing list, Slack group, StackOverflow) is: JsonDeserializer with …
Configuring Kafka Connect to log REST HTTP messages to a separate file
Kafka’s Connect API is a wondrous way of easily bringing data in and out of Apache Kafka without having to write a line of code. By choosing a …
Oracle GoldenGate -> Kafka Connect - “Failed to serialize Avro data”
tl;dr Make sure that key.converter.schema.registry.url and value.converter.schema.registry.url are specified, and that there are no trailing …
Kafka Connect - java.lang.IncompatibleClassChangeError
I hit this error running Kafka Connect HDFS connector from Confluent Platform v3.1.1 on BigDataLite 4.6: [oracle@bigdatalite ~]$ connect-standalone …
Kafka Connect JDBC - Oracle - Number of groups must be positive
There are various reasons for this error, but the one I hit was that the table name is case sensitive, and returned from Oracle by the JDBC driver in …
Kafka Connect - HDFS with Hive Integration - SchemaProjectorException - Schema version required
I’ve been doing some noodling around with Confluent’s Kafka Connect recently, as part of gaining a wider understanding into Kafka. If …