Kafka Connect CLI tricks
ERROR: Invalid interpolation format for “command” option in service…
Doing some funky Docker Compose stuff, including:
Flatten CDC records in KSQL
The problem - nested messages in Kafka
Data comes into Kafka in many shapes and sizes. Sometimes it’s from CDC tools, and may be nested like this:
Exploring JMX with jmxterm
- Check out the jmxterm repository
- Download jmxterm from https://docs.cyclopsgroup.org/jmxterm
Accessing Kafka Docker containers’ JMX from host
Sending multiline messages to Kafka
Window Timestamps in KSQL / Integration with Elasticsearch
Where I’m speaking in the rest of 2018
Kafka Listeners - Explained
(This was cross-posted on the Confluent.io blog)
This question comes up on StackOverflow and such places a lot, so here’s something to try and help.
tl;dr : You need to set advertised.listeners
(or KAFKA_ADVERTISED_LISTENERS
if you’re using Docker images) to the external address (host/IP) so that clients can correctly connect to it. Otherwise they’ll try to connect to the internal host address–and if that’s not reachable then problems ensue.
Put another way, courtesy of Spencer Ruport:
LISTENERS
are what interfaces Kafka binds to.ADVERTISED_LISTENERS
are how clients can connect.