$ kafkacat -b broker:29092 -t mytestopic -C -e -q| wc -l
3
What is Kafka Connect?
Kafka Connect is the integration API for Apache Kafka. Check out this video for an overview of what Kafka Connect enables you to do, and how to do it.
Counting the number of messages in a Kafka topic
There’s ways, and then there’s ways, to count the number of records/events/messages in a Kafka topic. Most of them are potentially inaccurate, or inefficient, or both. Here’s one that falls into the potentially inefficient category, using kafkacat
to read all the messages and pipe to wc
which with the -l
will tell you how many lines there are, and since each message is a line, how many messages you have in the Kafka topic:
Poking around the search engines in Google Chrome
Google Chrome automagically adds sites that you visit which support searching to a list of custom search engines. For each one you can set a keyword which activates it, so based on the above list if I want to search Amazon I can just type a
<tab>
and then my search term

🤖Building a Telegram bot with Apache Kafka, Go, and ksqlDB
I had the pleasure of presenting at DataEngBytes recently, and am delighted to share with you the 🗒️ slides, 👾 code, and 🎥 recording of my ✨brand new talk✨:
Telegram bot - BOT_COMMAND_INVALID
A tiny snippet since I wasted 10 minutes going around the houses on this one…
tl;dr: If you try to create a command that is not in lower case (e.g. Alert
not alert
) then the setMyCommands
API will return BOT_COMMAND_INVALID
Learning Golang (some rough notes) - S02E09 - Processing chunked responses before EOF is reached
The server sends Transfer-Encoding: chunked
data, and you want to work with the data as you get it, instead of waiting for the server to finish, the EOF to fire, and then process the data?
Learning Golang (some rough notes) - S02E08 - Checking Kafka advertised.listeners with Go
Learning Golang (some rough notes) - S02E07 - Splitting Go code into separate source files and building a binary executable
Learning Golang (some rough notes) - S02E06 - Putting the Producer in a function and handling errors in a Go routine
When I set out to learn Go one of the aims I had in mind was to write a version of this little Python utility which accompanies a blog I wrote recently about understanding and diagnosing problems with Kafka advertised listeners. Having successfully got Producer, Consumer, and AdminClient API examples working, it is now time to turn to that task.