š¤Building a Telegram bot with Apache Kafka, Go, and ksqlDB
I had the pleasure of presenting at DataEngBytes recently, and am delighted to share with you the šļø slides, š¾ code, and š„ recording of my āØbrand new talkāØ:
Learning Golang (some rough notes) - S02E09 - Processing chunked responses before EOF is reached
The server sends Transfer-Encoding: chunked data, and you want to work with the data as you get it, instead of waiting for the server to finish, the EOF to fire, and then process the data?
Learning Golang (some rough notes) - S02E08 - Checking Kafka advertised.listeners with Go
At the beginning of all this my aim was to learn something new (Go), and use it to write a version of a utility that Iād previously hacked together in Python that checks your Apache Kafka broker configuration for possible problems with the infamous advertised.listeners setting. Check out a blog that I wrote which explains all about Apache Kafka and listener configuration.
You can find the code at https://github.com/rmoff/kafka-listeners Itās been a fun journey, and now I am pleased to be able to show the results of it.
Learning Golang (some rough notes) - S02E07 - Splitting Go code into separate source files and building a binary executable
So far Iāve been running all my code either in the Go Tour sandbox, using Go Playground, or from a single file in VS Code. My explorations in the previous article ended up with a a source file that was starting to get a little bit unwieldily, so letās take a look at how that can be improved.
Within my most recent code, I have the main function and the doProduce function, which is fine when collapsed down:
Learning Golang (some rough notes) - S02E06 - Putting the Producer in a function and handling errors in a Go routine
When I set out to learn Go one of the aims I had in mind was to write a version of this little Python utility which accompanies a blog I wrote recently about understanding and diagnosing problems with Kafka advertised listeners. Having successfully got Producer, Consumer, and AdminClient API examples working, it is now time to turn to that task.
Learning Golang (some rough notes) - S02E05 - Kafka Go AdminClient
Learning Golang (some rough notes) - S02E04 - Kafka Go Consumer (Function-based)
Last time I looked at creating my first Apache Kafka consumer in Go, which used the now-deprecated channel-based consumer. Whilst idiomatic for Go, it has some issues which mean that the function-based consumer is recommended for use instead. So letās go and use it!
Instead of reading from the Events() channel of the consumer, we read events using the Poll() function with a timeout. The way we handle events (a switch based on their type) is the same:
Learning Golang (some rough notes) - S02E03 - Kafka Go Consumer (Channel-based)
Having written my first Kafka producer in Go, and even added error handling to it, the next step was to write a consumer. It follows closely the pattern of Producer code I finished up with previously, using the channel-based approach for the Consumer:
Learning Golang (some rough notes) - S02E02 - Adding error handling to the Producer
I looked last time at the very bare basics of writing a Kafka producer using Go. It worked, but only with everything lined up and pointing the right way. There was no error handling of any sorts. Letās see about fixing this now.
Learning Golang (some rough notes) - S02E00 - Kafka and Go
With the first leg of my journey with Go done (starting from a very rudimentary base), the next step for me was to bring it into my current area of interest and work - Apache Kafka.
Learning Golang (some rough notes) - S01E10 - Concurrency (Web Crawler)
In the previous exercise I felt my absence of a formal CompSci background with the introduction of Binary Sorted Trees, and now I am concious of it again with learning about mutex. Iād heard of them before, mostly when Oracle performance folk were talking about wait types - TIL it stands for mutual exclusion!
Learning Golang (some rough notes) - S01E09 - Concurrency (Channels, Goroutines)
A Tour of Go : Goroutines was OK but as with some previous material I headed over to Go by example for clearer explanations.
Learning Golang (some rough notes) - S01E08 - Images
This is based on the Picture generator from the Slices exercise.
Learning Golang (some rough notes) - S01E07 - Readers
Iām not intending to pick holes in the Tourā¦but itās not helping itself ;-)
For an introductory text, it makes a ton of assumptions about the user. Here it introduces Readers, and the explanation is goodābut the example code looks like this:
Learning Golang (some rough notes) - S01E06 - Errors
Like Interfaces, the Tour didnāt really do it for me on Errors either. Too absract, and not enough explanation of the code examples for my liking. It also doesnāt cover the errors package which other tutorial do. Iām not clear if thatās because the errors package isnāt used much, or the Tour focusses only on teaching the raw basics.
Learning Golang (some rough notes) - S01E05 - Interfaces
This page really threw me, for several reasons:
-
The text notes that thereās an error (so why donāt they fix it?)
-
The provided code doesnāt run (presumably because of the above error)
-
Itās not clear if this is a deliberate error to illustrate a point, or just a snafu
Learning Golang (some rough notes) - S01E04 - Function Closures
So far the Tour has been š¤ and š§ and even š¤Ø but function closures had me š¤Æ ā¦
Each of the words on the page made sense but strung together in a sentence didnāt really make any sense to me.
Learning Golang (some rough notes) - S01E03 - Maps
Implement WordCount
This is probably bread-and-butter for any seasoned programmer, but I enjoyed the simple process and satisfaction of breaking the problem down into steps to solve using what the tutorial had just covered. Sketching out the logic in pseudo-code first, I figured that I wanted to do this:
Learning Golang (some rough notes) - S01E02 - Slices
Slices made sense, until I got to Slice length and capacity. Two bits puzzled me in this code:
