Dec 17, 2018

Pull new version of multiple Docker images

Tiny little snippet this one. Given a list of images:

$ docker images|grep confluent
confluentinc/cp-enterprise-kafka                5.0.0               d0c5528d7f99        3 months ago        600MB
confluentinc/cp-kafka                           5.0.0               373a4e31e02e        3 months ago        558MB
confluentinc/cp-zookeeper                       5.0.0               3cab14034c43        3 months ago        558MB
confluentinc/cp-ksql-server                     5.0.0               691bc3c1991f        4 months ago        493MB
confluentinc/cp-ksql-cli                        5.0.0               e521f3e787d6        4 months ago        488MB
…

Now there’s a new version available, and you want to pull down all the latest ones for it:

docker images|grep "^confluentinc"|awk '{print $1}'|xargs -Ifoo docker pull foo:5.1.0

Dec 15, 2018

Docker Tips and Tricks with Kafka Connect, ksqlDB, and Kafka

A few years ago a colleague of mine told me about this thing called Docker, and I must admit I dismissed it as a fad…how wrong was I. Docker, and Docker Compose, are one of my key tools of the trade. With them I can build self-contained environments for tutorials, demos, conference talks etc. Tear it down, run it again, without worrying that somewhere a local config changed and will break things.

Dec 12, 2018

Streaming data from Oracle into Kafka

This is a short summary discussing what the options are for integrating Oracle RDBMS into Kafka, as of December 2018 (refreshed June 2020). For a more detailed background to why and how at a broader level for all databases (not just Oracle) see this blog and this talk. What techniques & tools are there? Franck Pachot has written up an excellent analysis of the options available here. As of June 2020, this is what the line-up looks like:

Dec 11, 2018

Tools I Use: iPad Pro

I’ve written recently about how I create the diagrams in my blog posts and talks, and from discussions around that, a couple of people were interested more broadly in how I use my iPad Pro. So, on the basis that if two people are interested maybe others are (and if no-one else is, I have a copy-and-paste answer to give to those two people) here we go. Kit iPad Pro 10.

Dec 10, 2018

So how DO you make those cool diagrams?

I write and speak lots about Kafka, and get a fair few questions from this. The most common question is actually nothing to do with Kafka, but instead: How do you make those cool diagrams? So here’s a short, and longer, answer! Update July 2019 I’ve moved away from Paper -> read more here tl;dr An iOS app called Paper, from a company called FiftyThree So, how DO you make those cool diagrams?

Dec 8, 2018

Get mtr working on the Mac

Install Not sure why the brew doesn’t work as it used to, but here’s how to get it working: brew install mtr sudo ln /usr/local/Cellar/mtr/0.92/sbin/mtr /usr/local/bin/mtr sudo ln /usr/local/Cellar/mtr/0.92/sbin/mtr-packet /usr/local/bin/mtr-packet (If you don’t do the two symbolic links (ln) you’ll get mtr: command not found or mtr: Failure to start mtr-packet: Invalid argument) Run sudo mtr google.com

Dec 3, 2018

Kafka Connect CLI tricks

I do lots of work with Kafka Connect, almost entirely in Distributed mode—even just with 1 node -> makes scaling out much easier when/if needed. Because I’m using Distributed mode, I use the Kafka Connect REST API to configure and manage it. Whilst others might use GUI REST tools like Postman etc, I tend to just use the commandline. Here are some useful snippets that I use all the time.

Nov 20, 2018