A short post to help Googlers. On a single-node sandbox Apache Kafka / Confluent Platform installation, I was getting this error
WARN [AdminClient clientId=adminclient-3] Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
I had correctly set the machine's hostname in my Kafka
but in Schema Registry, Connect etc I had not, and so they were using the default (
AdminClient bit comes in because when they launch each creates its own internal topics.
Based on my
/etc/hosts we can see
localhost has a different IP from the hostname (
rmoff@proxmox01 ~> cat /etc/hosts 127.0.0.1 localhost.localdomain localhost 192.168.10.250 proxmox01.moffatt.me proxmox01
Thus, Kafka was listening on one IP (192.168.10.25):
rmoff@proxmox01 ~> sudo netstat -plnt|grep 9092 tcp6 0 0 192.168.10.250:9092 :::* LISTEN 30345/java
But Schema Registry and Kafka Connect were trying (and failing) to connect to it on another (
127.0.0.1). With the appropriate files fixed (
schema-registry.properties) all was well with the world!
I also hit a hostname/networking related error earlier in this process, which stopped Kafka launching entirely:
kafka.common.KafkaException: Socket server failed to bind to proxmox01.moffatt.me:9092: Cannot assign requested address.
Turns out my
/etc/hosts was fubar - it had the wrong IP address listed for the hostname. Instead of
192.168.10.250 proxmox01.moffatt.me proxmox01
192.168.1.250 proxmox01.moffatt.me proxmox01
which was wrong. Fixing this solved the problem.