Debezium & MySQL v8 : Public Key Retrieval Is Not Allowed

I started hitting problems when trying Debezium against MySQL v8. When creating the connector:

curl -i -X PUT -H "Accept:application/json" \
    -H  "Content-Type:application/json" http://localhost:8083/connectors/source-debezium-mysql-00/config \
    -d '{
          "connector.class": "io.debezium.connector.mysql.MySqlConnector",
          "database.hostname": "mysql",
          "database.port": "3306",
          "database.user": "debezium",
          "database.password": "dbz",
          "database.server.id": "42",
          "database.server.name": "asgard",
          "table.whitelist": "demo.customers",
          "database.history.kafka.bootstrap.servers": "kafka:29092",
          "database.history.kafka.topic": "asgard.dbhistory.demo" ,
          "include.schema.changes": "true"
    }'

I’d get the error

{
    "error_code": 400,
    "message": "Connector configuration is invalid and contains the following 1 error(s):\nUnable to connect: Public Key Retrieval is not allowed\nYou can also find the above list of errors at the endpoint `/{connectorType}/config/validate`"
}

The fix for this, courtesy of Jiri on the Debezium gitter.im chat room, is to add to the connector configuraton database.allowPublicKeyRetrieval=true:

curl -i -X PUT -H "Accept:application/json" \
    -H  "Content-Type:application/json" http://localhost:8083/connectors/source-debezium-mysql-00/config \
    -d '{
          "connector.class": "io.debezium.connector.mysql.MySqlConnector",
          "database.hostname": "mysql",
          "database.port": "3306",
          "database.user": "debezium",
          "database.password": "dbz",
          "database.server.id": "42",
          "database.allowPublicKeyRetrieval":"true",
          "database.server.name": "asgard",
          "table.whitelist": "demo.customers",
          "database.history.kafka.bootstrap.servers": "kafka:29092",
          "database.history.kafka.topic": "asgard.dbhistory.demo" ,
          "include.schema.changes": "true"
    }'

After this the connector was created successfully, but immediately FAILED with the following error in the Kafka Connect worker log:

 
org.apache.kafka.connect.errors.ConnectException: Failed to authenticate to the MySQL database at mysql:3306 with user 'debezium'
[…]
com.github.shyiko.mysql.binlog.network.AuthenticationException: Client does not support authentication protocol requested by server; consider upgrading MySQL client

A bit of Googling threw up this issue on GitHub with a solution that worked—add WITH mysql_native_password to the authentication settings for the Debezium user:

ALTER USER 'debezium'@'%' IDENTIFIED WITH mysql_native_password BY 'dbz';

After that, the connector ran successfully 👍

Robin Moffatt

Robin Moffatt works on the DevRel team at Confluent. He likes writing about himself in the third person, eating good breakfasts, and drinking good beer.