Nov 19, 2024
Nov 19, 2024

Streaming Data from Postgres to Snowflake with CDC and Decodable

Note
 This post originally appeared on the Decodable blog.

In my last blog post I looked at why you might need CDC. In this post I’m going to put it into practice with probably the most common use case—extracting data from an operational transactional database to store somewhere else for analytics. I’m going to show Postgres to Snowflake, but the pattern is the same for pretty much any combination, such as MySQL to BigQuery, SQL Server to Redshift, and so on.

Nov 20, 2019
Nov 20, 2019

Streaming data from SQL Server to Kafka to Snowflake ❄️ with Kafka Connect

Snowflake is the data warehouse built for the cloud, so let’s get all ☁️ cloudy and stream some data from Kafka running in Confluent Cloud to Snowflake! What I’m showing also works just as well for an on-premises Kafka cluster. I’m using SQL Server as an example data source, with Debezium to capture and stream and changes from it into Kafka. I’m assuming that you’ve signed up for Confluent Cloud and Snowflake and are the proud owner of credentials for both.