.NET Core is gaining popularity as it has the power of the original .NET Framework running in multiple platforms. Kafka is rapidly becoming the standard messaging technology in software industry. This post is a simple introduction on how to give the first steps with .NET Core and Kafka.

.NET Core Confluent Kafka driver makes use of the famous librdkafka, written in C++, which is the base of many Kafka drivers for non-JVM programming languages like C++, C#, Python and Node.

In this post we will create two simple console applications in .NET Core, one for the producer and another one for the consumer.

For that, make sure to have .NET Core SDK installed in your environment. For OSX users:

brew tap caskroom/cask
brew cask install dotnet

Creating the producer application

mkdir KafkaProducer
cd KafkaProducer
dotnet new console

This will create the project file (KafkaProducer.csproj) and a Program.cs file containing a sample Hello World application.

The next step is to add Confluent.Kafka NuGet package.

dotnet add package Confluent.Kafka

Now let’s create a producer code that allows one to write some messages that are written to a Kafka topic.

Creating the consumer application

The consumer application will then pick the messages from the same topic and write them to console output. The process to create the consumer application is also very simple.

mkdir KafkaConsumer
cd KafkaConsumer
dotnet new console
dotnet add package Confluent.Kafka

And the application code is written as:

Testing everything together

We can use Docker to facilitate the bootstrap of a Kafka cluster locally and create a topic named hello-topic.

Start the cluster

docker-compose up -d

In the producer project

dotnet build -c Release
dotnet bin/Release/netcoreapp2.0/KafkaProducer.dll

The build might show some warnings about librdkafka not being targeted for .NET Core 2.0, but they are actually a false-positive, please refer to the explanation here.

In the consumer project

dotnet build -c Release
dotnet bin/Release/netcoreapp2.0/KafkaConsumer.dll

You should now be able to write some messages in the producer side and see them in the consumer side.

Wrap-up

And this is it. The .NET Core Confluent.Kafka is built on top of powerful native features of the C# language to deliver a simple API. The Confluent.Kafka is a wrapper around LibRdKafka, which is actively maintained and very reliable for production use.