Apache Kafka is an open-source message broker project developed by the Apache Software Foundation written in Scala. The project aims to provide a high-throughput, low-latency platform capable of handling hundreds of megabytes of reads and writes per second from thousands of clients.

Following tutorial shows how to download and install Apache Kafka on Windows and perform a start/stop of the installed instance.

If you want to learn more about how to connect to Apache Kafka – head on over to the Spring Kafka tutorials page.

It is important to note that Kafka will not work without Apache ZooKeeper, which is essentially a distributed hierarchical key-value store. Like Kafka, ZooKeeper is a software project of the Apache Software Foundation. Kafka uses ZooKeeper for electing a controller, cluster membership, topic configuration, quotas and ACLs.

Note that for running Kafka and ZooKeeper, a Java Runtime Environment needs to be installed and configured (with JAVA_HOME correctly set). If you are not sure if Java is installed on your machine, open a console and execute the following command:

java -version

Download and Setup ZooKeeper

Head over to the Apache ZooKeeper download page and and click on the download link in the Download section. This will redirect to a mirror site, click on the suggested mirror link and from the index select the stable directory as shown below. Download the gzipped TAR file.

At the moment of writing this tutorial, the latest stable release was zookeeper-3.4.9.

Extract the archive that was downloaded in the previous step. The extracted root directory should contain a number of files and subdirectories as shown below. From now on we will refer to this directory as: [zookeeper_install_dir].

Follow the below steps in order to setup a minimal working ZooKeeper configuration:

1. Navigate to the ZooKeeper configuration directory located under [zookeeper_install_dir]/conf.
2. Copy the file zoo_sample.cfg and rename to zoo.cfg.
3. Open the newly created zoo.cfg in a text editor.
4. Find the “dataDir=/tmp/zookeeper” entry and change it to “dataDir=C:/temp/zookeeper”. Make sure to use forward slashes in the path name!
5. Next, set the ‘ZOOKEEPER_HOME’ and corresponding ‘PATH’ environment variables. Click the Windows Start button and then type “env” without quotes in the search box. Select the Edit environment variables for your account entry, this will open the environment variables window.
   • Add a new variable using “ZOOKEEPER_HOME” as name and “[zookeeper_install_dir]” as value. Click “OK” to to save.
   • Edit (or add if it doesn’t exist) the variable with name “PATH” and add “;%ZOOKEEPER_HOME%/bin” to the end of the value. Click “OK” to save.

Now that ZooKeeper is configured, let’s go ahead and start it. Open a command prompt by clicking on the Windows Start button and typing “cmd” followed by pressing “ENTER”. Use the following command to startup ZooKeeper:

zkserver

By default, ZooKeeper will generate a number of log statements at start-up as shown below. One of the log entries will mention ‘binding to port 0.0.0.0/0.0.0.0:2181’. This indicates that ZooKeeper was successfully started.

Download and Setup Kafka

Open the Kafka releases page which contains the latest binary downloads. Kafka is written in Scala, which is a programming language that has full support for functional programming. Scala source code is intended to be compiled to Java bytecode so that the resulting executable code runs on a Java virtual machine.

You’ll notice that the release page contains multiple versions of Scala for a specific Kafka release. This only matters if you are using Scala yourself. If not the case, go ahead and choose the highest supported version.

At the moment of writing the latest stable release was kafka_2.11-0.10.0.1.tgz.

Extract the gzipped TAR file, downloaded in the previous step. The extracted root directory should contain a number of files and subdirectories as shown below. From now on we will refer to this directory as: [kafka_install_dir].

Make sure to extract to a directory path that does not contain spaces.

Follow the below steps in order to setup a minimal working Kakfa configuration:

1. Navigate to the Kafka configuration directory located under [kafka_install_dir]/config.
2. Edit the file server.properties in a text editor.
3. Find the “log.dirs=/tmp/kafka-logs” entry and change it to “log.dirs=C:/temp/kafka-logs”. Make sure to use forward slashes in the path name!

Make sure Zookeeper is up and running before starting Kafka.

In order to start Kafka, open a command prompt by clicking on the Windows Start button and typing “cmd” followed by pressing “ENTER”. Navigate to the [kafka_install_dir]. Use following command to startup Kafka:

./bin/windows/kafka-server-start.bat ./config/server.properties

When starting, Kafka will generate a number of log statements as shown below. The last log entries will mention ‘[Kafka Server 01], started’. This means that a Kafka instance is up and running.

This concludes installing ZooKeeper and Kafka on Windows.

If you found this post helpful or have any questions or remarks, please leave a comment.