KOP - Kafka On Pulsar is an interesting project that adds Kafka capabilities to Pulsar. I did play around with KOP release prior to 0.3.0 and unfortunately it supported only Kafka protocol versions 0 to 2. This seemed to have [changed]https://web.archive.org/web/2/https://github.com/streamnative/kop/pull/176) with KOP version 0.3.0. I have not had a chance to try it out.

Nevertheless, to get KOP working is straight forward. On your broker nodes add the following to broker.conf. For listeners parameter substitute correct value for each node:

messagingProtocols=kafka
protocolHandlerDirectory=/opt/pulsar/protohandlers
allowAutoTopicCreationType=partitioned
listeners=PLAINTEXT://broker1.example.com:9092

Place KOP’s .nar file into /opt/pulsar/protohandlers. The directory does not matter, as long as Pulsar can access it. Do not forget to change ownership and/or permissions, if necessary. Follow up by restarting broker process. That’s it.

Now grab Kafka binaries and try testing the functionality by running producer…

[root@broker1 ~]# bin/kafka-console-producer.sh --broker-list broker1.example.com:9092 --topic kafkatest --producer-property security.protocol=PLAINTEXT

…and consumer:

[root@broker1 ~]# bin/kafka-console-consumer.sh --bootstrap-server broker1.example.com:9092 --topic kafkatest --from-beginning --consumer-property security.protocol=PLAINTEXT