Apache Kafka for Python Developers

https://www.youtube.com/playlist?list=PLjfRmoYoxpNrs0VmIq6mOTqXP52RfZdRf

Install and run Apache Kafka & integration with Python using Kafka-Python

$ pip install kafka-python

main.py

from time import sleep
from json import dumps
from kafka import KafkaProducer

topic_name='hello_world'
producer = KafkaProducer(bootstrap_servers=['localhost:9092'],value_serializer=lambda x: dumps(x).encode('utf-8'))

for e in range(1000):
    data = {'number' : e}
    print(data)
    producer.send(topic_name, value=data)
    sleep(5)

// С первого раза (наверное) будет ошибка
$ python main.py

Multiple Producer & Multiple Consumer in a Kafka Topic

Broker Cluster and Zookeeper in Kafka

Topics, partitions, and offsets in Kafka

Kafka Cluster with Multiple Brokers

Topic with Replication in Multiple Broker Kafka Cluster