How does Broadcast Nested Loop Join work in Spark?
January 8, 2021How to merge multiple output files from MapReduce or Spark jobs to one?
January 13, 2021There are 2 ways to get the list of available brokers in a Kafka cluster. Both with the help of scripts from zookeeper.
Zookeeper manages the leader election and other coordination things for a Kafka cluster. So Zookeeper has a list of all the Kafka brokers in the cluster.
Using zookeeper-shell.sh
zookeeper-shell.sh <zookeeper-host>:2181 ls /brokers/ids
You would see something like below
WATCHER:: WatchedEvent state:SyncConnected type:None path:null [0, 1, 2]
This means that you have 3 alive brokers – 0, 1 and 2
Using zkCli.sh
Use zkCli.sh to login to Zookeeper client
zkCli.sh -server <zookeeper-host>:2181
Below will list all the available brokers in Kafka cluster
ls /brokers/ids