Listing Kafka Topics – 列出Kafka主题

最后修改: 2020年 8月 28日

中文/混合/英文(键盘快捷键：t)

1. Overview

1.概述

In this quick tutorial, we’ll learn how we can list all the topics in an Apache Kafka cluster.

在这个快速教程中，我们将学习如何列出Apache Kafka集群中的所有主题。

First, we’ll set up a single-node Apache Kafka and Zookeeper cluster. Then we’ll ask that cluster about its topics.

首先，我们将建立一个单节点的Apache Kafka和Zookeeper集群。然后，我们将询问该集群的主题。

2. Setting Up Kafka

2.设置Kafka

Before listing all the topics in a Kafka cluster, let’s set up a test single-node Kafka cluster in three steps:

在列出Kafka集群中的所有主题之前，让我们分三步建立一个测试的单节点Kafka集群。

  • Downloading Kafka and Zookeeper
  • Starting Zookeeper Service
  • Starting Kafka Service

First, we should make sure to download the right Kafka version from the Apache site. Once the download finishes, we should extract the downloaded archive:

首先，我们应该确保从Apache网站下载正确的Kafka版本。一旦下载完成，我们应该提取下载的存档。

$ tar xvf kafka_2.13-2.6.0.tgz

Kafka is using Apache Zookeeper to manage its cluster metadata, so we need a running Zookeeper cluster.

Kafka使用Apache Zookeeper来管理其集群元数据，所以我们需要一个正在运行的Zookeeper集群。

For test purposes, we can run a single-node Zookeeper instance using the zookeeper-server-start.sh script in the bin directory:

为了测试，我们可以使用bin目录下的zookeeper-server-start.sh脚本运行一个单节点Zookeeper实例。

$ cd kafka_2.13-2.6.0 # extracted directory
$ ./bin/zookeeper-server-start.sh config/zookeeper.properties

This will start a Zookeeper service listening on port 2181. After that, we can use another script to run the Kafka server:

这将启动一个在2181端口监听的Zookeeper服务。之后，我们可以使用另一个脚本来运行Kafka服务器。

$ ./bin/kafka-server-start.sh config/server.properties

After awhile, a Kafka broker will start. Let’s add a few topics to this simple cluster:

一段时间后，一个Kafka代理将启动。让我们向这个简单的集群添加一些主题。

$ bin/kafka-topics.sh --create --topic users.registrations --replication-factor 1 \
  --partitions 2  --zookeeper localhost:2181
$ bin/kafka-topics.sh --create --topic users.verfications --replication-factor 1 \
  --partitions 2  --zookeeper localhost:2181

Now that everything is ready, let’s see how we can list Kafka topics.

现在一切都准备好了，让我们看看如何列出Kafka主题。

3. Listing Topics

3.上市主题

To list all the Kafka topics in a cluster, we can use the bin/kafka-topics.sh shell script bundled in the downloaded Kafka distribution. All we have to do is to pass the –list option, along with the information about the cluster. For instance, we can pass the Zookeeper service address:

要列出一个集群中的所有Kafka主题，我们可以使用下载的Kafka发行版中捆绑的bin/kafka-topics.shshell脚本。我们所要做的就是传递-list 选项，以及关于集群的信息。例如，我们可以传递Zookeeper服务地址。

$ bin/kafka-topics.sh --list --zookeeper localhost:2181
users.registrations
users.verfications

As shown above, the –list option tells the kafka-topics.sh shell script to list all the topics. In this case, we have two topics to store user-related events. If there’s no topic in the cluster, then the command will return silently without any result.

如上所示，-list选项告诉kafka-topics.sh shell脚本列出所有的主题。在这种情况下，我们有两个主题来存储用户相关的事件。如果集群中没有话题，那么该命令将无声地返回，没有任何结果。

Also, in order to talk to the Kafka cluster, we need to pass the Zookeeper service URL using the –zookeeper option.

此外，为了与Kafka集群对话，我们需要使用-zookeeper 选项传递Zookeeper服务的URL。

It’s even possible to pass the Kafka cluster address directly using the –bootstrap-server option:

甚至可以使用-bootstrap-server选项直接传递Kafka集群地址。

$ ./bin/kafka-topics.sh --bootstrap-server=localhost:9092 --list
users.registrations
users.verfications

Our single-instance Kafka cluster listens to the 9092 port, so we specified “localhost:9092” as the bootstrap server. Put simply, bootstrap servers are Kafka brokers.

我们的单实例Kafka集群监听9092端口，所以我们指定“localhost:9092”作为引导服务器。简单地说，引导服务器是Kafka的经纪人。

If we don’t pass the information necessary to talk to a Kafka cluster, the kafka-topics.sh shell script will complain with an error:

如果我们没有传递与Kafka集群对话所需的信息，kafka-topics.sh shell脚本就会抱怨出错。

$ ./bin/kafka-topics.sh --list
Exception in thread "main" java.lang.IllegalArgumentException: Only one of --bootstrap-server or --zookeeper must be specified
        at kafka.admin.TopicCommand$TopicCommandOptions.checkArgs(TopicCommand.scala:721)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:52)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)

As shown above, the shell scripts require us to pass either the –bootstrap-server or –zookeeper option.

如上所示，shell脚本要求我们传递-bootstrap-server或-zookeeper选项。

4. Topic Details

4.主题详情

Once we’ve found a list of topics, we can take a peek at the details of one specific topic. To do that, we can use the “–describe –topic <topic name>” combination of options:

一旦我们找到了一个主题列表，我们就可以偷看一个特定主题的细节。要做到这一点，我们可以使用“-describe -topic <topic name>”选项的组合。

$ ./bin/kafka-topics.sh --bootstrap-server=localhost:9092 --describe --topic users.registrations
Topic: users.registrations      PartitionCount: 2       ReplicationFactor: 1    Configs: segment.bytes=1073741824
        Topic: users.registrations      Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: users.registrations      Partition: 1    Leader: 0       Replicas: 0     Isr: 0

These details include information about the specified topic, such as the number of partitions and replicas, among others. Similar to other commands, we must pass the cluster information or Zookeeper address. Otherwise, we won’t be able to talk to the cluster.

这些细节包括关于指定主题的信息，如分区和复制的数量等等。与其他命令类似，我们必须传递集群信息或Zookeeper地址。否则，我们将无法与集群对话。

5. Conclusion

5.总结

In this brief article, we learned how to list all the topics in a Kafka cluster. Along the way, we saw how to set up a simple, single-node Kafka cluster.

在这篇简短的文章中，我们学习了如何列出一个Kafka集群中的所有主题。一路走来，我们看到了如何建立一个简单的、单节点的Kafka集群。

Currently, Apache Kafka uses Zookeeper to manage its cluster metadata. This, however, will change shortly as part of KIP-500, as Kafka is going to have its own metadata quorum.

目前，Apache Kafka使用Zookeeper来管理其集群元数据。但是，作为KIP-500的一部分，这种情况将很快发生变化，因为Kafka将拥有自己的元数据配额。