1. Overview
1.概述
Client applications that use Apache Kafka would usually fall into either of the two categories, namely producers and consumers. Both producers and consumers require that the underlying Kafka server is up and running before they can start their job of producing and consuming, respectively.
使用Apache Kafka的客户端应用程序通常属于两类中的任何一类,即生产者和消费者。生产者和消费者都需要底层的Kafka服务器启动并运行,然后才能分别开始生产和消费的工作。
In this article, we’ll learn a few strategies to determine if the Kafka server is running.
在这篇文章中,我们将学习一些策略来确定Kafka服务器是否正在运行。
2. Using Zookeeper Commands
2.使用Zookeeper命令
One of the quickest ways to find out if there are active brokers is by using Zookeeper’s dump command. The dump command is one of the 4LW commands available to administer a Zookeeper server.
了解是否有活跃的经纪人的最快速方法之一是使用Zookeeper的dump命令。dump命令是可用于管理Zookeeper服务器的4LW命令之一。
Let’s go ahead and use the nc command to send the dump command over the Zookeeper server that’s listening at the 2181 port:
让我们继续使用nc命令,通过正在监听2181端口的Zookeeper服务器发送转储命令。
$ echo dump | nc localhost 2181 | grep -i broker | xargs
/brokers/ids/0
On executing the command, we see a list of ephemeral broker ids registered with the Zookeeper server. If no ephemeral ids exist, then none of the broker nodes is running.
在执行该命令时,我们看到一个在Zookeeper服务器上注册的临时代理ID的列表。如果没有短暂的ID存在,那么没有一个经纪人节点正在运行。
Further, it’s important to note that the dump command needs to be explicitly allowed in the configuration usually available within either zookeeper.properties or zoo.cfg configuration file:
此外,需要注意的是,dump命令需要在zookeeper.properties或zoo.cfg配置文件中明确允许配置。
lw.commands.whitelist=dump
Alternatively, we can also use Zookeeper APIs to find the list of active brokers.
另外,我们也可以使用Zookeeper APIs来查找活动经纪人的列表。
3. Using Apache Kafka’s AdminClient
3.使用Apache Kafka的AdminClient
If our producers or consumers are Java applications, then we can use Apache Kafka’s AdminClient class to find out if the Kafka server is up or not.
如果我们的生产者或消费者是Java应用程序,那么我们可以使用Apache Kafka的AdminClient类来了解Kafka服务器是否已经启动。
Let’s define the KafkaAdminClient class to wrap an instance of the AdminClient class so that we can quickly test our code:
让我们定义KafkaAdminClient类来包裹AdminClient类的一个实例,这样我们就可以快速测试我们的代码。
public class KafkaAdminClient {
private final AdminClient client;
public KafkaAdminClient(String bootstrap) {
Properties props = new Properties();
props.put("bootstrap.servers", bootstrap);
props.put("request.timeout.ms", 3000);
props.put("connections.max.idle.ms", 5000);
this.client = AdminClient.create(props);
}
}
Next, let’s define the verifyConnection() method in the KafkaAdminClient class to verify if the client can connect with running broker servers:
接下来,让我们在KafkaAdminClient类中定义verifyConnection()方法,以验证客户端是否能与正在运行的代理服务器连接。
public boolean verifyConnection() throws ExecutionException, InterruptedException {
Collection<Node> nodes = this.client.describeCluster()
.nodes()
.get();
return nodes != null && nodes.size() > 0;
}
Finally, let’s put our code to test by connecting to a running Kafka cluster:
最后,让我们通过连接到一个正在运行的Kafka集群来测试我们的代码。
@Test
void givenKafkaIsRunning_whenCheckedForConnection_thenConnectionIsVerified() throws Exception {
boolean alive = kafkaAdminClient.verifyConnection();
assertThat(alive).isTrue();
}
4. Using the kcat Utility
4.使用kcat程序
We can use the kcat (formerly kafkacat) command to find out if there are running Kafka broker nodes. To do so, let’s use the -L option to show the metadata of an existing topic:
我们可以使用kcat(以前的kafkacat)命令来了解是否有正在运行的Kafka代理节点。为此,让我们使用-L选项来显示一个现有主题的元数据。
$ kcat -b localhost:9092 -t demo-topic -L
Metadata for demo-topic (from broker -1: localhost:9092/bootstrap):
1 brokers:
broker 0 at 192.168.1.53:9092 (controller)
1 topics:
topic "demo-topic" with 1 partitions:
partition 0, leader 0, replicas: 0, isrs: 0
Next, let’s execute the same command when the broker nodes are down:
接下来,让我们在经纪人节点关闭时执行同样的命令。
$ kcat -b localhost:9092 -t demo-topic -L -m 1
%3|1660579562.937|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Connect to ipv4#127.0.0.1:9092 failed: Connection refused (after 1ms in state CONNECT)
% ERROR: Failed to acquire metadata: Local: Broker transport failure (Are the brokers reachable? Also try increasing the metadata timeout with -m <timeout>?)
For this case, we get the “Connection refused” error as there are no running broker nodes. Additionally, we must note that we were able to fail fast by restricting the request timeout to 1 second with the -m option.
在这种情况下,我们得到了 “连接拒绝 “的错误,因为没有正在运行的代理节点。此外,我们必须注意到,通过使用-m选项将请求超时限制为1秒,我们能够快速失败。
5. Using UI Tools
5.使用用户界面工具
We can rely on UI tools such as Offset Explorer for experimental POC projects that don’t require automated checks. However, this approach is not recommended if we want to verify the state of broker nodes for enterprise-grade Kafka clients.
对于不需要自动检查的实验性POC项目,我们可以依靠诸如Offset Explorer等UI工具。但是,如果我们想为企业级Kafka客户端验证代理节点的状态,则不建议使用这种方法。
Let’s use the Offset Explorer to connect to the Kafka cluster using the Zookeeper host and port details:
让我们使用偏移量资源管理器,使用Zookeeper主机和端口详细信息连接到Kafka集群:
We can see the list of running brokers on the left side pane. That’s it. We got it at a click of a button.
我们可以在左侧窗格中看到正在运行的经纪商列表。这就是了。我们点击一下按钮就得到了它。
6. Conclusion
6.结语
In this tutorial, we explored a few command-line approaches using Zookeeper commands, Apache’s AdminClient, and the kcat utility, followed by a UI-based approach to determine whether the Kafka server is up or not.
在本教程中,我们使用Zookeeper命令、Apache的AdminClient和kcat工具探索了一些命令行方法,然后是基于UI的方法来确定Kafka服务器是否启动。
As always, the complete source code for the tutorial is available over on GitHub.
一如既往,该教程的完整源代码可在GitHub上获得over。