kafka-consumer-groups

Learn how to use the kafka-consumer-groups tool.

The kafka-consumer-groups tool can be used to list all consumer groups,
describe a consumer group, delete consumer group info, or reset consumer group offsets. The
following topic gives an overview on how to describe or reset consumer group offsets.

Describe Offsets

This tool is primarily used for describing consumer groups and debugging any consumer
offset issues, like consumer lag. The output from the tool shows the log and consumer
offsets for each partition connected to the consumer group that is being described. You can
see at a glance which consumers are current with their partition and which ones are lagging.
From there, you can determine which partitions (and likely the corresponding brokers) are
slow.

Using the tool on secure and unsecure clusters differs slightly. On secure clusters, you
have to use the command-config option together with an appropriate property
file.

Describing offsets on an unsecure cluster

kafka-consumer-groups --bootstrap-server [HOST]:9092 --describe --group [CONSUMER GROUP]

Use the following command to describe offsets committed to Kafka:

GROUP   TOPIC  PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG     OWNER
group1   topic1     0          1               3               2       test-consumer-group_postamac.local-1456198719410-29ccd54f-0

Output Example:

Describing offsets on a secure cluster

In order to describe offsets on a secure Kafka cluster, the consumer-groups tool has
to be run with the command-config option. The
command-config option specifies the property file that contains the
necessary configurations to run the tool on a secure cluster. Which properties are
configured in this file depends on the security configuration of your cluster.

Example client.properties
file:

exclude.internal.topics=false
security.protocol = SASL_SSL
sasl.kerberos.service.name = kafka
ssl.truststore.location = /var/private/ssl/kafka.client.truststore.jks
ssl.truststore.password = test1234

This
example shows what properties you have to set when both Kerberos and TLS/SSL are
configured.

1. Pass the jaas.conf file location as a JVM
   parameter.

   export KAFKA_OPTS='-Djava.security.auth.login.config=[PATH TO JAAS.CONF]

2. Create a client.properties file.

   Use the example above. Make
   changes as necessary.

3. Run the tool with the command-config
   option.

   kafka-consumer-groups --bootstrap-server [HOST]:9093 --describe --command-config client.properties --group [CONSUMER GROUP]

To describe offsets do the following: