Hi there,
The Kafka servers in question were 0.11.0.0 this morning. I upgraded to
0.11.0.1 after discovering this issue but I don’t see any change. Scala
2.11.
A client using the new-style consumer groups is not consuming any new
messages because its current committed offset is out of range. I shut down
the clients to ensure the consumer group was inactive and checked the
offset state:
$ kafka-consumer-groups.sh --new-consumer --bootstrap-server
kafka.service:9092 --group router —describe | egrep 'TOPIC|app-router-logs’
Note: This will only show information about consumers that use the Java
consumer API (non-ZooKeeper-based consumers).
Consumer group 'router' has no active members.
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
LAG CONSUMER-ID HOST
CLIENT-ID
app-router-logs 0 728660106 2324523934
1595863828 - -
-
app-router-logs 1 728342585 2324526718
1596184133 - -
-
app-router-logs 2 728698812 2324430267
1595731455 - -
-
Resetting the offsets acts like it worked, but doesn’t:
$ kafka-consumer-groups.sh --new-consumer --bootstrap-server
kafka.service:9092 --group router --reset-offsets --topic app-router-logs
--to-latest
Note: This will only show information about consumers that use the Java
consumer API (non-ZooKeeper-based consumers).
TOPIC PARTITION NEW-OFFSET
app-router-logs 2 2325666004
app-router-logs 1 2325765832
app-router-logs 0 2325763132
kafka-consumer-groups.sh --new-consumer --bootstrap-server
kafka.service:9092 --group router --describe | egrep 'TOPIC|app-router-logs'
Note: This will only show information about consumers that use the Java
consumer API (non-ZooKeeper-based consumers).
Consumer group 'router' has no active members.
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET
LAG CONSUMER-ID HOST
CLIENT-ID
app-router-logs 0 728660106 2326329513
1597669407 - -
-
app-router-logs 1 728342585 2326334386
1597991801 - -
-
app-router-logs 2 728698812 2326237143
1597538331 - -
-
I’ve also tried —to-earliest, —to-current, and individually setting each
partition via —to-offset and there’s no difference. What’s going on here?
-shargan