[ 
https://issues.apache.org/jira/browse/KAFKA-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David van Geest updated KAFKA-6758:
-----------------------------------
    Description: 
*To reproduce:*
 * Use the default config for `group.id` of "" (the empty string)
 * Use the default config for `enable.auto.commit` of `true`
 * Use manually assigned partitions with `assign`

*Actual (unexpected) behaviour:*

Consumer offsets are stored for the "" group. Example:

{{~ $ /opt/kafka/kafka_2.11-0.11.0.2/bin/kafka-consumer-groups.sh 
--bootstrap-server localhost:9092 --describe --group ""}}
 {{Note: This will only show information about consumers that use the Java 
consumer API (non-ZooKeeper-based consumers).}}

{{Consumer group '' has no active members.}}

{{TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID}}
 {{my_topic 54 7859593 7865082 5489 - - -}}
 {{my_topic 5 14252813 14266419 13606 - - -}}
 {{my_topic 39 19099099 19122441 23342 - - -}}
 {{my_topic 43 16434573 16449180 14607 - - -.}}

....

 

However, the "" is not a real group. It doesn't show up with:

{{~ $ /opt/kafka/kafka_2.11-0.11.0.2/bin/kafka-consumer-groups.sh 
--bootstrap-server localhost:9092 --list}}

You also can't do dynamic partition assignment with it - if you try to 
`subscribe` when using the default "" group ID, you get:

{{AbstractCoordinator: Attempt to join group  failed due to fatal error: The 
configured groupId is invalid}}

*Better behaviours:*

(any of these would be preferable, in my opinion)
 * Don't commit offsets with the "" group, and log a warning telling the user 
that `enable.auto.commit = true` is meaningless in this situation. This is what 
I would have expected.
 * Don't have a default `group.id`. Some of my reading indicates that the new 
consumer basically needs a `group.id` to function. If so, force users to choose 
a group ID so that they're more aware of what will happen.
 * Have a default `group.id` of `default`, and make it a real consumer group. 
That is, it shows up in lists of groups, it has dynamic partitioning, etc.

As a user, when I don't set `group.id` I expect that I'm not using consumer 
groups. This is confirmed to me by listing the consumer groups on the broker 
and not seeing anything. Therefore, I expect that there will be no offset 
tracking in Kafka.

In my specific application, I was wanting `auto.offset.reset` to kick in so 
that a failed consumer would start at the `latest` offset. However, it started 
at this unexpectedly stored offset instead.

 

  was:
*To reproduce:*
 * Use the default config for `group.id` of "" (the empty string)
 * Use the default config for `enable.auto.commit` of `true`
 * Use manually assigned partitions with `assign`

*Actual (unexpected) behaviour:*

Consumer offsets are stored for the "" group. Example:

{{~ $ /opt/kafka/kafka_2.11-0.11.0.2/bin/kafka-consumer-groups.sh 
--bootstrap-server localhost:9092 --describe --group ""}}
 {{Note: This will only show information about consumers that use the Java 
consumer API (non-ZooKeeper-based consumers).Consumer group '' has no active 
members.TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST 
CLIENT-ID}}
 {{my_topic 54 7859593 7865082 5489 - - -}}
 {{my_topic 5 14252813 14266419 13606 - - -}}
 {{my_topic 39 19099099 19122441 23342 - - -}}
 {{my_topic 43 16434573 16449180 14607 - - -.}}

....

 

However, the "" is not a real group. It doesn't show up with:

{{~ $ /opt/kafka/kafka_2.11-0.11.0.2/bin/kafka-consumer-groups.sh 
--bootstrap-server localhost:9092 --list}}

You also can't do dynamic partition assignment with it - if you try to 
`subscribe` when using the default "" group ID, you get:

{{AbstractCoordinator: Attempt to join group  failed due to fatal error: The 
configured groupId is invalid}}

*Better behaviours:*

(any of these would be preferable, in my opinion)
 * Don't commit offsets with the "" group, and log a warning telling the user 
that `enable.auto.commit = true` is meaningless in this situation. This is what 
I would have expected.
 * Don't have a default `group.id`. Some of my reading indicates that the new 
consumer basically needs a `group.id` to function. If so, force users to choose 
a group ID so that they're more aware of what will happen.
 * Have a default `group.id` of `default`, and make it a real consumer group. 
That is, it shows up in lists of groups, it has dynamic partitioning, etc.

As a user, when I don't set `group.id` I expect that I'm not using consumer 
groups. This is confirmed to me by listing the consumer groups on the broker 
and not seeing anything. Therefore, I expect that there will be no offset 
tracking in Kafka.

In my specific application, I was wanting `auto.offset.reset` to kick in so 
that a failed consumer would start at the `latest` offset. However, it started 
at this unexpectedly stored offset instead.

 


> Default "" consumer group tracks committed offsets, but is otherwise not a 
> real group
> -------------------------------------------------------------------------------------
>
>                 Key: KAFKA-6758
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6758
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions: 0.11.0.2
>            Reporter: David van Geest
>            Priority: Major
>
> *To reproduce:*
>  * Use the default config for `group.id` of "" (the empty string)
>  * Use the default config for `enable.auto.commit` of `true`
>  * Use manually assigned partitions with `assign`
> *Actual (unexpected) behaviour:*
> Consumer offsets are stored for the "" group. Example:
> {{~ $ /opt/kafka/kafka_2.11-0.11.0.2/bin/kafka-consumer-groups.sh 
> --bootstrap-server localhost:9092 --describe --group ""}}
>  {{Note: This will only show information about consumers that use the Java 
> consumer API (non-ZooKeeper-based consumers).}}
> {{Consumer group '' has no active members.}}
> {{TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST 
> CLIENT-ID}}
>  {{my_topic 54 7859593 7865082 5489 - - -}}
>  {{my_topic 5 14252813 14266419 13606 - - -}}
>  {{my_topic 39 19099099 19122441 23342 - - -}}
>  {{my_topic 43 16434573 16449180 14607 - - -.}}
> ....
>  
> However, the "" is not a real group. It doesn't show up with:
> {{~ $ /opt/kafka/kafka_2.11-0.11.0.2/bin/kafka-consumer-groups.sh 
> --bootstrap-server localhost:9092 --list}}
> You also can't do dynamic partition assignment with it - if you try to 
> `subscribe` when using the default "" group ID, you get:
> {{AbstractCoordinator: Attempt to join group  failed due to fatal error: The 
> configured groupId is invalid}}
> *Better behaviours:*
> (any of these would be preferable, in my opinion)
>  * Don't commit offsets with the "" group, and log a warning telling the user 
> that `enable.auto.commit = true` is meaningless in this situation. This is 
> what I would have expected.
>  * Don't have a default `group.id`. Some of my reading indicates that the new 
> consumer basically needs a `group.id` to function. If so, force users to 
> choose a group ID so that they're more aware of what will happen.
>  * Have a default `group.id` of `default`, and make it a real consumer group. 
> That is, it shows up in lists of groups, it has dynamic partitioning, etc.
> As a user, when I don't set `group.id` I expect that I'm not using consumer 
> groups. This is confirmed to me by listing the consumer groups on the broker 
> and not seeing anything. Therefore, I expect that there will be no offset 
> tracking in Kafka.
> In my specific application, I was wanting `auto.offset.reset` to kick in so 
> that a failed consumer would start at the `latest` offset. However, it 
> started at this unexpectedly stored offset instead.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to