[jira] [Commented] (KAFKA-2720) Periodic purging groups in the coordinator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334787#comment-15334787
 ] 

ASF GitHub Bot commented on KAFKA-2720:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/1513


> Periodic purging groups in the coordinator
> --
>
> Key: KAFKA-2720
> URL: https://issues.apache.org/jira/browse/KAFKA-2720
> Project: Kafka
>  Issue Type: Sub-task
>  Components: consumer
>Reporter: Guozhang Wang
>Assignee: Jason Gustafson
> Fix For: 0.10.1.0
>
>
> Currently the coordinator removes the group (i.e. both removing it from the 
> cache and writing the tombstone message on its local replica without waiting 
> for ack) once it becomes an empty group.
> This can lead to a few issues such as 1) group removal and creation churns 
> when a group with very few members are being rebalanced, 2) if the local 
> write is failed / not propagated to other followers, they can only be removed 
> again when a new coordinator is migrated and detects the group has no members 
> already.
> We could instead piggy-back the periodic offset expiration along with the 
> group purging as well which removes any groups that had no corresponding 
> offsets in the cache any more.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2720) Periodic purging groups in the coordinator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334164#comment-15334164
 ] 

ASF GitHub Bot commented on KAFKA-2720:
---

GitHub user hachikuji opened a pull request:

https://github.com/apache/kafka/pull/1513

MINOR: Follow-up from KAFKA-2720 with comment/style fixes



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hachikuji/kafka followup-for-kafka-2720

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1513.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1513


commit d013a1c8c1a77d3f6c2d8eace99419bec961ab1d
Author: Jason Gustafson 
Date:   2016-06-16T17:02:55Z

MINOR: Follow-up from KAFKA-2720 with comment/style fixes




> Periodic purging groups in the coordinator
> --
>
> Key: KAFKA-2720
> URL: https://issues.apache.org/jira/browse/KAFKA-2720
> Project: Kafka
>  Issue Type: Sub-task
>  Components: consumer
>Reporter: Guozhang Wang
>Assignee: Jason Gustafson
> Fix For: 0.10.1.0
>
>
> Currently the coordinator removes the group (i.e. both removing it from the 
> cache and writing the tombstone message on its local replica without waiting 
> for ack) once it becomes an empty group.
> This can lead to a few issues such as 1) group removal and creation churns 
> when a group with very few members are being rebalanced, 2) if the local 
> write is failed / not propagated to other followers, they can only be removed 
> again when a new coordinator is migrated and detects the group has no members 
> already.
> We could instead piggy-back the periodic offset expiration along with the 
> group purging as well which removes any groups that had no corresponding 
> offsets in the cache any more.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2720) Periodic purging groups in the coordinator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332949#comment-15332949
 ] 

ASF GitHub Bot commented on KAFKA-2720:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/1427


> Periodic purging groups in the coordinator
> --
>
> Key: KAFKA-2720
> URL: https://issues.apache.org/jira/browse/KAFKA-2720
> Project: Kafka
>  Issue Type: Sub-task
>  Components: consumer
>Reporter: Guozhang Wang
>Assignee: Jason Gustafson
> Fix For: 0.10.1.0
>
>
> Currently the coordinator removes the group (i.e. both removing it from the 
> cache and writing the tombstone message on its local replica without waiting 
> for ack) once it becomes an empty group.
> This can lead to a few issues such as 1) group removal and creation churns 
> when a group with very few members are being rebalanced, 2) if the local 
> write is failed / not propagated to other followers, they can only be removed 
> again when a new coordinator is migrated and detects the group has no members 
> already.
> We could instead piggy-back the periodic offset expiration along with the 
> group purging as well which removes any groups that had no corresponding 
> offsets in the cache any more.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2720) Periodic purging groups in the coordinator

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299416#comment-15299416
 ] 

ASF GitHub Bot commented on KAFKA-2720:
---

GitHub user hachikuji opened a pull request:

https://github.com/apache/kafka/pull/1427

KAFKA-2720 [WIP]: expire group metadata when all offsets have expired



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hachikuji/kafka KAFKA-2720

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1427.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1427


commit f077556496c7c8e2da43a2989efb545d6a0e7de5
Author: Jason Gustafson 
Date:   2016-05-25T03:45:13Z

KAFKA-2720 [WIP]: expire group metadata when all offsets have expired




> Periodic purging groups in the coordinator
> --
>
> Key: KAFKA-2720
> URL: https://issues.apache.org/jira/browse/KAFKA-2720
> Project: Kafka
>  Issue Type: Sub-task
>  Components: consumer
>Reporter: Guozhang Wang
>Assignee: Jason Gustafson
> Fix For: 0.10.1.0
>
>
> Currently the coordinator removes the group (i.e. both removing it from the 
> cache and writing the tombstone message on its local replica without waiting 
> for ack) once it becomes an empty group.
> This can lead to a few issues such as 1) group removal and creation churns 
> when a group with very few members are being rebalanced, 2) if the local 
> write is failed / not propagated to other followers, they can only be removed 
> again when a new coordinator is migrated and detects the group has no members 
> already.
> We could instead piggy-back the periodic offset expiration along with the 
> group purging as well which removes any groups that had no members already.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)