[GitHub] kafka pull request #3108: KAFKA-5247: Materialize committed offsets in offse...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/3108


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] kafka pull request #3108: KAFKA-5247: Materialize committed offsets in offse...

[apurvam]

GitHub user apurvam opened a pull request:

https://github.com/apache/kafka/pull/3108

KAFKA-5247: Materialize committed offsets in offset order

With this patch, offset commits are always materialized according to the 
order of the commit records in the offsets topic. 

Before this patch, transactional offset commits were materialized in 
transaction order. However, the log cleaner will always preserve the record 
with the greatest offset. This meant that if there was a mix of offset commits 
from a consumer and a transactional producer, then it we would switch from 
transactional order to offset order after cleaning, resulting in an 
inconsistent state.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apurvam/kafka 
KAFKA-5247-materialize-committed-offsets-in-offset-order

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3108.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3108


commit f6efd565023d440c6d15091609442ff61ad6f85a
Author: Apurva Mehta 
Date:   2017-05-19T18:31:48Z

KAFKA-5247 materialize offset commits in offset order

Updated the GroupMetadata to keep track of the offset in the
__consumer_offsets topic for the commit record for a given offset commit.
We only update the offsets cache when a given offset is committed if the
offset of the commit record in the offsets topic is greater than the offset
of the existing materialized offset.

This way, if we have a mix of transactional and non transactional offset
commits for the same group, we will always materialize the offset
commtis in offset order.

commit 20ee45422130f197791600891a9872826d510ca7
Author: Apurva Mehta 
Date:   2017-05-19T22:35:27Z

Update the return values of the GroupMetadata.remove* methods

commit 2fd79d1680711cdd746233dfbeaea957e65e67d8
Author: Apurva Mehta 
Date:   2017-05-19T23:49:08Z

Minor cleanups and added unit tests

commit 7e5f2820809d9a085333e1fa97efd13207e5a4e0
Author: Apurva Mehta 
Date:   2017-05-20T00:02:13Z

Remove erroneous comment




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---