[jira] [Commented] (KAFKA-4081) Consumer API consumer new interface commitSyn does not verify the validity of offset

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15657766#comment-15657766
 ] 

ASF GitHub Bot commented on KAFKA-4081:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/1827


> Consumer API consumer new interface commitSyn does not verify the validity of 
> offset
> 
>
> Key: KAFKA-4081
> URL: https://issues.apache.org/jira/browse/KAFKA-4081
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 0.9.0.1
>Reporter: lifeng
>Assignee: Mickael Maison
> Fix For: 0.10.2.0
>
>
> Consumer API consumer new interface commitSyn synchronization update offset, 
> for the illegal offset successful return, illegal offset<0 or offset>hw



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4081) Consumer API consumer new interface commitSyn does not verify the validity of offset

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510415#comment-15510415
 ] 

ASF GitHub Bot commented on KAFKA-4081:
---

GitHub user mimaison reopened a pull request:

https://github.com/apache/kafka/pull/1827

KAFKA-4081: Consumer API consumer new interface commitSync does not v…

…erify the validity of offset

Commit throws InvalidOffsetException if the offset is negative

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mimaison/kafka KAFKA-4081

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1827.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1827


commit 7ec08416868180422dd6df8e921c9f6027218bfa
Author: Mickael Maison 
Date:   2016-09-19T17:22:34Z

KAFKA-4081: Consumer API consumer new interface commitSync does not verify 
the validity of offset

Commit will now fail with an IllegalArgumentException when trying to commit 
an offset that is either negative or larger than the consumer's current position

commit e1c1674b8f661e4063e4d5bc04a86ad069a3f317
Author: Edoardo Comar 
Date:   2016-09-21T16:14:36Z

KAFKA-4081: Consumer API consumer new interface commitSync does not
verify the validity of offset

Commit will now fail with an IllegalArgumentException when trying to
commit an offset that is either negative or larger than the consumer's
current position




> Consumer API consumer new interface commitSyn does not verify the validity of 
> offset
> 
>
> Key: KAFKA-4081
> URL: https://issues.apache.org/jira/browse/KAFKA-4081
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 0.9.0.1
>Reporter: lifeng
>Assignee: Mickael Maison
>
> Consumer API consumer new interface commitSyn synchronization update offset, 
> for the illegal offset successful return, illegal offset<0 or offset>hw



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4081) Consumer API consumer new interface commitSyn does not verify the validity of offset

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504492#comment-15504492
 ] 

ASF GitHub Bot commented on KAFKA-4081:
---

Github user mimaison closed the pull request at:

https://github.com/apache/kafka/pull/1827


> Consumer API consumer new interface commitSyn does not verify the validity of 
> offset
> 
>
> Key: KAFKA-4081
> URL: https://issues.apache.org/jira/browse/KAFKA-4081
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 0.9.0.1
>Reporter: lifeng
>Assignee: Mickael Maison
>
> Consumer API consumer new interface commitSyn synchronization update offset, 
> for the illegal offset successful return, illegal offset<0 or offset>hw



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4081) Consumer API consumer new interface commitSyn does not verify the validity of offset

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467915#comment-15467915
 ] 

ASF GitHub Bot commented on KAFKA-4081:
---

GitHub user mimaison opened a pull request:

https://github.com/apache/kafka/pull/1827

KAFKA-4081: Consumer API consumer new interface commitSync does not v…

…erify the validity of offset

Commit throws InvalidOffsetException if the offset is negative

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mimaison/kafka KAFKA-4081

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1827.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1827


commit 334f82f60416c511665519362ce23d4ab22cc5fd
Author: Mickael Maison 
Date:   2016-09-01T16:41:16Z

KAFKA-4081: Consumer API consumer new interface commitSync does not verify 
the validity of offset

Commit throws InvalidOffsetException if the offset is negative




> Consumer API consumer new interface commitSyn does not verify the validity of 
> offset
> 
>
> Key: KAFKA-4081
> URL: https://issues.apache.org/jira/browse/KAFKA-4081
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 0.9.0.1
>Reporter: lifeng
>Assignee: Mickael Maison
>
> Consumer API consumer new interface commitSyn synchronization update offset, 
> for the illegal offset successful return, illegal offset<0 or offset>hw



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4081) Consumer API consumer new interface commitSyn does not verify the validity of offset

[Jason Gustafson (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15447017#comment-15447017
 ] 

Jason Gustafson commented on KAFKA-4081:


I think it definitely makes sense to reject negative values (we currently use 
-1 to indicate an invalid offset which leads to inconsistent behavior as noted 
by [~mimaison]), but rejecting offsets greater than hw seems more challenging 
since neither the client (nor the server accepting the offset commit) will 
necessarily have an up-to-date value. We could use the last hw returned from a 
previous fetch, but it might be stale by the time the user attempts to commit 
offsets. Perhaps it would make more sense to reject _any_ offset which is 
greater than the current position? It would still be possible to commit an 
invalid offset, but not without an explicit seek to that offset. However, there 
is a compatibility concern for use cases which use the consumer only for access 
to the offset API, which could be seen in offset tooling.

> Consumer API consumer new interface commitSyn does not verify the validity of 
> offset
> 
>
> Key: KAFKA-4081
> URL: https://issues.apache.org/jira/browse/KAFKA-4081
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 0.9.0.1
>Reporter: lifeng
>
> Consumer API consumer new interface commitSyn synchronization update offset, 
> for the illegal offset successful return, illegal offset<0 or offset>hw



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4081) Consumer API consumer new interface commitSyn does not verify the validity of offset

[lifeng (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15440830#comment-15440830
 ] 

lifeng commented on KAFKA-4081:
---

thanks

Our new consumer api Each interface has been tested, quality department raised 
the issue, they think that if kafka as production systems, should have robust.

> Consumer API consumer new interface commitSyn does not verify the validity of 
> offset
> 
>
> Key: KAFKA-4081
> URL: https://issues.apache.org/jira/browse/KAFKA-4081
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 0.9.0.1
>Reporter: lifeng
>
> Consumer API consumer new interface commitSyn synchronization update offset, 
> for the illegal offset successful return, illegal offset<0 or offset>hw



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-4081) Consumer API consumer new interface commitSyn does not verify the validity of offset

[Mickael Maison (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439258#comment-15439258
 ] 

Mickael Maison commented on KAFKA-4081:
---

The same happens with 0.10.0.1. Looking at the offset commit logic, the value 
of the offset is not checked at all.

It seems some code (ConsumerCoordinator:730) attributes a special meaning to 
negative offsets. 
One inconsistency I can find is that in the client that commits a negative 
offset, calls to committed() will return the negative value whereas in clients 
that just fetch the committed offset it returns nothing. This is because when 
we commit, it updates the SubscriptionState. This can be seen by using 
ConsumerGroupCommand, it shows "unknown" for topics that have a negative offset 
committed.

Are there use cases we'd break by forcing offsets to be valid when committed ? 
[~ijuma]



> Consumer API consumer new interface commitSyn does not verify the validity of 
> offset
> 
>
> Key: KAFKA-4081
> URL: https://issues.apache.org/jira/browse/KAFKA-4081
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Affects Versions: 0.9.0.1
>Reporter: lifeng
>
> Consumer API consumer new interface commitSyn synchronization update offset, 
> for the illegal offset successful return, illegal offset<0 or offset>hw



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)