[
https://issues.apache.org/jira/browse/KAFKA-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15770874#comment-15770874
]
Vahid Hashemian commented on KAFKA-4547:
----------------------------------------
I have not been able to reproduce this issue in my Ubuntu or Windows
environment.
I checked out [0.10.0.1|https://github.com/apache/kafka/releases/tag/0.10.0.1]
and [0.10.1.0|https://github.com/apache/kafka/releases/tag/0.10.1.0] releases,
built from source, and followed the steps above (for the consumer I used
default properties), but in step B above I always get 2 2 2 when I run the
code. The trunk gives me the same result too.
[~pnakhe] Could you share more info about your environment or any particular
setting you are using that gives you the unexpected result? Thanks.
> Consumer.position returns incorrect results for Kafka 0.10.1.0 client
> ---------------------------------------------------------------------
>
> Key: KAFKA-4547
> URL: https://issues.apache.org/jira/browse/KAFKA-4547
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 0.10.1.0
> Environment: Windows Kafka 0.10.1.0
> Reporter: Pranav Nakhe
> Assignee: Vahid Hashemian
> Labels: clients
> Fix For: 0.10.2.0
>
>
> Consider the following code -
> KafkaConsumer<String, String> consumer = new
> KafkaConsumer<String, String>(props);
> List<TopicPartition> listOfPartitions = new ArrayList();
> for (int i = 0; i <
> consumer.partitionsFor("IssueTopic").size(); i++) {
> listOfPartitions.add(new TopicPartition("IssueTopic",
> i));
> }
> consumer.assign(listOfPartitions);
> consumer.pause(listOfPartitions);
> consumer.seekToEnd(listOfPartitions);
> // consumer.resume(listOfPartitions); -- commented out
> for(int i = 0; i < listOfPartitions.size(); i++) {
>
> System.out.println(consumer.position(listOfPartitions.get(i)));
> }
>
> I have created a topic IssueTopic with 3 partitions with a single replica on
> my single node kafka installation (0.10.1.0)
> The behavior noticed for Kafka client 0.10.1.0 as against Kafka client
> 0.10.0.1
> A) Initially when there are no messages on IssueTopic running the above
> program returns
> 0.10.1.0
> 0
> 0
> 0
> 0.10.0.1
> 0
> 0
> 0
> B) Next I send 6 messages and see that the messages have been evenly
> distributed across the three partitions. Running the above program now
> returns
> 0.10.1.0
> 0
> 0
> 2
> 0.10.0.1
> 2
> 2
> 2
> Clearly there is a difference in behavior for the 2 clients.
> Now after seekToEnd call if I make a call to resume (uncomment the resume
> call in code above) then the behavior is
> 0.10.1.0
> 2
> 2
> 2
> 0.10.0.1
> 2
> 2
> 2
> This is an issue I came across when using the spark kafka integration for
> 0.10. When I use kafka 0.10.1.0 I started seeing this issue. I had raised a
> pull request to resolve that issue [SPARK-18779] but when looking at the
> kafka client implementation/documentation now it seems the issue is with
> kafka and not with spark. There does not seem to be any documentation which
> specifies/implies that we need to call resume after seekToEnd for position to
> return the correct value. Also there is a clear difference in the behavior in
> the two kafka client implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
