Sorry, wrong ML.

On Fri, Jul 24, 2015 at 7:07 PM, Cody Koeninger <c...@koeninger.org> wrote:

> Are you intending to be mailing the spark list or the kafka list?
>
> On Fri, Jul 24, 2015 at 11:56 AM, Stevo Slavić <ssla...@gmail.com> wrote:
>
>> Hello Cody,
>>
>> I'm not sure we're talking about same thing.
>>
>> Since you're mentioning streams I guess you were referring to current
>> high level consumer, while I'm talking about new yet unreleased high level
>> consumer.
>>
>> Poll I was referring to is
>> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L715
>>
>> ConsumerRecord offset I was referring to is
>> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerRecord.java#L22
>>
>> poll returns ConsumerRecords, per TopicPartition collection of
>> ConsumerRecord. And in example I gave ConsumerRecord offset would be 0.
>>
>> Commit I was referring to is
>> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L805
>>
>> After processing read ConsumerRecord, commit expects me to submit not
>> offset 0 but offset 1...
>>
>> Kind regards,
>> Stevo Slavic
>>
>> On Fri, Jul 24, 2015 at 6:31 PM, Cody Koeninger <c...@koeninger.org>
>> wrote:
>>
>>> Well... there are only 2 hard problems in computer science: naming
>>> things, cache invalidation, and off-by-one errors.
>>>
>>> The direct stream implementation isn't asking you to "commit" anything.
>>> It's asking you to provide a starting point for the stream on startup.
>>>
>>> Because offset ranges are inclusive start, exclusive end, it's pretty
>>> natural to use the end of the previous offset range as the beginning of the
>>> next.
>>>
>>>
>>> On Fri, Jul 24, 2015 at 11:13 AM, Stevo Slavić <ssla...@gmail.com>
>>> wrote:
>>>
>>>> Hello Apache Kafka community,
>>>>
>>>> Say there is only one topic with single partition and a single message
>>>> on it.
>>>> Result of calling a poll with new consumer will return ConsumerRecord
>>>> for that message and it will have offset of 0.
>>>>
>>>> After processing message, current KafkaConsumer implementation expects
>>>> one to commit not offset 0 as processed, but to commit offset 1 - next
>>>> offset/position one would like to consume.
>>>>
>>>> Does this sound strange to you as well?
>>>>
>>>> Wondering couldn't this offset+1 handling for next position to read
>>>> been done in one place, in KafkaConsumer implementation or broker or
>>>> whatever, instead of every user of KafkaConsumer having to do it.
>>>>
>>>> Kind regards,
>>>> Stevo Slavic.
>>>>
>>>
>>>
>>
>

Reply via email to