Hi Jagadish,
I may not express my questions clearly.
Here is what I want to know. When MessageCollector.send is called in
process method, if sending fail and fail again, under this situation is it
possible to cause data loss ( continue to fetch and process messages, but can’t
send them out, at the same time offset is still forwarding and checkpointing ).
Thanks very much.
————————
Qi Shu
> 在 2017年5月11日，15:35，Jagadish Venkatraman <[email protected]> 写道：
>
> Hi Qi,
>
>>> If one record can’t be sent out all the time, then the consumer
> will still fetch messages or not, and what about the offset checkpointing?
>
> Polling / fetching messages from the consumer (in case of Kafka) happens in
> a separate thread.
>
> Samza offers an at-least once processing guarantee with zero data loss.
>
> I'm not sure I understand your specific question about checkpointing?
>
>
> On Thu, May 11, 2017 at 12:28 AM, 舒琦 <[email protected]> wrote:
>
>> Hi,
>>
>> Below is the description about checkpointing.
>>
>> 『Checkpointing is guaranteed to only cover events that are fully
>> processed. It happens only when there are no pending
>> process()/processAsync() or WindowableTask.window() invocations. All
>> preceding invocations happen-before checkpointing and checkpointing
>> happens-before all subsequent invocations.』
>>
>> If one record can’t be sent out all the time, then the consumer
>> will still fetch messages or not, and what about the offset checkpointing?
>>
>> Thanks!
>>
>> ————————
>> Qi Shu
>
>
>
>
> --
> Jagadish V,
> Graduate Student,
> Department of Computer Science,
> Stanford University
