Hi!
I have not used the Kafka Offset Checker before, maybe someone who worked
with that can chime in.
Greetings,
Stephan
On Wed, Aug 3, 2016 at 4:59 PM, Janardhan Reddy <janardhan.re...@olacabs.com
> wrote:
> I can see that offsets are stored in zookeeper and are not returned when i
> query through kafka offset checker.
>
> Can you please tell me how to monitor kafka flink consumer lag for 0.8
> flink kafka consumer.
>
> On Wed, Aug 3, 2016 at 3:29 PM, Stephan Ewen <se...@apache.org> wrote:
>
>> Hi!
>>
>> Just check in the code. The 0.8 FlinkKafkaConsumer should always commit
>> offsets, regardless of whether checkpointing is enables. The 0.9
>> FlinkKafkaConsumer actually does not do any periodic offset committing when
>> checkpointing is disabled.
>>
>> Greetings,
>> Stephan
>>
>> On Wed, Aug 3, 2016 at 7:36 AM, Janardhan Reddy <
>> janardhan.re...@olacabs.com> wrote:
>>
>>> Checkpointing wasn't enabled in the streaming job, but the offsets
>>> should have been committed to zookeeper.
>>>
>>> But we don't see the offsets being written to zookeeper.
>>>
>>> On Tue, Aug 2, 2016 at 7:41 PM, Till Rohrmann <trohrm...@apache.org>
>>> wrote:
>>>
>>>> Hi Janardhan,
>>>>
>>>> Flink should commit the current offsets to Zookeeper whenever a
>>>> checkpoint has been completed. In case that you disabled checkpointing,
>>>> then the offsets will be periodically committed to ZooKeeper. The default
>>>> value is 60s.
>>>>
>>>> Could it be that there wasn't yet a completed checkpoint? Which version
>>>> of Flink are you using?
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Tue, Aug 2, 2016 at 7:26 PM, Janardhan Reddy <
>>>> janardhan.re...@olacabs.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> When the run the following command i am getting that no topic is
>>>>> available for that consumer group. i am suing
>>>>> flink-connector-kafka-0.8_${scala.version}(2.11).
>>>>>
>>>>> ./bin/kafka-consumer-groups.sh --zookeeper <> --group <> --describe
>>>>>
>>>>> No topic available for consumer group provided
>>>>>
>>>>>
>>>>> Does the kafka consumer commit offset to the broker always ? Do we
>>>>> need to enable checkpointing for the offsets to be committed ?
>>>>>
>>>>>
>>>>> Thanks
>>>>>
>>>>
>>>>
>>>
>>
>
