Cody, thanks for the response.
So Kafka direct stream actually has consumer on both the driver and executor?
Can you please provide more details? Thank you very much!
From: Cody Koeninger [mailto:c...@koeninger.org]
Sent: 2016年10月12日 20:10
To: Haopu Wang
Subject: Re: Kafka integration: get existing Kafka messages?
its set to none for the executors, because otherwise they wont do exactly what
the driver told them to do.
you should be able to set up the driver consumer to determine batches however
you want, though.
On Wednesday, October 12, 2016, Haopu Wang <hw...@qilinsoft.com> wrote:
I want to read the existing Kafka messages and then subscribe new stream
But I find "auto.offset.reset" property is always set to "none" in KafkaUtils.
Does that mean I cannot specify "earliest" property value when create direct