Re: Kafka integration: get existing Kafka messages?

2016-10-14 Thread Cody Koeninger
an you please provide more details? Thank you very much! >> >> >> >> ________ >> >> From: Cody Koeninger [mailto:c...@koeninger.org] >> Sent: 2016年10月12日 20:10 >> To: Haopu Wang >> Cc: user@spark.apache.org >>

RE: Kafka integration: get existing Kafka messages?

2016-10-14 Thread Haopu Wang
pic2 0 2 after polling for 512 == -Original Message- From: Cody Koeninger [mailto:c...@koeninger.org] Sent: 2016年10月13日 9:31 To: Haopu Wang Cc: user@spark.apache.org Subject: Re: Kafka integration: get existing Kafka messages? Look at the presentation and blog post linked from https://github.co

Re: Kafka integration: get existing Kafka messages?

2016-10-12 Thread Cody Koeninger
2016年10月12日 20:10 > To: Haopu Wang > Cc: user@spark.apache.org > Subject: Re: Kafka integration: get existing Kafka messages? > > > > its set to none for the executors, because otherwise they wont do exactly > what the driver told them to do. > > > > you should be a

RE: Kafka integration: get existing Kafka messages?

2016-10-12 Thread Haopu Wang
Cc: user@spark.apache.org Subject: Re: Kafka integration: get existing Kafka messages? its set to none for the executors, because otherwise they wont do exactly what the driver told them to do. you should be able to set up the driver consumer to determine batches however you want, though

Re: Kafka integration: get existing Kafka messages?

2016-10-12 Thread Cody Koeninger
its set to none for the executors, because otherwise they wont do exactly what the driver told them to do. you should be able to set up the driver consumer to determine batches however you want, though. On Wednesday, October 12, 2016, Haopu Wang wrote: > Hi, > > > > I want