Spark Streaming: Is it possible to schedule multiple active batches?

Is it possible to have the scheduler schedule the next batch even if the previous batch has not completed yet? I'd like to schedule up to 3 batches at the same time if this is possible.

Spark Streaming: My kafka receivers are not consuming in parallel

Hello Spark users, We are setting up our fist bach of spark streaming pipelines. And I am running into an issue which I am not sure how to resolve, but seems like should be fairly trivial. I am using receiver-mode Kafka consumer that comes with Spark, and running in standalone mode. I've setup

Re: Spark Streaming: My kafka receivers are not consuming in parallel

to the same partition, unless you specify the "--new-producer" parameter. Thanks, Jorge On Wed, Feb 3, 2016 at 12:44 PM, Jorge Rodriguez <jo...@bloomreach.com> wrote: > Hello Spark users, > > We are setting up our fist bach of spark streaming pipelines. And I am > runn