Re: Spark Streaming, very slow processing and increasing scheduling delay of kafka input stream

2016-03-10 Thread Andy Davidson
gmail.com> Cc: "user @spark" <user@spark.apache.org> Subject: Re: Spark Streaming, very slow processing and increasing scheduling delay of kafka input stream > Hi Vinti, > > All of your tasks are failing based on the screen shots provided. > > I think a few m

Re: Spark Streaming, very slow processing and increasing scheduling delay of kafka input stream

2016-03-10 Thread Todd Nist
Hi Vinti, All of your tasks are failing based on the screen shots provided. I think a few more details would be helpful. Is this YARN or a Standalone cluster? How much overall memory is on your cluster? On each machine where workers and executors are running? Are you using the Direct