Do you have the logs of the driver? Does that give any exceptions?
TD
On Fri, Mar 27, 2015 at 12:24 PM, Chen Song wrote:
> I ran a spark streaming job.
>
> 100 executors
> 30G heap per executor
> 4 cores per executor
>
> The version I used is 1.3.0-cdh5.1.0.
>
> The job is reading from a direct
I ran a spark streaming job.
100 executors
30G heap per executor
4 cores per executor
The version I used is 1.3.0-cdh5.1.0.
The job is reading from a directory on HDFS (with files incoming
continuously) and does some join on the data. I set batch interval to be 15
minutes and the job worked fine
