Re: Reducing parallelism leads to NoResourceAvailableException

Hi Ufuk, > On Apr 28, 2016, at 1:32am, Ufuk Celebi wrote: > > Hey Ken! > > That should not happen. Can you check the web interface for two things: > > - How many available slots are advertized on the landing page > (localhost:8081) when you submit your job? I’m running this

Re: Reducing parallelism leads to NoResourceAvailableException

> On Apr 28, 2016, at 1:32am, Aljoscha Krettek wrote: > > Hi, > is this a streaming or batch job? Batch. > If it is a batch job, are you using either collect() or print() on a DataSet? Definitely not a print(). Don’t know about collect(), since the job is created via

Re: Reducing parallelism leads to NoResourceAvailableException

Hey Ken! That should not happen. Can you check the web interface for two things: - How many available slots are advertized on the landing page (localhost:8081) when you submit your job? - Can you check the actual parallelism of the submitted job (it should appear as a FAILED job in the web

Re: Reducing parallelism leads to NoResourceAvailableException

Hi, is this a streaming or batch job? If it is a batch job, are you using either collect() or print() on a DataSet? Cheers, Aljoscha On Thu, 28 Apr 2016 at 00:52 Ken Krugler wrote: > Hi all, > > In trying out different settings for performance, I run into a job

Reducing parallelism leads to NoResourceAvailableException

Hi all, In trying out different settings for performance, I run into a job failure case that puzzles me. I’d done a run with a parallelism of 20 (-p 20 via CLI), and the job ran successfully, on a cluster with 40 slots. I then tried with -p 15, and it failed with: