I have a streaming job that uses a lot of memory. Capacity scheduler lets me set the mapred.job.map.memory.mb property to something high like 2560. The job then takes 5 slots (512mb slot) for every map task. I have noticed that it appears to actually start many java processes that look like tasks (I've seen as many as 12) on my slaves to service my map tasks. What are all the extra java processes for (are these considered slots?) they use memory but no cpu, are they just place holders?
- capacity - resource based scheduling Corbin Hoenes
- Re: capacity - resource based scheduling Harsh J
- Re: capacity - resource based scheduling Corbin Hoenes