I have a 5 node cluster with about 12GB of RAM at the disposal of the resource manager. When you are running your job take a look at the scheduler section of your resource manager UI. It will give you a hint whether you are starved of task slots or the queue is being overloaded or it is because your underlying map-reduce job is not getting enough memory for a yarn container.
Limiting the maximum memory for a single container might allow multiple containers to get instantiated. This in turn will increase the number of jobs you can run simultaneously on yarn at the expense of smaller yarn containers. On Thu, Feb 11, 2016 at 10:17 AM, tkg_cangkul <[email protected]> wrote: > hi Nitin Kumar, > > thx for your reply. i've been try this way but still have the same issue. > can u tell me how much RM memory in your small cluster. > i've set my RM memory to 5Gb but still has the same issue > > On 11/02/16 10:52, Nitin Kumar wrote: > >> I faced a similar problem when running on a small test cluster. I'm not >> aware about the memory requirements of your underlying map-reduce job, but >> a prime cause of this 'deadlock' behaviour is because oozie map-reduce >> jobs >> are launched by an oozie launcher job (which is a map only job). This >> launcher job takes up task slots in the hadoop queue (the 'default' queue >> is used by default) and stays alive till the underlying map-reduce job >> terminates. If the the underlying map-reduce job gets no task slots it >> will >> wait for resources to be released and the launcher will wait for the job >> to >> terminate. >> >> A way to mitigate this issue is by splitting your default queue into 2 or >> more queues. Set the oozie.launcher.mapred.job.queue.name and >> mapred.job.queue.name properties to different queues and you should be >> fine. >> >> On Thu, Feb 11, 2016 at 9:06 AM, satish saley <[email protected]> >> wrote: >> >> Could you please share the Hadoop logs for your job? >>> >>> >>> Sent from Yahoo Mail for iPhone <https://yho.com/footer0> >>> >>> >>> On Wednesday, February 10, 2016, 7:33 PM, tkg_cangkul < >>> [email protected]> wrote: >>> >>> i've been trying to run oozie from virtual machine. but the process stuck >>> at running state. i've set my memory of RM to 4gb and my Vcore is 2. is >>> there any configuration that i've missed? >>> pls help >>> >>> [image: capture from my resource manager] >>> >>> >>> >
