How to see total pending containers ?

Hi, Is there a way to see total pending containers in a cluster,so that we know how far behind we are with etl ? There is a "pending containers" field on the scheduler page under "dr. who" table,but that is always zero. -- Thanks, Ashwin

Why resource requests are normalized in RM ?

Hi, Anyone knows why resource requests from AMs are normalized to be multiples of yarn.scheduler.minimum-allocation-mb which is 1G by default ? Also is there any problem with reducing yarn.scheduler.minimum-allocation-mb to less than 1G ? /** * Utility method to normalize a list of resource r

assignMultiple and continuous scheduling in Fair scheduler

Hi, I see these two knobs in Fair scheduler - 'assignMultiple' and 'continuos scheduling'. 1. Are there performance benefits using them ? What are the cons ? 2. Also is there any problem with 'continuous scheduling',I'm asking because this is not mentioned in the FS doc ? -- Thanks, Ashwin

Re: Differences between HistoryServer and Yarn TimeLine server?

APREDUCE-5858) to track this issue. > > > On Wed, Apr 23, 2014 at 11:25 PM, Ashwin Shankar < > ashwinshanka...@gmail.com> wrote: > >> Hi Zhijie, >> There seems to two umbrella jiras for this - YARN-321 and YARN-1530,can >> you please let me know what is the

Re: Differences between HistoryServer and Yarn TimeLine server?

Hi Zhijie, There seems to two umbrella jiras for this - YARN-321 and YARN-1530,can you please let me know what is the difference ? Is timeline server finally going to be YARN321+YARN1530 ? You mentioned that MR is going to integrated with timeline server,is there a jira I can watch ? Thanks, Ashw

Re: Setting debug log level for individual daemons

; On Wed, Apr 16, 2014 at 2:06 AM, Ashwin Shankar > wrote: > >> Thanks Gordon and Stanley, but this would require us to bounce the >> process. >> Is there a way to change log levels without bouncing the process ? >> >> >> >> On Tue, Apr 15, 2014 at 3

Re: Setting debug log level for individual daemons

ile. > > log4j.logger.org.apache.hadoop.yarn.server.resourcemanager=DEBUG,console > > > On Tue, Apr 15, 2014 at 8:33 AM, Ashwin Shankar > wrote: > >> Hi, >> How do we set log level to debug for lets say only Resource manager >> and not the other hadoop daemons ? >> >> -- >> Thanks, >>

Setting debug log level for individual daemons

Hi, How do we set log level to debug for lets say only Resource manager and not the other hadoop daemons ? -- Thanks, Ashwin

Resetting dead datanodes list

Hi, Hadoop-1's name node UI displays dead datanodes even if those instances are terminated and are not part of the cluster anymore. Is there a way to reset the dead datenode list without bouncing namenode ? This would help me in my script(which would run nightly) which parses the html page,termina

Re: Job fails if I change HADOOP_USER_NAME

awei.com> wrote: > Hi > > > > The below stack trace is generic for any am launcher failed to launch. Can > debug on AM container logs, so get proper stacktrace.? > > > > > > Thanks & Regards > > Rohith Sharma K S > > > > *From:* Ashwin Sha

Job fails if I change HADOOP_USER_NAME

Hi, I'm writing a new feature in Fair scheduler and wanted to test it out by running jobs submitted by different users from my laptop. My sleep job runs fine as long as the user name is my mac user name. If I change my hadoop user name by setting HADOOP_USER_NAME, my jobs fail with the exception *