the capacityscheduler jar in share/hadoop/mapreduce/*resourcemanager*.jar already has a default capacity-default.xml in it. I think maybe this is a packaging issue.
2012/6/22 Prajakta Kalmegh <pkalm...@gmail.com> > Hi Vinod > > I am actually using the 'mvn package -Pdist -DskipTests -Dtar' command as > described in the BUILDING.txt file after cloning Hadoop. Then I am using > the eclipse goal with mvn to set it up for eclipse as described in < > http://wiki.apache.org/hadoop/EclipseEnvironment> page. To setup YARN, I > am > following instructions in < > > http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/SingleCluster.html > >. > > Finally setting up the env variables and starting the > namenode/datanode/resourcemanager/nodemanager and historyserver using > appropriate daemons. > > I just saw the below mentioned JIRA and tried commenting out the code for > CapacityScheduler. Since this worked, I thought it might be an issue with > using CapacityScheduler. Any idea on the root cause? > > Even now, if I add the property in my yarn-site.xml, the job hangs or gives > the below error. I am new to Hadoop (used it before but never setup for > making changes to it) so not sure if I am doing something wrong in > building/setting it up. > > Regards, > Prajakta > > On Fri, Jun 22, 2012 at 2:36 AM, Vinod Kumar Vavilapalli < > vino...@hortonworks.com> wrote: > > > > > This has got nothing to do with the scheduler. > > > > I believe this has got to do with some compilation issue. How did you > > build hadoop? > > > > Also, I found that the repo at github (which is a mirror of git repo at > > apache) doesn't always pick all the commits immediately. You are better > off > > checking out from svn ( > > http://hadoop.apache.org/common/version_control.html) > > > > HTH, > > +Vinod > > > > > > On Jun 21, 2012, at 9:34 AM, Prajakta Kalmegh wrote: > > > > > Okay, the below problem was fixed after commenting out the property to > > use > > > CapacityScheduler from my yarn-site.xml. I found a reference to this in > > > this JIRA <https://issues.apache.org/jira/browse/MAPREDUCE-4339>. YARN > > now > > > uses the default FIFO scheduler. > > > > > > Is the Dispatcher used in the CapacityScheduler's (or any scheduler's) > > > flow? > > > > > > Thanks in advance. > > > Regards, > > > Prajakta > > > > > > > > > > > > > > > > > > On Thu, Jun 21, 2012 at 10:53 AM, Prajakta Kalmegh <pkalm...@gmail.com > > >wrote: > > > > > >> Hi Jagat > > >> > > >> Thanks for your reply. I am not using Pig. I have the latest hadoop > > >> running cloned from github trunk. Actually I was able to execute my > > >> programs until I refreshed my github foked copy yesterday and ran a > > build > > >> on it. :( > > >> > > >> I figured out from a debug that the lines that are creating a problem > > are: > > >> ---------------- > > >> //service to log job history events > > >> EventHandler<JobHistoryEvent> historyService = > > >> createJobHistoryHandler(context); > > >> > > >> > > > dispatcher.register(org.apache.hadoop.mapreduce.jobhistory.EventType.class, > > >> historyService); > > >> ---------------- > > >> in the MRAppMaster.java class. The reason being the register method > > >> from org.apache.hadoop.yarn.event.Dispatcher is not > > >> accepting EventHandler<JobHistoryEvent> as the second argument. Not > sure > > >> why. > > >> > > >> Regards, > > >> Prajakta > > >> > > >> > > >> > > >> On Thu, Jun 21, 2012 at 10:22 AM, Jagat Singh <jagatsi...@gmail.com > > >wrote: > > >> > > >>> Seems you are using Pig with Hadoop 0.23 or 2.0 version. > > >>> > > >>> Can you quickly recompile pig with 23 option and try this again. > > >>> > > >>> > > >>> > > >>> On Thu, Jun 21, 2012 at 10:02 AM, Prajakta Kalmegh < > pkalm...@gmail.com > > >>>> wrote: > > >>> > > >>>> Hi > > >>>> > > >>>> I am getting the following error while trying to execute any example > > >>>> (wordcount, terasort etc): > > >>>> 12/06/21 09:52:39 INFO mapreduce.Job: Running job: > > >>> job_1340251923324_0001 > > >>>> 12/06/21 09:52:45 INFO mapreduce.Job: Job job_1340251923324_0001 > > >>> running in > > >>>> uber mode : false > > >>>> 12/06/21 09:52:45 INFO mapreduce.Job: map 0% reduce 0% > > >>>> 12/06/21 09:52:45 INFO mapreduce.Job: Job job_1340251923324_0001 > > failed > > >>>> with state FAILED due to: Application application_1340251923324_0001 > > >>> failed > > >>>> 1 times due to AM Container for appattempt_1340251923324_0001_000001 > > >>> exited > > >>>> with exitCode: 1 due to: > > >>>> .Failing this attempt.. Failing the application. > > >>>> 12/06/21 09:52:45 INFO mapreduce.Job: Counters: 0 > > >>>> ----------------------------- > > >>>> > > >>>> *The contents of containers logs are pasted below:* > > >>>> 2012-06-21 09:52:43,856 INFO [main] > > >>>> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster > > for > > >>>> application appattempt_1340251923324_0001_000001 > > >>>> 2012-06-21 09:52:44,625 FATAL [main] > > >>>> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting > > >>> MRAppMaster > > >>>> java.lang.Error: Unresolved compilation problems: > > >>>> The method register(Class<? extends Enum>, EventHandler) in the type > > >>>> Dispatcher is not applicable for the arguments (Class<EventType>, > > >>>> EventHandler<JobHistoryEvent>) > > >>>> org.apache.hadoop.mapreduce.jobhistory.EventType cannot be resolved > to > > >>> a > > >>>> type > > >>>> > > >>>> at > > >>>> > > >>> > > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.init(MRAppMaster.java:261) > > >>>> at > > >>>> > > >>> > > > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1049) > > >>>> at java.security.AccessController.doPrivileged(Native Method) > > >>>> at javax.security.auth.Subject.doAs(Subject.java:415) > > >>>> at > > >>>> > > >>>> > > >>> > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) > > >>>> at > > >>>> > > >>>> > > >>> > > > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1046) > > >>>> at > > >>>> > > >>> > > > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1010) > > >>>> 2012-06-21 09:52:44,628 INFO [Thread-0] > > >>>> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster > received a > > >>>> signal. Signaling RMCommunicator and JobHistoryEventHandler. > > >>>> > > >>>> ---------------------- > > >>>> > > >>>> Any idea what would be going wrong? > > >>>> > > >>>> Regards, > > >>>> Prajakta > > >>>> > > >>> > > >> > > >> > > > > >
