Hi Vinod I am actually using the 'mvn package -Pdist -DskipTests -Dtar' command as described in the BUILDING.txt file after cloning Hadoop. Then I am using the eclipse goal with mvn to set it up for eclipse as described in < http://wiki.apache.org/hadoop/EclipseEnvironment> page. To setup YARN, I am following instructions in < http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/SingleCluster.html >.
Finally setting up the env variables and starting the namenode/datanode/resourcemanager/nodemanager and historyserver using appropriate daemons. I just saw the below mentioned JIRA and tried commenting out the code for CapacityScheduler. Since this worked, I thought it might be an issue with using CapacityScheduler. Any idea on the root cause? Even now, if I add the property in my yarn-site.xml, the job hangs or gives the below error. I am new to Hadoop (used it before but never setup for making changes to it) so not sure if I am doing something wrong in building/setting it up. Regards, Prajakta On Fri, Jun 22, 2012 at 2:36 AM, Vinod Kumar Vavilapalli < vino...@hortonworks.com> wrote: > > This has got nothing to do with the scheduler. > > I believe this has got to do with some compilation issue. How did you > build hadoop? > > Also, I found that the repo at github (which is a mirror of git repo at > apache) doesn't always pick all the commits immediately. You are better off > checking out from svn ( > http://hadoop.apache.org/common/version_control.html) > > HTH, > +Vinod > > > On Jun 21, 2012, at 9:34 AM, Prajakta Kalmegh wrote: > > > Okay, the below problem was fixed after commenting out the property to > use > > CapacityScheduler from my yarn-site.xml. I found a reference to this in > > this JIRA <https://issues.apache.org/jira/browse/MAPREDUCE-4339>. YARN > now > > uses the default FIFO scheduler. > > > > Is the Dispatcher used in the CapacityScheduler's (or any scheduler's) > > flow? > > > > Thanks in advance. > > Regards, > > Prajakta > > > > > > > > > > > > On Thu, Jun 21, 2012 at 10:53 AM, Prajakta Kalmegh <pkalm...@gmail.com > >wrote: > > > >> Hi Jagat > >> > >> Thanks for your reply. I am not using Pig. I have the latest hadoop > >> running cloned from github trunk. Actually I was able to execute my > >> programs until I refreshed my github foked copy yesterday and ran a > build > >> on it. :( > >> > >> I figured out from a debug that the lines that are creating a problem > are: > >> ---------------- > >> //service to log job history events > >> EventHandler<JobHistoryEvent> historyService = > >> createJobHistoryHandler(context); > >> > >> > dispatcher.register(org.apache.hadoop.mapreduce.jobhistory.EventType.class, > >> historyService); > >> ---------------- > >> in the MRAppMaster.java class. The reason being the register method > >> from org.apache.hadoop.yarn.event.Dispatcher is not > >> accepting EventHandler<JobHistoryEvent> as the second argument. Not sure > >> why. > >> > >> Regards, > >> Prajakta > >> > >> > >> > >> On Thu, Jun 21, 2012 at 10:22 AM, Jagat Singh <jagatsi...@gmail.com > >wrote: > >> > >>> Seems you are using Pig with Hadoop 0.23 or 2.0 version. > >>> > >>> Can you quickly recompile pig with 23 option and try this again. > >>> > >>> > >>> > >>> On Thu, Jun 21, 2012 at 10:02 AM, Prajakta Kalmegh <pkalm...@gmail.com > >>>> wrote: > >>> > >>>> Hi > >>>> > >>>> I am getting the following error while trying to execute any example > >>>> (wordcount, terasort etc): > >>>> 12/06/21 09:52:39 INFO mapreduce.Job: Running job: > >>> job_1340251923324_0001 > >>>> 12/06/21 09:52:45 INFO mapreduce.Job: Job job_1340251923324_0001 > >>> running in > >>>> uber mode : false > >>>> 12/06/21 09:52:45 INFO mapreduce.Job: map 0% reduce 0% > >>>> 12/06/21 09:52:45 INFO mapreduce.Job: Job job_1340251923324_0001 > failed > >>>> with state FAILED due to: Application application_1340251923324_0001 > >>> failed > >>>> 1 times due to AM Container for appattempt_1340251923324_0001_000001 > >>> exited > >>>> with exitCode: 1 due to: > >>>> .Failing this attempt.. Failing the application. > >>>> 12/06/21 09:52:45 INFO mapreduce.Job: Counters: 0 > >>>> ----------------------------- > >>>> > >>>> *The contents of containers logs are pasted below:* > >>>> 2012-06-21 09:52:43,856 INFO [main] > >>>> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster > for > >>>> application appattempt_1340251923324_0001_000001 > >>>> 2012-06-21 09:52:44,625 FATAL [main] > >>>> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting > >>> MRAppMaster > >>>> java.lang.Error: Unresolved compilation problems: > >>>> The method register(Class<? extends Enum>, EventHandler) in the type > >>>> Dispatcher is not applicable for the arguments (Class<EventType>, > >>>> EventHandler<JobHistoryEvent>) > >>>> org.apache.hadoop.mapreduce.jobhistory.EventType cannot be resolved to > >>> a > >>>> type > >>>> > >>>> at > >>>> > >>> > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.init(MRAppMaster.java:261) > >>>> at > >>>> > >>> > org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1049) > >>>> at java.security.AccessController.doPrivileged(Native Method) > >>>> at javax.security.auth.Subject.doAs(Subject.java:415) > >>>> at > >>>> > >>>> > >>> > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) > >>>> at > >>>> > >>>> > >>> > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1046) > >>>> at > >>>> > >>> > org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1010) > >>>> 2012-06-21 09:52:44,628 INFO [Thread-0] > >>>> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a > >>>> signal. Signaling RMCommunicator and JobHistoryEventHandler. > >>>> > >>>> ---------------------- > >>>> > >>>> Any idea what would be going wrong? > >>>> > >>>> Regards, > >>>> Prajakta > >>>> > >>> > >> > >> > >
