Re: Why jobtracker.jsp can safely call a non-thread-safe method of JT?

These are all getter apis which if you look through the implementations call some synchronized api's. Its just that they dont need to synchronized across all the completed/retired/running jobs for the web ui. thanks mahadev On Wed, Feb 16, 2011 at 12:47 AM, Min Zhou wrote: > Anyone can help me w

Re: Unable to build MR-279 due to unresolved ivy/maven dependencies ?

Stephen, The README is a little old. Well need to change the instructions in the README on installing the avro maven plugin. We'll change that today. Thanks for pointing it out. Youll have to install the avro maven plugin from https://github.com/phunt/avro-maven-plugin. Please let us know if yo

Re: Queries on MRv2

Praveen, Answers in line: > > Q) What happens if an ApplicationMaster asks a NM to launch a container and > then releases the container in the allocate call later? The Application Master only releases the container once the container is done. > > Q) So, the NM watches the UNIX Process/Container

Re: Queries on MRv2

s the container once the container > is done. > > Thanks, > Praveen > > On Wed, Jun 15, 2011 at 8:59 AM, Mahadev Konar wrote: > >> Praveen, >>  Answers in line: >> >> > >> > Q) What happens if an ApplicationMaster asks a NM to launch a con

Re: Building and Deploying MRv2

I just checked in Tom's patch for updated INSTALL instructions. Hope that helps. thanks mahadev On Wed, Jun 15, 2011 at 7:49 AM, Thomas Graves wrote: > > > On 6/15/11 8:54 AM, "Praveen Sripati" wrote: > >> Hi, >> >> I am trying to build and deploy MRv2 and following the instructions in the >> I

Re: MR-279 branch: Problem opening files on the namenode web console (need to pick patches from trunk to resolve)

HI Amhed, We are still working to start using trunk hdfs and trunk common for MR-279 (mapreduce). I think we are almost there and should be ready next week. For now we can probably skip fixing it since we are anyway going to move to trunk hdfs. Does that seem reasonable? thanks mahadev On Fri, J

Re: Jiras need review

Hi Josh, Ill review the jira's by EOD today. We should clean up our PA's (currently we have 102 of them) and get it down to reasonable/maintainable/searchable number. thanks mahadev On Tue, Jul 12, 2011 at 9:13 AM, Josh Wills wrote: > Please add me (jwills) to that query, which should grab MR-26

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

Just an update. Some good news. I have a patch for MR-279 to start working with trunk/common and trunk/hdfs (some minor fixes). I am facing some issues with secuirty which should be sorted out by EOD today. We should be ready to start the process of merging soon. thanks mahadev On Jul 12, 201

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

Wed, Jul 27, 2011 at 9:46 AM, Mahadev Konar wrote: > Just an update. > > Some good news. I have a patch for MR-279 to start working with trunk/common > and trunk/hdfs (some minor fixes). I am facing some issues with secuirty > which should be sorted out by EOD today. > We

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

, Todd Lipcon wrote: > Hey Mahadev, > > Have you been running findbugs/RAT on the branch? Other test-patch like > things? > > -Todd > > On Fri, Jul 29, 2011 at 12:06 AM, Mahadev Konar > wrote: > >> HI all, >> For those working on MR-279. >> >&g

Re: trouble building MR-279 branch

Thanks for pointing out Ravi. We are working on fixing it. The jars that you mention are created as part of MR-279 build. For building MR-279, all you have to do in mvn clean install -DskipTests and that should work (given you have already installed common and hdfs from trunk). thanks mahadev O

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

Hi all, We are almost done with fixing findbugs issues. Once done, we plan to start a vote to merge MR-279 into trunk. If you plan to review, this would be a good time. We hope to start the vote next week sometime. thanks mahadev On Fri, Jul 29, 2011 at 3:14 PM, Mahadev Konar wrote: > G

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

ime. We hope to start the vote next week sometime. > > > >thanks > >mahadev > > > >On Fri, Jul 29, 2011 at 3:14 PM, Mahadev Konar > >wrote: > > > >> Good point! We are working on this. Hopefully should get this done > >>within a >

Re: Not able to execute MRv2

Hi Praveen, This is an issue because of incompatible versions of avro. This occurs if you have avro 1.3.2 in your cp. We use avro 1.4. For the time being you can remove any occurence of avro 1.3.2. jar in your classpath and that should get you out of the error below. For fixing it in the righ

Re: Can't find main class when run ResourceManager or NodeManager

Matei, Are you sure you are using the latest common from trunk? The ClientCache was recently added within the last few weeks. Looks like its using some old version of hadoop common. Also the sl4j errors should be fixed in the latest MR-279 branch. thanks mahadev On Aug 11, 2011, at 1:52 P

Re: Can't find main class when run ResourceManager or NodeManager

or@0.0.0.0:8088 > 2011-08-11 15:25:44,775 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager > metrics system... > 2011-08-11 15:25:44,776 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping metrics source > QueueMetrics,q0=default > 2011

[VOTE] Merge MR-279 to trunk.

Hi all, We are excited to let you know that we have MR-279 ready to be merged to trunk. I have uploaded necessary details on https://issues.apache.org/jira/browse/MAPREDUCE-279. Please take a look and vote. Clearly I am +1 on it. thanks mahadev

Re: [VOTE] Merge MR-279 to trunk.

waway branch first (right > now) to ensure we don't mess it up, so please don't be surprised to see us do > this 'twice'. :) > > Arun > > On Aug 16, 2011, at 2:14 PM, Mahadev Konar wrote: > >> Hi all, >> >> We are excited to let you

Re: Building MR2

Eli, I think its protoc. Do you have protoc installed? mahadev On Thu, Aug 18, 2011 at 10:28 AM, Eli Collins wrote: > Hey gang, > > I'm running mvn clean install from the hadoop-mapreduce directory on > an  svn co of trunk (hadoop-common) and get the following: > > [ERROR] Failed to execute goa

Re: Notes for working on mapreduce trunk after the MR-279 merge.

I am seeing this issue again. Todd, do we need to fix something else? I committed 2 patches one yest and one today and I cant get it on the git branch. Something off with git pulling from svn. Any other fix we need to do? thanks mahadev On Aug 18, 2011, at 10:05 AM, Vinod Kumar Vavilapalli wr

Re: Notes for working on mapreduce trunk after the MR-279 merge.

Fri, Aug 19, 2011 at 10:28 AM, Todd Lipcon wrote: >> Let me ping Jukka and see if he has any idea why it's not auto-updating >> anymore. >> >> -Todd >> >> On Fri, Aug 19, 2011 at 10:26 AM, Mahadev Konar >> wrote: >>> I am seeing this issue

Re: DistCpV2 in 0.23

+1 for a seperate module in hadoop-mapreduce-project. I think hadoop-mapreduce-client might not be right place for it. We might have to pick a new maven module under hadoop-mapreduce-project that could host streaming/distcp/hadoop archives. thanks mahadev On Thu, Aug 25, 2011 at 11:04 AM, Alejand

Fwd: Hadoop-Mapreduce-trunk-Commit - Build # 789 - Still Failing

Looks like we might need to clean the ivy cache on the build machine? mahadev Begin forwarded message: > From: Apache Jenkins Server > Date: August 25, 2011 12:32:40 PM PDT > To: mapreduce-dev@hadoop.apache.org > Subject: Hadoop-Mapreduce-trunk-Commit - Build # 789 - Still Failing > Reply-To: m

Re: DistCpV2 in 0.23

e rewrite and did not want to remove it until users are familiarized > with new one. > > On 8/26/11 12:51 AM, "Todd Lipcon" wrote: > > Maybe a separate toplevel for hadoop-tools? Stuff like RAID could go > in there as well - ie tools that are downstream of MR and/or H

Re: Trunk and 0.23 build failing with clean .m2 directory

Bobby, You are right. The test-patch uses mvn compile. Please file a jira. It should be a minor change: thanks mahadev On Mon, Aug 29, 2011 at 10:34 AM, Robert Evans wrote: > Thanks Alejandro, > > That really clears things up. Is the a JIRA you know of to change test-patch > to do mvn test -Ds

Re: Jenkins's Links to FindBugs warnings not useful

Thanks Giri! mahadev On Fri, Sep 2, 2011 at 10:42 AM, Giridharan Kesavan wrote: > artifact archive location for mr per-comitt job on jenkins updated. > -Giri > > On Fri, Sep 2, 2011 at 7:33 AM, Vinod Kumar Vavilapalli > wrote: >> None of the links to the warnings related to FindBugs by Jenkins

Re: Mavenization for MR1 - HADOOP-7590

Yes, I think we need this, both for 0.23 and 0.24. The current form of build is causing a lot of issues. Folks are not aware of checking ant after running maven and unknowingly break the builds. thanks mahadev On Sep 7, 2011, at 7:16 AM, Alejandro Abdelnur wrote: > Do we still want/need to

Re: Hadoop Tools Layout (was Re: DistCpV2 in 0.23)

I like the idea of having tools as a seperate module and I dont think that it will be a dumping ground unless we choose to make one of it. +1 for hadoop tools module under trunk. thanks mahadev On Wed, Sep 7, 2011 at 11:18 AM, Alejandro Abdelnur wrote: > Agreed, we should not have a dumping gro

Re: A complicated procedure to Run MRV2.

Nice Prashanth. We've been trying to add documents to make it easier for users to deploy/test/admin mrv2. Here is a doc on setting up a single node cluster. It would be great if you'd want to add/refine it. http://s.apache.org/mrv2_setup Also, there are some more to write: https://issues.apache.o

Trunk commit builds.

Looks like the trunk commit builds are failing/crashing. https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Commit/1060/ I suspect its: MAPREDUCE-3033. Hitesh, could you please take a look? We definitely need to mavenize MRv1 as soon as possible. thanks mahadev

Re: Trunk commit builds.

Thanks Hitesh! mahadev On Mon, Oct 10, 2011 at 11:34 AM, Hitesh Shah wrote: > Opened MAPREDUCE-3158 for fixing the builds. Seems like the change for > default value of framework name to yarn from classic broke a couple of the > MRv1 tests. > > -- Hitesh > > On Oct 9, 2011,

Re: Yarn NodeManager services

Shuffle is an auxiliary service in the NM. You might want to take a look at AuxServices.java in the NodeManager codebase. Also, you can add config parameters to yarn.nodemanager.aux-services mapreduce.shuffle (config knob in NodeManagers) to add more configurable services in the NodeMan

Re: Application/Job is hanging! validate

Kamesh, This is what will happen if you do not have enough resource in the cluster to run a job. Though there could be an improvement that a job fails with not enough resource in the cluster but thats not what happens right now. thanks mahadev On Mon, Oct 17, 2011 at 6:25 AM, Kamesh wrote: > H

Re: Hadoop-Mapreduce-trunk-Commit - Build # 1175 - Still Failing

Adding Giri. He's fixing some issues with the autoreconf and other software installations. mahadev On Fri, Oct 28, 2011 at 11:12 AM, Todd Lipcon wrote: > Actually, it looks like the libssl0.9.8 package is installed but not > the libssl-dev package. > > Rajiv, would you mind if I installed the li

Re: MapReduce and MPI

Ralph, I think there is a lot of interest in MPI over Hadoop. If you are counting votes here's my +1. mahadev On Thu, Dec 1, 2011 at 3:31 AM, Ralph Castain wrote: > Hi folks > > I'm a lead developer on the Open MPI project, and recently joined the Hadoop > community to help with Hamster. A cou

Re: Hadoop-Mapreduce-trunk-Commit - Build # 1437 - Still Failing

Todd, There is a jira for it. https://issues.apache.org/jira/browse/MAPREDUCE-3541 Ravi will be posting a patch ASAP. thanks mahadev On Tue, Dec 13, 2011 at 10:48 AM, Todd Lipcon wrote: > Is anyone working on this? Been failing a few days and my inbox is sad :( > > -Todd > > -- Forwa

Re: Yarn build error

Milind, The build command has changed for Yarn. Now its uniform all over hadoop. You can run: mvn clean package -DskipTests -Pdist -Dtar -Dmaven.javadoc.skip=true to create tarballs now. mahadev On Thu, Jan 12, 2012 at 2:25 PM, wrote: > Folks, > > I am getting the "Error reading assemblie

Re: How is MRv2 fundamentally changed?

Hi Jie, You might want to read through: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/YARN.html and http://developer.yahoo.com/blogs/hadoop/posts/2011/02/mapreduce-nextgen/ for more information on the architecture. Itll help you understand the major differences between

Re: New subproject logos

+1 mahadev On 6/26/09 12:13 AM, "Chris Douglas" wrote: > +1 > > On Thu, Jun 25, 2009 at 11:42 PM, Nigel Daley wrote: >> Here are some logos for the new subprojects >> http://www.flickr.com/photos/88199...@n00/3661433605/ >> >> Please vote +1 if you like 'em and -1 if you don't. >> >> Cheer

[jira] [Created] (MAPREDUCE-2403) MR-279: Improve job history event handling in AM to log to HDFS

Type: Improvement Components: mrv2 Reporter: Mahadev konar Assignee: Krishna Ramachandran Fix For: 0.23.0 Improve the job history event handling in the application master to log to HDFS in the staging directory for the job and also move it to the

[jira] [Created] (MAPREDUCE-2405) MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)

Issue Type: Improvement Components: mrv2 Reporter: Mahadev konar Assignee: Greg Roelofs Fix For: 0.23.0 "Port" MAPREDUCE-1220 to MRv2. This is an optimization for small jobs wherein all tasks run on the same node in the same JVM

[jira] [Resolved] (MAPREDUCE-2405) MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)

[ https://issues.apache.org/jira/browse/MAPREDUCE-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2405. -- Resolution: Fixed I just pushed this to MR-279 branch. thanks greg. > MR-

[jira] [Resolved] (MAPREDUCE-2432) MR-279: Install sanitized poms for downstream sanity

[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2432. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this. thanks

[jira] [Created] (MAPREDUCE-2438) MR-279: WebApp for Job History

: 0.23.0 Reporter: Mahadev konar Assignee: Krishna Ramachandran Add webapp for job history server in MR-279 branch. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAPREDUCE-2439) MR-279: Fix YarnRemoteException to give more details.

Components: mrv2 Reporter: Mahadev konar Assignee: Siddharth Seth Fix For: 0.23.0 Fix YarnRemoteException to add more details. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (MAPREDUCE-2424) MR-279: counters/UI/etc. for uber-AppMaster (in-cluster LocalJobRunner for MRv2)

[ https://issues.apache.org/jira/browse/MAPREDUCE-2424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2424. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this. thanks

[jira] [Resolved] (MAPREDUCE-2440) MR-279: Name clashes in TypeConverter

[ https://issues.apache.org/jira/browse/MAPREDUCE-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2440. -- Resolution: Fixed I just committed this to MR-279. thanks luke. > MR-279: N

[jira] [Resolved] (MAPREDUCE-2433) MR-279: YARNApplicationConstants hard code app master jar version

[ https://issues.apache.org/jira/browse/MAPREDUCE-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2433. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this to MR-279

[jira] [Resolved] (MAPREDUCE-2458) MR-279: Rename sanitized pom.xml in build directory to work around IDE bug

[ https://issues.apache.org/jira/browse/MAPREDUCE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2458. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

[jira] [Resolved] (MAPREDUCE-2480) MR-279: mr app should not depend on hard-coded version of shuffle

[ https://issues.apache.org/jira/browse/MAPREDUCE-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2480. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

[jira] [Resolved] (MAPREDUCE-2500) MR 279: PB factories are not thread safe

[ https://issues.apache.org/jira/browse/MAPREDUCE-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2500. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this to MR-279

[jira] [Resolved] (MAPREDUCE-2504) MR 279: race in JobHistoryEventHandler stop

[ https://issues.apache.org/jira/browse/MAPREDUCE-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2504. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

[jira] [Resolved] (MAPREDUCE-2509) MR-279: Fix NPE in UI for pending attempts

[ https://issues.apache.org/jira/browse/MAPREDUCE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2509. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this to MR-279

[jira] [Resolved] (MAPREDUCE-2522) MR 279: Security for JobHistory service

[ https://issues.apache.org/jira/browse/MAPREDUCE-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2522. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this to MR-279

[jira] [Resolved] (MAPREDUCE-2534) MR-279: Fix CI breaking hard coded version in jobclient pom

[ https://issues.apache.org/jira/browse/MAPREDUCE-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2534. -- Resolution: Fixed Hadoop Flags: [Reviewed] > MR-279: Fix CI breaking h

[jira] [Resolved] (MAPREDUCE-2527) MR-279: Metrics for MRAppMaster

[ https://issues.apache.org/jira/browse/MAPREDUCE-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2527. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this to MR-279

[jira] [Resolved] (MAPREDUCE-2532) MR-279: Metrics for NodeManager

[ https://issues.apache.org/jira/browse/MAPREDUCE-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2532. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

[jira] [Resolved] (MAPREDUCE-2556) MR 279: NodeStatus.getNodeHealthStatus().setBlah broken

[ https://issues.apache.org/jira/browse/MAPREDUCE-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2556. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this. thanks sid

[jira] [Resolved] (MAPREDUCE-2566) MR 279: YarnConfiguration should reloadConfiguration if instantiated with a non YarnConfiguration object

[ https://issues.apache.org/jira/browse/MAPREDUCE-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2566. -- Resolution: Fixed I just pushed this. thanks sid! > MR 279: YarnConfigurat

[jira] [Resolved] (MAPREDUCE-2580) MR 279: RM UI should redirect finished jobs to History UI

[ https://issues.apache.org/jira/browse/MAPREDUCE-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2580. -- Resolution: Fixed This has been committed to the branch. > MR 279: RM UI sho

[jira] [Resolved] (MAPREDUCE-2707) ProtoOverHadoopRpcEngine without using TunnelProtocol over WritableRpc

[ https://issues.apache.org/jira/browse/MAPREDUCE-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2707. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

[jira] [Resolved] (MAPREDUCE-2776) MR 279: Fix some of the yarn findbug warnings

[ https://issues.apache.org/jira/browse/MAPREDUCE-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2776. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this. thanks sid

[jira] [Created] (MAPREDUCE-2860) Fix log4j logging in the maven test cases.

Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 At present the logging in the new test cases is broken because surefire isnt able to find the log4j properties file. -- This message is automatically generated by JIRA. For more information on JIRA

[jira] [Created] (MAPREDUCE-2867) Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.

Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 TestApplicationCleanup in resourcemanager/applicationsmanager doesnt do anything. There is already a test in resourcemanager/TestApplicationCleanup which tests

[jira] [Resolved] (MAPREDUCE-2867) Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.

[ https://issues.apache.org/jira/browse/MAPREDUCE-2867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2867. -- Resolution: Fixed Removed the file via svn commit revision 1160340. > Rem

[jira] [Resolved] (MAPREDUCE-2848) Upgrade avro to 1.5.2

[ https://issues.apache.org/jira/browse/MAPREDUCE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2848. -- Resolution: Fixed checked in as part of MAPREDUCE-2868. > Upgrade avro to 1.

[jira] [Created] (MAPREDUCE-2870) MR tests are failing webapps/hdfs not found in CLASSPATH

Reporter: Mahadev konar Fix For: 0.23.0 While running ant tests, the tests are failing with the following trace: {noformat} webapps/hdfs not found in CLASSPATH java.io.FileNotFoundException: webapps/hdfs not found in CLASSPATH at

[jira] [Created] (MAPREDUCE-2877) Add missing Apache license header in some files in MR and also add the rat plugin to the poms.

-2877 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 Some of the files in MR have a missing Apache header files. We also need to add the apache-rat plugin to be able to run rat

[jira] [Created] (MAPREDUCE-2886) Fix Javadoc warnings in MapReduce.

Fix Javadoc warnings in MapReduce. -- Key: MAPREDUCE-2886 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2886 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar

[jira] [Created] (MAPREDUCE-2908) Fix findbugs warnings in Map Reduce.

Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 In the current trunk/0.23 codebase there are 5 findbugs warnings which cause the precommit CI builds to -1 the patches. -- This message is automatically

[jira] [Created] (MAPREDUCE-2916) Ivy build for MRv1 fails with bad organization for common daemon.

Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 This jira is to ignore ivy resolve errors because of bad poms in common daemons. -- This message is automatically generated by JIRA. For more information on JIRA, see: http

[jira] [Resolved] (MAPREDUCE-2926) 500 Error in ResourceManager UI

[ https://issues.apache.org/jira/browse/MAPREDUCE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2926. -- Resolution: Duplicate I just verified. I dont see the issue anymore. Marking it as

[jira] [Created] (MAPREDUCE-2937) Errors in Application failures are not shown in the client trace.

Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Priority: Critical Fix For: 0.23.0, 0.24.0 The client side does not show enough information on why the job failed. Here is step to reproduce it: 1) set the scheduler

[jira] [Created] (MAPREDUCE-2961) Increase the default threadpool fize for container launching in the application master.

Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 Currently the default threadpool size is 10 for launching containers in

[jira] [Created] (MAPREDUCE-2963) TestMRJobs hangs waiting to connect to history server.

Affects Versions: 0.23.0 Reporter: Mahadev konar Priority: Critical Fix For: 0.23.0 TestMRJobs is hanging waiting to connect to history server. I will post the logs next. -- This message is automatically generated by JIRA. For more information on JIRA, see: http

[jira] [Created] (MAPREDUCE-2975) ResourceManager Delegate is not getting initialized with yarn-site.xml as default configuration.

/MAPREDUCE-2975 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 MAPREDUCE-2937 accidentally changes ResourceMgrDelegate so

[jira] [Created] (MAPREDUCE-2990) Health Report on Resource Manager UI is null if the NM's are all healthy.

p Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar The web UI on the RM for the link Nodes shows that Health-report as null when the NM is healthy. This is a simple fix where in we can check for null in NodesPage

[jira] [Resolved] (MAPREDUCE-2964) mapreduce trunk build fails with compile-mapred-test ant target

[ https://issues.apache.org/jira/browse/MAPREDUCE-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2964. -- Resolution: Fixed Just fixed this. No longer an issue on trunk builds

[jira] [Created] (MAPREDUCE-3017) The Web UI shows FINISHED for killed/successful/failed jobs.

: Bug Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 The RM web ui shows FINISHED status for all the jobs even if they failed/killed or were successful. This should be fixed. Only the

[jira] [Created] (MAPREDUCE-3018) Streaming jobs with -file option fail to run.

Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 Streaming jobs fail to run with the -file option. hadoop jar streaming.jar -input input.txt -output Out -mapper "mapper.sh"

[jira] [Created] (MAPREDUCE-3073) Build failure for MRv1 caused due to changes to MRConstants.

: Bug Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 When runnning ant -Dresolvers=internal binary, the build seems to be failing with: [javac] public class JobTracker implements

[jira] [Created] (MAPREDUCE-3093) Write tests for data locality in MRv2.

Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 We should add tests to make sure data locality is in place in MRv2 (with respect to the capacity scheduler and also the matching/ask of containers in the MR AM). -- This

[jira] Created: (MAPREDUCE-730) allow relative paths to be created inside archives.

Components: harchive Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.21.0 This jira is a shadow jira for mapreduce changes related to HADOOP-3663. Archives currently stores the full path from the input sources - since it allows multiple sources and regular

[jira] Created: (MAPREDUCE-1010) Adding tests for changes in archives.

Adding tests for changes in archives. - Key: MAPREDUCE-1010 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1010 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar

[jira] Resolved: (MAPREDUCE-1111) JT Jetty UI not working if we run mumak.sh off packaged distribution directory.

[ https://issues.apache.org/jira/browse/MAPREDUCE-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to both

[jira] Resolved: (MAPREDUCE-1010) Adding tests for changes in archives.

[ https://issues.apache.org/jira/browse/MAPREDUCE-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-1010. -- Resolution: Fixed This was committed as a part of HADOOP-6097. > Adding te

[jira] Created: (MAPREDUCE-1428) Make block size and the size of archive created files configurable.

Issue Type: Improvement Components: harchive Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.22.0 Currently the block size used by archives is the default block size of the hdfs filesystem. We need to make it configurable so that the

[jira] Created: (MAPREDUCE-1458) Test case for regular expression in Hadoop Archives.

Components: test Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.22.0 We need a test case for regular expressions with hadoop archives. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the

[jira] Created: (MAPREDUCE-1474) forrest docs for achives is out of date.

Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.22.0 The docs for archives are out of date. The new docs that were checked into hadoop common were lost because of the project split. -- This message is automatically generated by JIRA. - You can

[jira] Created: (MAPREDUCE-1514) Add documentation on permissions, limitations, error handling for archives.

/Reduce Issue Type: Improvement Components: documentation Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.22.0 add documentaion on permissions aspect of archives and other limitations that it might have. Also add documentation on

[jira] Created: (MAPREDUCE-1590) Move HarFileSystem from Hadoop Common to Mapreduce tools.

: Improvement Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.22.0 Keeping HarFileSystem in Hadoop Common has been a mistake since we sometimes cannot make changes to archives without breaking build across common and mapreduce. Also, it would be good to

[jira] Resolved: (MAPREDUCE-1509) Partition size of Hadoop Archives should be configurable

[ https://issues.apache.org/jira/browse/MAPREDUCE-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-1509. -- Resolution: Duplicate resolving this as duplicate of MAPREDUCE-1428. > Partit

[jira] Created: (MAPREDUCE-1615) ant test on trunk does not compile.

ant test on trunk does not compile. --- Key: MAPREDUCE-1615 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1615 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar

[jira] Resolved: (MAPREDUCE-142) Race condition in DistributedCache

[ https://issues.apache.org/jira/browse/MAPREDUCE-142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-142. - Resolution: Cannot Reproduce This should be fixed in the current trunk. please reopen

[jira] Created: (MAPREDUCE-1943) Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes

Issue Type: Improvement Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.22.0 We have come across issues in production clusters wherein users abuse counters, statusreport messages and split sizes. One such case was when one of the users

[jira] Created: (MAPREDUCE-1960) Limit the size of jobconf.

Limit the size of jobconf. -- Key: MAPREDUCE-1960 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1960 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Mahadev konar Assignee

[jira] Created: (MAPREDUCE-1965) Add info for job failure on jobtracker UI.

: Mahadev konar Assignee: Mahadev konar Attachments: MAPREDUCE-1965-yahoo-hadoop-0.20S.patch MAPREDUCE-1521 added a filed to jobstatus to mark reason for failures of the job. This information needs to be displayed on the jobtracker UI. -- This message is automatically generated

[jira] [Created] (MAPREDUCE-3099) Add docs for setting up a single node MRv2 cluster.

Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure

[jira] [Resolved] (MAPREDUCE-3110) Hadoop-Mapreduce-0.23-Build - Build # 29 - Still Failing

[ https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3110. -- Resolution: Duplicate Duplicate of MAPREDUCE-3054. > Had

[jira] [Reopened] (MAPREDUCE-3064) 27 unit test failures with Invalid "mapreduce.jobtracker.address" configuration value for JobTracker: "local"

[ https://issues.apache.org/jira/browse/MAPREDUCE-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar reopened MAPREDUCE-3064: -- Reopening the issue, we are still having issues with the build: https

  1   2   >