[jira] [Created] (MAPREDUCE-3853) TestLinuxResourceCalculatorPlugin is failing on trunk and 0.23 branch.

2012-02-10 Thread Mahadev konar (Created) (JIRA)
Issue Type: Bug Components: mrv2 Reporter: Mahadev konar Fix For: 0.23.2 Looks like the test is failing: https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-0.23-Build/188/console -- This message is automatically generated by JIRA. If you

[jira] [Created] (MAPREDUCE-3822) TestJobCounters is failing intermittently on trunk and 0.23.

2012-02-06 Thread Mahadev konar (Created) (JIRA)
: Bug Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.1 TestJobCounters fails sometimes on trunk. I have tracked it down to stats issue in FileSystem. Still working on it. -- This message is automatically generated

[jira] [Created] (MAPREDUCE-3774) yarn-default.xml should be moved to hadoop-yarn-common.

2012-01-31 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.1 yarn-default.xml right now resides in hadoop-yarn-server-common jars which is not the right thing to do since this jar might not be needed in some

[jira] [Created] (MAPREDUCE-3762) Resource Manager fails to come up with default capacity scheduler configs.

2012-01-30 Thread Mahadev konar (Created) (JIRA)
/Reduce Issue Type: Bug Components: mrv2 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Thanks to [~harip] for pointing out the issue. This is the stack trace for bringing up RM with default CS configs: {code

[jira] [Created] (MAPREDUCE-3743) Documentation for Log Aggregation and log retrieval.

2012-01-27 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Priority: Critical Retrieving logs in 0.23 is very different from what 0.20.* does. This is a very new feature which will require good documentation for users to get used to it. Lets make sure we have some

[jira] [Created] (MAPREDUCE-3733) Add Apache License Header to hadoop-distcp/pom.xml

2012-01-26 Thread Mahadev konar (Created) (JIRA)
Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.1 Looks like I missed the Apache Headers in the review. Adding it now. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA

[jira] [Resolved] (MAPREDUCE-3733) Add Apache License Header to hadoop-distcp/pom.xml

2012-01-26 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3733. -- Resolution: Fixed I went ahead and committed this. Add Apache

[jira] [Created] (MAPREDUCE-3735) Add distcp jar to the distribution (tar)

2012-01-26 Thread Mahadev konar (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 Distcp jar isnt getting added to the tarball as of now. We need to add it along with archives/streaming and others. -- This message

[jira] [Created] (MAPREDUCE-3717) JobClient test jar has missing files to run all the test programs.

2012-01-24 Thread Mahadev konar (Created) (JIRA)
Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 Looks like MAPREDUCE-3582 forgot to move couple of files from the ant builds. The current test

[jira] [Resolved] (MAPREDUCE-3525) Shuffle benchmark is nearly 1.5x slower in 0.23

2012-01-22 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3525. -- Resolution: Fixed Fix Version/s: 0.23.1 Fixed via MAPREDUCE-3641. Thanks

[jira] [Created] (MAPREDUCE-3701) Delete HadoopYarnRPC from 0.23 branch.

2012-01-19 Thread Mahadev konar (Created) (JIRA)
Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.1 HadoopYarnRPC file exists in 0.23 (should have been removed with the new HadoopYarnProtoRPC). Trunk does not have this issue. -- This message is automatically generated by JIRA. If you think it was sent

[jira] [Resolved] (MAPREDUCE-3647) Pipes job fails with Illegal text protocol

2012-01-16 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3647. -- Resolution: Not A Problem Looks like the issue has been fixed. No longer an issue

[jira] [Created] (MAPREDUCE-3679) AM logs and others should not automatically refresh after every 1 second.

2012-01-16 Thread Mahadev konar (Created) (JIRA)
/Reduce Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Mahadev konar If you are looking through the logs for AM or containers, the page is automatically refreshed after 1 second or so which makes it problematic to search through the page or debug using

Re: How is MRv2 fundamentally changed?

2012-01-16 Thread Mahadev Konar
Hi Jie, You might want to read through: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/YARN.html and http://developer.yahoo.com/blogs/hadoop/posts/2011/02/mapreduce-nextgen/ for more information on the architecture. Itll help you understand the major differences

[jira] [Created] (MAPREDUCE-3649) Job End notification gives an error on calling back.

2012-01-09 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.1 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 When calling job end notification for oozie the AM fails with the following trace: {noformat} 2012-01-09 23:45:41,732 WARN

[jira] [Created] (MAPREDUCE-3624) bin/yarn script adds jdk tools.jar to the classpath.

2012-01-05 Thread Mahadev konar (Created) (JIRA)
Reporter: Mahadev konar Assignee: Mahadev konar Thanks to Roman for pointing it out. Looks like we have the following lines in bin/yarn: {code} CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar {code} We dont really have a dependency on the tools jar. We should remove

[jira] [Created] (MAPREDUCE-3608) MAPREDUCE-3522 commit causes compilation to fail

2011-12-29 Thread Mahadev konar (Created) (JIRA)
: mrv2 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.1 There are compilation errors after MAPREDUCE-3522 was committed. Some more changes were need to webapps to fix the compilation issue. -- This message is automatically generated by JIRA

[jira] [Created] (MAPREDUCE-3587) The deployment tarball should have different directories for yarn jars and mapreduce jars.

2011-12-20 Thread Mahadev konar (Created) (JIRA)
Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Currently all the jars in the mr tarball go to share/hadoop/mapreduce. The jars should be split into: share/hadoop/yarn and share/hadoop/mapreduce for clear

[jira] [Resolved] (MAPREDUCE-3177) mapreduce tar layout does not conform new layout

2011-12-16 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3177. -- Resolution: Duplicate Duplicate of MAPREDUCE-3366. mapreduce tar

Re: Hadoop-Mapreduce-trunk-Commit - Build # 1437 - Still Failing

2011-12-13 Thread Mahadev Konar
Todd, There is a jira for it. https://issues.apache.org/jira/browse/MAPREDUCE-3541 Ravi will be posting a patch ASAP. thanks mahadev On Tue, Dec 13, 2011 at 10:48 AM, Todd Lipcon t...@cloudera.com wrote: Is anyone working on this? Been failing a few days and my inbox is sad :( -Todd

[jira] [Created] (MAPREDUCE-3543) Mavenize Gridmix.

2011-12-13 Thread Mahadev konar (Created) (JIRA)
Mavenize Gridmix. - Key: MAPREDUCE-3543 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3543 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Mahadev konar Fix

[jira] [Created] (MAPREDUCE-3513) Capacity Scheduler web UI has a spelling mistake for Memory.

2011-12-05 Thread Mahadev konar (Created) (JIRA)
: Bug Reporter: Mahadev konar Priority: Trivial The web page for capacity scheduler has a column named Memopry Total, a spelling mistake which needs to be fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your

Re: MapReduce and MPI

2011-12-01 Thread Mahadev Konar
Ralph, I think there is a lot of interest in MPI over Hadoop. If you are counting votes here's my +1. mahadev On Thu, Dec 1, 2011 at 3:31 AM, Ralph Castain r...@open-mpi.org wrote: Hi folks I'm a lead developer on the Open MPI project, and recently joined the Hadoop community to help with

[jira] [Created] (MAPREDUCE-3488) Streaming jobs are failing because the main class isnt set in the pom files.

2011-11-30 Thread Mahadev konar (Created) (JIRA)
Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 Streaming jobs are failing since the main MANIFEST file isnt being set in the pom files. -- This message is automatically

[jira] [Created] (MAPREDUCE-3443) Oozie jobs are running as oozie user even though they create the jobclient as doAs.

2011-11-21 Thread Mahadev konar (Created) (JIRA)
Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 Oozie is having issues with job submission, since it does

[jira] [Resolved] (MAPREDUCE-3308) MR builds failing due to download failure

2011-10-30 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3308. -- Resolution: Fixed This is no longer an issue. The build works fine. As for the tar

Re: Hadoop-Mapreduce-trunk-Commit - Build # 1175 - Still Failing

2011-10-28 Thread Mahadev Konar
Adding Giri. He's fixing some issues with the autoreconf and other software installations. mahadev On Fri, Oct 28, 2011 at 11:12 AM, Todd Lipcon t...@cloudera.com wrote: Actually, it looks like the libssl0.9.8 package is installed but not the libssl-dev package. Rajiv, would you mind if I

[jira] [Resolved] (MAPREDUCE-3295) TestAMAuthorization failing on branch 0.23.

2011-10-28 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3295. -- Resolution: Fixed TestAMAuthorization failing on branch 0.23

[jira] [Created] (MAPREDUCE-3292) In secure mode job submission fails with Provider org.apache.hadoop.mapreduce.security.token.JobTokenIndentifier$Renewer not found.

2011-10-27 Thread Mahadev konar (Created) (JIRA)
-3292 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3292 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority

[jira] [Created] (MAPREDUCE-3295) TestAMAuthorization failing on branch 0.23.

2011-10-27 Thread Mahadev konar (Created) (JIRA)
Reporter: Mahadev konar Fix For: 0.23.0 The test seems to fail both on Mac and linux. Trace in the next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org

[jira] [Resolved] (MAPREDUCE-3288) Mapreduce 23 builds failing

2011-10-27 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3288. -- Resolution: Fixed Just fixed this on hudson. A successful build is still dependent

[jira] [Created] (MAPREDUCE-3242) Trunk compilation broken with bad interaction from MAPREDUCE-3070 and MAPREDUCE-3239.

2011-10-21 Thread Mahadev konar (Created) (JIRA)
Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 Looks like patch command threw away some of the changes when I committed MAPREDUCE

[jira] [Resolved] (MAPREDUCE-3234) Locality scheduling broken due to mismatch between IPs and hosts

2011-10-20 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3234. -- Resolution: Not A Problem Locality scheduling broken due to mismatch between

[jira] [Created] (MAPREDUCE-3225) Killing an unkown job throws NPE.

2011-10-19 Thread Mahadev konar (Created) (JIRA)
: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 On a job -kill of an unkown job, the code currently throws a NPE. Stack trace on the next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly

[jira] [Created] (MAPREDUCE-3203) Fix some javac warnings in MRAppMaster.

2011-10-18 Thread Mahadev konar (Created) (JIRA)
Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 MAPREDUCE-2762 accidentally introduced a couple of javac warning. This jira is to fix some of them in MRAppMaster. We have plenty more to fix but I dont intend to fix them all

Re: Application/Job is hanging! validate

2011-10-17 Thread Mahadev Konar
Kamesh, This is what will happen if you do not have enough resource in the cluster to run a job. Though there could be an improvement that a job fails with not enough resource in the cluster but thats not what happens right now. thanks mahadev On Mon, Oct 17, 2011 at 6:25 AM, Kamesh

Re: Yarn NodeManager services

2011-10-11 Thread Mahadev Konar
Shuffle is an auxiliary service in the NM. You might want to take a look at AuxServices.java in the NodeManager codebase. Also, you can add config parameters to property nameyarn.nodemanager.aux-services/name valuemapreduce.shuffle/value /property (config knob in NodeManagers) to add

[jira] [Created] (MAPREDUCE-3170) Trunk builds are failing.

2011-10-11 Thread Mahadev konar (Created) (JIRA)
Trunk builds are failing. - Key: MAPREDUCE-3170 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3170 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Arun C

Re: Trunk commit builds.

2011-10-10 Thread Mahadev Konar
:11 PM, Mahadev Konar wrote: Looks like the trunk commit builds are failing/crashing. https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Commit/1060/ I suspect its: MAPREDUCE-3033. Hitesh, could you please take a look? We definitely need to mavenize MRv1

[jira] [Reopened] (MAPREDUCE-3064) 27 unit test failures with Invalid mapreduce.jobtracker.address configuration value for JobTracker: local

2011-09-27 Thread Mahadev konar (Reopened) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar reopened MAPREDUCE-3064: -- Reopening the issue, we are still having issues with the build: https

[jira] [Created] (MAPREDUCE-3093) Write tests for data locality in MRv2.

2011-09-26 Thread Mahadev konar (JIRA)
Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 We should add tests to make sure data locality is in place in MRv2 (with respect to the capacity scheduler and also the matching/ask of containers in the MR AM

[jira] [Created] (MAPREDUCE-3099) Add docs for setting up a single node MRv2 cluster.

2011-09-26 Thread Mahadev konar (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure

[jira] [Created] (MAPREDUCE-3073) Build failure for MRv1 caused due to changes to MRConstants.

2011-09-22 Thread Mahadev konar (JIRA)
: Bug Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 When runnning ant -Dresolvers=internal binary, the build seems to be failing with: [javac] public class JobTracker implements

[jira] [Created] (MAPREDUCE-3017) The Web UI shows FINISHED for killed/successful/failed jobs.

2011-09-15 Thread Mahadev konar (JIRA)
: Bug Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 The RM web ui shows FINISHED status for all the jobs even if they failed/killed or were successful. This should be fixed. Only

[jira] [Created] (MAPREDUCE-3018) Streaming jobs with -file option fail to run.

2011-09-15 Thread Mahadev konar (JIRA)
Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 Streaming jobs fail to run with the -file option. hadoop jar streaming.jar -input input.txt -output Out -mapper mapper.sh -reducer NONE

[jira] [Created] (MAPREDUCE-2990) Health Report on Resource Manager UI is null if the NM's are all healthy.

2011-09-12 Thread Mahadev konar (JIRA)
/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar The web UI on the RM for the link Nodes shows that Health-report as null when the NM is healthy. This is a simple fix where in we can check for null in NodesPage.java

[jira] [Resolved] (MAPREDUCE-2964) mapreduce trunk build fails with compile-mapred-test ant target

2011-09-12 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2964. -- Resolution: Fixed Just fixed this. No longer an issue on trunk builds. mapreduce

[jira] [Created] (MAPREDUCE-2961) Increase the default threadpool fize for container launching in the application master.

2011-09-08 Thread Mahadev konar (JIRA)
Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 Currently the default threadpool size is 10 for launching containers

[jira] [Created] (MAPREDUCE-2963) TestMRJobs hangs waiting to connect to history server.

2011-09-08 Thread Mahadev konar (JIRA)
Affects Versions: 0.23.0 Reporter: Mahadev konar Priority: Critical Fix For: 0.23.0 TestMRJobs is hanging waiting to connect to history server. I will post the logs next. -- This message is automatically generated by JIRA. For more information on JIRA, see: http

Re: Mavenization for MR1 - HADOOP-7590

2011-09-07 Thread Mahadev Konar
Yes, I think we need this, both for 0.23 and 0.24. The current form of build is causing a lot of issues. Folks are not aware of checking ant after running maven and unknowingly break the builds. thanks mahadev On Sep 7, 2011, at 7:16 AM, Alejandro Abdelnur wrote: Do we still want/need to

Re: Hadoop Tools Layout (was Re: DistCpV2 in 0.23)

2011-09-07 Thread Mahadev Konar
I like the idea of having tools as a seperate module and I dont think that it will be a dumping ground unless we choose to make one of it. +1 for hadoop tools module under trunk. thanks mahadev On Wed, Sep 7, 2011 at 11:18 AM, Alejandro Abdelnur t...@cloudera.com wrote: Agreed, we should not

[jira] [Resolved] (MAPREDUCE-2926) 500 Error in ResourceManager UI

2011-09-06 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2926. -- Resolution: Duplicate I just verified. I dont see the issue anymore. Marking

Re: Jenkins's Links to FindBugs warnings not useful

2011-09-02 Thread Mahadev Konar
Thanks Giri! mahadev On Fri, Sep 2, 2011 at 10:42 AM, Giridharan Kesavan gkesa...@hortonworks.com wrote: artifact archive location for mr per-comitt job on jenkins updated. -Giri On Fri, Sep 2, 2011 at 7:33 AM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: None of the links to the

[jira] [Created] (MAPREDUCE-2916) Ivy build for MRv1 fails with bad organization for common daemon.

2011-08-30 Thread Mahadev konar (JIRA)
Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 This jira is to ignore ivy resolve errors because of bad poms in common daemons. -- This message is automatically generated by JIRA. For more information on JIRA, see: http

Re: Trunk and 0.23 build failing with clean .m2 directory

2011-08-29 Thread Mahadev Konar
Bobby, You are right. The test-patch uses mvn compile. Please file a jira. It should be a minor change: thanks mahadev On Mon, Aug 29, 2011 at 10:34 AM, Robert Evans ev...@yahoo-inc.com wrote: Thanks Alejandro, That really clears things up. Is the a JIRA you know of to change test-patch to

[jira] [Created] (MAPREDUCE-2908) Fix findbugs warnings in Map Reduce.

2011-08-29 Thread Mahadev konar (JIRA)
Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 In the current trunk/0.23 codebase there are 5 findbugs warnings which cause the precommit CI builds to -1 the patches. -- This message is automatically

[jira] [Created] (MAPREDUCE-2886) Fix Javadoc warnings in MapReduce.

2011-08-26 Thread Mahadev konar (JIRA)
Fix Javadoc warnings in MapReduce. -- Key: MAPREDUCE-2886 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2886 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar

Re: DistCpV2 in 0.23

2011-08-25 Thread Mahadev Konar
+1 for a seperate module in hadoop-mapreduce-project. I think hadoop-mapreduce-client might not be right place for it. We might have to pick a new maven module under hadoop-mapreduce-project that could host streaming/distcp/hadoop archives. thanks mahadev On Thu, Aug 25, 2011 at 11:04 AM,

Fwd: Hadoop-Mapreduce-trunk-Commit - Build # 789 - Still Failing

2011-08-25 Thread Mahadev Konar
Looks like we might need to clean the ivy cache on the build machine? mahadev Begin forwarded message: From: Apache Jenkins Server jenk...@builds.apache.org Date: August 25, 2011 12:32:40 PM PDT To: mapreduce-dev@hadoop.apache.org Subject: Hadoop-Mapreduce-trunk-Commit - Build # 789 - Still

[jira] [Created] (MAPREDUCE-2867) Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.

2011-08-22 Thread Mahadev konar (JIRA)
Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 TestApplicationCleanup in resourcemanager/applicationsmanager doesnt do anything. There is already a test in resourcemanager/TestApplicationCleanup which tests

[jira] [Resolved] (MAPREDUCE-2867) Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.

2011-08-22 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2867. -- Resolution: Fixed Removed the file via svn commit revision 1160340. Remove

[jira] [Resolved] (MAPREDUCE-2848) Upgrade avro to 1.5.2

2011-08-22 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2848. -- Resolution: Fixed checked in as part of MAPREDUCE-2868. Upgrade avro to 1.5.2

[jira] [Created] (MAPREDUCE-2870) MR tests are failing webapps/hdfs not found in CLASSPATH

2011-08-22 Thread Mahadev konar (JIRA)
Reporter: Mahadev konar Fix For: 0.23.0 While running ant tests, the tests are failing with the following trace: {noformat} webapps/hdfs not found in CLASSPATH java.io.FileNotFoundException: webapps/hdfs not found in CLASSPATH

Re: Notes for working on mapreduce trunk after the MR-279 merge.

2011-08-19 Thread Mahadev Konar
I am seeing this issue again. Todd, do we need to fix something else? I committed 2 patches one yest and one today and I cant get it on the git branch. Something off with git pulling from svn. Any other fix we need to do? thanks mahadev On Aug 18, 2011, at 10:05 AM, Vinod Kumar Vavilapalli

Re: Notes for working on mapreduce trunk after the MR-279 merge.

2011-08-19 Thread Mahadev Konar
, Todd Lipcon t...@cloudera.com wrote: Let me ping Jukka and see if he has any idea why it's not auto-updating anymore. -Todd On Fri, Aug 19, 2011 at 10:26 AM, Mahadev Konar maha...@hortonworks.com wrote: I am seeing this issue again. Todd, do we need to fix something else? I committed 2

[jira] [Created] (MAPREDUCE-2860) Fix log4j logging in the maven test cases.

2011-08-18 Thread Mahadev konar (JIRA)
Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 At present the logging in the new test cases is broken because surefire isnt able to find the log4j properties file. -- This message is automatically generated by JIRA. For more information on JIRA

Re: [VOTE] Merge MR-279 to trunk.

2011-08-17 Thread Mahadev Konar
) to ensure we don't mess it up, so please don't be surprised to see us do this 'twice'. :) Arun On Aug 16, 2011, at 2:14 PM, Mahadev Konar wrote: Hi all, We are excited to let you know that we have MR-279 ready to be merged to trunk. I have uploaded necessary details on https

[VOTE] Merge MR-279 to trunk.

2011-08-16 Thread Mahadev Konar
Hi all, We are excited to let you know that we have MR-279 ready to be merged to trunk. I have uploaded necessary details on https://issues.apache.org/jira/browse/MAPREDUCE-279. Please take a look and vote. Clearly I am +1 on it. thanks mahadev

Re: Can't find main class when run ResourceManager or NodeManager

2011-08-11 Thread Mahadev Konar
, 2011 at 2:07 PM, Mahadev Konar maha...@hortonworks.com wrote: Matei,  Are you sure you are using the latest common from trunk?  The ClientCache was recently added within the last few weeks.  Looks like its using some old version of hadoop common.  Also the sl4j errors should be fixed

Re: Not able to execute MRv2

2011-08-09 Thread Mahadev Konar
Hi Praveen, This is an issue because of incompatible versions of avro. This occurs if you have avro 1.3.2 in your cp. We use avro 1.4. For the time being you can remove any occurence of avro 1.3.2. jar in your classpath and that should get you out of the error below. For fixing it in the

Re: trouble building MR-279 branch

2011-08-05 Thread Mahadev Konar
Thanks for pointing out Ravi. We are working on fixing it. The jars that you mention are created as part of MR-279 build. For building MR-279, all you have to do in mvn clean install -DskipTests and that should work (given you have already installed common and hdfs from trunk). thanks mahadev

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-08-05 Thread Mahadev Konar
Hi all, We are almost done with fixing findbugs issues. Once done, we plan to start a vote to merge MR-279 into trunk. If you plan to review, this would be a good time. We hope to start the vote next week sometime. thanks mahadev On Fri, Jul 29, 2011 at 3:14 PM, Mahadev Konar maha

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-08-05 Thread Mahadev Konar
, Mahadev Konar maha...@hortonworks.com wrote: Hi all, We are almost done with fixing findbugs issues. Once done, we plan to start a vote to merge MR-279 into trunk. If you plan to review, this would be a good time. We hope to start the vote next week sometime. thanks mahadev On Fri, Jul 29

[jira] [Resolved] (MAPREDUCE-2776) MR 279: Fix some of the yarn findbug warnings

2011-08-03 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2776. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this. thanks sid

[jira] [Resolved] (MAPREDUCE-2707) ProtoOverHadoopRpcEngine without using TunnelProtocol over WritableRpc

2011-07-29 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2707. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-07-29 Thread Mahadev Konar
On Wed, Jul 27, 2011 at 9:46 AM, Mahadev Konar maha...@hortonworks.com wrote: Just an update. Some good news. I have a patch for MR-279 to start working with trunk/common and trunk/hdfs (some minor fixes). I am facing some issues with secuirty which should be sorted out by EOD today. We should

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-07-29 Thread Mahadev Konar
, Todd Lipcon wrote: Hey Mahadev, Have you been running findbugs/RAT on the branch? Other test-patch like things? -Todd On Fri, Jul 29, 2011 at 12:06 AM, Mahadev Konar maha...@hortonworks.comwrote: HI all, For those working on MR-279. I just committed a patch to MR-279 to make Next

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-07-27 Thread Mahadev Konar
Just an update. Some good news. I have a patch for MR-279 to start working with trunk/common and trunk/hdfs (some minor fixes). I am facing some issues with secuirty which should be sorted out by EOD today. We should be ready to start the process of merging soon. thanks mahadev On Jul 12,

Re: Jiras need review

2011-07-12 Thread Mahadev Konar
Hi Josh, Ill review the jira's by EOD today. We should clean up our PA's (currently we have 102 of them) and get it down to reasonable/maintainable/searchable number. thanks mahadev On Tue, Jul 12, 2011 at 9:13 AM, Josh Wills jwi...@cloudera.com wrote: Please add me (jwills) to that query,

Re: MR-279 branch: Problem opening files on the namenode web console (need to pick patches from trunk to resolve)

2011-07-08 Thread Mahadev Konar
HI Amhed, We are still working to start using trunk hdfs and trunk common for MR-279 (mapreduce). I think we are almost there and should be ready next week. For now we can probably skip fixing it since we are anyway going to move to trunk hdfs. Does that seem reasonable? thanks mahadev On Fri,

Re: Building and Deploying MRv2

2011-06-15 Thread Mahadev Konar
I just checked in Tom's patch for updated INSTALL instructions. Hope that helps. thanks mahadev On Wed, Jun 15, 2011 at 7:49 AM, Thomas Graves tgra...@yahoo-inc.com wrote: On 6/15/11 8:54 AM, Praveen Sripati praveensrip...@gmail.com wrote: Hi, I am trying to build and deploy MRv2 and

Re: Queries on MRv2

2011-06-14 Thread Mahadev Konar
Praveen, Answers in line: Q) What happens if an ApplicationMaster asks a NM to launch a container and then releases the container in the allocate call later? The Application Master only releases the container once the container is done. Q) So, the NM watches the UNIX Process/Containers

Re: Queries on MRv2

2011-06-14 Thread Mahadev Konar
the container is done. Thanks, Praveen On Wed, Jun 15, 2011 at 8:59 AM, Mahadev Konar maha...@apache.org wrote: Praveen,  Answers in line: Q) What happens if an ApplicationMaster asks a NM to launch a container and then releases the container in the allocate call later? The Application

[jira] [Resolved] (MAPREDUCE-2580) MR 279: RM UI should redirect finished jobs to History UI

2011-06-09 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2580. -- Resolution: Fixed This has been committed to the branch. MR 279: RM UI should

[jira] [Resolved] (MAPREDUCE-2566) MR 279: YarnConfiguration should reloadConfiguration if instantiated with a non YarnConfiguration object

2011-06-08 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2566. -- Resolution: Fixed I just pushed this. thanks sid! MR 279: YarnConfiguration

[jira] [Resolved] (MAPREDUCE-2500) MR 279: PB factories are not thread safe

2011-05-17 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2500. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this to MR-279

[jira] [Resolved] (MAPREDUCE-2504) MR 279: race in JobHistoryEventHandler stop

2011-05-17 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2504. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

[jira] [Resolved] (MAPREDUCE-2509) MR-279: Fix NPE in UI for pending attempts

2011-05-17 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2509. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this to MR-279

[jira] [Resolved] (MAPREDUCE-2480) MR-279: mr app should not depend on hard-coded version of shuffle

2011-05-10 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2480. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

[jira] [Resolved] (MAPREDUCE-2433) MR-279: YARNApplicationConstants hard code app master jar version

2011-04-28 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2433. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this to MR-279

[jira] [Resolved] (MAPREDUCE-2458) MR-279: Rename sanitized pom.xml in build directory to work around IDE bug

2011-04-28 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2458. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

[jira] [Resolved] (MAPREDUCE-2440) MR-279: Name clashes in TypeConverter

2011-04-18 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2440. -- Resolution: Fixed I just committed this to MR-279. thanks luke. MR-279: Name

Re: Unable to build MR-279 due to unresolved ivy/maven dependencies ?

2011-04-15 Thread Mahadev Konar
Stephen, The README is a little old. Well need to change the instructions in the README on installing the avro maven plugin. We'll change that today. Thanks for pointing it out. Youll have to install the avro maven plugin from https://github.com/phunt/avro-maven-plugin. Please let us know if

[jira] [Resolved] (MAPREDUCE-2424) MR-279: counters/UI/etc. for uber-AppMaster (in-cluster LocalJobRunner for MRv2)

2011-04-15 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2424. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this. thanks

[jira] [Created] (MAPREDUCE-2438) MR-279: WebApp for Job History

2011-04-14 Thread Mahadev konar (JIRA)
: 0.23.0 Reporter: Mahadev konar Assignee: Krishna Ramachandran Add webapp for job history server in MR-279 branch. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAPREDUCE-2439) MR-279: Fix YarnRemoteException to give more details.

2011-04-14 Thread Mahadev konar (JIRA)
Components: mrv2 Reporter: Mahadev konar Assignee: Siddharth Seth Fix For: 0.23.0 Fix YarnRemoteException to add more details. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (MAPREDUCE-2432) MR-279: Install sanitized poms for downstream sanity

2011-04-13 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2432. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this. thanks

[jira] [Resolved] (MAPREDUCE-2405) MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)

2011-04-07 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2405. -- Resolution: Fixed I just pushed this to MR-279 branch. thanks greg. MR-279

[jira] [Created] (MAPREDUCE-2405) MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)

2011-03-25 Thread Mahadev konar (JIRA)
Issue Type: Improvement Components: mrv2 Reporter: Mahadev konar Assignee: Greg Roelofs Fix For: 0.23.0 Port MAPREDUCE-1220 to MRv2. This is an optimization for small jobs wherein all tasks run on the same node in the same JVM/container

Re: Why jobtracker.jsp can safely call a non-thread-safe method of JT?

2011-02-16 Thread Mahadev Konar
These are all getter apis which if you look through the implementations call some synchronized api's. Its just that they dont need to synchronized across all the completed/retired/running jobs for the web ui. thanks mahadev On Wed, Feb 16, 2011 at 12:47 AM, Min Zhou coderp...@gmail.com wrote:

  1   2   >