Re: [DISCUSS] Release numbering for stable 2.8 and beyond

2015-04-27 Thread Sandy Ryza
My understanding was that the main reason that we labeled 2.0 alpha and 2.1 beta is that we wanted flexibility to make breaking API changes. Is that the case now with 2.7? I.e. do we have APIs labeled as Public / Stable that we want freedom to change in 2.8? If not, I definitely don't think the

Re: [DISCUSS] branch-1

2015-05-08 Thread Sandy Ryza
+1 to auto-closing branch-1 issues. -Sandy On Fri, May 8, 2015 at 11:14 AM, Karthik Kambatla wrote: > Closing out the JIRAs as "Auto Closed" or "Closed due to Inactivity" seems > reasonable to me. For branch-1, we can be more aggressive. We should > probably do the same less aggressively for ot

Re: Max Maps for default queues in FairScheduler

2012-10-18 Thread Sandy Ryza
You're correct that there's no way to put a hard limit on the number of maps or reduces for a given user, and a user can potentially consume all of the cluster resources. However, if there are multiple users contending for resources, the scheduler makes an effort to schedule tasks equally, so it w

Writing tests for command line tools in MR2

2012-12-30 Thread Sandy Ryza
I am trying to write a test for "mapred queue -info" (for MAPREDUCE-4896). The test requires a running RM ask for the information. I noticed that TestJobClient and TestMRJobClient, which test other mapreduce command line tools, are both @Ignored. Does anybody know whether these are ignored becau

mapreduce.jobtracker.address vs. yarn.resourcemanager.address

2013-02-08 Thread Sandy Ryza
Is there a reason why mapreduce.jobtracker.address is not deprecated to yarn.resourcemanager.address? thanks, Sandy

Re: mapreduce.jobtracker.address vs. yarn.resourcemanager.address

2013-02-10 Thread Sandy Ryza
xist, hence the non-deprecation. > >> > >> I am not sure we formally did, but once it is cast in stone that MR1 > >> cannot be run with Hadoop 2.0 *(by which time we should remove JT/TT > etc.), > >> then we can deprecate these config

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-12 Thread Sandy Ryza
+1 (non-binding) Built from source on Ubuntu and ran a couple different jobs under the fair scheduler. On Tue, Feb 12, 2013 at 8:52 PM, Konstantin Boudnik wrote: > On Tue, Feb 12, 2013 at 07:44PM, Konstantin Boudnik wrote: > > We've created BigTop stack with 2.0.3 as the base. Ran YCSB, slive, a

tests in mapreduce.lib excluded in jenkins?

2013-02-25 Thread Sandy Ryza
A recent patch of mine (https://issues.apache.org/jira/browse/MAPREDUCE-4994) broke a couple of tests, but the Hadoop QA build ( https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3321/testReport/) didn't catch anything wrong. It looks like the tests under mapred.lib and mapreduce.lib, such a

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-16 Thread Sandy Ryza
+1 (non-binding) Built from source and ran sample jobs concurrently with the fair scheduler on a single node cluster. On Fri, Apr 12, 2013 at 2:56 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I would > like to release. > > The RC is a

Re: [VOTE] Release Apache Hadoop 0.23.7

2013-04-17 Thread Sandy Ryza
+1 (non-binding) Built from source and ran a couple of MR examples on a single node cluster. -Sandy On Wed, Apr 17, 2013 at 12:03 PM, Siddharth Seth wrote: > +1 (binding). > Verified checksums and signature. > Built from source tar, deployed a single node cluster (CapacityScheduler) > and tried

Re: sort phase in hadoop mapper

2013-04-18 Thread Sandy Ryza
Hi Samaneh, If you want to see the map outputs post sort/shuffle, the easiest way is probably to use an IdentityReducer and inspect the job. Can you be more specific on what you need to disable the sort phase for? Sorting is used in part to group map outputs and route them to the correct reducer

Re: sort phase in hadoop mapper

2013-04-18 Thread Sandy Ryza
sible to skip sort in some cases.For example if we have only one reducer > is it possible to skip the sorting and just flush the data directly to the > reducer ? > > Samaneh > > > > On Thu, Apr 18, 2013 at 8:46 PM, Sandy Ryza > wrote: > > > Hi Samaneh, > >

Re: ConfigKeys wrappers for MapReduce source code base

2013-04-21 Thread Sandy Ryza
Hi Jay, In trunk, these keys are in MRJobConfig (extended by JobContext) and MRConfig. I'm not sure what the state of things are in branch-1. Agreed that it would be nice for the naming convention to be consistent with HDFS, but personally I like *Config more than *ConfigKeys better because the

Re: Versions - Confusion

2013-04-27 Thread Sandy Ryza
The 2.0 series contains security as well. -Sandy On Sat, Apr 27, 2013 at 5:30 AM, Suresh S wrote: > Thank you very much sir. great explaination. > > 0.20.20X is renamed to 1.0. So, 1.0 series is coming with security. > > What about 2.0 series? It is with security or without? > > *Regards* >

Re: Hadoop MapReduce High Availability

2013-04-29 Thread Sandy Ryza
Hi Augusto, In Hadoop 2, ResourceManager HA is being worked on under YARN-128 and YARN-149. There's a design doc for RM recovery on the latter. Hadoop 1's MapReduce high availability story is kind of fragmented. Cloudera distribution has JobTracker HA based on the HA libraries available in Hado

Re: [VOTE] Plan to create release candidate for 0.23.8

2013-05-19 Thread Sandy Ryza
+1 (non-binding) On Sun, May 19, 2013 at 1:22 PM, Derek Dagit wrote: > +1 (non-binding) > > On May 17, 2013, at 4:14 PM, "Thomas Graves" > wrote: > > > Hello all, > > > > We've had a few critical issues come up in 0.23.7 that I think warrants a > > 0.23.8 release. The main one is MAPREDUCE-521

Re: Clarifications on MAPREDUCE-5183

2013-05-21 Thread Sandy Ryza
Hi Niranjan, Your understanding is correct. -Sandy On Tue, May 21, 2013 at 1:02 AM, maisnam ns wrote: > Hi, > > I was looking into this issue but would be happy if someone could clarify > some of my doubts. > > Is the issue related to the given below snapshot of log: > > org.apache.hadoop.map

Re: Clarifications on MAPREDUCE-5183

2013-05-21 Thread Sandy Ryza
If you're planning to fix it, it would probably look prettiest to keep the percentage sign and have the numbers between 0 and 100. -Sandy On Tue, May 21, 2013 at 10:55 AM, maisnam ns wrote: > Thanks Sandy Ryza > > > > > On Tue, May 21, 2013 at 11:20 PM, Sandy Ryza &g

Re: [VOTE] Release Apache Hadoop 0.23.8

2013-06-03 Thread Sandy Ryza
+1 (non-binding). Did a full build from source and ran a few sample jobs on a pseudo-distributed cluster. -Sandy On Mon, Jun 3, 2013 at 11:29 AM, Kihwal Lee wrote: > +1 I've downloaded & built the RC and ran several tests on a single node > cluster. > > Kihwal > > On 5/28/13 11:00 AM, "Thomas

Re: [VOTE] Release Apache Hadoop 2.0.5-alpha (rc2)

2013-06-03 Thread Sandy Ryza
+1 (non-binding) Built from source, ran a few sample jobs on a pseudo-distributed cluster On Mon, Jun 3, 2013 at 3:22 PM, Chris Douglas wrote: > +1 > > Checksum and signature match, ran some unit tests, checked diff > against 2.0.4-alpha. > > Thanks for seeing this through, Cos. -C > > On Mon,

mapred.child.ulimit in MR2

2013-06-18 Thread Sandy Ryza
Hi yarn-dev/mapreduce-dev, Is there a reason that mapred.child.ulimit no longer has an effect in MR2? Should it be added back in? thanks for any help, -Sandy

Re: mapred.child.ulimit in MR2

2013-06-21 Thread Sandy Ryza
are worried about backwards compatibility > we can add it back in. It is not that hard, all it did was add 'ulimt -v > ' to the shell script that launched the task, except on windows. > > --Bobby > > On 6/18/13 3:56 PM, "Sandy Ryza" wrote: >

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-06-28 Thread Sandy Ryza
For YARN-791, if we can come to consensus on the correct approach, I can try to have a patch ASAP. -Sandy On Fri, Jun 28, 2013 at 12:03 PM, Hitesh Shah wrote: > Hi Arun, > > From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may > potentially change the apis. They can implemented

Re: How to Free-up a Map Slot without Killing the Entire Job?

2013-06-29 Thread Sandy Ryza
The fair scheduler does preemption as well. -Sandy On Sat, Jun 29, 2013 at 12:05 PM, Steve Loughran wrote: > there's a scheduler in contrib/ that does pre-emption. look here > > > https://github.com/apache/hadoop-common/tree/branch-0.22/mapreduce/src/contrib/dynamic-scheduler > > On 28 June 201

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-08-19 Thread Sandy Ryza
Vinod, your thinking makes sense to me. My two cents are that we should hold off on fixes until 2.1.1-beta. Unless there are downstream projects that need it to work for integration testing. -Sandy On Mon, Aug 19, 2013 at 2:49 PM, Vinod Kumar Vavilapalli wrote: > > Thanks for the clarificatio

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-08-21 Thread Sandy Ryza
+1 (non-binding) Built from source, ran jobs on a pseudo-distributed cluster with the Fair Scheduler. -Sandy On Tue, Aug 20, 2013 at 7:27 PM, Arun C Murthy wrote: > Thanks for the heads up Aaron, I've changed fix-version of HDFS-4763 to > 2.1.1-beta for now. > > Committers - please be careful

JobClient.getRootQueues returns default, not root?

2013-10-02 Thread Sandy Ryza
Apparently JobClient.getRootQueues returns the default queue, not the root queue. Is this the correct behavior? It might make sense for the FIFO scheduler, but not for the Fair and Capacity schedulers. thanks for any guidance -Sandy

Re: [VOTE] Release Apache Hadoop 2.2.0

2013-10-08 Thread Sandy Ryza
+1 (non-binding) Built from source and ran a few jobs on a pseudo-distributed cluster with the Fair Scheduler. On Tue, Oct 8, 2013 at 6:48 AM, Thomas Graves wrote: > +1. > > Downloaded, verified signature/md5, CHANGES.txt, NOTICE, LICENSE, README, > release notes, built the source tar ball, an

pipes not working in MR2?

2013-10-10 Thread Sandy Ryza
I'm unable to get a simple hadoop pipes job working in MR2, and got the sense it hasn't been working for a while. Does anybody have any insight into what's going on? Has anybody used them successfully recently? thanks for any help, Sandy

streaming documentation in Hadoop 2?

2013-10-14 Thread Sandy Ryza
Hi All, I noticed that the hadoop streaming documentation does not exist in the Hadoop 2 source tree, and also cannot be found on the internet. Is this on purpose? I found this wiki page http://wiki.apache.org/hadoop/HadoopStreaming - is that where doc is supposed to go? As this page isn't tie

Re: streaming documentation in Hadoop 2?

2013-10-14 Thread Sandy Ryza
d file a jira for it. The > wiki content here could be a good starting point. > > On Mon, Oct 14, 2013 at 2:56 PM, Sandy Ryza > wrote: > > Hi All, > > > > I noticed that the hadoop streaming documentation does not exist in the > > Hadoop 2 source tree, and also

Re: Next releases

2013-11-08 Thread Sandy Ryza
Starting afresh with 2.2.1 and keeping it as small as possible sounds reasonable to me. Would love to get 2.3 out soon. To that end, how would people feel about having code and/or feature freeze and/or ship dates? We've been way behind out goals for recent releases. Having actual targets on the

Re: Next releases

2013-11-13 Thread Sandy Ryza
N-1333. Support blacklisting in the Fair Scheduler YARN-1109. Demote NodeManager "Sending out status for container" logs to debug (haosdent via Sandy Ryza) YARN-1388. Fair Scheduler page always displays blank fair share +1 to doing releases at some fixed time interval. -Sandy On W

Re: Re-swizzle 2.3

2014-01-27 Thread Sandy Ryza
We should hold off commits until that's done, right? On Mon, Jan 27, 2014 at 1:07 PM, Arun C Murthy wrote: > Yep, on it as we speak. :) > > > Arun > > On Jan 27, 2014, at 12:36 PM, Jason Lowe wrote: > > > Thanks, Arun. Are there plans to update the Fix Versions and > CHANGES.txt accordingly?

Re: Doubt in Yarn Scheduler.

2014-01-27 Thread Sandy Ryza
Hi Suresh, The schedulers used in MR1 are not compatible with the schedulers used in MR2 / YARN. Check out ResourceScheduler.java for the updated interface. I believe that speculative execution still works the same way. -Sandy On Sat, Jan 25, 2014 at 9:47 AM, Suresh S wrote: > Hello Friends

Re: Re-swizzle 2.3

2014-01-28 Thread Sandy Ryza
Going forward with commits because it seems like others have been doing so On Mon, Jan 27, 2014 at 1:31 PM, Sandy Ryza wrote: > We should hold off commits until that's done, right? > > > On Mon, Jan 27, 2014 at 1:07 PM, Arun C Murthy wrote: > >> Yep, on it as

Re: Re-swizzle 2.3

2014-02-06 Thread Sandy Ryza
+1 to reverting those JIRAs from branch-2.3. As YARN-1689 is fixing a problem caused by YARN-1493 I think we can revert it in branch-2.3 as well. I think we should leave them in branch-2 for now. We can revert if 2.4 is imminent and they're holding it up, but hopefully the issues they caused wil

Re: The code for the Shuffle phase of MapReduce

2014-02-10 Thread Sandy Ryza
Hi Pramod, For the reduce side, also check out Shuffle.java. -Sandy On Mon, Feb 10, 2014 at 2:54 AM, Pramod Biligiri wrote: > Hi, > I'm beginning to look at the code for the Shuffle phase of MapReduce, for > an academic project. > > I wanted to confirm if I have started at the right location:

Re: [VOTE] Release Apache Hadoop 2.3.0

2014-02-13 Thread Sandy Ryza
+1 (non-binding) Built from source and ran jobs on a pseudo-distributed cluster with the Fair Scheduler On Wed, Feb 12, 2014 at 7:56 PM, Xuan Gong wrote: > +1 (non-binding) > > downloaded the source tar ball, built, ran a number of MR jobs on a > single-node cluster and checked the job history

Re: Thinking ahead to 2.4

2014-03-20 Thread Sandy Ryza
Looking forward to the release. What's the state of the application history server? Do we have security, documentation, and are APIs stable? If not, do we have a plan for how to make this clear to users? What about the timeline store? thanks, Sandy On Fri, Mar 14, 2014 at 8:23 PM, Arun C Mur

Re: [VOTE] Release Apache Hadoop 2.4.0

2014-03-31 Thread Sandy Ryza
What's the state of the application history server? Do we have security, documentation, and are APIs stable? If any of these are missing, do we have a plan for how to make this clear to users? What about the timeline store? thanks, Sandy On Mon, Mar 31, 2014 at 2:22 AM, Arun C Murthy wrote:

Re: [VOTE] Release Apache Hadoop 2.4.0

2014-03-31 Thread Sandy Ryza
eline > store is for storing per-framework data only, while we eventually hope to > move the generic data there as well. > > Thanks, > Zhijie > > > > > On Mon, Mar 31, 2014 at 11:02 AM, Sandy Ryza >wrote: > > > What's the state of the application hist

Re: [VOTE] Release Apache Hadoop 2.4.0

2014-04-03 Thread Sandy Ryza
While the Scheduler Load Simulator isn't part of YARN's core, it's a tool that YARN includes, and it's broken entirely in the current RC. YARN-1726 seems to me like something worth including in the release. -Sandy On Thu, Apr 3, 2014 at 11:12 AM, Xuan Gong wrote: > +1 non-binding > > Built fr

Re: Thinking ahead

2014-04-12 Thread Sandy Ryza
+1 for starting to think about 2.5. Early June seems a little early to me - we had talked about a quarterly release cadence and this would be about half that. I'm having trouble editing the wiki, but I think Timeline Server stability (e.g. security and locking down APIs) should go on that list.

Re: Thinking ahead

2014-04-13 Thread Sandy Ryza
if it meant extending the target release date a couple weeks - I of course had no intention to volunteer anybody for it. On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy wrote: > Sandy, > > On Apr 12, 2014, at 10:09 AM, Sandy Ryza wrote: > > I'm having trouble editing the w

Re: Thinking ahead

2014-04-13 Thread Sandy Ryza
to point out that the proposal to merge AHS into trunk stated an intent to tie ends up on security. On Sun, Apr 13, 2014 at 7:50 PM, Sandy Ryza wrote: > Unfortunately I don't have the bandwidth to take on ATS security at this > time. My (I now understand mistaken) impression that i

Re: hadoop-2.5 - June end?

2014-06-10 Thread Sandy Ryza
That sounds reasonable to me. -Sandy On Mon, Jun 9, 2014 at 9:39 AM, Arun C Murthy wrote: > Folks, > > As you can see from the Roadmap wiki, it looks like several items are > still a bit away from being ready. > > I think rather than wait for them, it will be useful to create an > intermedia

Re: Plans of moving towards JDK7 in trunk

2014-06-23 Thread Sandy Ryza
Andrew, correct me if I'm misunderstanding, but the incompatible change that would require a major version bump is dropping support for JDK6. On Mon, Jun 23, 2014 at 1:53 PM, sanjay Radia wrote: > > On Jun 21, 2014, at 8:01 AM, Andrew Wang wrote: > > > This is why I'd like to keep my original

Re: [VOTE] Change by-laws on release votes: 5 days instead of 7

2014-06-24 Thread Sandy Ryza
+1 (binding) -Sandy On Tue, Jun 24, 2014 at 7:53 AM, Devaraj K wrote: > +1 > > Thanks > Devaraj K > > > On Tue, Jun 24, 2014 at 2:23 PM, Arun C Murthy > wrote: > > > Folks, > > > > As discussed, I'd like to call a vote on changing our by-laws to change > > release votes from 7 days to 5. > >

Re: Moving to JDK7, JDK8 and new major releases

2014-06-24 Thread Sandy Ryza
While we haven't codified this in our compatibility guidelines, dropping a Java version seems to me like change that needs to happen alongside a major release. In plain talk, it has the ability to break everything for users who aren't doing anything particularly unreasonable. I don't think we sho

Re: Where is the map input transfered to the 'map worker'/container?

2014-07-06 Thread Sandy Ryza
Hi Christian, I'm not sure the exact code path, but HDFS, not MapReduce, is in charge of getting the bytes from the remote node. -Sandy On Mon, Jun 23, 2014 at 1:57 AM, Christian Grote < cgr...@mail.uni-paderborn.de> wrote: > Hey, > > I'm looking for the place where the actual map input is tra

Re: [DISCUSS] Assume Private-Unstable for classes that are not annotated

2014-07-22 Thread Sandy Ryza
That policy makes sense to me. We should still label things @Private of course so that it can be reflected in the documentation. -Sandy On Tue, Jul 22, 2014 at 2:54 PM, Karthik Kambatla wrote: > Hi devs > > As you might have noticed, we have several classes and methods in them that > are not

Re: [VOTE] Migration from subversion to git for version control

2014-08-09 Thread Sandy Ryza
+1 (binding) On Fri, Aug 8, 2014 at 7:57 PM, Karthik Kambatla wrote: > I have put together this proposal based on recent discussion on this topic. > > Please vote on the proposal. The vote runs for 7 days. > >1. Migrate from subversion to git for version control. >2. Force-push to be di

Re: [VOTE] Release Apache Hadoop 2.5.1 RC0

2014-09-10 Thread Sandy Ryza
+1 (binding) Ran a pseudo-distributed cluster with the Fair Scheduler and ran some MapReduce example jobs. -Sandy On Wed, Sep 10, 2014 at 12:15 PM, Alejandro Abdelnur wrote: > Thanks Karthik. > > +1. > > + verified MD5 for source tarball > + verified signature for source tarball > + successful

[jira] [Created] (MAPREDUCE-4655) MergeManager.reserve can OutOfMemoryError if more than 10% of max memory is used on non-MapOutputs

2012-09-13 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4655: - Summary: MergeManager.reserve can OutOfMemoryError if more than 10% of max memory is used on non-MapOutputs Key: MAPREDUCE-4655 URL: https://issues.apache.org/jira/browse

[jira] [Created] (MAPREDUCE-4659) Confusing output when running "hadoop version" from one hadoop installation when HADOOP_HOME points to another

2012-09-14 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4659: - Summary: Confusing output when running "hadoop version" from one hadoop installation when HADOOP_HOME points to another Key: MAPREDUCE-4659 URL: https://issues.apach

[jira] [Created] (MAPREDUCE-4676) Add test for job history cleaner

2012-09-21 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4676: - Summary: Add test for job history cleaner Key: MAPREDUCE-4676 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4676 Project: Hadoop Map/Reduce Issue

[jira] [Created] (MAPREDUCE-4680) Job history cleaner should only check timestamps of files in old enough directories

2012-09-24 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4680: - Summary: Job history cleaner should only check timestamps of files in old enough directories Key: MAPREDUCE-4680 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4680

[jira] [Created] (MAPREDUCE-4722) LocalJobRunner random ID should not be chosen with Math.abs(rand.nextInt())

2012-10-11 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4722: - Summary: LocalJobRunner random ID should not be chosen with Math.abs(rand.nextInt()) Key: MAPREDUCE-4722 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4722

[jira] [Created] (MAPREDUCE-4723) Fix warnings found by findbugs 2

2012-10-11 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4723: - Summary: Fix warnings found by findbugs 2 Key: MAPREDUCE-4723 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4723 Project: Hadoop Map/Reduce Issue

[jira] [Created] (MAPREDUCE-4725) Setting local variables to null causes findbugs 2 warnings

2012-10-15 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4725: - Summary: Setting local variables to null causes findbugs 2 warnings Key: MAPREDUCE-4725 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4725 Project: Hadoop

[jira] [Created] (MAPREDUCE-4726) Empty catch blocks cause findbugs 2 warnings

2012-10-15 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4726: - Summary: Empty catch blocks cause findbugs 2 warnings Key: MAPREDUCE-4726 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4726 Project: Hadoop Map/Reduce

[jira] [Resolved] (MAPREDUCE-4725) Setting local variables to null causes findbugs 2 warnings

2012-10-15 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved MAPREDUCE-4725. --- Resolution: Duplicate > Setting local variables to null causes findbugs 2 warni

[jira] [Resolved] (MAPREDUCE-4726) Empty catch blocks cause findbugs 2 warnings

2012-10-15 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved MAPREDUCE-4726. --- Resolution: Duplicate > Empty catch blocks cause findbugs 2 warni

[jira] [Created] (MAPREDUCE-4778) Fair scheduler event log is only written if directory exists on HDFS

2012-11-07 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4778: - Summary: Fair scheduler event log is only written if directory exists on HDFS Key: MAPREDUCE-4778 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4778 Project

[jira] [Created] (MAPREDUCE-4795) TestDelegationTokenRenewal should not use static variables in Renewer

2012-11-13 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4795: - Summary: TestDelegationTokenRenewal should not use static variables in Renewer Key: MAPREDUCE-4795 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4795

[jira] [Created] (MAPREDUCE-4845) ClusterStatus.getMaxMemory() and getUsedMemory() exist in MR1 but not MR2

2012-12-04 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4845: - Summary: ClusterStatus.getMaxMemory() and getUsedMemory() exist in MR1 but not MR2 Key: MAPREDUCE-4845 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4845

[jira] [Created] (MAPREDUCE-4846) Some JobQueueInfo methods are public in MR1 but protected in MR2

2012-12-04 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4846: - Summary: Some JobQueueInfo methods are public in MR1 but protected in MR2 Key: MAPREDUCE-4846 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4846 Project

[jira] [Created] (MAPREDUCE-4856) TestJobOutputCommitter uses same directory as TestJobCleanup

2012-12-06 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4856: - Summary: TestJobOutputCommitter uses same directory as TestJobCleanup Key: MAPREDUCE-4856 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4856 Project: Hadoop

[jira] [Reopened] (MAPREDUCE-4549) Distributed cache conflicts breaks backwards compatability

2012-12-10 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reopened MAPREDUCE-4549: --- > Distributed cache conflicts breaks backwards compatabil

[jira] [Created] (MAPREDUCE-4880) TaskImpl interprets death of second attempt as RetroactiveKilledTransition even when first attempt succeeds

2012-12-15 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4880: - Summary: TaskImpl interprets death of second attempt as RetroactiveKilledTransition even when first attempt succeeds Key: MAPREDUCE-4880 URL: https://issues.apache.org/jira

[jira] [Resolved] (MAPREDUCE-4880) TaskImpl interprets death of second attempt as RetroactiveKilledTransition even when first attempt succeeds

2012-12-17 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved MAPREDUCE-4880. --- Resolution: Not A Problem It looks like this was already fixed in MAPREDUCE-4607

[jira] [Created] (MAPREDUCE-4896) "mapred queue -info" spits out ugly exception when queue does not exist

2012-12-20 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4896: - Summary: "mapred queue -info" spits out ugly exception when queue does not exist Key: MAPREDUCE-4896 URL: https://issues.apache.org/jira/browse/MAPR

[jira] [Created] (MAPREDUCE-4907) TrackerDistributedCacheManager issues too many getFileStatus calls

2012-12-28 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4907: - Summary: TrackerDistributedCacheManager issues too many getFileStatus calls Key: MAPREDUCE-4907 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4907 Project

[jira] [Resolved] (MAPREDUCE-4655) MergeManager.reserve can OutOfMemoryError if more than 10% of max memory is used on non-MapOutputs

2013-01-03 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved MAPREDUCE-4655. --- Resolution: Invalid > MergeManager.reserve can OutOfMemoryError if more than 10%

[jira] [Created] (MAPREDUCE-4922) Request with multiple data local nodes can cause NPE in AppSchedulingInfo

2013-01-07 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4922: - Summary: Request with multiple data local nodes can cause NPE in AppSchedulingInfo Key: MAPREDUCE-4922 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4922

[jira] [Created] (MAPREDUCE-4923) Add toString method to TaggedInputSplit

2013-01-07 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4923: - Summary: Add toString method to TaggedInputSplit Key: MAPREDUCE-4923 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4923 Project: Hadoop Map/Reduce

[jira] [Created] (MAPREDUCE-4929) mapreduce.task.timeout is ignored

2013-01-09 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4929: - Summary: mapreduce.task.timeout is ignored Key: MAPREDUCE-4929 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4929 Project: Hadoop Map/Reduce Issue

[jira] [Created] (MAPREDUCE-4933) MR1 merger asks for length of file it just wrote before flushing it

2013-01-10 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4933: - Summary: MR1 merger asks for length of file it just wrote before flushing it Key: MAPREDUCE-4933 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4933 Project

[jira] [Created] (MAPREDUCE-4949) Enable multiple pi jobs to run in parallel

2013-01-18 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4949: - Summary: Enable multiple pi jobs to run in parallel Key: MAPREDUCE-4949 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4949 Project: Hadoop Map/Reduce

[jira] [Created] (MAPREDUCE-4951) Container preemption interpreted as task failure

2013-01-20 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4951: - Summary: Container preemption interpreted as task failure Key: MAPREDUCE-4951 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4951 Project: Hadoop Map/Reduce

[jira] [Created] (MAPREDUCE-4952) FSSchedulerNode is always instantiated with a 0 virtual core capacity

2013-01-20 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4952: - Summary: FSSchedulerNode is always instantiated with a 0 virtual core capacity Key: MAPREDUCE-4952 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4952

[jira] [Resolved] (MAPREDUCE-4952) FSSchedulerNode is always instantiated with a 0 virtual core capacity

2013-01-20 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved MAPREDUCE-4952. --- Resolution: Invalid Sorry, this should be YARN JIRA - filed YARN-350 instead

[jira] [Created] (MAPREDUCE-4962) jobdetails.jsp uses display name instead of real name to get counters

2013-01-25 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4962: - Summary: jobdetails.jsp uses display name instead of real name to get counters Key: MAPREDUCE-4962 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4962

[jira] [Created] (MAPREDUCE-4965) In merge, no ordering defined for CompressAwarePath

2013-01-28 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4965: - Summary: In merge, no ordering defined for CompressAwarePath Key: MAPREDUCE-4965 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4965 Project: Hadoop Map

[jira] [Created] (MAPREDUCE-4970) Child tasks create security audit log files

2013-01-31 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4970: - Summary: Child tasks create security audit log files Key: MAPREDUCE-4970 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4970 Project: Hadoop Map/Reduce

[jira] [Created] (MAPREDUCE-4984) AM leaves unwanted node/rack requests after container has been assigned

2013-02-06 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4984: - Summary: AM leaves unwanted node/rack requests after container has been assigned Key: MAPREDUCE-4984 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4984

[jira] [Resolved] (MAPREDUCE-4984) AM leaves unwanted node/rack requests after container has been assigned

2013-02-06 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved MAPREDUCE-4984. --- Resolution: Duplicate > AM leaves unwanted node/rack requests after container

[jira] [Created] (MAPREDUCE-4994) Can't submit local job with hadoop jar -jt local

2013-02-08 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4994: - Summary: Can't submit local job with hadoop jar -jt local Key: MAPREDUCE-4994 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4994 Project: Hadoop Map/R

[jira] [Resolved] (MAPREDUCE-4922) Request with multiple data local nodes can cause NPE in AppSchedulingInfo

2013-02-08 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved MAPREDUCE-4922. --- Resolution: Won't Fix This is the expected behavior > Requ

[jira] [Created] (MAPREDUCE-4997) Deprecate mapreduce.jobtracker.address

2013-02-10 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-4997: - Summary: Deprecate mapreduce.jobtracker.address Key: MAPREDUCE-4997 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4997 Project: Hadoop Map/Reduce

[jira] [Reopened] (MAPREDUCE-4643) Make job-history cleanup-period configurable

2013-02-12 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reopened MAPREDUCE-4643: --- > Make job-history cleanup-period configura

[jira] [Reopened] (MAPREDUCE-4994) -jt generic command line option does not work

2013-02-13 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reopened MAPREDUCE-4994: --- > -jt generic command line option does not w

[jira] [Created] (MAPREDUCE-5008) Merger progress miscounts with respect to EOF_MARKER

2013-02-15 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-5008: - Summary: Merger progress miscounts with respect to EOF_MARKER Key: MAPREDUCE-5008 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5008 Project: Hadoop Map

[jira] [Created] (MAPREDUCE-5013) JobStatus compatibility: MR2 missing constructors from MR1

2013-02-19 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-5013: - Summary: JobStatus compatibility: MR2 missing constructors from MR1 Key: MAPREDUCE-5013 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5013 Project: Hadoop

[jira] [Created] (MAPREDUCE-5021) Add an addDirectoryToClassPath method DistributedCache

2013-02-21 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-5021: - Summary: Add an addDirectoryToClassPath method DistributedCache Key: MAPREDUCE-5021 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5021 Project: Hadoop Map

[jira] [Created] (MAPREDUCE-5036) Default shuffle handler port should not be 8080

2013-02-27 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-5036: - Summary: Default shuffle handler port should not be 8080 Key: MAPREDUCE-5036 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5036 Project: Hadoop Map/Reduce

[jira] [Resolved] (MAPREDUCE-4933) MR1 final merge asks for length of file it just wrote before flushing it

2013-02-27 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza resolved MAPREDUCE-4933. --- Resolution: Fixed > MR1 final merge asks for length of file it just wrote bef

[jira] [Created] (MAPREDUCE-5038) mapred CombineFileInputFormat does not work on non-splittable files

2013-02-28 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-5038: - Summary: mapred CombineFileInputFormat does not work on non-splittable files Key: MAPREDUCE-5038 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5038 Project

[jira] [Created] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters

2013-03-05 Thread Sandy Ryza (JIRA)
Sandy Ryza created MAPREDUCE-5047: - Summary: keep.failed.task.files=true causes job failure on secure clusters Key: MAPREDUCE-5047 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047 Project

  1   2   >