On Aug 3, 2011, at 3:58 PM, Koji Noguchi wrote:
> Can we add
> * HADOOP-6942 Ability for having user's classes take precedence over the
> system classes for tasks' classpath
>
Thanks Koji.
HADOOP-6942 is in the 'MRv1 exceptions' category since MRv2 doesn't have this
issue at all.
> It's pretty bad in a sense that it's not on trunk but 0.20.204 and CDH3 both
> have this *using different parameters*.
This was originally part of the 0.20.203 release, but what/how Cloudera chooses
to put in CDH3 is beyond the scope of this discussion... :)
Arun
>
> Koji
>
> On 8/3/11 2:14 PM, "Matt Foley" <mfo...@hortonworks.com> wrote:
>
>>>
>>> So, how are we doing with 0.20.204 content and trunk given the above
>>> proposal? Very well, in fact. Matt, Suresh and I have done a detailed
>>> analysis (separate email), please take a look.
>>
>>
>> Here's the analysis of changes in 204 vs trunk. We believe that ALL the
>> changes in 204 are either already in trunk or do not need to be in trunk.
>> Here is the list of 204 items not already in trunk, and their
>> categorization.
>>
>> Please, if anyone thinks we've missed something, bring it to my attention
>> and if it isn't already in trunk we will get it into trunk as expeditiously
>> as possible.
>> Thanks,
>> --Matt
>>
>> Not needed for trunk:
>> HDFS-1758. Make Web UI JSP pages thread safe. (Tanping via suresh) (Not a
>> problem in trunk)
>> HDFS-2057. Wait time to terminate the threads causes unit tests to take
>> longer time. (Bharath Mundlapudi via suresh) (Not a problem in trunk)
>> HDFS-1878. TestHDFSServerPorts unit test failure - race condition in
>> FSNamesystem.close() causes NullPointerException without serious
>> consequence. (mattf) (not a problem in trunk)
>> HDFS-1842. Handle editlog opcode conflict with 0.20.203 during upgrade, by
>> throwing an error to indicate the editlog needs to be empty. (suresh)
>> (Handled by HDFS-1824)
>> HDFS-2218. Disable TestHdfsProxy.testHdfsProxyInterface in automated test
>> suite for 0.20-security-204 release. (Matt Foley) (Not a problem in trunk)
>> HDFS-2044. TestQueueProcessingStatistics failing automatic test due to
>> timing issues. (mattf) (not a problem in trunk)
>> HADOOP-7459. Remove jdk-1.6.0 dependency check from rpm. (omalley) (replaced
>> by HDFS-2156)
>> HADOOP-7398. Suppress warnings about use of HADOOP_HOME. (omalley) (not a
>> problem in trunk)
>> HADOOP-7369. Fix permissions in tarball for sbin/* and libexec/* (omalley)
>> (not a problem in trunk)
>> HADOOP-7373. Fix {start,stop}-{dfs,mapred} and hadoop-daemons.sh from trying
>> to use the wrong bin directory. (omalley) (not a problem in trunk)
>> HADOOP-7475. Fix hadoop-setup-single-node.sh to reflect new layout. (eyang
>> via omalley) (not a problem in trunk)
>>
>> Back port from trunk:
>> MAPREDUCE-2524. Port reduce failure reporting semantics from trunk, to fail
>> faulty maps more aggressively. (Thomas Graves via cdouglas)
>> MAPREDUCE-2479. Move distributed cache cleanup to a background task,
>> backporting MAPREDUCE-1568. (Robert Joseph Evans via cdouglas)
>> HDFS-2023. Backport of NPE for File.list and File.listFiles. Merged ports
>> of HADOOP-7322, HDFS-1934, HADOOP-7342, and HDFS-2019. (Bharath Mundlapudi
>> via mattf)
>> HADOOP-7248. Update eclipse target to generate .classpath from ivy config.
>> (Thomas Graves and Tom White via cdouglas) (from HADOOP-6407)
>> HADOOP-7274. Fix typos in IOUtils. (Jonathan Eagles via cdouglas) (from
>> HADOOP-7057)
>> HADOOP-7277. Add generation of run configurations to eclipse target.
>> (Jeffrey Naisbitt and Philip Zeyliger via cdouglas) (from HADOOP-5911)
>> HADOOP-7364. TestMiniMRDFSCaching fails if test.build.dir is set to
>> something other than build/test. (Thomas Graves via mahadev) (from
>> MAPREDUCE-2575)
>>
>> Already in trunk for MRV2:
>> MAPREDUCE-2558. Add queue-level metrics 0.20-security branch (test fixes)
>> (Jeffrey Naisbitt via mahadev)
>> MAPREDUCE-2418. Show job errors in JobHistory page. (Siddharth Seth via
>> acmurthy) (Already in MR279)
>> MAPREDUCE-2411. Force an exception when the queue has an invalid name or its
>> ACLs are misconfigured. (Dick King via cdouglas) (Already in MR279)
>>
>> Waiting for MR279 merge:
>> MAPREDUCE-2429. Validate JVM in TaskUmbilicalProtocol. (Siddharth Seth via
>> acmurthy)
>> MAPREDUCE-2447. Fix Child.java to set Task.jvmContext sooner to avoid corner
>> cases in error handling. (Siddharth Seth via acmurthy)
>>
>> MapReduce v1 exceptions:
>> MAPREDUCE-2415. Distribute the user task logs on to multiple disks.
>> (Bharath Mundlapudi via omalley)
>> MAPREDUCE-2413. TaskTracker should handle disk failures by reinitializing
>> itself. (Ravi Gummadi and Jagane Sundar via omalley)
>> MAPREDUCE-2621. TestCapacityScheduler fails with "Queue "q1" does not
>> exist". (Sherry Chen via mahadev)
>> MAPREDUCE-2535. Fix NPE in JobClient caused by retirement. (Robert Joseph
>> Evans via cdouglas)
>> MAPREDUCE-2555. Avoid sprious logging from completedtasks. (Thomas Graves
>> via cdouglas)
>> MAPREDUCE-2443. Fix TaskAspect for TaskUmbilicalProtocol.ping(..).
>> (Siddharth Seth via szetszwo)
>>
>>
>>
>>
>>
>>
>> On Wed, Aug 3, 2011 at 2:02 PM, Arun C Murthy <a...@hortonworks.com> wrote:
>>
>>> On Aug 2, 2011, at 4:28 AM, Steve Loughran wrote:
>>>> I'm getting confused about release roadmaps right now
>>>>
>>>> Is there somewhere that lists the (proposed) timetable for the 0.20.204,
>>> 0.20.205, 0.22, 0.23 releases?
>>>>
>>>
>>> Since I was among the people who started the 'security on 0.20' thread, I
>>> apologize for the lack of clarity on the roadmap and timelines.
>>>
>>> Eric did send out a note on the motivation for sustaining releases (
>>> http://s.apache.org/hadoop-sustenance-releases) - which I believe is
>>> important to keep Hadoop installations going. However, I accept that there
>>> has been very poor clarity on the process for inclusion of content to the
>>> sustaining releases and the timelines.
>>>
>>> Here is my proposal for the community process for sustaining releases
>>> moving forward:
>>> # The Release Manager should clearly communicate, upfront, the expected
>>> timeline for each upcoming release.
>>> # Anyone interested in contributing to the release submits a patch to
>>> trunk, if applicable*, and to the branch-0.20-security branch. Then talk to
>>> the Release Manager for inclusion via the normal process.
>>>
>>> The RM is responsible for release content, timelines and for enforcing that
>>> each change should have an equivalent for trunk, as applicable, before
>>> inclusion.
>>>
>>> If this makes sense, I'll add this to the wiki to record it.
>>>
>>> ----
>>>
>>> So, how are we doing with 0.20.204 content and trunk given the above
>>> proposal? Very well, in fact. Matt, Suresh and I have done a detailed
>>> analysis (separate email), please take a look.
>>>
>>> thanks,
>>> Arun
>>>
>>> *Please note the exception that we have agreed that MRv2 is the way forward
>>> (http://s.apache.org/mr1) , and thus, MR1 patches might not be ported
>>> as-is to trunk in some cases such as fixes to JobTracker/TaskTracker.
>>> However, this doesn't mean changes to the runtime (i.e. map task, reduce
>>> task, sort, shuffle etc.) are exempt from the rules proposed above.
>>>
>>>
>>>
>
