http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/CHANGES.0.18.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/CHANGES.0.18.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/CHANGES.0.18.0.md new file mode 100644 index 0000000..7df47c0 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/CHANGES.0.18.0.md @@ -0,0 +1,337 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop Changelog + +## Release 0.18.0 - 2008-08-22 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3837](https://issues.apache.org/jira/browse/HADOOP-3837) | hadop streaming does not use progress reporting to detect hung tasks | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3808](https://issues.apache.org/jira/browse/HADOOP-3808) | [HOD] Include job tracker RPC in notes attribute after job submission | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3703](https://issues.apache.org/jira/browse/HADOOP-3703) | [HOD] logcondense needs to use the new pattern of output in hadoop dfs -lsr | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-3683](https://issues.apache.org/jira/browse/HADOOP-3683) | Hadoop dfs metric FilesListed shows number of files listed instead of operations | Major | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3665](https://issues.apache.org/jira/browse/HADOOP-3665) | WritableComparator newKey() fails for NullWritable | Minor | io | Lukas Vlcek | Chris Douglas | +| [HADOOP-3610](https://issues.apache.org/jira/browse/HADOOP-3610) | [HOD] HOD does not automatically create a cluster directory for the script option | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-3598](https://issues.apache.org/jira/browse/HADOOP-3598) | Map-Reduce framework needlessly creates temporary \_${taskid} directories for Maps | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3569](https://issues.apache.org/jira/browse/HADOOP-3569) | KFS input stream read() returns 4 bytes instead of 1 | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3512](https://issues.apache.org/jira/browse/HADOOP-3512) | Split map/reduce tools into separate jars | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-3486](https://issues.apache.org/jira/browse/HADOOP-3486) | Change default for initial block report to 0 sec and document it in hadoop-defaults.xml | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-3483](https://issues.apache.org/jira/browse/HADOOP-3483) | [HOD] Improvements with cluster directory handling | Major | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3459](https://issues.apache.org/jira/browse/HADOOP-3459) | Change dfs -ls listing to closely match format on Linux | Major | . | Mukund Madhugiri | Mukund Madhugiri | +| [HADOOP-3452](https://issues.apache.org/jira/browse/HADOOP-3452) | fsck exit code would be better if non-zero when FS corrupt | Minor | . | Pete Wyckoff | Lohit Vijayarenu | +| [HADOOP-3417](https://issues.apache.org/jira/browse/HADOOP-3417) | JobClient should not have a static configuration for cli parsing | Major | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-3405](https://issues.apache.org/jira/browse/HADOOP-3405) | Make mapred internal classes package-local | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-3390](https://issues.apache.org/jira/browse/HADOOP-3390) | Remove deprecated ClientProtocol.abandonFileInProgress() | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3379](https://issues.apache.org/jira/browse/HADOOP-3379) | Document the "stream.non.zero.exit.status.is.failure" knob for streaming | Blocker | documentation | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-3329](https://issues.apache.org/jira/browse/HADOOP-3329) | DatanodeDescriptor objects stored in FSImage may be out dated. | Major | . | Tsz Wo Nicholas Sze | dhruba borthakur | +| [HADOOP-3317](https://issues.apache.org/jira/browse/HADOOP-3317) | add default port for hdfs namenode | Minor | . | Doug Cutting | Doug Cutting | +| [HADOOP-3310](https://issues.apache.org/jira/browse/HADOOP-3310) | Lease recovery for append | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3283](https://issues.apache.org/jira/browse/HADOOP-3283) | Need a mechanism for data nodes to update generation stamps. | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3265](https://issues.apache.org/jira/browse/HADOOP-3265) | Remove deprecated API getFileCacheHints | Major | fs | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3226](https://issues.apache.org/jira/browse/HADOOP-3226) | Run combiner when merging spills from map output | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3193](https://issues.apache.org/jira/browse/HADOOP-3193) | Discovery of corrupt block reported in name node log | Minor | . | Robert Chansler | Chris Douglas | +| [HADOOP-3184](https://issues.apache.org/jira/browse/HADOOP-3184) | HOD gracefully exclude "bad" nodes during ring formation | Major | contrib/hod | Marco Nicosia | Hemanth Yamijala | +| [HADOOP-3113](https://issues.apache.org/jira/browse/HADOOP-3113) | DFSOututStream.flush() should flush data to real block file on DataNode. | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3095](https://issues.apache.org/jira/browse/HADOOP-3095) | Validating input paths and creating splits is slow on S3 | Major | fs, fs/s3 | Tom White | Tom White | +| [HADOOP-3035](https://issues.apache.org/jira/browse/HADOOP-3035) | Data nodes should inform the name-node about block crc errors. | Major | . | Konstantin Shvachko | Lohit Vijayarenu | +| [HADOOP-2909](https://issues.apache.org/jira/browse/HADOOP-2909) | Improve IPC idle connection management | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-2865](https://issues.apache.org/jira/browse/HADOOP-2865) | FsShell.ls() should print file attributes first then the path name. | Major | . | Konstantin Shvachko | Edward J. Yoon | +| [HADOOP-2797](https://issues.apache.org/jira/browse/HADOOP-2797) | Withdraw CRC upgrade from HDFS | Critical | . | Robert Chansler | Raghu Angadi | +| [HADOOP-2703](https://issues.apache.org/jira/browse/HADOOP-2703) | New files under lease (before close) still shows up as MISSING files/blocks in fsck | Minor | . | Koji Noguchi | Lohit Vijayarenu | +| [HADOOP-2656](https://issues.apache.org/jira/browse/HADOOP-2656) | Support for upgrading existing cluster to facilitate appends to HDFS files | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2585](https://issues.apache.org/jira/browse/HADOOP-2585) | Automatic namespace recovery from the secondary image. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2188](https://issues.apache.org/jira/browse/HADOOP-2188) | RPC should send a ping rather than use client timeouts | Major | ipc | Owen O'Malley | Hairong Kuang | +| [HADOOP-2181](https://issues.apache.org/jira/browse/HADOOP-2181) | Input Split details for maps should be logged | Minor | . | Lohit Vijayarenu | Amareshwari Sriramadasu | +| [HADOOP-2065](https://issues.apache.org/jira/browse/HADOOP-2065) | Replication policy for corrupted block | Major | . | Koji Noguchi | Lohit Vijayarenu | +| [HADOOP-1702](https://issues.apache.org/jira/browse/HADOOP-1702) | Reduce buffer copies when data is written to DFS | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-544](https://issues.apache.org/jira/browse/HADOOP-544) | Replace the job, tip and task ids with objects. | Major | . | Owen O'Malley | Enis Soztutar | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3502](https://issues.apache.org/jira/browse/HADOOP-3502) | Quota API needs documentation in Forrest | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3460](https://issues.apache.org/jira/browse/HADOOP-3460) | SequenceFileAsBinaryOutputFormat | Minor | . | Koji Noguchi | Koji Noguchi | +| [HADOOP-3336](https://issues.apache.org/jira/browse/HADOOP-3336) | Direct a subset of namenode RPC events for audit logging | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3307](https://issues.apache.org/jira/browse/HADOOP-3307) | Archives in Hadoop. | Major | fs | Mahadev konar | Mahadev konar | +| [HADOOP-3250](https://issues.apache.org/jira/browse/HADOOP-3250) | Extend FileSystem API to allow appending to files | Major | fs | dhruba borthakur | Tsz Wo Nicholas Sze | +| [HADOOP-3246](https://issues.apache.org/jira/browse/HADOOP-3246) | FTP client over HDFS | Major | util | Ankur | Ankur | +| [HADOOP-3230](https://issues.apache.org/jira/browse/HADOOP-3230) | Add command line access to named counters | Major | scripts | Tom White | Tom White | +| [HADOOP-3221](https://issues.apache.org/jira/browse/HADOOP-3221) | Need a "LineBasedTextInputFormat" | Major | . | Milind Bhandarkar | Amareshwari Sriramadasu | +| [HADOOP-3188](https://issues.apache.org/jira/browse/HADOOP-3188) | compaction utility for directories | Major | . | Robert Chansler | Robert Chansler | +| [HADOOP-3187](https://issues.apache.org/jira/browse/HADOOP-3187) | Quotas for name space management | Major | . | Robert Chansler | Hairong Kuang | +| [HADOOP-3074](https://issues.apache.org/jira/browse/HADOOP-3074) | URLStreamHandler for the DFS | Major | util | Christophe Taton | Christophe Taton | +| [HADOOP-3061](https://issues.apache.org/jira/browse/HADOOP-3061) | Writable for single byte and double | Major | io | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-3022](https://issues.apache.org/jira/browse/HADOOP-3022) | Fast Cluster Restart | Major | . | Robert Chansler | Konstantin Shvachko | +| [HADOOP-1915](https://issues.apache.org/jira/browse/HADOOP-1915) | adding counters methods using String (as opposed to Enum) | Minor | . | Alejandro Abdelnur | Tom White | +| [HADOOP-1328](https://issues.apache.org/jira/browse/HADOOP-1328) | Hadoop Streaming needs to provide a way for the stream plugin to update global counters | Major | . | Runping Qi | Tom White | +| [HADOOP-930](https://issues.apache.org/jira/browse/HADOOP-930) | Add support for reading regular (non-block-based) files from S3 in S3FileSystem | Major | fs | Tom White | Tom White | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3599](https://issues.apache.org/jira/browse/HADOOP-3599) | The new setCombineOnceOnly shouldn't take a JobConf, since it is a method on JobConf | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-3572](https://issues.apache.org/jira/browse/HADOOP-3572) | setQuotas usage interface has some minor bugs. | Minor | . | Mahadev konar | Hairong Kuang | +| [HADOOP-3547](https://issues.apache.org/jira/browse/HADOOP-3547) | Improve documentation about distributing native libraries via DistributedCache | Blocker | documentation | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-3535](https://issues.apache.org/jira/browse/HADOOP-3535) | IOUtils.close needs better documentation | Blocker | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-3532](https://issues.apache.org/jira/browse/HADOOP-3532) | Create build targets to create api change reports using jdiff | Major | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-3501](https://issues.apache.org/jira/browse/HADOOP-3501) | deprecate InMemoryFileSystem | Major | fs | Doug Cutting | Doug Cutting | +| [HADOOP-3492](https://issues.apache.org/jira/browse/HADOOP-3492) | add forrest documentation for user archives | Blocker | . | Mahadev konar | Mahadev konar | +| [HADOOP-3467](https://issues.apache.org/jira/browse/HADOOP-3467) | The javadoc for FileSystem.deleteOnExit should have more description | Blocker | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3464](https://issues.apache.org/jira/browse/HADOOP-3464) | [HOD] HOD can improve error messages by reporting failures on compute nodes back to hod client | Major | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-3455](https://issues.apache.org/jira/browse/HADOOP-3455) | IPC.Client synchronisation looks weak | Major | ipc | Steve Loughran | Hairong Kuang | +| [HADOOP-3448](https://issues.apache.org/jira/browse/HADOOP-3448) | Add some more hints of the problem when datanode and namenode don't match | Minor | . | Steve Loughran | Steve Loughran | +| [HADOOP-3434](https://issues.apache.org/jira/browse/HADOOP-3434) | Retain cause of bind failure in Server.bind | Major | . | Steve Loughran | Steve Loughran | +| [HADOOP-3429](https://issues.apache.org/jira/browse/HADOOP-3429) | Increase the buffersize for the streaming parent java process's streams | Major | . | Devaraj Das | Amareshwari Sriramadasu | +| [HADOOP-3406](https://issues.apache.org/jira/browse/HADOOP-3406) | Document controls for profiling maps & reduces | Blocker | documentation | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-3400](https://issues.apache.org/jira/browse/HADOOP-3400) | Facilitate creation of temporary files in HDFS | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3398](https://issues.apache.org/jira/browse/HADOOP-3398) | ReduceTask::closestPowerOf2 is inefficient | Trivial | . | Chris Douglas | Chris Douglas | +| [HADOOP-3377](https://issues.apache.org/jira/browse/HADOOP-3377) | Use StringUtils#replaceAll instead of | Trivial | . | Brice Arnould | Brice Arnould | +| [HADOOP-3369](https://issues.apache.org/jira/browse/HADOOP-3369) | Fast block processing during name-node startup. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3366](https://issues.apache.org/jira/browse/HADOOP-3366) | Shuffle/Merge improvements | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3364](https://issues.apache.org/jira/browse/HADOOP-3364) | Faster image and log edits loading. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3355](https://issues.apache.org/jira/browse/HADOOP-3355) | Configuration should accept decimal and hexadecimal values | Major | conf | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3350](https://issues.apache.org/jira/browse/HADOOP-3350) | distcp should permit users to limit the number of maps | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3345](https://issues.apache.org/jira/browse/HADOOP-3345) | Enhance the hudson-test-patch target | Minor | build | Nigel Daley | Nigel Daley | +| [HADOOP-3334](https://issues.apache.org/jira/browse/HADOOP-3334) | Move lease handling codes out from FSNamesystem | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3332](https://issues.apache.org/jira/browse/HADOOP-3332) | improving the logging during shuffling | Blocker | . | Runping Qi | Devaraj Das | +| [HADOOP-3308](https://issues.apache.org/jira/browse/HADOOP-3308) | Improve QuickSort by excluding values eq the pivot from the partition | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3297](https://issues.apache.org/jira/browse/HADOOP-3297) | The way in which ReduceTask/TaskTracker gets completion events during shuffle can be improved | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-3295](https://issues.apache.org/jira/browse/HADOOP-3295) | Allow TextOutputFormat to use configurable separators | Minor | io | Zheng Shao | Zheng Shao | +| [HADOOP-3277](https://issues.apache.org/jira/browse/HADOOP-3277) | hod should better errors message when deallocate is fired on non allocated directory. | Minor | contrib/hod | Karam Singh | | +| [HADOOP-3274](https://issues.apache.org/jira/browse/HADOOP-3274) | The default constructor of BytesWritable should not create a 100-byte array. | Minor | io | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3270](https://issues.apache.org/jira/browse/HADOOP-3270) | Constant DatanodeCommand should be stored in static fianl immutable variables. | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3254](https://issues.apache.org/jira/browse/HADOOP-3254) | FSNamesystem.gotHeartbeat(..., Object[] xferResults, Object[] deleteList) should not use Object[] as pass-by-reference parameters | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3177](https://issues.apache.org/jira/browse/HADOOP-3177) | Expose DFSOutputStream.fsync API though the FileSystem interface | Major | . | dhruba borthakur | Tsz Wo Nicholas Sze | +| [HADOOP-3164](https://issues.apache.org/jira/browse/HADOOP-3164) | Use FileChannel.transferTo() when data is read from DataNode. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3144](https://issues.apache.org/jira/browse/HADOOP-3144) | better fault tolerance for corrupted text files | Major | . | Joydeep Sen Sarma | Zheng Shao | +| [HADOOP-3096](https://issues.apache.org/jira/browse/HADOOP-3096) | Improve documentation about the Task Execution Environment in the Map-Reduce tutorial | Blocker | documentation | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-3058](https://issues.apache.org/jira/browse/HADOOP-3058) | Hadoop DFS to report more replication metrics | Minor | metrics | Marco Nicosia | Lohit Vijayarenu | +| [HADOOP-3013](https://issues.apache.org/jira/browse/HADOOP-3013) | fsck to show (checksum) corrupted files | Major | . | Koji Noguchi | Lohit Vijayarenu | +| [HADOOP-2987](https://issues.apache.org/jira/browse/HADOOP-2987) | Keep two generations of fsimage | Major | . | Robert Chansler | Konstantin Shvachko | +| [HADOOP-2910](https://issues.apache.org/jira/browse/HADOOP-2910) | Throttle IPC Client/Server during bursts of requests or server slowdown | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-2867](https://issues.apache.org/jira/browse/HADOOP-2867) | Add a task's cwd to it's LD\_LIBRARY\_PATH | Major | . | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-2857](https://issues.apache.org/jira/browse/HADOOP-2857) | libhdfs: no way to set JVM args other than classpath | Minor | . | Craig Macdonald | Craig Macdonald | +| [HADOOP-2799](https://issues.apache.org/jira/browse/HADOOP-2799) | Replace org.apache.hadoop.io.Closeable with java.io.Closeable | Minor | io | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2762](https://issues.apache.org/jira/browse/HADOOP-2762) | Better documentation of controls for memory limits on hadoop daemons and Map-Reduce tasks | Blocker | documentation, scripts | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-2661](https://issues.apache.org/jira/browse/HADOOP-2661) | Replicator log should include block id | Minor | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2461](https://issues.apache.org/jira/browse/HADOOP-2461) | Configuration should trim property names and accept decimal, hexadecimal, and octal numbers | Minor | conf | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2154](https://issues.apache.org/jira/browse/HADOOP-2154) | Non-interleaved checksums would optimize block transfers. | Major | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-2019](https://issues.apache.org/jira/browse/HADOOP-2019) | DistributedFileCache should support .tgz files in addition to jars and zip files | Major | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-4](https://issues.apache.org/jira/browse/HADOOP-4) | tool to mount dfs on linux | Major | . | John Xing | Pete Wyckoff | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3947](https://issues.apache.org/jira/browse/HADOOP-3947) | TaskTrackers fail to connect back upon a re-init action | Blocker | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-3901](https://issues.apache.org/jira/browse/HADOOP-3901) | CLASSPATH in bin/hadoop script is set incorrectly for cygwin | Blocker | scripts | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3897](https://issues.apache.org/jira/browse/HADOOP-3897) | SecondaryNameNode fails with NullPointerException | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3884](https://issues.apache.org/jira/browse/HADOOP-3884) | eclipse plugin build is broken with current eclipse versions | Blocker | contrib/eclipse-plugin | Doug Cutting | Doug Cutting | +| [HADOOP-3865](https://issues.apache.org/jira/browse/HADOOP-3865) | SecondaryNameNode runs out of memory | Blocker | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3855](https://issues.apache.org/jira/browse/HADOOP-3855) | Fix import of MiniDFSCluster in TestCompressedEmptyMapOutputs.java | Blocker | test | Arun C Murthy | Arun C Murthy | +| [HADOOP-3827](https://issues.apache.org/jira/browse/HADOOP-3827) | Jobs with empty map-outputs and intermediate compression fail | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3806](https://issues.apache.org/jira/browse/HADOOP-3806) | Remove debug message from Quicksort | Trivial | . | Chris Douglas | | +| [HADOOP-3798](https://issues.apache.org/jira/browse/HADOOP-3798) | compile-core-test fails to compile | Blocker | build | Mukund Madhugiri | Mukund Madhugiri | +| [HADOOP-3794](https://issues.apache.org/jira/browse/HADOOP-3794) | KFS implementation needs to return directory modification time | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3776](https://issues.apache.org/jira/browse/HADOOP-3776) | NPE in NameNode with unknown blocks | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3774](https://issues.apache.org/jira/browse/HADOOP-3774) | Typos in shell output | Blocker | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3762](https://issues.apache.org/jira/browse/HADOOP-3762) | Task tracker died due to OOM | Blocker | . | Runping Qi | Doug Cutting | +| [HADOOP-3755](https://issues.apache.org/jira/browse/HADOOP-3755) | the gridmix scripts do not work with hod 0.4 | Major | . | Runping Qi | Runping Qi | +| [HADOOP-3752](https://issues.apache.org/jira/browse/HADOOP-3752) | Audit logging fails to record rename | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-3743](https://issues.apache.org/jira/browse/HADOOP-3743) | -libjars, -files and -archives options do not work with 0.18 | Blocker | . | Mahadev konar | Amareshwari Sriramadasu | +| [HADOOP-3737](https://issues.apache.org/jira/browse/HADOOP-3737) | CompressedWritable throws OutOfMemoryError | Major | io | Grant Glouser | Grant Glouser | +| [HADOOP-3724](https://issues.apache.org/jira/browse/HADOOP-3724) | Namenode does not start due to exception throw while saving Image | Blocker | . | Lohit Vijayarenu | dhruba borthakur | +| [HADOOP-3718](https://issues.apache.org/jira/browse/HADOOP-3718) | KFS: write(int v) API writes out an integer rather than a byte | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3716](https://issues.apache.org/jira/browse/HADOOP-3716) | KFS listStatus() returns NULL on empty directories | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3706](https://issues.apache.org/jira/browse/HADOOP-3706) | CompositeInputFormat: Unable to wrap custom InputFormats | Major | . | Jingkei Ly | Jingkei Ly | +| [HADOOP-3693](https://issues.apache.org/jira/browse/HADOOP-3693) | Fix documentation for Archives, distcp and native libraries | Blocker | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3692](https://issues.apache.org/jira/browse/HADOOP-3692) | Fix documentation for Cluster setup and Quick start guides | Blocker | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3691](https://issues.apache.org/jira/browse/HADOOP-3691) | Fix mapred docs | Blocker | documentation | Amareshwari Sriramadasu | Jothi Padmanabhan | +| [HADOOP-3688](https://issues.apache.org/jira/browse/HADOOP-3688) | Fix up HDFS docs | Blocker | . | Robert Chansler | Robert Chansler | +| [HADOOP-3677](https://issues.apache.org/jira/browse/HADOOP-3677) | Problems with generation stamp upgrade | Blocker | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-3673](https://issues.apache.org/jira/browse/HADOOP-3673) | Deadlock in Datanode RPC servers | Blocker | . | dhruba borthakur | Tsz Wo Nicholas Sze | +| [HADOOP-3670](https://issues.apache.org/jira/browse/HADOOP-3670) | JobTracker running out of heap space | Blocker | . | Christian Kunz | Amareshwari Sriramadasu | +| [HADOOP-3668](https://issues.apache.org/jira/browse/HADOOP-3668) | Clean up HOD documentation | Blocker | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-3653](https://issues.apache.org/jira/browse/HADOOP-3653) | test-patch target not working on hudson.zones.apache.org due to HADOOP-3480 | Blocker | . | Nigel Daley | Brice Arnould | +| [HADOOP-3649](https://issues.apache.org/jira/browse/HADOOP-3649) | ArrayIndexOutOfBounds in FSNamesystem.getBlockLocationsInternal | Blocker | . | Arun C Murthy | Lohit Vijayarenu | +| [HADOOP-3647](https://issues.apache.org/jira/browse/HADOOP-3647) | Corner-case in IFile leads to failed tasks | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3645](https://issues.apache.org/jira/browse/HADOOP-3645) | MetricsTimeVaryingRate returns wrong value for metric\_avg\_time | Blocker | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3639](https://issues.apache.org/jira/browse/HADOOP-3639) | Exception when closing DFSClient while multiple files are open | Blocker | . | Benjamin Gufler | Benjamin Gufler | +| [HADOOP-3635](https://issues.apache.org/jira/browse/HADOOP-3635) | Uncaught exception in DataBlockScanner | Blocker | . | Koji Noguchi | Tsz Wo Nicholas Sze | +| [HADOOP-3630](https://issues.apache.org/jira/browse/HADOOP-3630) | CompositeRecordReader: key and values can be in uninitialized state if files being joined have no records | Major | . | Jingkei Ly | Chris Douglas | +| [HADOOP-3615](https://issues.apache.org/jira/browse/HADOOP-3615) | DatanodeProtocol.versionID should be 16L | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3606](https://issues.apache.org/jira/browse/HADOOP-3606) | Update streaming documentation | Blocker | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3604](https://issues.apache.org/jira/browse/HADOOP-3604) | Reduce stuck at shuffling phase | Blocker | . | Runping Qi | Arun C Murthy | +| [HADOOP-3603](https://issues.apache.org/jira/browse/HADOOP-3603) | Setting spill threshold to 100% fails to detect spill for records | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-3597](https://issues.apache.org/jira/browse/HADOOP-3597) | SortValidator always uses the default file system irrespective of the actual input | Major | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-3593](https://issues.apache.org/jira/browse/HADOOP-3593) | Update MapRed tutorial | Blocker | documentation | Devaraj Das | Devaraj Das | +| [HADOOP-3590](https://issues.apache.org/jira/browse/HADOOP-3590) | Null pointer exception in JobTracker when the task tracker is not yet resolved | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-3588](https://issues.apache.org/jira/browse/HADOOP-3588) | Bug report for archives | Blocker | . | Hairong Kuang | Mahadev konar | +| [HADOOP-3586](https://issues.apache.org/jira/browse/HADOOP-3586) | keep combiner backward compatible with earlier versions of hadoop | Blocker | . | Olga Natkovich | Chris Douglas | +| [HADOOP-3580](https://issues.apache.org/jira/browse/HADOOP-3580) | Using a har file as input for the Sort example fails | Blocker | . | Jothi Padmanabhan | Mahadev konar | +| [HADOOP-3576](https://issues.apache.org/jira/browse/HADOOP-3576) | hadoop dfs -mv throws NullPointerException | Blocker | . | Lohit Vijayarenu | Tsz Wo Nicholas Sze | +| [HADOOP-3575](https://issues.apache.org/jira/browse/HADOOP-3575) | clover target broken after src restructuring | Minor | build | Nigel Daley | Nigel Daley | +| [HADOOP-3571](https://issues.apache.org/jira/browse/HADOOP-3571) | ArrayIndexOutOfBoundsException in BlocksMap$BlockInfo.setPrevious | Blocker | . | Tsz Wo Nicholas Sze | Konstantin Shvachko | +| [HADOOP-3564](https://issues.apache.org/jira/browse/HADOOP-3564) | Sometime after successful hod allocation datanode fails to come up with java.net.BindException for dfs.datanode.ipc.address | Blocker | contrib/hod | Karam Singh | Vinod Kumar Vavilapalli | +| [HADOOP-3561](https://issues.apache.org/jira/browse/HADOOP-3561) | With trash enabled, 'hadoop fs -rmr .' still fully deletes the working dir | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-3560](https://issues.apache.org/jira/browse/HADOOP-3560) | Archvies sometimes create empty part files. | Blocker | . | Mahadev konar | Mahadev konar | +| [HADOOP-3559](https://issues.apache.org/jira/browse/HADOOP-3559) | test-libhdfs fails on linux | Blocker | . | Mukund Madhugiri | Lohit Vijayarenu | +| [HADOOP-3552](https://issues.apache.org/jira/browse/HADOOP-3552) | forrest doc for hadoop commands | Blocker | documentation | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-3548](https://issues.apache.org/jira/browse/HADOOP-3548) | The tools.jar is not included in the distribution | Blocker | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-3546](https://issues.apache.org/jira/browse/HADOOP-3546) | TaskTracker re-initialization gets stuck in cleaning up | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3545](https://issues.apache.org/jira/browse/HADOOP-3545) | archive is failing with "Illegal Capacity" error | Blocker | . | Jothi Padmanabhan | Mahadev konar | +| [HADOOP-3544](https://issues.apache.org/jira/browse/HADOOP-3544) | The command "archive" is missing in the example in docs/hadoop\_archives.html (and pdf) | Blocker | documentation | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-3542](https://issues.apache.org/jira/browse/HADOOP-3542) | Hadoop archives should not create \_logs file in the final archive directory. | Blocker | . | Mahadev konar | Mahadev konar | +| [HADOOP-3539](https://issues.apache.org/jira/browse/HADOOP-3539) | Cygwin: cygpath displays an error message in running bin/hadoop script | Blocker | scripts | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3537](https://issues.apache.org/jira/browse/HADOOP-3537) | Datanode not starting up with java.lang.StringIndexOutOfBoundsException in NetworkTopology.remove | Blocker | . | Koji Noguchi | Hairong Kuang | +| [HADOOP-3534](https://issues.apache.org/jira/browse/HADOOP-3534) | The namenode ignores ioexceptions in close | Blocker | . | Owen O'Malley | Tsz Wo Nicholas Sze | +| [HADOOP-3533](https://issues.apache.org/jira/browse/HADOOP-3533) | The api to JobTracker and TaskTracker have changed incompatibly | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-3531](https://issues.apache.org/jira/browse/HADOOP-3531) | Hod does not report job tracker failure on hod client side when job tracker fails to come up | Blocker | contrib/hod | Karam Singh | Hemanth Yamijala | +| [HADOOP-3528](https://issues.apache.org/jira/browse/HADOOP-3528) | Metrics FilesCreated and files\_deleted metrics do not match. | Blocker | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3523](https://issues.apache.org/jira/browse/HADOOP-3523) | [HOD] If a job does not exist in Torque's list of jobs, HOD allocate on previously allocated directory fails. | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3521](https://issues.apache.org/jira/browse/HADOOP-3521) | Hadoop mapreduce task metrics, unable to send metrics data. | Blocker | . | Eric Yang | Arun C Murthy | +| [HADOOP-3520](https://issues.apache.org/jira/browse/HADOOP-3520) | Generation stamp upgrade fails TestDFSUpgradeFromImage | Blocker | . | Konstantin Shvachko | dhruba borthakur | +| [HADOOP-3519](https://issues.apache.org/jira/browse/HADOOP-3519) | NPE in DFS FileSystem rename | Blocker | . | Tom White | Hairong Kuang | +| [HADOOP-3517](https://issues.apache.org/jira/browse/HADOOP-3517) | The last InMemory merge may be missed | Blocker | . | Devaraj Das | Arun C Murthy | +| [HADOOP-3516](https://issues.apache.org/jira/browse/HADOOP-3516) | TestHarFileSystem.testArchives fails with NullPointerException | Blocker | test | Nigel Daley | Subru Krishnan | +| [HADOOP-3513](https://issues.apache.org/jira/browse/HADOOP-3513) | Improve NNThroughputBenchmark log messages. | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3511](https://issues.apache.org/jira/browse/HADOOP-3511) | Namenode should not restore the root's quota if the quota was not in the image | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3509](https://issues.apache.org/jira/browse/HADOOP-3509) | FSNamesystem.close() throws NullPointerException | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3505](https://issues.apache.org/jira/browse/HADOOP-3505) | omissions in HOD documentation | Blocker | contrib/hod, documentation | Ari Rabkin | Vinod Kumar Vavilapalli | +| [HADOOP-3503](https://issues.apache.org/jira/browse/HADOOP-3503) | Race condition when client and namenode start block recovery simultaneously | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3496](https://issues.apache.org/jira/browse/HADOOP-3496) | TestHarFileSystem.testArchives fails | Blocker | fs | Amareshwari Sriramadasu | Tom White | +| [HADOOP-3493](https://issues.apache.org/jira/browse/HADOOP-3493) | TestStreamingFailure fails. | Major | . | Amareshwari Sriramadasu | Lohit Vijayarenu | +| [HADOOP-3491](https://issues.apache.org/jira/browse/HADOOP-3491) | Name-node shutdown causes InterruptedException in ResolutionMonitor | Major | . | Konstantin Shvachko | Lohit Vijayarenu | +| [HADOOP-3489](https://issues.apache.org/jira/browse/HADOOP-3489) | NPE in SafeModeMonitor | Major | . | Konstantin Shvachko | Lohit Vijayarenu | +| [HADOOP-3487](https://issues.apache.org/jira/browse/HADOOP-3487) | Balancer should not allocate a thread per block move | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3480](https://issues.apache.org/jira/browse/HADOOP-3480) | Need to update Eclipse template to reflect current trunk | Blocker | build | Tsz Wo Nicholas Sze | Brice Arnould | +| [HADOOP-3476](https://issues.apache.org/jira/browse/HADOOP-3476) | Code cleanup needed in fuse-dfs | Major | . | Tsz Wo Nicholas Sze | Pete Wyckoff | +| [HADOOP-3471](https://issues.apache.org/jira/browse/HADOOP-3471) | TestIndexedSort sometimes fails | Major | test | Chris Douglas | Chris Douglas | +| [HADOOP-3468](https://issues.apache.org/jira/browse/HADOOP-3468) | Compile error: FTPFileSystem.java:26: cannot access org.apache.commons.net.ftp.FTP | Blocker | fs | Tsz Wo Nicholas Sze | Ankur | +| [HADOOP-3463](https://issues.apache.org/jira/browse/HADOOP-3463) | hadoop scripts don't change directory to hadoop\_home | Critical | scripts | Owen O'Malley | Owen O'Malley | +| [HADOOP-3454](https://issues.apache.org/jira/browse/HADOOP-3454) | Text.find incorrectly searches beyond the end of the buffer | Major | . | Chad Whipkey | Chad Whipkey | +| [HADOOP-3453](https://issues.apache.org/jira/browse/HADOOP-3453) | ipc.Client.close() throws NullPointerException | Major | ipc | Tsz Wo Nicholas Sze | Hairong Kuang | +| [HADOOP-3451](https://issues.apache.org/jira/browse/HADOOP-3451) | test-libhdfs fails on Linux | Blocker | test | Mukund Madhugiri | Lohit Vijayarenu | +| [HADOOP-3443](https://issues.apache.org/jira/browse/HADOOP-3443) | map outputs should not be renamed between partitions | Critical | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-3440](https://issues.apache.org/jira/browse/HADOOP-3440) | TaskRunner creates a symlink with name 'null' if a file is added to DistributedCache without fragment | Minor | . | Abhijit Bagri | Devaraj Das | +| [HADOOP-3435](https://issues.apache.org/jira/browse/HADOOP-3435) | test-patch fail if sh != bash | Major | . | Brice Arnould | Brice Arnould | +| [HADOOP-3427](https://issues.apache.org/jira/browse/HADOOP-3427) | In ReduceTask::fetchOutputs, wait for result can be improved slightly | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-3424](https://issues.apache.org/jira/browse/HADOOP-3424) | the value returned from getPartition should be checked to make sure it is in the range 0..#reduces-1 | Major | . | Owen O'Malley | Chris Douglas | +| [HADOOP-3419](https://issues.apache.org/jira/browse/HADOOP-3419) | TestFsck fails once in a while on WINDOWS/LINUX | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3418](https://issues.apache.org/jira/browse/HADOOP-3418) | NameNode does not restart if parent directory of a "FileUnderConstruction" is deleted. | Blocker | . | Raghu Angadi | Tsz Wo Nicholas Sze | +| [HADOOP-3413](https://issues.apache.org/jira/browse/HADOOP-3413) | SequenceFile.Reader doesn't use the Serialization framework | Critical | io | Arun C Murthy | Tom White | +| [HADOOP-3410](https://issues.apache.org/jira/browse/HADOOP-3410) | KFS implementation needs to return file modification time | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3409](https://issues.apache.org/jira/browse/HADOOP-3409) | NameNode should save the root inode into fsimage | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3408](https://issues.apache.org/jira/browse/HADOOP-3408) | Change FSNamesytem status metrics to IntValue | Major | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3403](https://issues.apache.org/jira/browse/HADOOP-3403) | Job tracker's ExpireTackers thread gets NullPointerException if a tasktracker is lost. | Blocker | . | Amareshwari Sriramadasu | Arun C Murthy | +| [HADOOP-3401](https://issues.apache.org/jira/browse/HADOOP-3401) | Update FileBench to use the "work" directory for SequenceFileOutputFormat | Major | test | Chris Douglas | Chris Douglas | +| [HADOOP-3399](https://issues.apache.org/jira/browse/HADOOP-3399) | Debug log not removed in ipc client | Trivial | ipc | Raghu Angadi | Raghu Angadi | +| [HADOOP-3396](https://issues.apache.org/jira/browse/HADOOP-3396) | Unit test TestDatanodeBlockScanner fails on Windows | Critical | . | Mukund Madhugiri | Lohit Vijayarenu | +| [HADOOP-3393](https://issues.apache.org/jira/browse/HADOOP-3393) | TestHDFSServerPorts fails on LINUX (NFS mounted directory) and on WINDOWS | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3391](https://issues.apache.org/jira/browse/HADOOP-3391) | HADOOP-3248 introduced a findbugs warning. | Minor | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3388](https://issues.apache.org/jira/browse/HADOOP-3388) | TestDatanodeBlockScanner failed while trying to corrupt replicas | Major | test | dhruba borthakur | dhruba borthakur | +| [HADOOP-3381](https://issues.apache.org/jira/browse/HADOOP-3381) | INode interlinks can multiply effect of memory leaks | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3376](https://issues.apache.org/jira/browse/HADOOP-3376) | [HOD] HOD should have a way to detect and deal with clusters that violate/exceed resource manager limits | Major | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-3375](https://issues.apache.org/jira/browse/HADOOP-3375) | Lease paths are sometimes not removed from LeaseManager.sortedLeasesByPath | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3371](https://issues.apache.org/jira/browse/HADOOP-3371) | MBeanUtil dumps stacktrace from registerMBean | Minor | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3365](https://issues.apache.org/jira/browse/HADOOP-3365) | SequenceFile.Sorter.MergeQueue.next does an unnecessary copy of the key | Major | io | Arun C Murthy | Devaraj Das | +| [HADOOP-3363](https://issues.apache.org/jira/browse/HADOOP-3363) | HDFS throws a InconsistentFSStateException when the name node starts up on a directory that isnt formatted | Blocker | . | Steve Loughran | Konstantin Shvachko | +| [HADOOP-3351](https://issues.apache.org/jira/browse/HADOOP-3351) | Fix history viewer | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3349](https://issues.apache.org/jira/browse/HADOOP-3349) | FSNamesystem.changeLease(src, dst) incorrectly updates the paths inside a lease | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3348](https://issues.apache.org/jira/browse/HADOOP-3348) | TestUrlStreamHandler hangs on LINUX | Major | fs | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3340](https://issues.apache.org/jira/browse/HADOOP-3340) | hadoop dfs metrics shows 0 | Major | metrics | Eric Yang | Lohit Vijayarenu | +| [HADOOP-3339](https://issues.apache.org/jira/browse/HADOOP-3339) | DFS Write pipeline does not detect defective datanode correctly if it times out. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3338](https://issues.apache.org/jira/browse/HADOOP-3338) | trunk doesn't compile after HADOOP-544 was committed | Blocker | . | Nigel Daley | Christophe Taton | +| [HADOOP-3337](https://issues.apache.org/jira/browse/HADOOP-3337) | Name-node fails to start because DatanodeInfo format changed. | Blocker | . | Konstantin Shvachko | Tsz Wo Nicholas Sze | +| [HADOOP-3335](https://issues.apache.org/jira/browse/HADOOP-3335) | 'make clean' in src/c++/libhdfs does 'rm -rf /\*' | Critical | build | Doug Cutting | Doug Cutting | +| [HADOOP-3333](https://issues.apache.org/jira/browse/HADOOP-3333) | job failing because of reassigning same tasktracker to failing tasks | Blocker | . | Christian Kunz | Jothi Padmanabhan | +| [HADOOP-3326](https://issues.apache.org/jira/browse/HADOOP-3326) | ReduceTask should not sleep for 200 ms while waiting for merge to finish | Major | . | Owen O'Malley | Sharad Agarwal | +| [HADOOP-3320](https://issues.apache.org/jira/browse/HADOOP-3320) | NPE in NetworkTopology.getDistance() | Blocker | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-3318](https://issues.apache.org/jira/browse/HADOOP-3318) | Hadoop streaming doesn't recognize "Darwin" as an OS but Soylatte (OpenJDK port to Mac) reports that rather than "Mac OS X" | Major | . | Sam Pullara | Sam Pullara | +| [HADOOP-3313](https://issues.apache.org/jira/browse/HADOOP-3313) | RPC::Invoker makes unnecessary calls to System.currentTimeMillis | Minor | . | Chris Douglas | Chris Douglas | +| [HADOOP-3309](https://issues.apache.org/jira/browse/HADOOP-3309) | Unit test fails on Windows: org.apache.hadoop.mapred.TestMiniMRDFSSort.unknown | Major | . | Mukund Madhugiri | Lohit Vijayarenu | +| [HADOOP-3301](https://issues.apache.org/jira/browse/HADOOP-3301) | Misleading error message when S3 URI contains hostname containing an underscore | Major | fs/s3 | Tom White | Tom White | +| [HADOOP-3300](https://issues.apache.org/jira/browse/HADOOP-3300) | FindBugs warnings in NetworkTopology | Major | . | Tom White | Tom White | +| [HADOOP-3299](https://issues.apache.org/jira/browse/HADOOP-3299) | org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException | Major | io | Jason | Chris Douglas | +| [HADOOP-3296](https://issues.apache.org/jira/browse/HADOOP-3296) | Some levels are skipped while creating the task cache in JobInProgress | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-3282](https://issues.apache.org/jira/browse/HADOOP-3282) | TestCheckpoint occasionally fails because of the port issues. | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3272](https://issues.apache.org/jira/browse/HADOOP-3272) | Reduce redundant copy of Block object in BlocksMap.map hash map | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3269](https://issues.apache.org/jira/browse/HADOOP-3269) | NameNode doesn't startup when restarted after running an MR job | Blocker | . | Devaraj Das | Tsz Wo Nicholas Sze | +| [HADOOP-3268](https://issues.apache.org/jira/browse/HADOOP-3268) | TestUrlStreamHandler.testFileUrls fails on Windows | Major | test | Nigel Daley | Christophe Taton | +| [HADOOP-3259](https://issues.apache.org/jira/browse/HADOOP-3259) | Configuration.substituteVars() needs to handle security exceptions | Trivial | conf | Steve Loughran | Edward J. Yoon | +| [HADOOP-3248](https://issues.apache.org/jira/browse/HADOOP-3248) | Improve Namenode startup performance | Major | . | girish vaitheeswaran | dhruba borthakur | +| [HADOOP-3240](https://issues.apache.org/jira/browse/HADOOP-3240) | TestJobShell should not create files in the current directory | Blocker | test | Tsz Wo Nicholas Sze | Mahadev konar | +| [HADOOP-3232](https://issues.apache.org/jira/browse/HADOOP-3232) | Datanodes time out | Critical | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-3203](https://issues.apache.org/jira/browse/HADOOP-3203) | TaskTracker::localizeJob doesn't provide the correct size to LocalDirAllocator | Major | . | Chris Douglas | Amareshwari Sriramadasu | +| [HADOOP-3176](https://issues.apache.org/jira/browse/HADOOP-3176) | Change lease record when a open-for-write-file gets renamed | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3160](https://issues.apache.org/jira/browse/HADOOP-3160) | remove exists() from ClientProtocol and NameNode | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3151](https://issues.apache.org/jira/browse/HADOOP-3151) | Hod should have better error messages. | Major | contrib/hod | Arkady Borkovsky | Vinod Kumar Vavilapalli | +| [HADOOP-3135](https://issues.apache.org/jira/browse/HADOOP-3135) | if the 'mapred.system.dir' in the client jobconf is different from the JobTracker's value job submission fails | Critical | . | Alejandro Abdelnur | Subru Krishnan | +| [HADOOP-3130](https://issues.apache.org/jira/browse/HADOOP-3130) | Shuffling takes too long to get the last map output. | Major | . | Runping Qi | Amar Kamat | +| [HADOOP-3127](https://issues.apache.org/jira/browse/HADOOP-3127) | rm /user/\<username\>/.Trash/\_\_\_\_ only moves it back to .Trash | Minor | fs | Koji Noguchi | Brice Arnould | +| [HADOOP-3119](https://issues.apache.org/jira/browse/HADOOP-3119) | Text.getBytes() | Trivial | . | Andrew Gudkov | Tim Nelson | +| [HADOOP-3109](https://issues.apache.org/jira/browse/HADOOP-3109) | RPC should accepted connections even when rpc queue is full (ie undo part of HADOOP-2910) | Blocker | . | Sanjay Radia | Hairong Kuang | +| [HADOOP-3101](https://issues.apache.org/jira/browse/HADOOP-3101) | 'bin/hadoop job' should display the help and silently exit | Minor | . | Amar Kamat | Edward J. Yoon | +| [HADOOP-3085](https://issues.apache.org/jira/browse/HADOOP-3085) | pushMetric() method of various metric util classes should catch exceptions | Major | metrics | Runping Qi | Chris Douglas | +| [HADOOP-3076](https://issues.apache.org/jira/browse/HADOOP-3076) | [HOD] If a cluster directory is specified as a relative path, an existing script.exitcode file will not be deleted. | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2961](https://issues.apache.org/jira/browse/HADOOP-2961) | [HOD] Hod expects port info though external host is not mentioned. | Minor | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2930](https://issues.apache.org/jira/browse/HADOOP-2930) | make {start,stop}-balancer.sh work even if hadoop-daemon.sh isn't in the PATH | Trivial | scripts | Spiros Papadimitriou | Spiros Papadimitriou | +| [HADOOP-2928](https://issues.apache.org/jira/browse/HADOOP-2928) | Remove deprecated methods getContentLength() in ClientProtocol, NameNode, FileSystem, DistributedFileSystem and DFSClient | Blocker | . | Tsz Wo Nicholas Sze | Lohit Vijayarenu | +| [HADOOP-2905](https://issues.apache.org/jira/browse/HADOOP-2905) | fsck -move triggers NPE in namenode | Major | . | Michael Bieniosek | Lohit Vijayarenu | +| [HADOOP-2793](https://issues.apache.org/jira/browse/HADOOP-2793) | Links for worst performing shuffle tasks are broken in Analyze Job. | Minor | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-2669](https://issues.apache.org/jira/browse/HADOOP-2669) | DFS client lost lease during writing into DFS files | Major | . | Runping Qi | dhruba borthakur | +| [HADOOP-2565](https://issues.apache.org/jira/browse/HADOOP-2565) | DFSPath cache of FileStatus can become stale | Major | . | Doug Cutting | Tsz Wo Nicholas Sze | +| [HADOOP-2427](https://issues.apache.org/jira/browse/HADOOP-2427) | Cleanup of mapred.local.dir after maptask is complete | Major | . | Lohit Vijayarenu | Amareshwari Sriramadasu | +| [HADOOP-2393](https://issues.apache.org/jira/browse/HADOOP-2393) | TaskTracker locks up removing job files within a synchronized method | Critical | . | Joydeep Sen Sarma | Amareshwari Sriramadasu | +| [HADOOP-2294](https://issues.apache.org/jira/browse/HADOOP-2294) | In hdfs.h , the comment says you release the result of a hdfsListDirectory with a freehdfsFileInfo, but should say hdfsFreeFileInfo | Trivial | . | Dick King | Craig Macdonald | +| [HADOOP-2132](https://issues.apache.org/jira/browse/HADOOP-2132) | Killing successfully completed jobs moves them to failed | Critical | . | Srikanth Kakani | Jothi Padmanabhan | +| [HADOOP-2095](https://issues.apache.org/jira/browse/HADOOP-2095) | Reducer failed due to Out ofMemory | Major | . | Runping Qi | Arun C Murthy | +| [HADOOP-1318](https://issues.apache.org/jira/browse/HADOOP-1318) | Do not fail completed maps on lost tasktrackers if '-reducer NONE' is specified | Minor | . | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-236](https://issues.apache.org/jira/browse/HADOOP-236) | job tracker should refuse connection from a task tracker with a different version number | Major | . | Hairong Kuang | Sharad Agarwal | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3100](https://issues.apache.org/jira/browse/HADOOP-3100) | Develop tests to test the DFS command line interface | Major | test | Mukund Madhugiri | Mukund Madhugiri | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3541](https://issues.apache.org/jira/browse/HADOOP-3541) | Namespace recovery from the secondary image should be documented. | Blocker | documentation | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2984](https://issues.apache.org/jira/browse/HADOOP-2984) | Distcp should have forrest documentation | Blocker | util | Owen O'Malley | Chris Douglas | +| [HADOOP-2632](https://issues.apache.org/jira/browse/HADOOP-2632) | Discussion of fsck operation in the permissions regime | Major | . | Robert Chansler | Robert Chansler | + +
http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/RELEASENOTES.0.18.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/RELEASENOTES.0.18.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/RELEASENOTES.0.18.0.md new file mode 100644 index 0000000..81278d1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/RELEASENOTES.0.18.0.md @@ -0,0 +1,545 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop 0.18.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-3837](https://issues.apache.org/jira/browse/HADOOP-3837) | *Major* | **hadop streaming does not use progress reporting to detect hung tasks** + +Changed streaming tasks to adhere to task timeout value specified in the job configuration. + + +--- + +* [HADOOP-3808](https://issues.apache.org/jira/browse/HADOOP-3808) | *Blocker* | **[HOD] Include job tracker RPC in notes attribute after job submission** + +Modified HOD to include the RPC port of the JobTracker in the 'notes' attribute of the resource manager. The RPC port is included as the string 'Mapred RPC Port:\<port number\>'. Tools that depend on the value of the notes attribute must change to parse this new value. + + +--- + +* [HADOOP-3703](https://issues.apache.org/jira/browse/HADOOP-3703) | *Blocker* | **[HOD] logcondense needs to use the new pattern of output in hadoop dfs -lsr** + +Modified logcondense.py to use the new format of hadoop dfs -lsr output. This version of logcondense would not work with previous versions of Hadoop and hence is incompatible. + + +--- + +* [HADOOP-3683](https://issues.apache.org/jira/browse/HADOOP-3683) | *Major* | **Hadoop dfs metric FilesListed shows number of files listed instead of operations** + +Change FileListed to getNumGetListingOps and add CreateFileOps, DeleteFileOps and AddBlockOps metrics. + + +--- + +* [HADOOP-3677](https://issues.apache.org/jira/browse/HADOOP-3677) | *Blocker* | **Problems with generation stamp upgrade** + +Simplify generation stamp upgrade by making is a local upgrade on datandodes. Deleted distributed upgrade. + + +--- + +* [HADOOP-3665](https://issues.apache.org/jira/browse/HADOOP-3665) | *Minor* | **WritableComparator newKey() fails for NullWritable** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-3610](https://issues.apache.org/jira/browse/HADOOP-3610) | *Blocker* | **[HOD] HOD does not automatically create a cluster directory for the script option** + +Modified HOD to automatically create a cluster directory if the one specified with the script command does not exist. + + +--- + +* [HADOOP-3598](https://issues.apache.org/jira/browse/HADOOP-3598) | *Blocker* | **Map-Reduce framework needlessly creates temporary \_${taskid} directories for Maps** + +Changed Map-Reduce framework to no longer create temporary task output directories for staging outputs if staging outputs isn't necessary. ${mapred.out.dir}/\_temporary/\_${taskid} + + +--- + +* [HADOOP-3569](https://issues.apache.org/jira/browse/HADOOP-3569) | *Minor* | **KFS input stream read() returns 4 bytes instead of 1** + +Fixed KFS to have read() read and return 1 byte instead of 4. + + +--- + +* [HADOOP-3564](https://issues.apache.org/jira/browse/HADOOP-3564) | *Blocker* | **Sometime after successful hod allocation datanode fails to come up with java.net.BindException for dfs.datanode.ipc.address** + +Modifed HOD to generate the dfs.datanode.ipc.address parameter in the hadoop-site.xml of datanodes that it launches. + + +--- + +* [HADOOP-3512](https://issues.apache.org/jira/browse/HADOOP-3512) | *Major* | **Split map/reduce tools into separate jars** + +Separated Distcp, Logalyzer and Archiver into a tools jar. + + +--- + +* [HADOOP-3486](https://issues.apache.org/jira/browse/HADOOP-3486) | *Major* | **Change default for initial block report to 0 sec and document it in hadoop-defaults.xml** + +Changed the default value of dfs.blockreport.initialDelay to be 0 seconds. + + +--- + +* [HADOOP-3483](https://issues.apache.org/jira/browse/HADOOP-3483) | *Major* | **[HOD] Improvements with cluster directory handling** + +Modified HOD to create a cluster directory if one does not exist and to auto-deallocate a cluster while reallocating it, if it is already dead. + + +--- + +* [HADOOP-3464](https://issues.apache.org/jira/browse/HADOOP-3464) | *Major* | **[HOD] HOD can improve error messages by reporting failures on compute nodes back to hod client** + +Implemented a mechanism to transfer HOD errors that occur on compute nodes to the submit node running the HOD client, so users have good feedback on why an allocation failed. + + +--- + +* [HADOOP-3460](https://issues.apache.org/jira/browse/HADOOP-3460) | *Minor* | **SequenceFileAsBinaryOutputFormat** + +Created SequenceFileAsBinaryOutputFormat to write raw bytes as keys and values to a SequenceFile. + + +--- + +* [HADOOP-3459](https://issues.apache.org/jira/browse/HADOOP-3459) | *Major* | **Change dfs -ls listing to closely match format on Linux** + +Changed the output of the "fs -ls" command to more closely match familiar Linux format. Applications that parse the command output should be reviewed. + + +--- + +* [HADOOP-3452](https://issues.apache.org/jira/browse/HADOOP-3452) | *Minor* | **fsck exit code would be better if non-zero when FS corrupt** + +Changed exit status of fsck to report whether the files system is healthy or corrupt. + + +--- + +* [HADOOP-3429](https://issues.apache.org/jira/browse/HADOOP-3429) | *Major* | **Increase the buffersize for the streaming parent java process's streams** + +Increased the size of the buffer used in the communication between the Java task and the Streaming process to 128KB. + + +--- + +* [HADOOP-3427](https://issues.apache.org/jira/browse/HADOOP-3427) | *Major* | **In ReduceTask::fetchOutputs, wait for result can be improved slightly** + +Changed shuffle scheduler policy to wait for notifications from shuffle threads before scheduling more. + + +--- + +* [HADOOP-3417](https://issues.apache.org/jira/browse/HADOOP-3417) | *Major* | **JobClient should not have a static configuration for cli parsing** + +Removed the public class org.apache.hadoop.mapred.JobShell. +Command line options -libjars, -files and -archives are moved to GenericCommands. Thus applications have to implement org.apache.hadoop.util.Tool to use the options. + + +--- + +* [HADOOP-3405](https://issues.apache.org/jira/browse/HADOOP-3405) | *Major* | **Make mapred internal classes package-local** + +Refactored previously public classes MapTaskStatus, ReduceTaskStatus, JobSubmissionProtocol, CompletedJobStatusStore to be package local. + + +--- + +* [HADOOP-3390](https://issues.apache.org/jira/browse/HADOOP-3390) | *Major* | **Remove deprecated ClientProtocol.abandonFileInProgress()** + +Removed deprecated ClientProtocol.abandonFileInProgress(). + + +--- + +* [HADOOP-3379](https://issues.apache.org/jira/browse/HADOOP-3379) | *Blocker* | **Document the "stream.non.zero.exit.status.is.failure" knob for streaming** + +Set default value for configuration property "stream.non.zero.exit.status.is.failure" to be "true". + + +--- + +* [HADOOP-3376](https://issues.apache.org/jira/browse/HADOOP-3376) | *Major* | **[HOD] HOD should have a way to detect and deal with clusters that violate/exceed resource manager limits** + +Modified HOD client to look for specific messages related to resource limit overruns and take appropriate actions - such as either failing to allocate the cluster, or issuing a warning to the user. A tool is provided, specific to Maui and Torque, that will set these specific messages. + + +--- + +* [HADOOP-3366](https://issues.apache.org/jira/browse/HADOOP-3366) | *Major* | **Shuffle/Merge improvements** + +Improved shuffle so that all fetched map-outputs are kept in-memory before being merged by stalling the shuffle so that the in-memory merge executes and frees up memory for the shuffle. + + +--- + +* [HADOOP-3355](https://issues.apache.org/jira/browse/HADOOP-3355) | *Major* | **Configuration should accept decimal and hexadecimal values** + +Added support for hexadecimal values in Configuration + + +--- + +* [HADOOP-3339](https://issues.apache.org/jira/browse/HADOOP-3339) | *Major* | **DFS Write pipeline does not detect defective datanode correctly if it times out.** + +Improved failure handling of last Data Node in write pipeline. + + +--- + +* [HADOOP-3336](https://issues.apache.org/jira/browse/HADOOP-3336) | *Major* | **Direct a subset of namenode RPC events for audit logging** + +Added a log4j appender that emits events from FSNamesystem for audit logging + + +--- + +* [HADOOP-3329](https://issues.apache.org/jira/browse/HADOOP-3329) | *Major* | **DatanodeDescriptor objects stored in FSImage may be out dated.** + +Changed format of file system image to not store locations of last block. + + +--- + +* [HADOOP-3326](https://issues.apache.org/jira/browse/HADOOP-3326) | *Major* | **ReduceTask should not sleep for 200 ms while waiting for merge to finish** + +Changed fetchOutputs() so that LocalFSMerger and InMemFSMergeThread threads are spawned only once. The thread gets notified when something is ready for merge. The merge happens when thresholds are met. + + +--- + +* [HADOOP-3317](https://issues.apache.org/jira/browse/HADOOP-3317) | *Minor* | **add default port for hdfs namenode** + +Changed the default port for "hdfs:" URIs to be 8020, so that one may simply use URIs of the form "hdfs\://example.com/dir/file". + + +--- + +* [HADOOP-3310](https://issues.apache.org/jira/browse/HADOOP-3310) | *Major* | **Lease recovery for append** + +Implemented Lease Recovery to sync the last bock of a file. Added ClientDatanodeProtocol for client trigging block recovery. Changed DatanodeProtocol to support block synchronization. Changed InterDatanodeProtocol to support block update. + + +--- + +* [HADOOP-3307](https://issues.apache.org/jira/browse/HADOOP-3307) | *Major* | **Archives in Hadoop.** + +Introduced archive feature to Hadoop. A Map/Reduce job can be run to create an archive with indexes. A FileSystem abstraction is provided over the archive. + + +--- + +* [HADOOP-3299](https://issues.apache.org/jira/browse/HADOOP-3299) | *Major* | **org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException** + +Changed the TextInputFormat and KeyValueTextInput classes to initialize the compressionCodecs member variable before dereferencing it. + + +--- + +* [HADOOP-3283](https://issues.apache.org/jira/browse/HADOOP-3283) | *Major* | **Need a mechanism for data nodes to update generation stamps.** + +Added an IPC server in DataNode and a new IPC protocol InterDatanodeProtocol. Added conf properties dfs.datanode.ipc.address and dfs.datanode.handler.count with defaults "0.0.0.0:50020" and 3, respectively. +Changed the serialization in DatanodeRegistration and DatanodeInfo, and therefore, updated the versionID in ClientProtocol, DatanodeProtocol, NamenodeProtocol. + + +--- + +* [HADOOP-3265](https://issues.apache.org/jira/browse/HADOOP-3265) | *Major* | **Remove deprecated API getFileCacheHints** + +Removed deprecated API getFileCacheHints + + +--- + +* [HADOOP-3246](https://issues.apache.org/jira/browse/HADOOP-3246) | *Major* | **FTP client over HDFS** + +Introduced an FTPFileSystem backed by Apache Commons FTPClient to directly store data into HDFS. + + +--- + +* [HADOOP-3232](https://issues.apache.org/jira/browse/HADOOP-3232) | *Critical* | **Datanodes time out** + +Changed 'du' command to run in a seperate thread so that it does not block user. + + +--- + +* [HADOOP-3230](https://issues.apache.org/jira/browse/HADOOP-3230) | *Major* | **Add command line access to named counters** + +Added command line tool "job -counter \<job-id\> \<group-name\> \<counter-name\>" to access counters. + + +--- + +* [HADOOP-3226](https://issues.apache.org/jira/browse/HADOOP-3226) | *Major* | **Run combiner when merging spills from map output** + +Changed policy for running combiner. The combiner may be run multiple times as the map's output is sorted and merged. Additionally, it may be run on the reduce side as data is merged. The old semantics are available in Hadoop 0.18 if the user calls: +job.setCombineOnlyOnce(true); + + +--- + +* [HADOOP-3221](https://issues.apache.org/jira/browse/HADOOP-3221) | *Major* | **Need a "LineBasedTextInputFormat"** + +Added org.apache.hadoop.mapred.lib.NLineInputFormat ,which splits N lines of input as one split. N can be specified by configuration property "mapred.line.input.format.linespermap", which defaults to 1. + + +--- + +* [HADOOP-3193](https://issues.apache.org/jira/browse/HADOOP-3193) | *Minor* | **Discovery of corrupt block reported in name node log** + +Added reporter to FSNamesystem stateChangeLog, and a new metric to track the number of corrupted replicas. + + +--- + +* [HADOOP-3187](https://issues.apache.org/jira/browse/HADOOP-3187) | *Major* | **Quotas for name space management** + +Introduced directory quota as hard limits on the number of names in the tree rooted at that directory. An administrator may set quotas on individual directories explicitly. Newly created directories have no associated quota. File/directory creations fault if the quota would be exceeded. The attempt to set a quota faults if the directory would be in violation of the new quota. + + +--- + +* [HADOOP-3184](https://issues.apache.org/jira/browse/HADOOP-3184) | *Major* | **HOD gracefully exclude "bad" nodes during ring formation** + +Modified HOD to handle master (NameNode or JobTracker) failures on bad nodes by trying to bring them up on another node in the ring. Introduced new property ringmaster.max-master-failures to specify the maximum number of times a master is allowed to fail. + + +--- + +* [HADOOP-3177](https://issues.apache.org/jira/browse/HADOOP-3177) | *Major* | **Expose DFSOutputStream.fsync API though the FileSystem interface** + +Added a new public interface Syncable which declares the sync() operation. FSDataOutputStream implements Syncable. If the wrappedStream in FSDataOutputStream is Syncalbe, calling FSDataOutputStream.sync() is equivalent to call wrappedStream.sync(). Otherwise, FSDataOutputStream.sync() is a no-op. Both DistributedFileSystem and LocalFileSystem support the sync() operation. + + +--- + +* [HADOOP-3164](https://issues.apache.org/jira/browse/HADOOP-3164) | *Major* | **Use FileChannel.transferTo() when data is read from DataNode.** + +Changed data node to use FileChannel.tranferTo() to transfer block data. + + +--- + +* [HADOOP-3135](https://issues.apache.org/jira/browse/HADOOP-3135) | *Critical* | **if the 'mapred.system.dir' in the client jobconf is different from the JobTracker's value job submission fails** + +Changed job submission protocol to not allow submission if the client's value of mapred.system.dir does not match the job tracker's. Deprecated JobConf.getSystemDir(); use JobClient.getSystemDir(). + + +--- + +* [HADOOP-3113](https://issues.apache.org/jira/browse/HADOOP-3113) | *Major* | **DFSOututStream.flush() should flush data to real block file on DataNode.** + +Added sync() method to FSDataOutputStream to really, really persist data in HDFS. InterDatanodeProtocol to implement this feature. + + +--- + +* [HADOOP-3095](https://issues.apache.org/jira/browse/HADOOP-3095) | *Major* | **Validating input paths and creating splits is slow on S3** + +Added overloaded method getFileBlockLocations(FileStatus, long, long). This is an incompatible change for FileSystem implementations which override getFileBlockLocations(Path, long, long). They should have the signature of this method changed to getFileBlockLocations(FileStatus, long, long) to work correctly. + + +--- + +* [HADOOP-3061](https://issues.apache.org/jira/browse/HADOOP-3061) | *Major* | **Writable for single byte and double** + +Introduced ByteWritable and DoubleWritable (implementing WritableComparable) implementations for Byte and Double. + + +--- + +* [HADOOP-3058](https://issues.apache.org/jira/browse/HADOOP-3058) | *Minor* | **Hadoop DFS to report more replication metrics** + +Added FSNamesystem status metrics. + + +--- + +* [HADOOP-3035](https://issues.apache.org/jira/browse/HADOOP-3035) | *Major* | **Data nodes should inform the name-node about block crc errors.** + +Changed protocol for transferring blocks between data nodes to report corrupt blocks to data node for re-replication from a good replica. + + +--- + +* [HADOOP-3013](https://issues.apache.org/jira/browse/HADOOP-3013) | *Major* | **fsck to show (checksum) corrupted files** + +fsck reports corrupt blocks in the system. + + +--- + +* [HADOOP-2909](https://issues.apache.org/jira/browse/HADOOP-2909) | *Major* | **Improve IPC idle connection management** + +Removed property ipc.client.maxidletime from the default configuration. The allowed idle time is twice ipc.client.connection.maxidletime. + + +--- + +* [HADOOP-2867](https://issues.apache.org/jira/browse/HADOOP-2867) | *Major* | **Add a task's cwd to it's LD\_LIBRARY\_PATH** + +Added task's cwd to its LD\_LIBRARY\_PATH. + + +--- + +* [HADOOP-2865](https://issues.apache.org/jira/browse/HADOOP-2865) | *Major* | **FsShell.ls() should print file attributes first then the path name.** + +Changed the output of the "fs -ls" command to more closely match familiar Linux format. Additional changes were made by HADOOP-3459. Applications that parse the command output should be reviewed. + + +--- + +* [HADOOP-2797](https://issues.apache.org/jira/browse/HADOOP-2797) | *Critical* | **Withdraw CRC upgrade from HDFS** + +Withdrew the upgrade-to-CRC facility. HDFS will no longer support upgrades from versions without CRCs for block data. Users upgrading from version 0.13 or earlier must first upgrade to an intermediate (0.14, 0.15, 0.16, 0.17) version before doing upgrade to version 0.18 or later. + + +--- + +* [HADOOP-2703](https://issues.apache.org/jira/browse/HADOOP-2703) | *Minor* | **New files under lease (before close) still shows up as MISSING files/blocks in fsck** + +Changed fsck to ignore files opened for writing. Introduced new option "-openforwrite" to explicitly show open files. + + +--- + +* [HADOOP-2656](https://issues.apache.org/jira/browse/HADOOP-2656) | *Major* | **Support for upgrading existing cluster to facilitate appends to HDFS files** + +Associated a generation stamp with each block. On data nodes, the generation stamp is stored as part of the file name of the block's meta-data file. + + +--- + +* [HADOOP-2585](https://issues.apache.org/jira/browse/HADOOP-2585) | *Major* | **Automatic namespace recovery from the secondary image.** + +Improved management of replicas of the name space image. If all replicas on the Name Node are lost, the latest check point can be loaded from the secondary Name Node. Use parameter "-importCheckpoint" and specify the location with "fs.checkpoint.dir." The directory structure on the secondary Name Node has changed to match the primary Name Node. + + +--- + +* [HADOOP-2427](https://issues.apache.org/jira/browse/HADOOP-2427) | *Major* | **Cleanup of mapred.local.dir after maptask is complete** + +The current working directory of a task, i.e. ${mapred.local.dir}/taskTracker/jobcache/\<jobid\>/\<task\_dir\>/work is cleanedup, as soon as the task is finished. + + +--- + +* [HADOOP-2188](https://issues.apache.org/jira/browse/HADOOP-2188) | *Major* | **RPC should send a ping rather than use client timeouts** + +Replaced timeouts with pings to check that client connection is alive. Removed the property ipc.client.timeout from the default Hadoop configuration. Removed the metric RpcOpsDiscardedOPsNum. + + +--- + +* [HADOOP-2181](https://issues.apache.org/jira/browse/HADOOP-2181) | *Minor* | **Input Split details for maps should be logged** + +Added logging for input splits in job tracker log and job history log. Added web UI for viewing input splits in the job UI and history UI. + + +--- + +* [HADOOP-2132](https://issues.apache.org/jira/browse/HADOOP-2132) | *Critical* | **Killing successfully completed jobs moves them to failed** + +Change "job -kill" to only allow a job that is in the RUNNING or PREP state to be killed. + + +--- + +* [HADOOP-2095](https://issues.apache.org/jira/browse/HADOOP-2095) | *Major* | **Reducer failed due to Out ofMemory** + +Reduced in-memory copies of keys and values as they flow through the Map-Reduce framework. Changed the storage of intermediate map outputs to use new IFile instead of SequenceFile for better compression. + + +--- + +* [HADOOP-2065](https://issues.apache.org/jira/browse/HADOOP-2065) | *Major* | **Replication policy for corrupted block** + +Added "corrupt" flag to LocatedBlock to indicate that all replicas of the block thought to be corrupt. + + +--- + +* [HADOOP-2019](https://issues.apache.org/jira/browse/HADOOP-2019) | *Major* | **DistributedFileCache should support .tgz files in addition to jars and zip files** + +Added support for .tar, .tgz and .tar.gz files in DistributedCache. File sizes are limited to 2GB. + + +--- + +* [HADOOP-1915](https://issues.apache.org/jira/browse/HADOOP-1915) | *Minor* | **adding counters methods using String (as opposed to Enum)** + +Provided a new method to update counters. "incrCounter(String group, String counter, long amount)" + + +--- + +* [HADOOP-1702](https://issues.apache.org/jira/browse/HADOOP-1702) | *Major* | **Reduce buffer copies when data is written to DFS** + +Reduced buffer copies as data is written to HDFS. The order of sending data bytes and control information has changed, but this will not be observed by client applications. + + +--- + +* [HADOOP-1328](https://issues.apache.org/jira/browse/HADOOP-1328) | *Major* | **Hadoop Streaming needs to provide a way for the stream plugin to update global counters** + +Introduced a way for a streaming process to update global counters and status using stderr stream to emit information. Use "reporter:counter:\<group\>,\<counter\>,\<amount\> " to update a counter. Use "reporter:status:\<message\>" to update status. + + +--- + +* [HADOOP-930](https://issues.apache.org/jira/browse/HADOOP-930) | *Major* | **Add support for reading regular (non-block-based) files from S3 in S3FileSystem** + +Added support for reading and writing native S3 files. Native S3 files are referenced using s3n URIs. See http://wiki.apache.org/hadoop/AmazonS3 for more details. + + +--- + +* [HADOOP-544](https://issues.apache.org/jira/browse/HADOOP-544) | *Major* | **Replace the job, tip and task ids with objects.** + +Introduced new classes JobID, TaskID and TaskAttemptID, which should be used instead of their string counterparts. Deprecated functions in JobClient, TaskReport, RunningJob, jobcontrol.Job and TaskCompletionEvent that use string arguments. Applications can use xxxID.toString() and xxxID.forName() methods to convert/restore objects to/from strings. + + +--- + +* [HADOOP-236](https://issues.apache.org/jira/browse/HADOOP-236) | *Major* | **job tracker should refuse connection from a task tracker with a different version number** + +Changed connection protocol job tracker and task tracker so that task tracker will not connect to a job tracker with a different build version. + + +--- + +* [HADOOP-4](https://issues.apache.org/jira/browse/HADOOP-4) | *Major* | **tool to mount dfs on linux** + +Introduced FUSE module for HDFS. Module allows mount of HDFS as a Unix filesystem, and optionally the export of that mount point to other machines. Writes are disabled. rmdir, mv, mkdir, rm are supported, but not cp, touch, and the like. Usage information is attached to the Jira record. + + + http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/CHANGES.0.18.1.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/CHANGES.0.18.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/CHANGES.0.18.1.md new file mode 100644 index 0000000..8fa57cb --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/CHANGES.0.18.1.md @@ -0,0 +1,74 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop Changelog + +## Release 0.18.1 - 2008-09-17 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4145](https://issues.apache.org/jira/browse/HADOOP-4145) | [HOD] Support an accounting plugin script for HOD | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3934](https://issues.apache.org/jira/browse/HADOOP-3934) | Update log4j from 1.2.13 to 1.2.15 | Blocker | . | Don Kim | Owen O'Malley | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4161](https://issues.apache.org/jira/browse/HADOOP-4161) | [HOD] Uncaught exceptions can potentially hang hod-client. | Major | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-4060](https://issues.apache.org/jira/browse/HADOOP-4060) | [HOD] Make HOD to roll log files on the client | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-4046](https://issues.apache.org/jira/browse/HADOOP-4046) | WritableComparator's constructor should be protected instead of private | Major | io | Owen O'Malley | Chris Douglas | +| [HADOOP-4040](https://issues.apache.org/jira/browse/HADOOP-4040) | Remove the hardcoded ipc.client.connection.maxidletime setting from the TaskTracker.Child.main() | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-4037](https://issues.apache.org/jira/browse/HADOOP-4037) | HadoopQA contrib -1 comments due to inconsistency in eclipse plugin declared jars | Blocker | . | Nigel Daley | Nigel Daley | +| [HADOOP-3995](https://issues.apache.org/jira/browse/HADOOP-3995) | renameTo(src, dst) does not restore src name in case of quota failure. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3940](https://issues.apache.org/jira/browse/HADOOP-3940) | Reduce often attempts in memory merge with no work | Minor | . | Chris Douglas | Chris Douglas | +| [HADOOP-3821](https://issues.apache.org/jira/browse/HADOOP-3821) | SequenceFile's Reader.decompressorPool or Writer.decompressorPool gets into an inconsistent state when calling close() more than once | Major | io | Peter Voss | Arun C Murthy | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/RELEASENOTES.0.18.1.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/RELEASENOTES.0.18.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/RELEASENOTES.0.18.1.md new file mode 100644 index 0000000..1367443 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/RELEASENOTES.0.18.1.md @@ -0,0 +1,38 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop 0.18.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-4060](https://issues.apache.org/jira/browse/HADOOP-4060) | *Blocker* | **[HOD] Make HOD to roll log files on the client** + +HOD client was modified to roll over client logs being written to the cluster directory. A new configuration parameter, hod.log-rollover-count, was introduced to specify how many rollover log files to retain. + + +--- + +* [HADOOP-3995](https://issues.apache.org/jira/browse/HADOOP-3995) | *Blocker* | **renameTo(src, dst) does not restore src name in case of quota failure.** + +In case of quota failure on HDFS, rename does not restore source filename. + + +
