Re: Branching 2.5

I looked in the log, it also looks like findbugs is OOMing: [java] Exception in thread main java.lang.OutOfMemoryError: GC overhead limit exceeded [java] at edu.umd.cs.findbugs.ba.Path.grow(Path.java:263) [java] at edu.umd.cs.findbugs.ba.Path.copyFrom(Path.java:113)

Re: Branching 2.5

Alright, dug around some more and I think it's that FINDBUGS_HOME is not being set correctly. I downloaded and extracted Findbugs 1.3.9, pointed FINDBUGS_HOME at it, and the build worked after that. I don't know what's up with the default maven build, it'd be great if someone could check. Can

[jira] [Created] (HDFS-6780) Batch the encryption zones listing API

Andrew Wang created HDFS-6780: - Summary: Batch the encryption zones listing API Key: HDFS-6780 URL: https://issues.apache.org/jira/browse/HDFS-6780 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-6781) Separate HDFS commands from CommandsManual.apt.vm

Akira AJISAKA created HDFS-6781: --- Summary: Separate HDFS commands from CommandsManual.apt.vm Key: HDFS-6781 URL: https://issues.apache.org/jira/browse/HDFS-6781 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-6782) Improve FS editlog logSync

Yi Liu created HDFS-6782: Summary: Improve FS editlog logSync Key: HDFS-6782 URL: https://issues.apache.org/jira/browse/HDFS-6782 Project: Hadoop HDFS Issue Type: Improvement Components:

[jira] [Created] (HDFS-6783) Fix not really wait for rescan in HDFS CacheReplicationModitor

Yi Liu created HDFS-6783: Summary: Fix not really wait for rescan in HDFS CacheReplicationModitor Key: HDFS-6783 URL: https://issues.apache.org/jira/browse/HDFS-6783 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-6785) Should not be able to create encryption zone using path to a non-directory file

Stephen Chu created HDFS-6785: - Summary: Should not be able to create encryption zone using path to a non-directory file Key: HDFS-6785 URL: https://issues.apache.org/jira/browse/HDFS-6785 Project:

[jira] [Created] (HDFS-6784) Avoid rescan twice in HDFS CacheReplicationMonitor for one FS Op if it calls setNeedsRescan multiple times.

Yi Liu created HDFS-6784: Summary: Avoid rescan twice in HDFS CacheReplicationMonitor for one FS Op if it calls setNeedsRescan multiple times. Key: HDFS-6784 URL: https://issues.apache.org/jira/browse/HDFS-6784

[jira] [Created] (HDFS-6786) Fix potential issue of cache refresh interval

Yi Liu created HDFS-6786: Summary: Fix potential issue of cache refresh interval Key: HDFS-6786 URL: https://issues.apache.org/jira/browse/HDFS-6786 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-6787) Remove duplicate code in FSDirectory#unprotectedConcat

Yi Liu created HDFS-6787: Summary: Remove duplicate code in FSDirectory#unprotectedConcat Key: HDFS-6787 URL: https://issues.apache.org/jira/browse/HDFS-6787 Project: Hadoop HDFS Issue Type: Bug

Build failed in Jenkins: Hadoop-Hdfs-trunk #1821

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1821/changes Changes: [wang] HDFS-6778. The extended attributes javadoc should simply refer to the user docs. Contributed by Charles Lamb. [wang] HDFS-6665. Add tests for XAttrs in combination with viewfs. Contributed by Stephen Chu.

Hadoop-Hdfs-trunk - Build # 1821 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1821/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 17677 lines...] [INFO] [INFO] ---

[jira] [Created] (HDFS-6789) TestDFSClientFailover.testFileContextDoesntDnsResolveLogicalURI and TestDFSClientFailover.testDoesntDnsResolveLogicalURI failing on jdk7

Rushabh S Shah created HDFS-6789: Summary: TestDFSClientFailover.testFileContextDoesntDnsResolveLogicalURI and TestDFSClientFailover.testDoesntDnsResolveLogicalURI failing on jdk7 Key: HDFS-6789 URL:

[jira] [Resolved] (HDFS-1060) Append/flush should support concurrent tailer use case

[ https://issues.apache.org/jira/browse/HDFS-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1060. Resolution: Won't Fix Closing as Won't Fix. If you need append, use 2.x+ Append/flush

[jira] [Resolved] (HDFS-1064) NN Availability - umbrella Jira

[ https://issues.apache.org/jira/browse/HDFS-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1064. Resolution: Fixed Closing this as fixed, given HA is in 2.x. NN Availability - umbrella

[jira] [Resolved] (HDFS-1069) Write HDFS wire protocols in AVRO IDL

[ https://issues.apache.org/jira/browse/HDFS-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1069. Resolution: Won't Fix Closing as Won't Fix. We've already switched to protobuf. Write

[jira] [Resolved] (HDFS-1066) Make HDFS tests pass using Avro RPC

[ https://issues.apache.org/jira/browse/HDFS-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1066. Resolution: Won't Fix Closing as Won't Fix. We've already switched to protobuf. Make

[jira] [Resolved] (HDFS-1086) Percentage bar in dfsnodelist.jsp shows wrong values

[ https://issues.apache.org/jira/browse/HDFS-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1086. Resolution: Incomplete UI has change a lot since this jira. Closing as stale. Percentage

[jira] [Resolved] (HDFS-1097) Many allocated memory pointera of libhdfs maybe null, which maybe cause coredump.

[ https://issues.apache.org/jira/browse/HDFS-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1097. Resolution: Fixed Stale, given how much of libhdfs has been rewritten. Many allocated

[jira] [Resolved] (HDFS-1105) Balancer improvement

[ https://issues.apache.org/jira/browse/HDFS-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1105. Resolution: Incomplete A lot has changed in balancer since this was written up. Closing as

[jira] [Resolved] (HDFS-1113) Allow users with write access to a directory to change ownership of its subdirectories/files

[ https://issues.apache.org/jira/browse/HDFS-1113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1113. Resolution: Fixed There are multiple ways to close this one. Pick your preferred option:

[jira] [Resolved] (HDFS-1115) SocketException: Protocol not available for some JVMs

[ https://issues.apache.org/jira/browse/HDFS-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1115. Resolution: Won't Fix SocketException: Protocol not available for some JVMs

[jira] [Resolved] (HDFS-6692) Add more HDFS encryption tests

[ https://issues.apache.org/jira/browse/HDFS-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-6692. --- Resolution: Fixed Fix Version/s: fs-encryption (HADOOP-10150 and HDFS-6134) Thanks for

[jira] [Created] (HDFS-6790) DFSUtil Should Use configuration.getPassword for SSL passwords

Larry McCay created HDFS-6790: - Summary: DFSUtil Should Use configuration.getPassword for SSL passwords Key: HDFS-6790 URL: https://issues.apache.org/jira/browse/HDFS-6790 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-1128) Allow TestDFSIO to process multiple files for each map

[ https://issues.apache.org/jira/browse/HDFS-1128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1128. Resolution: Fixed Allow TestDFSIO to process multiple files for each map

[jira] [Resolved] (HDFS-1135) Please sort the various options alphabetically in the output of hadoop fs -help

[ https://issues.apache.org/jira/browse/HDFS-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1135. Resolution: Fixed Please sort the various options alphabetically in the output of hadoop

[jira] [Resolved] (HDFS-1137) Name node is using the write-ahead log improperly

[ https://issues.apache.org/jira/browse/HDFS-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1137. Resolution: Not a Problem Name node is using the write-ahead log improperly

[jira] [Resolved] (HDFS-1139) Forward-port TestFileAppend4 tests from 20-append branch

[ https://issues.apache.org/jira/browse/HDFS-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1139. Resolution: Fixed Forward-port TestFileAppend4 tests from 20-append branch

[jira] [Created] (HDFS-6791) A block could remain under replicated if all of its replicas are on decommissioned nodes

Ming Ma created HDFS-6791: - Summary: A block could remain under replicated if all of its replicas are on decommissioned nodes Key: HDFS-6791 URL: https://issues.apache.org/jira/browse/HDFS-6791 Project:

[jira] [Resolved] (HDFS-1166) 0.21 nightly snapshot build has dependency on 0.22 snapshot

[ https://issues.apache.org/jira/browse/HDFS-1166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1166. Resolution: Fixed Likely fixed. 0.21 nightly snapshot build has dependency on 0.22

[jira] [Resolved] (HDFS-1179) Implement a file change log

[ https://issues.apache.org/jira/browse/HDFS-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1179. Resolution: Duplicate I'm just going to resolve this as a dupe of HDFS-6634. Implement a

[jira] [Resolved] (HDFS-1382) A transient failure with edits log and a corrupted fstime together could lead to a data loss

[ https://issues.apache.org/jira/browse/HDFS-1382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1382. Resolution: Fixed A transient failure with edits log and a corrupted fstime together could

[jira] [Resolved] (HDFS-1225) Block lost when primary crashes in recoverBlock

[ https://issues.apache.org/jira/browse/HDFS-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1225. Resolution: Incomplete append got overhauled in 2.x. closing. Block lost when primary

[jira] [Resolved] (HDFS-1224) Stale connection makes node miss append

[ https://issues.apache.org/jira/browse/HDFS-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1224. Resolution: Incomplete append got overhauled in 2.x. closing. Stale connection makes node

[jira] [Resolved] (HDFS-1228) CRC does not match when retrying appending a partial block

[ https://issues.apache.org/jira/browse/HDFS-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1228. Resolution: Won't Fix append got overhauled in 2.x. Closing. CRC does not match when

[jira] [Resolved] (HDFS-1231) Generation Stamp mismatches, leading to failed append

[ https://issues.apache.org/jira/browse/HDFS-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1231. Resolution: Won't Fix append got overhauled in 2.x. closing. Generation Stamp mismatches,

[jira] [Resolved] (HDFS-1243) 0.20 append: Replication tests in TestFileAppend4 should not expect immediate replication

[ https://issues.apache.org/jira/browse/HDFS-1243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1243. Resolution: Fixed Probably stale. 0.20 append: Replication tests in TestFileAppend4

[jira] [Resolved] (HDFS-1238) A block is stuck in ongoingRecovery due to exception not propagated

[ https://issues.apache.org/jira/browse/HDFS-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1238. Resolution: Incomplete Stale. A block is stuck in ongoingRecovery due to exception not

[jira] [Resolved] (HDFS-1244) Misc improvements to TestFileAppend2

[ https://issues.apache.org/jira/browse/HDFS-1244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1244. Resolution: Fixed Probably stale. Misc improvements to TestFileAppend2

[jira] [Created] (HDFS-6793) Missing changes in HftpFileSystem when Reintroduce dfs.http.port / dfs.https.port in branch-2

Juan Yu created HDFS-6793: - Summary: Missing changes in HftpFileSystem when Reintroduce dfs.http.port / dfs.https.port in branch-2 Key: HDFS-6793 URL: https://issues.apache.org/jira/browse/HDFS-6793 Project:

[jira] [Resolved] (HDFS-1248) Misc cleanup/logging improvements for branch-20-append

[ https://issues.apache.org/jira/browse/HDFS-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1248. Resolution: Won't Fix append was overhauled in 2.x. Closing. Misc cleanup/logging

[jira] [Resolved] (HDFS-1251) Unnecessary log messages are printed in info level

[ https://issues.apache.org/jira/browse/HDFS-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1251. Resolution: Fixed Unnecessary log messages are printed in info level

[jira] [Resolved] (HDFS-1264) 0.20: OOME in HDFS client made an unrecoverable HDFS block

[ https://issues.apache.org/jira/browse/HDFS-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1264. Resolution: Fixed 0.20: OOME in HDFS client made an unrecoverable HDFS block

[jira] [Resolved] (HDFS-1281) Fix '$bin' path duplication in setup scripts

[ https://issues.apache.org/jira/browse/HDFS-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1281. Resolution: Duplicate I missed this one and have been able to duplicate it. I'll fix as

[jira] [Resolved] (HDFS-1294) StreamFile is a datanode servlet but it is misplaced in the namenode package

[ https://issues.apache.org/jira/browse/HDFS-1294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1294. Resolution: Won't Fix streamfile appears to be deprecated. closing as wont' fix.

[jira] [Resolved] (HDFS-1326) Provide pluggable mechanism for securing datanodes

[ https://issues.apache.org/jira/browse/HDFS-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1326. Resolution: Duplicate All of the basic hooks should be there in HADOOP-9902 now that the

[jira] [Resolved] (HDFS-1327) TestNameNodeMetrics fails under windows.

[ https://issues.apache.org/jira/browse/HDFS-1327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1327. Resolution: Fixed Almost certainly stale. TestNameNodeMetrics fails under windows.

[jira] [Resolved] (HDFS-1328) ant eclipse-files doesn't load all library necessary to make eclipse happy

[ https://issues.apache.org/jira/browse/HDFS-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1328. Resolution: Incomplete Maven makes this stale. Closing. ant eclipse-files doesn't load

Re: Branching 2.5

Thanks to Andrew's patch on HADOOP-10910, I am able to build an RC. On Wed, Jul 30, 2014 at 1:59 AM, Ted Yu yuzhih...@gmail.com wrote: Adding bui...@apache.org Cheers On Jul 30, 2014, at 12:52 AM, Andrew Wang andrew.w...@cloudera.com wrote: Alright, dug around some more and I think

[jira] [Resolved] (HDFS-1344) trunk HDFS build breakage with build.properties (resolvers=internal)

[ https://issues.apache.org/jira/browse/HDFS-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1344. Resolution: Incomplete Likely stale. trunk HDFS build breakage with build.properties

[jira] [Resolved] (HDFS-1350) make datanodes do graceful shutdown

[ https://issues.apache.org/jira/browse/HDFS-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1350. Resolution: Won't Fix Based upon context, this appears to have been a 0.20-append issue...

[jira] [Resolved] (HDFS-1380) The append pipeline does not followed TSP principal

[ https://issues.apache.org/jira/browse/HDFS-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1380. Resolution: Incomplete Closing as stale. The append pipeline does not followed TSP

[jira] [Resolved] (HDFS-1366) reduce namenode startup time by optimising checkBlockInfo while loading fsimage

[ https://issues.apache.org/jira/browse/HDFS-1366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1366. Resolution: Fixed reduce namenode startup time by optimising checkBlockInfo while loading

[jira] [Resolved] (HDFS-1384) NameNode should give client the first node in the pipeline from different rack other than that of excludedNodes list in the same rack.

[ https://issues.apache.org/jira/browse/HDFS-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1384. Resolution: Incomplete Closing as stale. NameNode should give client the first node in

[jira] [Resolved] (HDFS-1390) Remove or support dfs.web.ugi

[ https://issues.apache.org/jira/browse/HDFS-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1390. Resolution: Fixed Essentially fixed. Closing. Remove or support dfs.web.ugi

[jira] [Resolved] (HDFS-1441) Infinite WAITING on dead s3n connection.

[ https://issues.apache.org/jira/browse/HDFS-1441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved HDFS-1441. Resolution: Not a Problem Infinite WAITING on dead s3n connection.

Build failed in Jenkins: Hadoop-Hdfs-trunk #1822

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1822/changes Changes: [wang] HADOOP-10910. Increase findbugs maxHeap size. (wang) [arp] HDFS-6768. Fix a few unit tests that use hard-coded port numbers. (Arpit Agarwal) [tucu] HADOOP-10756. KMS audit log should consolidate successful

Hadoop-Hdfs-trunk - Build # 1822 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1822/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 13692 lines...] main: [mkdir] Created dir: