[jira] [Commented] (HDFS-6945) ExcessBlocks metric may not be decremented if there are no over replicated blocks

2014-08-27 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111913#comment-14111913 ] Akira AJISAKA commented on HDFS-6945: - The number of excess blocks is incremented but

[jira] [Commented] (HDFS-6945) ExcessBlocks metric may not be decremented if there are no over replicated blocks

2014-08-27 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111920#comment-14111920 ] Akira AJISAKA commented on HDFS-6945: - I propose to add the function to remove the

[jira] [Commented] (HDFS-6944) Archival Storage: add a test framework for testing different migration scenarios

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111930#comment-14111930 ] Jing Zhao commented on HDFS-6944: - Actually it will be better to move the retry logic out

[jira] [Updated] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread James Thomas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Thomas updated HDFS-6865: --- Attachment: HDFS-6865.6.patch [~tlipcon], thanks for the review. Made all of your suggested changes

[jira] [Updated] (HDFS-6944) Archival Storage: add a test framework for testing different migration scenarios

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6944: Attachment: HDFS-6944.002.patch Update the patch. Archival Storage: add a test framework for testing

[jira] [Updated] (HDFS-6944) Archival Storage: add retry and termination logic for Mover

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6944: Summary: Archival Storage: add retry and termination logic for Mover (was: Archival Storage: add a test

[jira] [Updated] (HDFS-6944) Archival Storage: add retry and termination logic for Mover

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6944: Description: This jira plans to 1) add retry and termination logic for Mover, 2) add a testing framework

[jira] [Commented] (HDFS-6879) Adding tracing to Hadoop RPC

2014-08-27 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111998#comment-14111998 ] Masatake Iwasaki commented on HDFS-6879: Thanks for the review comments [~cmccabe]!

[jira] [Commented] (HDFS-6879) Adding tracing to Hadoop RPC

2014-08-27 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112004#comment-14112004 ] Masatake Iwasaki commented on HDFS-6879: {noformat} -1 core tests. The patch failed

[jira] [Updated] (HDFS-6879) Adding tracing to Hadoop RPC

2014-08-27 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HDFS-6879: --- Attachment: HDFS-6879-1.patch attaching updated patch. Adding tracing to Hadoop RPC

[jira] [Assigned] (HDFS-6945) ExcessBlocks metric may not be decremented if there are no over replicated blocks

2014-08-27 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA reassigned HDFS-6945: --- Assignee: Akira AJISAKA ExcessBlocks metric may not be decremented if there are no over

[jira] [Updated] (HDFS-6945) ExcessBlocks metric may not be decremented if there are no over replicated blocks

2014-08-27 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated HDFS-6945: Attachment: HDFS-6945.patch Attaching a patch. ExcessBlocks metric may not be decremented if

[jira] [Updated] (HDFS-6945) ExcessBlocks metric may not be decremented if there are no over replicated blocks

2014-08-27 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated HDFS-6945: Target Version/s: 2.6.0 Status: Patch Available (was: Open) ExcessBlocks metric may

[jira] [Commented] (HDFS-6879) Adding tracing to Hadoop RPC

2014-08-27 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112023#comment-14112023 ] Masatake Iwasaki commented on HDFS-6879: * removed whitespace changes * fixed

[jira] [Updated] (HDFS-6945) excessReplicateMap can increase infinitely

2014-08-27 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated HDFS-6945: Description: I'm seeing ExcessBlocks metric increases to more than 300K in some clusters, however,

[jira] [Commented] (HDFS-6945) excessReplicateMap can increase infinitely

2014-08-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112096#comment-14112096 ] Hadoop QA commented on HDFS-6945: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6879) Adding tracing to Hadoop RPC

2014-08-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112097#comment-14112097 ] Hadoop QA commented on HDFS-6879: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112100#comment-14112100 ] Hadoop QA commented on HDFS-6865: - {color:red}-1 overall{color}. Here are the results of

[jira] [Created] (HDFS-6952) Update Rack Aware documentation and/or script

2014-08-27 Thread Chris Horrocks (JIRA)
Chris Horrocks created HDFS-6952: Summary: Update Rack Aware documentation and/or script Key: HDFS-6952 URL: https://issues.apache.org/jira/browse/HDFS-6952 Project: Hadoop HDFS Issue Type:

[jira] [Created] (HDFS-6953) HDFS file append failing in single node configuration

2014-08-27 Thread Vladislav Falfushinsky (JIRA)
Vladislav Falfushinsky created HDFS-6953: Summary: HDFS file append failing in single node configuration Key: HDFS-6953 URL: https://issues.apache.org/jira/browse/HDFS-6953 Project: Hadoop

[jira] [Updated] (HDFS-6953) HDFS file append failing in single node configuration

2014-08-27 Thread Vladislav Falfushinsky (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Falfushinsky updated HDFS-6953: - Description: The following issue happens in both fully distributed and single

[jira] [Updated] (HDFS-6953) HDFS file append failing in single node configuration

2014-08-27 Thread Vladislav Falfushinsky (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Falfushinsky updated HDFS-6953: - Attachment: test_hdfs.c Main.java hdfs-site.xml

[jira] [Updated] (HDFS-6953) HDFS file append failing in single node configuration

2014-08-27 Thread Vladislav Falfushinsky (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladislav Falfushinsky updated HDFS-6953: - Description: The following issue happens in both fully distributed and single

[jira] [Assigned] (HDFS-6948) DN rejects blocks if it has older UC block

2014-08-27 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne reassigned HDFS-6948: Assignee: Eric Payne DN rejects blocks if it has older UC block

[jira] [Moved] (HDFS-6954) With crypto, no native lib systems are too verbose

2014-08-27 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer moved HADOOP-11011 to HDFS-6954: - Key: HDFS-6954 (was: HADOOP-11011) Project: Hadoop HDFS

[jira] [Updated] (HDFS-6954) With crypto, no native lib systems are too verbose

2014-08-27 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6954: --- Component/s: encryption With crypto, no native lib systems are too verbose

[jira] [Assigned] (HDFS-6954) With crypto, no native lib systems are too verbose

2014-08-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb reassigned HDFS-6954: -- Assignee: Charles Lamb With crypto, no native lib systems are too verbose

[jira] [Commented] (HDFS-6954) With crypto, no native lib systems are too verbose

2014-08-27 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112288#comment-14112288 ] Allen Wittenauer commented on HDFS-6954: distcp is even worse: {code} $ bin/hadoop

[jira] [Commented] (HDFS-6833) DirectoryScanner should not register a deleting block with memory of DataNode

2014-08-27 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112299#comment-14112299 ] Yongjun Zhang commented on HDFS-6833: - HI [~sinchii], Thanks for the new revision and

[jira] [Commented] (HDFS-6833) DirectoryScanner should not register a deleting block with memory of DataNode

2014-08-27 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112301#comment-14112301 ] Yongjun Zhang commented on HDFS-6833: - BTW, I saw why your fix addressed the exception:

[jira] [Resolved] (HDFS-6936) The HttpFSFileSystem should also return the isLazyPersist flag in the FileStatus

2014-08-27 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-6936. - Resolution: Duplicate Fix Version/s: HDFS-6581 Fixed by HDFS-6921. The HttpFSFileSystem

[jira] [Resolved] (HDFS-6921) Add LazyPersist flag to FileStatus

2014-08-27 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-6921. - Resolution: Fixed Fix Version/s: HDFS-6581 Hadoop Flags: Reviewed [~vinayrpet],

[jira] [Work started] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-08-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-6951 started by Charles Lamb. Saving namespace and restarting NameNode will remove existing encryption zones

[jira] [Updated] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-08-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6951: --- Attachment: HDFS-6951.001.patch @schu, Nice find. The problem is that when we create a compact

[jira] [Updated] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-08-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6951: --- Fix Version/s: (was: 3.0.0) Target Version/s: 3.0.0 Status: Patch Available

[jira] [Updated] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-08-27 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-6867: Attachment: HDFS-6867-design-20140827.pdf New design document to outline the two steps of recovering blocks

[jira] [Updated] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-08-27 Thread Zhe Zhang (JIRA)
Type: Bug Components: hdfs-client Reporter: Colin Patrick McCabe Assignee: Zhe Zhang Attachments: HDFS-6867-20140827.patch, HDFS-6867-design-20140820.pdf, HDFS-6867-design-20140821.pdf, HDFS-6867-design-20140822.pdf, HDFS-6867-design-20140827.pdf

[jira] [Updated] (HDFS-6694) TestPipelinesFailover.testPipelineRecoveryStress tests fail intermittently with various symptoms

2014-08-27 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-6694: Priority: Critical (was: Blocker) Committed to trunk. I am downgrading the priority for now.

[jira] [Updated] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-08-27 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-6867: Attachment: HDFS-6867-20140827.patch For DFSOutputStream, do pipeline recovery for a single block

[jira] [Commented] (HDFS-6694) TestPipelinesFailover.testPipelineRecoveryStress tests fail intermittently with various symptoms

2014-08-27 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112469#comment-14112469 ] Yongjun Zhang commented on HDFS-6694: - Thanks [~arpitagarwal]. The recent failure I

[jira] [Updated] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread James Thomas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Thomas updated HDFS-6865: --- Attachment: HDFS-6865.7.patch Turns out TestBlockToken was a related failure. The others seem

[jira] [Updated] (HDFS-6908) incorrect snapshot directory diff generated by snapshot deletion

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6908: Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed Status: Resolved (was:

[jira] [Commented] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112511#comment-14112511 ] Hadoop QA commented on HDFS-6865: - {color:red}-1 overall{color}. Here are the results of

[jira] [Resolved] (HDFS-6911) Archival Storage: check if a block is already scheduled in Mover

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-6911. - Resolution: Fixed Hadoop Flags: Reviewed I've committed this. Thanks Nicholas! Archival Storage:

[jira] [Updated] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread James Thomas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Thomas updated HDFS-6865: --- Attachment: HDFS-6865.8.patch Fixed patch. Byte array native checksumming on client side (HDFS

[jira] [Commented] (HDFS-6944) Archival Storage: add retry and termination logic for Mover

2014-08-27 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112538#comment-14112538 ] Tsz Wo Nicholas Sze commented on HDFS-6944: --- Wrong file? There are some

[jira] [Commented] (HDFS-6892) Add XDR packaging method for each NFS request

2014-08-27 Thread Brandon Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112569#comment-14112569 ] Brandon Li commented on HDFS-6892: -- I've committed the patch. Thank you, [~wheat9]. Add

[jira] [Updated] (HDFS-6938) Cleanup javac warnings in FSNamesystem

2014-08-27 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6938: - Priority: Trivial (was: Major) Cleanup javac warnings in FSNamesystem

[jira] [Commented] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112573#comment-14112573 ] stack commented on HDFS-6865: - I tried this patch on a small cluster doing TestHLogPE. All

[jira] [Updated] (HDFS-6938) Cleanup javac warnings in FSNamesystem

2014-08-27 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6938: - Summary: Cleanup javac warnings in FSNamesystem (was: Cleanup javac warnings in FSNamesystem.java)

[jira] [Updated] (HDFS-6938) Cleanup javac warnings in FSNamesystem

2014-08-27 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6938: - Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed Status: Resolved

[jira] [Updated] (HDFS-6938) Cleanup javac warnings in FSNamesystem

2014-08-27 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6938: - Fix Version/s: (was: 2.6.0) 3.0.0 Cleanup javac warnings in FSNamesystem

[jira] [Commented] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread James Thomas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112595#comment-14112595 ] James Thomas commented on HDFS-6865: [~stack], thanks for taking a look. So you saw no

[jira] [Updated] (HDFS-6892) Add XDR packaging method for each NFS request

2014-08-27 Thread Brandon Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-6892: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Add

[jira] [Updated] (HDFS-6892) Add XDR packaging method for each NFS request

2014-08-27 Thread Brandon Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-6892: - Fix Version/s: 2.6.0 Add XDR packaging method for each NFS request

[jira] [Updated] (HDFS-6915) hadoop fs -text of zero-length file causes EOFException

2014-08-27 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated HDFS-6915: - Attachment: HDFS-6915.201408271824.txt hadoop fs -text of zero-length file causes EOFException

[jira] [Commented] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112599#comment-14112599 ] stack commented on HDFS-6865: - bq. So you saw no effect when native was not available, right?

[jira] [Updated] (HDFS-6915) hadoop fs -text of zero-length file causes EOFException

2014-08-27 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated HDFS-6915: - Target Version/s: 3.0.0, 2.6.0 Status: Patch Available (was: Open) hadoop fs -text of

[jira] [Updated] (HDFS-6944) Archival Storage: add retry and termination logic for Mover

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6944: Attachment: HDFS-6944.003.patch Yes, I forgot to update my code base for HDFS-6911. Update the patch to fix

[jira] [Commented] (HDFS-6800) Determine how Datanode layout changes should interact with rolling upgrade

2014-08-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112636#comment-14112636 ] Colin Patrick McCabe commented on HDFS-6800: Thanks for all the reviews, guys.

[jira] [Updated] (HDFS-6920) Archival Storage: check the storage type of delNodeHintStorage when deleting a replica

2014-08-27 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6920: -- Attachment: h6920_20140828.patch h6920_20140828.patch: adds some tests. Archival

[jira] [Updated] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-08-27 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-6867: Attachment: HDFS-6867-20140827-2.patch This patch pauses the DataStreamer thread while waiting

[jira] [Commented] (HDFS-2975) Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash + restart).

2014-08-27 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112648#comment-14112648 ] Uma Maheswara Rao G commented on HDFS-2975: --- Thanks a lot for working on this

[jira] [Updated] (HDFS-6376) Distcp data between two HA clusters requires another configuration

2014-08-27 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6376: - Attachment: HDFS-6376.011.patch Distcp data between two HA clusters requires another configuration

[jira] [Commented] (HDFS-6940) Initial refactoring to allow ConsensusNode implementation

2014-08-27 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112651#comment-14112651 ] Aaron T. Myers commented on HDFS-6940: -- Not entirely sure what was unclear here. I

[jira] [Commented] (HDFS-6915) hadoop fs -text of zero-length file causes EOFException

2014-08-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112676#comment-14112676 ] Hadoop QA commented on HDFS-6915: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-08-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112681#comment-14112681 ] Hadoop QA commented on HDFS-6951: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6867) For DFSOutputStream, do pipeline recovery for a single block in the background

2014-08-27 Thread Hadoop QA (JIRA)
of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12664672/HDFS-6867-20140827.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include

[jira] [Commented] (HDFS-6944) Archival Storage: add retry and termination logic for Mover

2014-08-27 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112709#comment-14112709 ] Tsz Wo Nicholas Sze commented on HDFS-6944: --- Thanks for the update. Some minor

[jira] [Updated] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-08-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6951: --- Attachment: HDFS-6951.001.patch Fix failing unit tests. Saving namespace and restarting NameNode

[jira] [Updated] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-08-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6951: --- Attachment: (was: HDFS-6951.001.patch) Saving namespace and restarting NameNode will remove

[jira] [Commented] (HDFS-6800) Determine how Datanode layout changes should interact with rolling upgrade

2014-08-27 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112704#comment-14112704 ] Arpit Agarwal commented on HDFS-6800: - First of all thanks for working on this

[jira] [Commented] (HDFS-6776) distcp from insecure cluster (source) to secure cluster (destination) doesn't work via webhdfs

2014-08-27 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112713#comment-14112713 ] Yongjun Zhang commented on HDFS-6776: - If all agree, I can still make the client side

[jira] [Updated] (HDFS-6946) TestBalancerWithSaslDataTransfer fails in trunk

2014-08-27 Thread Stephen Chu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Chu updated HDFS-6946: -- Attachment: HDFS-6946.1.patch I ran TestBalancerWithSaslDataTransfer multiple times with a build of

[jira] [Updated] (HDFS-6946) TestBalancerWithSaslDataTransfer fails in trunk

2014-08-27 Thread Stephen Chu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Chu updated HDFS-6946: -- Status: Patch Available (was: Open) TestBalancerWithSaslDataTransfer fails in trunk

[jira] [Created] (HDFS-6955) DN should reserve disk space for a full block when creating tmp files

2014-08-27 Thread Arpit Agarwal (JIRA)
Arpit Agarwal created HDFS-6955: --- Summary: DN should reserve disk space for a full block when creating tmp files Key: HDFS-6955 URL: https://issues.apache.org/jira/browse/HDFS-6955 Project: Hadoop HDFS

[jira] [Commented] (HDFS-6776) distcp from insecure cluster (source) to secure cluster (destination) doesn't work via webhdfs

2014-08-27 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112768#comment-14112768 ] Haohui Mai commented on HDFS-6776: -- bq. I've tried to quickly scan the discussion. I don't

[jira] [Commented] (HDFS-6800) Determine how Datanode layout changes should interact with rolling upgrade

2014-08-27 Thread James Thomas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112783#comment-14112783 ] James Thomas commented on HDFS-6800: [~arpitagarwal], thanks for the response. Good

[jira] [Updated] (HDFS-6773) MiniDFSCluster should skip edit log fsync by default

2014-08-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6773: --- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-6773) MiniDFSCluster should skip edit log fsync by default

2014-08-27 Thread Stephen Chu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112802#comment-14112802 ] Stephen Chu commented on HDFS-6773: --- Thank you, Colin! MiniDFSCluster should skip edit

[jira] [Commented] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112801#comment-14112801 ] Hadoop QA commented on HDFS-6865: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6705) Create an XAttr that disallows the HDFS admin from accessing a file

2014-08-27 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112804#comment-14112804 ] Andrew Wang commented on HDFS-6705: --- Hi Charles, thanks for working on this. Had some

[jira] [Updated] (HDFS-6944) Archival Storage: add retry and termination logic for Mover

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6944: Attachment: HDFS-6944.004.patch Thanks Nicholas for the review! Update the patch to address the comments.

[jira] [Updated] (HDFS-4486) Add log category for long-running DFSClient notices

2014-08-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-4486: --- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch

[jira] [Updated] (HDFS-6902) FileWriter should be closed in finally block in BlockReceiver#receiveBlock()

2014-08-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6902: --- Resolution: Fixed Fix Version/s: 2.6.0 Target Version/s: 2.6.0

[jira] [Created] (HDFS-6956) Allow dynamically changing the tracing level in Hadoop servers

2014-08-27 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-6956: -- Summary: Allow dynamically changing the tracing level in Hadoop servers Key: HDFS-6956 URL: https://issues.apache.org/jira/browse/HDFS-6956 Project:

[jira] [Created] (HDFS-6957) Allow DFSClient to manually specify that a request should be traced

2014-08-27 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-6957: -- Summary: Allow DFSClient to manually specify that a request should be traced Key: HDFS-6957 URL: https://issues.apache.org/jira/browse/HDFS-6957 Project:

[jira] [Commented] (HDFS-6920) Archival Storage: check the storage type of delNodeHintStorage when deleting a replica

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112844#comment-14112844 ] Jing Zhao commented on HDFS-6920: - +1. I will commit it shortly. Archival Storage: check

[jira] [Commented] (HDFS-6879) Adding tracing to Hadoop RPC

2014-08-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112853#comment-14112853 ] Colin Patrick McCabe commented on HDFS-6879: bq. I think host means not

[jira] [Resolved] (HDFS-6920) Archival Storage: check the storage type of delNodeHintStorage when deleting a replica

2014-08-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-6920. - Resolution: Fixed Hadoop Flags: Reviewed I've committed this. Thanks Nicholas! Archival Storage:

[jira] [Updated] (HDFS-6705) Create an XAttr that disallows the HDFS admin from accessing a file

2014-08-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6705: --- Status: Patch Available (was: In Progress) Create an XAttr that disallows the HDFS admin from

[jira] [Updated] (HDFS-6843) Create FSDataInputStream FSDataOutputStream isEncrypted() method

2014-08-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6843: --- Target Version/s: 3.0.0 (was: fs-encryption (HADOOP-10150 and HDFS-6134)) Affects Version/s:

[jira] [Updated] (HDFS-6843) Create FSDataInputStream FSDataOutputStream isEncrypted() method

2014-08-27 Thread Charles Lamb (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6843: --- Status: Patch Available (was: In Progress) Create FSDataInputStream FSDataOutputStream

[jira] [Commented] (HDFS-6865) Byte array native checksumming on client side (HDFS changes)

2014-08-27 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112857#comment-14112857 ] Todd Lipcon commented on HDFS-6865: --- For the logging, this sounds like a good thing to

[jira] [Updated] (HDFS-6879) Adding tracing to Hadoop RPC

2014-08-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6879: --- Resolution: Fixed Fix Version/s: 2.6.0 Target Version/s: 2.6.0

[jira] [Updated] (HDFS-6879) Adding tracing to Hadoop RPC

2014-08-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6879: --- Assignee: Masatake Iwasaki Adding tracing to Hadoop RPC

[jira] [Commented] (HDFS-6843) Create FSDataInputStream FSDataOutputStream isEncrypted() method

2014-08-27 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112872#comment-14112872 ] Andrew Wang commented on HDFS-6843: --- Overall LGTM, just one nit: let's not c+p the

[jira] [Commented] (HDFS-6800) Determine how Datanode layout changes should interact with rolling upgrade

2014-08-27 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112876#comment-14112876 ] Arpit Agarwal commented on HDFS-6800: - {{signalRollingUpgrade}} will also need to

[jira] [Comment Edited] (HDFS-6800) Determine how Datanode layout changes should interact with rolling upgrade

2014-08-27 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112876#comment-14112876 ] Arpit Agarwal edited comment on HDFS-6800 at 8/27/14 9:22 PM: --

[jira] [Commented] (HDFS-4257) The ReplaceDatanodeOnFailure policies could have a forgiving option

2014-08-27 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112878#comment-14112878 ] Colin Patrick McCabe commented on HDFS-4257: It's been a more than a week since

  1   2   >