[jira] [Updated] (HDFS-8862) Improve BlockManager#excessReplicateMap

[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8862: - Attachment: HDFS-8862.001.patch Improve BlockManager#excessReplicateMap ---

[jira] [Updated] (HDFS-8862) Improve BlockManager#excessReplicateMap

[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8862: - Status: Patch Available (was: Open) Improve BlockManager#excessReplicateMap

[jira] [Commented] (HDFS-8857) Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure

[ https://issues.apache.org/jira/browse/HDFS-8857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659334#comment-14659334 ] Kai Zheng commented on HDFS-8857: - +1. Erasure Coding: Fix ArrayIndexOutOfBoundsException

[jira] [Updated] (HDFS-8774) Implement FileSystem and InputStream API for libhdfspp

[ https://issues.apache.org/jira/browse/HDFS-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8774: - Attachment: HDFS-8774.000.patch Implement FileSystem and InputStream API for libhdfspp

[jira] [Updated] (HDFS-8744) Erasure Coding: the number of chunks in packet is not updated when writing parity data

[ https://issues.apache.org/jira/browse/HDFS-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8744: - Attachment: (was: HDFS-8744.000.patch) Erasure Coding: the number of chunks in packet is not updated

[jira] [Updated] (HDFS-8744) Erasure Coding: the number of chunks in packet is not updated when writing parity data

[ https://issues.apache.org/jira/browse/HDFS-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8744: - Attachment: HDFS-8744.000.patch Erasure Coding: the number of chunks in packet is not updated when

[jira] [Commented] (HDFS-8499) Refactor BlockInfo class hierarchy with static helper class

[ https://issues.apache.org/jira/browse/HDFS-8499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659499#comment-14659499 ] Hadoop QA commented on HDFS-8499: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Comment Edited] (HDFS-8792) Improve BlockManager#postponedMisreplicatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659302#comment-14659302 ] Yi Liu edited comment on HDFS-8792 at 8/6/15 1:32 AM: -- Thanks

[jira] [Created] (HDFS-8862) Improve BlockManager#excessReplicateMap

Yi Liu created HDFS-8862: Summary: Improve BlockManager#excessReplicateMap Key: HDFS-8862 URL: https://issues.apache.org/jira/browse/HDFS-8862 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659521#comment-14659521 ] Vinayakumar B commented on HDFS-8815: - bq. It is good to avoid required for new

[jira] [Commented] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659533#comment-14659533 ] Hudson commented on HDFS-8815: -- FAILURE: Integrated in Hadoop-trunk-Commit #8270 (See

[jira] [Commented] (HDFS-8857) Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure

[ https://issues.apache.org/jira/browse/HDFS-8857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659363#comment-14659363 ] Li Bo commented on HDFS-8857: - Thanks Kai for the review. I have just committed the code to

[jira] [Comment Edited] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659526#comment-14659526 ] Vinayakumar B edited comment on HDFS-8815 at 8/6/15 5:44 AM: -

[jira] [Commented] (HDFS-8851) datanode fails to start due to a bad disk

[ https://issues.apache.org/jira/browse/HDFS-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659431#comment-14659431 ] Wang Hao commented on HDFS-8851: here is the total log STARTUP_MSG: java = 1.7.0_71

[jira] [Updated] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-8815: Affects Version/s: (was: 2.8.0) DFS getStoragePolicy implementation using single RPC call

[jira] [Updated] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-8815: Issue Type: Improvement (was: Bug) DFS getStoragePolicy implementation using single RPC call

[jira] [Commented] (HDFS-8861) Remove unnecessary log from method FSNamesystem.getCorruptFiles

[ https://issues.apache.org/jira/browse/HDFS-8861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659344#comment-14659344 ] Hadoop QA commented on HDFS-8861: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8854) Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

[ https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659374#comment-14659374 ] Kai Zheng commented on HDFS-8854: - [~zhz] [~walter.k.su], Yes it looks good to me to

[jira] [Commented] (HDFS-8792) Improve BlockManager#postponedMisreplicatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659321#comment-14659321 ] Yi Liu commented on HDFS-8792: -- I move {{BlockManager#excessReplicateMap}} to HDFS-8862.

[jira] [Work started] (HDFS-8859) Improve DataNode (ReplicaMap) memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8859 started by Yi Liu. Improve DataNode (ReplicaMap) memory footprint to save about 45%

[jira] [Commented] (HDFS-8792) Improve BlockManager#postponedMisreplicatedBlocks and BlockManager#excessReplicateMap

[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659302#comment-14659302 ] Yi Liu commented on HDFS-8792: -- Thanks [~cmccabe] for the review. {quote} This JavaDoc

[jira] [Commented] (HDFS-8827) Erasure Coding: When namenode processes over replicated striped block, NPE will be occur in ReplicationMonitor

[ https://issues.apache.org/jira/browse/HDFS-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659310#comment-14659310 ] Takuya Fukudome commented on HDFS-8827: --- Thanks for your comments, [~walter.k.su],

[jira] [Commented] (HDFS-8854) Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

[ https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659493#comment-14659493 ] Zhe Zhang commented on HDFS-8854: - Thanks Walter and Kai for the discussion. Let's move the

[jira] [Commented] (HDFS-8851) datanode fails to start due to a bad disk

[ https://issues.apache.org/jira/browse/HDFS-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659519#comment-14659519 ] Wang Hao commented on HDFS-8851: I think the bad data dir should not be added in

[jira] [Updated] (HDFS-8792) Improve BlockManager#postponedMisreplicatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8792: - Summary: Improve BlockManager#postponedMisreplicatedBlocks (was: Improve

[jira] [Updated] (HDFS-8792) Improve BlockManager#postponedMisreplicatedBlocks and BlockManager#excessReplicateMap

[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8792: - Attachment: HDFS-8792.003.patch Update the new patch. Improve BlockManager#postponedMisreplicatedBlocks and

[jira] [Commented] (HDFS-8854) Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

[ https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659325#comment-14659325 ] Walter Su commented on HDFS-8854: - bq. Should we split out the patch in common? Can't run

[jira] [Updated] (HDFS-8859) Improve DataNode (ReplicaMap) memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8859: - Priority: Critical (was: Major) Improve DataNode (ReplicaMap) memory footprint to save about 45%

[jira] [Commented] (HDFS-8857) Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure

[ https://issues.apache.org/jira/browse/HDFS-8857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659335#comment-14659335 ] Kai Zheng commented on HDFS-8857: - +1. Erasure Coding: Fix ArrayIndexOutOfBoundsException

[jira] [Commented] (HDFS-8792) Improve BlockManager#postponedMisreplicatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659512#comment-14659512 ] Hadoop QA commented on HDFS-8792: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8862) Improve BlockManager#excessReplicateMap

[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659513#comment-14659513 ] Hadoop QA commented on HDFS-8862: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-8815: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Commented] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654873#comment-14654873 ] Li Bo commented on HDFS-8704: - The failure of

[jira] [Commented] (HDFS-8772) fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails

[ https://issues.apache.org/jira/browse/HDFS-8772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654875#comment-14654875 ] Hadoop QA commented on HDFS-8772: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Assigned] (HDFS-8855) Webhdfs client leaks active NameNode connections

[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou reassigned HDFS-8855: --- Assignee: Xiaobing Zhou Webhdfs client leaks active NameNode connections

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654892#comment-14654892 ] Vinayakumar B commented on HDFS-8833: - bq. Remove isStriped from INodeFile header and

[jira] [Commented] (HDFS-6682) Add a metric to expose the timestamp of the oldest under-replicated block

[ https://issues.apache.org/jira/browse/HDFS-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654899#comment-14654899 ] Akira AJISAKA commented on HDFS-6682: - Thanks Allen, Andrew, and Yi for the discussion.

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654904#comment-14654904 ] Zhe Zhang commented on HDFS-8833: - Good idea Vinay. I think shrinking

[jira] [Commented] (HDFS-1148) Convert FSDataset to ReadWriteLock

[ https://issues.apache.org/jira/browse/HDFS-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654926#comment-14654926 ] Hadoop QA commented on HDFS-1148: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Created] (HDFS-8858) DU should be re-executed if the target directory exists

Akira AJISAKA created HDFS-8858: --- Summary: DU should be re-executed if the target directory exists Key: HDFS-8858 URL: https://issues.apache.org/jira/browse/HDFS-8858 Project: Hadoop HDFS

[jira] [Commented] (HDFS-8858) DU should be re-executed if the target directory exists

[ https://issues.apache.org/jira/browse/HDFS-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654980#comment-14654980 ] Masatake Iwasaki commented on HDFS-8858: This seems to be related to HDFS-8791.

[jira] [Commented] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654983#comment-14654983 ] Yi Liu commented on HDFS-8856: -- +1, thanks Arpit. I found a nit in

[jira] [Commented] (HDFS-8857) Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure

[ https://issues.apache.org/jira/browse/HDFS-8857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14655051#comment-14655051 ] Hadoop QA commented on HDFS-8857: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14655078#comment-14655078 ] Hadoop QA commented on HDFS-8856: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone

[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8853: - Attachment: HDFS-8853-HDFS-7285-01.patch Attached an initial patch . Please review. Erasure Coding:

[jira] [Updated] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone

[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated HDFS-8853: - Status: Patch Available (was: Open) Erasure Coding: Provide ECSchema validation when creating ECZone

[jira] [Updated] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-8815: - Attachment: HDFS-8815-004.patch DFS getStoragePolicy implementation using single

[jira] [Commented] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14655135#comment-14655135 ] Surendra Singh Lilhore commented on HDFS-8815: -- Thanks [~vinayrpet] for

[jira] [Commented] (HDFS-8802) dfs.checksum.type is not described in hdfs-default.xml

[ https://issues.apache.org/jira/browse/HDFS-8802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14655137#comment-14655137 ] Gururaj Shetty commented on HDFS-8802: -- Hi [~ozawa] Kindly review and let me know for

[jira] [Commented] (HDFS-8701) WebHDFS write commands fail when running multiple DataNodes on one machine

[ https://issues.apache.org/jira/browse/HDFS-8701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14655139#comment-14655139 ] Jagadesh Kiran N commented on HDFS-8701: hi [~erwaman] , i tried with 3 datanodes

[jira] [Commented] (HDFS-8844) TestHDFSCLI does not cleanup the test directory

[ https://issues.apache.org/jira/browse/HDFS-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14655172#comment-14655172 ] Hudson commented on HDFS-8844: -- ABORTED: Integrated in Hadoop-Yarn-trunk #1008 (See

[jira] [Commented] (HDFS-8844) TestHDFSCLI does not cleanup the test directory

[ https://issues.apache.org/jira/browse/HDFS-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14655175#comment-14655175 ] Hudson commented on HDFS-8844: -- ABORTED: Integrated in Hadoop-Yarn-trunk-Java8 #278 (See

[jira] [Updated] (HDFS-8854) Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

[ https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8854: Summary: Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs (was: Erasure coding: Move

[jira] [Created] (HDFS-8859) Improve DataNode (ReplicaMap) memory footprint to save about 45%

Yi Liu created HDFS-8859: Summary: Improve DataNode (ReplicaMap) memory footprint to save about 45% Key: HDFS-8859 URL: https://issues.apache.org/jira/browse/HDFS-8859 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-8854) Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

[ https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8854: Attachment: HDFS-8854.00.patch bq. Maybe we can keep ECSchema class in COMMON, and refactor HDFS codes to

[jira] [Commented] (HDFS-8844) TestHDFSCLI does not cleanup the test directory

[ https://issues.apache.org/jira/browse/HDFS-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658192#comment-14658192 ] Hudson commented on HDFS-8844: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #267 (See

[jira] [Commented] (HDFS-8844) TestHDFSCLI does not cleanup the test directory

[ https://issues.apache.org/jira/browse/HDFS-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658271#comment-14658271 ] Hudson commented on HDFS-8844: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2205 (See

[jira] [Commented] (HDFS-8855) Webhdfs client leaks active NameNode connections

[ https://issues.apache.org/jira/browse/HDFS-8855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658298#comment-14658298 ] Bob Hansen commented on HDFS-8855: -- This has the same description as HDFS-7597, although

[jira] [Commented] (HDFS-8844) TestHDFSCLI does not cleanup the test directory

[ https://issues.apache.org/jira/browse/HDFS-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658355#comment-14658355 ] Hudson commented on HDFS-8844: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #275 (See

[jira] [Commented] (HDFS-8844) TestHDFSCLI does not cleanup the test directory

[ https://issues.apache.org/jira/browse/HDFS-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658391#comment-14658391 ] Hudson commented on HDFS-8844: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2224 (See

[jira] [Commented] (HDFS-8792) Improve BlockManager#postponedMisreplicatedBlocks and BlockManager#excessReplicateMap

[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658477#comment-14658477 ] Colin Patrick McCabe commented on HDFS-8792: {code} 530 /** The

[jira] [Updated] (HDFS-8854) Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

[ https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8854: Affects Version/s: HDFS-7285 Target Version/s: HDFS-7285 Status: Patch Available (was:

[jira] [Updated] (HDFS-8854) Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

[ https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8854: Attachment: HDFS-8854-HDFS-7285.00.patch Submitting duplicate patch, renamed for Jenkins. Erasure coding:

[jira] [Commented] (HDFS-8750) FIleSystem does not honor Configuration.getClassLoader() while loading FileSystem implementations

[ https://issues.apache.org/jira/browse/HDFS-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658513#comment-14658513 ] Colin Patrick McCabe commented on HDFS-8750: Is this a compatible change? It

[jira] [Commented] (HDFS-8854) Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

[ https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658515#comment-14658515 ] Zhe Zhang commented on HDFS-8854: - Took a quite glance, overall structure looks pretty

[jira] [Commented] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658558#comment-14658558 ] Haohui Mai commented on HDFS-8856: -- Is it possible to return {{leasesById.size()}}

[jira] [Commented] (HDFS-8815) DFS getStoragePolicy implementation using single RPC call

[ https://issues.apache.org/jira/browse/HDFS-8815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658561#comment-14658561 ] Arpit Agarwal commented on HDFS-8815: - +1 the v004 patch LGTM. It is good to avoid

[jira] [Updated] (HDFS-7597) DNs should not open new NN connections when webhdfs clients seek

[ https://issues.apache.org/jira/browse/HDFS-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bob Hansen updated HDFS-7597: - Attachment: HDFS-7597.diff Refactored the diffs to move changes over to hadoop-client project DNs should

[jira] [Updated] (HDFS-7597) DNs should not open new NN connections when webhdfs clients seek

[ https://issues.apache.org/jira/browse/HDFS-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bob Hansen updated HDFS-7597: - Attachment: (was: HDFS-7597.diff) DNs should not open new NN connections when webhdfs clients seek

[jira] [Updated] (HDFS-7597) DNs should not open new NN connections when webhdfs clients seek

[ https://issues.apache.org/jira/browse/HDFS-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bob Hansen updated HDFS-7597: - Attachment: HDFS-7597.patch DNs should not open new NN connections when webhdfs clients seek

[jira] [Updated] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8856: Attachment: HDFS-8856.02.patch Thanks for the reviews. v2 patch attached. bq. Is it possible to

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658614#comment-14658614 ] Jing Zhao commented on HDFS-8833: - Thanks for the discussion guys. Some thoughts here: One

[jira] [Updated] (HDFS-8838) Tolerate datanode failures in DFSStripedOutputStream when the data length is small

[ https://issues.apache.org/jira/browse/HDFS-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8838: -- Attachment: HDFS-8838-HDFS-7285-000.patch Tolerate datanode failures in

[jira] [Updated] (HDFS-8838) Tolerate datanode failures in DFSStripedOutputStream when the data length is small

[ https://issues.apache.org/jira/browse/HDFS-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8838: -- Attachment: (was: h8838_20150804.patch) Tolerate datanode failures in

[jira] [Commented] (HDFS-6407) new namenode UI, lost ability to sort columns in datanode tab

[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658652#comment-14658652 ] Chang Li commented on HDFS-6407: [~wheat9] thanks for the patch! The code looks good. One

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658662#comment-14658662 ] Zhe Zhang commented on HDFS-8833: - Thanks Jing for sharing the thoughts! I agree that a

[jira] [Updated] (HDFS-8643) Add snapshot names list to SnapshottableDirectoryStatus

[ https://issues.apache.org/jira/browse/HDFS-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-8643: --- Attachment: HDFS-8643-01.patch Add snapshot names list to SnapshottableDirectoryStatus

[jira] [Commented] (HDFS-8643) Add snapshot names list to SnapshottableDirectoryStatus

[ https://issues.apache.org/jira/browse/HDFS-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658710#comment-14658710 ] Rakesh R commented on HDFS-8643: Thanks [~xyao] for the reviews and the pointer to

[jira] [Commented] (HDFS-8550) Erasure Coding: Fix FindBugs Multithreaded correctness Warning

[ https://issues.apache.org/jira/browse/HDFS-8550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658711#comment-14658711 ] Rakesh R commented on HDFS-8550: Thanks for sharing the link, I could see all the warnings

[jira] [Commented] (HDFS-8486) DN startup may cause severe data loss

[ https://issues.apache.org/jira/browse/HDFS-8486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658729#comment-14658729 ] Xiaoyu Yao commented on HDFS-8486: -- Thanks Arpit. The branch-2.6 patch LGTM, +1. DN

[jira] [Commented] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658770#comment-14658770 ] Xiaoyu Yao commented on HDFS-8856: -- Patch v2 looks good to me. +1 One nit: you can use a

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658785#comment-14658785 ] Jing Zhao commented on HDFS-8833: - Yes. The hybrid approach is actually the same way we

[jira] [Commented] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658799#comment-14658799 ] Haohui Mai commented on HDFS-8856: -- {code} +private CollectionLong getFiles() { +

[jira] [Commented] (HDFS-8643) Add snapshot names list to SnapshottableDirectoryStatus

[ https://issues.apache.org/jira/browse/HDFS-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658800#comment-14658800 ] Jing Zhao commented on HDFS-8643: - Instead of directly adding name list to

[jira] [Commented] (HDFS-8854) Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs

[ https://issues.apache.org/jira/browse/HDFS-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658811#comment-14658811 ] Hadoop QA commented on HDFS-8854: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Updated] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8856: Attachment: HDFS-8856.03.patch Make LeaseManager#countPath O(1)

[jira] [Commented] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658871#comment-14658871 ] Arpit Agarwal commented on HDFS-8856: - I don't see a good reason to return a modifiable

[jira] [Commented] (HDFS-8838) Tolerate datanode failures in DFSStripedOutputStream when the data length is small

[ https://issues.apache.org/jira/browse/HDFS-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658872#comment-14658872 ] Hadoop QA commented on HDFS-8838: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8696) Reduce the variances of latency of WebHDFS

[ https://issues.apache.org/jira/browse/HDFS-8696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658875#comment-14658875 ] Jun Yin commented on HDFS-8696: --- On unpatched try $ curl -s -L

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658902#comment-14658902 ] Andrew Wang commented on HDFS-8833: --- I think EC schema differs from storage policies

[jira] [Assigned] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reassigned HDFS-8808: --- Assignee: Zhe Zhang dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

[jira] [Commented] (HDFS-8808) dfs.image.transfer.bandwidthPerSec should not apply to -bootstrapStandby

[ https://issues.apache.org/jira/browse/HDFS-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658936#comment-14658936 ] Zhe Zhang commented on HDFS-8808: - Thanks [~ggop] for reporting the problem.

[jira] [Commented] (HDFS-8775) SASL support for data transfer protocol in libhdfspp

[ https://issues.apache.org/jira/browse/HDFS-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658938#comment-14658938 ] Haohui Mai commented on HDFS-8775: -- bq. I think it would make things more maintainable to

[jira] [Commented] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658940#comment-14658940 ] Haohui Mai commented on HDFS-8856: -- +1 pending jenkins. Make LeaseManager#countPath O(1)

[jira] [Commented] (HDFS-7597) DNs should not open new NN connections when webhdfs clients seek

[ https://issues.apache.org/jira/browse/HDFS-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658962#comment-14658962 ] Hadoop QA commented on HDFS-7597: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8856) Make LeaseManager#countPath O(1)

[ https://issues.apache.org/jira/browse/HDFS-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658979#comment-14658979 ] Hadoop QA commented on HDFS-8856: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658978#comment-14658978 ] Jing Zhao commented on HDFS-8833: - bq. QFS I believe hardcodes 6,3 with a 64KB cell size.

[jira] [Commented] (HDFS-8486) DN startup may cause severe data loss

[ https://issues.apache.org/jira/browse/HDFS-8486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659030#comment-14659030 ] Arpit Agarwal commented on HDFS-8486: - Thanks [~xyao], will hold off committing for a

[jira] [Commented] (HDFS-8499) Refactor BlockInfo class hierarchy with static helper class

[ https://issues.apache.org/jira/browse/HDFS-8499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659049#comment-14659049 ] Zhe Zhang commented on HDFS-8499: - Thanks! Refactor BlockInfo class hierarchy with static

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659059#comment-14659059 ] Andrew Wang commented on HDFS-8833: --- Why does cell size blow up the configuration space?

  1   2   >