[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942004#comment-14942004 ] Yi Liu commented on HDFS-9053: -- Thanks [~szetszwo], good comment, I ever considered it carefully too. I want

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14942004#comment-14942004 ] Yi Liu edited comment on HDFS-9053 at 10/3/15 12:47 AM: Thanks [~szetszwo], good

[jira] [Commented] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers

[ https://issues.apache.org/jira/browse/HDFS-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14940679#comment-14940679 ] Yi Liu commented on HDFS-9180: -- Thanks [~jingzhao] for the work. I have spent some time to familiar with what

[jira] [Updated] (HDFS-8988) Use LightWeightHashSet instead of LightWeightLinkedSet in BlockManager#excessReplicateMap

[ https://issues.apache.org/jira/browse/HDFS-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8988: - Description: {code} public final Map excessReplicateMap = new HashMap<>(); {code}

[jira] [Updated] (HDFS-8988) Use LightWeightHashSet instead of LightWeightLinkedSet in BlockManager#excessReplicateMap

[ https://issues.apache.org/jira/browse/HDFS-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8988: - Description: {code} public final Map excessReplicateMap = new HashMap<>(); {code}

[jira] [Updated] (HDFS-8988) Use LightWeightHashSet instead of LightWeightLinkedSet in BlockManager#excessReplicateMap

[ https://issues.apache.org/jira/browse/HDFS-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8988: - Issue Type: Improvement (was: Sub-task) Parent: (was: HDFS-8793) > Use LightWeightHashSet instead of

[jira] [Resolved] (HDFS-9183) Cleanup the findbugs and other issues after HDFS EC merged to trunk.

[ https://issues.apache.org/jira/browse/HDFS-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-9183. -- Resolution: Duplicate Assignee: (was: Yi Liu) > Cleanup the findbugs and other issues after HDFS EC

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936183#comment-14936183 ] Yi Liu edited comment on HDFS-9053 at 10/1/15 1:33 AM: --- Thanks [~szetszwo] for the

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934634#comment-14934634 ] Yi Liu edited comment on HDFS-9053 at 10/1/15 1:32 AM: --- Thanks for the comments,

[jira] [Updated] (HDFS-9182) Cleanup the findbugs and other issues after HDFS EC merged to trunk.

[ https://issues.apache.org/jira/browse/HDFS-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9182: - Assignee: Uma Maheswara Rao G (was: Yi Liu) > Cleanup the findbugs and other issues after HDFS EC merged to

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934598#comment-14934598 ] Yi Liu edited comment on HDFS-9053 at 10/1/15 1:30 AM: --- Thanks Jing for the further

[jira] [Created] (HDFS-9183) Cleanup the findbugs and other issues after HDFS EC merged to trunk.

Yi Liu created HDFS-9183: Summary: Cleanup the findbugs and other issues after HDFS EC merged to trunk. Key: HDFS-9183 URL: https://issues.apache.org/jira/browse/HDFS-9183 Project: Hadoop HDFS

[jira] [Created] (HDFS-9182) Cleanup the findbugs and other issues after HDFS EC merged to trunk.

Yi Liu created HDFS-9182: Summary: Cleanup the findbugs and other issues after HDFS EC merged to trunk. Key: HDFS-9182 URL: https://issues.apache.org/jira/browse/HDFS-9182 Project: Hadoop HDFS

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939144#comment-14939144 ] Yi Liu commented on HDFS-9053: -- {quote} the new class Node is. There is a "this" reference to the outer class.

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053.004.patch Update patch to change BTree#Node as *static* inner class, then can save a

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934689#comment-14934689 ] Yi Liu edited comment on HDFS-9053 at 9/29/15 6:18 AM: --- [~jingzhao], you have given a

[jira] [Commented] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934836#comment-14934836 ] Yi Liu commented on HDFS-8859: -- Thanks Uma, since the new patch only removes the unused import, based on the

[jira] [Updated] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8859: - Attachment: HDFS-8859.006.patch Update patch to remove unnecessary import. > Improve DataNode ReplicaMap memory

[jira] [Commented] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934697#comment-14934697 ] Yi Liu commented on HDFS-8859: -- Thanks Uma. There is an unused import, I will remove it in the new version

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934689#comment-14934689 ] Yi Liu edited comment on HDFS-9053 at 9/29/15 6:55 AM: --- [~jingzhao], you have given a

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934689#comment-14934689 ] Yi Liu edited comment on HDFS-9053 at 9/29/15 6:20 AM: --- [~jingzhao], you have given a

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934689#comment-14934689 ] Yi Liu commented on HDFS-9053: -- [~jingzhao], you have given a good review, and it hits the two places I ever

[jira] [Commented] (HDFS-9141) Thread leak in Datanode#refreshVolumes

[ https://issues.apache.org/jira/browse/HDFS-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934783#comment-14934783 ] Yi Liu commented on HDFS-9141: -- +1 pending Jenkins, Thanks Uma. > Thread leak in Datanode#refreshVolumes >

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14935212#comment-14935212 ] Yi Liu commented on HDFS-9053: -- The three test failures are not related. > Support large directories

[jira] [Created] (HDFS-9174) Fix the latest findbugs of FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread

Yi Liu created HDFS-9174: Summary: Fix the latest findbugs of FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread Key: HDFS-9174 URL: https://issues.apache.org/jira/browse/HDFS-9174

[jira] [Updated] (HDFS-9174) Fix the latest findbugs of FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread

[ https://issues.apache.org/jira/browse/HDFS-9174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9174: - Attachment: HDFS-9174.001.patch > Fix the latest findbugs of FSOutputSummer.tracer and >

[jira] [Updated] (HDFS-9174) Fix the latest findbugs of FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread

[ https://issues.apache.org/jira/browse/HDFS-9174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9174: - Status: Patch Available (was: Open) > Fix the latest findbugs of FSOutputSummer.tracer and >

[jira] [Created] (HDFS-9176) TestDirectoryScanner#testThrottling often fails.

Yi Liu created HDFS-9176: Summary: TestDirectoryScanner#testThrottling often fails. Key: HDFS-9176 URL: https://issues.apache.org/jira/browse/HDFS-9176 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053.003.patch > Support large directories efficiently using B-Tree >

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934862#comment-14934862 ] Yi Liu commented on HDFS-9053: -- Update the patch to address Jing's comments: 1. Call {{addOrReplace}} directly

[jira] [Updated] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8859: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934863#comment-14934863 ] Yi Liu commented on HDFS-8859: -- Committed to trunk and branch-2, thanks [~szetszwo], [~umamaheswararao],

[jira] [Commented] (HDFS-9141) Thread leak in Datanode#refreshVolumes

[ https://issues.apache.org/jira/browse/HDFS-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14935228#comment-14935228 ] Yi Liu commented on HDFS-9141: -- The test failures are unrelated, and checkstyle is about the file contains too

[jira] [Updated] (HDFS-9141) Thread leak in Datanode#refreshVolumes

[ https://issues.apache.org/jira/browse/HDFS-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9141: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-9174) Fix findbugs warnings in FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread

[ https://issues.apache.org/jira/browse/HDFS-9174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936154#comment-14936154 ] Yi Liu commented on HDFS-9174: -- Thanks [~wheat9] and [~templedf] for the review, and thanks Haohui for the

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Description: This is a long standing issue, we were trying to improve this in the past. Currently we use an

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Description: This is a long standing issue, we were trying to improve this in the past. Currently we use an

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936229#comment-14936229 ] Yi Liu commented on HDFS-9053: -- Have fixed following errors we found so far: 1. the time complexity of

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936183#comment-14936183 ] Yi Liu commented on HDFS-9053: -- Thanks [~szetszwo] for the comments. {quote} here are quite a few errors in

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934598#comment-14934598 ] Yi Liu edited comment on HDFS-9053 at 9/30/15 1:29 AM: --- Thanks Jing for the further

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936183#comment-14936183 ] Yi Liu edited comment on HDFS-9053 at 9/30/15 3:48 AM: --- Thanks [~szetszwo] for the

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936183#comment-14936183 ] Yi Liu edited comment on HDFS-9053 at 9/30/15 4:04 AM: --- Thanks [~szetszwo] for the

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936183#comment-14936183 ] Yi Liu edited comment on HDFS-9053 at 9/30/15 3:55 AM: --- Thanks [~szetszwo] for the

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934634#comment-14934634 ] Yi Liu commented on HDFS-9053: -- Thanks for the comments, Nicholas. {quote} Where are the numbers, especially

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934634#comment-14934634 ] Yi Liu edited comment on HDFS-9053 at 9/29/15 5:15 AM: --- Thanks for the comments,

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934598#comment-14934598 ] Yi Liu commented on HDFS-9053: -- Thanks Jing for the further review. {quote} Yeah, sorry for the delay.

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934598#comment-14934598 ] Yi Liu edited comment on HDFS-9053 at 9/29/15 4:48 AM: --- Thanks Jing for the further

[jira] [Commented] (HDFS-9147) Fix the setting of visibleLength in ExternalBlockReader

[ https://issues.apache.org/jira/browse/HDFS-9147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14909952#comment-14909952 ] Yi Liu commented on HDFS-9147: -- +1, the jenkins seems having issues, let me re-trigger. > Fix the setting of

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14910048#comment-14910048 ] Yi Liu commented on HDFS-9053: -- Hi [~jingzhao], could you help to do further reviews? Thanks a lot. > Support

[jira] [Commented] (HDFS-9106) Transfer failure during pipeline recovery causes permanent write failures

[ https://issues.apache.org/jira/browse/HDFS-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907789#comment-14907789 ] Yi Liu commented on HDFS-9106: -- Looks good, +1, thanks [~kihwal]. Let's wait for Jing to see whether he has

[jira] [Commented] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14908008#comment-14908008 ] Yi Liu commented on HDFS-8859: -- The one test failure is not related. > Improve DataNode ReplicaMap memory

[jira] [Commented] (HDFS-9132) Pass genstamp to ReplicaAccessorBuilder

[ https://issues.apache.org/jira/browse/HDFS-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906011#comment-14906011 ] Yi Liu commented on HDFS-9132: -- Thanks Colin for the work. One question, this new added {{ReplicaAccessor}}

[jira] [Updated] (HDFS-8739) Move DFSClient to client implementation package

[ https://issues.apache.org/jira/browse/HDFS-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8739: - Parent Issue: HDFS-6200 (was: HDFS-8048) > Move DFSClient to client implementation package >

[jira] [Updated] (HDFS-8739) Move DFSClient to hadoop-hdfs-client

[ https://issues.apache.org/jira/browse/HDFS-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8739: - Summary: Move DFSClient to hadoop-hdfs-client (was: Move DFSClient to client implementation package) > Move

[jira] [Commented] (HDFS-8739) Move DFSClient to client implementation package

[ https://issues.apache.org/jira/browse/HDFS-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14905975#comment-14905975 ] Yi Liu commented on HDFS-8739: -- Yes, I think we can move it into {{hadoop-hdfs-client}} directly and put in

[jira] [Commented] (HDFS-9133) ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF

[ https://issues.apache.org/jira/browse/HDFS-9133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906000#comment-14906000 ] Yi Liu commented on HDFS-9133: -- Thanks Colin for the work. *1.* This comment is for {{ExternalBlockReader}},

[jira] [Commented] (HDFS-9137) DeadLock between DataNode#refreshVolumes and BPOfferService#registrationSucceeded

[ https://issues.apache.org/jira/browse/HDFS-9137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906020#comment-14906020 ] Yi Liu commented on HDFS-9137: -- Thanks Uma, agree with that it's potential risk that can cause deadlock.

[jira] [Comment Edited] (HDFS-9133) ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF

[ https://issues.apache.org/jira/browse/HDFS-9133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906000#comment-14906000 ] Yi Liu edited comment on HDFS-9133 at 9/24/15 1:19 PM: --- Thanks Colin for the work.

[jira] [Comment Edited] (HDFS-9133) ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF

[ https://issues.apache.org/jira/browse/HDFS-9133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906000#comment-14906000 ] Yi Liu edited comment on HDFS-9133 at 9/24/15 1:52 PM: --- Thanks Colin for the work.

[jira] [Comment Edited] (HDFS-9133) ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF

[ https://issues.apache.org/jira/browse/HDFS-9133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906000#comment-14906000 ] Yi Liu edited comment on HDFS-9133 at 9/24/15 1:45 PM: --- Thanks Colin for the work.

[jira] [Commented] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907431#comment-14907431 ] Yi Liu commented on HDFS-8859: -- Update the patch: 1. address Uma and Brahma 's comments. 2. cleanup the

[jira] [Updated] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8859: - Attachment: HDFS-8859.005.patch > Improve DataNode ReplicaMap memory footprint to save about 45% >

[jira] [Commented] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907426#comment-14907426 ] Yi Liu commented on HDFS-8859: -- Yes, Thanks Brahma for the comment, the default value of {{trackModification}}

[jira] [Commented] (HDFS-8859) Improve DataNode ReplicaMap memory footprint to save about 45%

[ https://issues.apache.org/jira/browse/HDFS-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907309#comment-14907309 ] Yi Liu commented on HDFS-8859: -- Thanks Uma for the review, let me update the patch to address your comment. >

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903725#comment-14903725 ] Yi Liu edited comment on HDFS-9053 at 9/23/15 12:40 AM: The Jenkins has some issue

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903725#comment-14903725 ] Yi Liu commented on HDFS-9053: -- The Jenkins has some issue and the test failures are not related. The reason

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902232#comment-14902232 ] Yi Liu commented on HDFS-9053: -- Update the patch to address the comments, also cleanup the

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053.002.patch > Support large directories efficiently using B-Tree >

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: (was: HDFS-9053.002.patch) > Support large directories efficiently using B-Tree >

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053.002.patch > Support large directories efficiently using B-Tree >

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053.002.patch > Support large directories efficiently using B-Tree >

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: (was: HDFS-9053.002.patch) > Support large directories efficiently using B-Tree >

[jira] [Commented] (HDFS-9107) Prevent NN's unrecoverable death spiral after full GC

[ https://issues.apache.org/jira/browse/HDFS-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900335#comment-14900335 ] Yi Liu commented on HDFS-9107: -- Thanks [~daryn], the issue seems critical. I have few comments: *1.* the

[jira] [Commented] (HDFS-9107) Prevent NN's unrecoverable death spiral after full GC

[ https://issues.apache.org/jira/browse/HDFS-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900352#comment-14900352 ] Yi Liu commented on HDFS-9107: -- Sorry I just see Steve's comments. {quote} cores on different sockets may

[jira] [Comment Edited] (HDFS-9107) Prevent NN's unrecoverable death spiral after full GC

[ https://issues.apache.org/jira/browse/HDFS-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900352#comment-14900352 ] Yi Liu edited comment on HDFS-9107 at 9/21/15 8:08 AM: --- Sorry I just see Steve's

[jira] [Commented] (HDFS-9106) Transfer failure during pipeline recovery causes permanent write failures

[ https://issues.apache.org/jira/browse/HDFS-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900647#comment-14900647 ] Yi Liu commented on HDFS-9106: -- Thanks [~kihwal] for working on this. I have few comments: *1.* {code} +

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901808#comment-14901808 ] Yi Liu commented on HDFS-9053: -- Thanks a lot for your review and spend lots of time on this, Jing! I will

[jira] [Commented] (HDFS-8739) Move DFSClient to client implementation package

[ https://issues.apache.org/jira/browse/HDFS-8739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877349#comment-14877349 ] Yi Liu commented on HDFS-8739: -- Should move it to hadoop-hdfs-client finally. > Move DFSClient to client

[jira] [Commented] (HDFS-9113) ErasureCodingWorker#processErasureCodingTasks should not fail to process remaining tasks due to one invalid ECTask

[ https://issues.apache.org/jira/browse/HDFS-9113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14877347#comment-14877347 ] Yi Liu commented on HDFS-9113: -- +1, thanks Uma. I think this is caused by HDFS-8899 where change to use a

[jira] [Assigned] (HDFS-7285) Erasure Coding Support inside HDFS

[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu reassigned HDFS-7285: Assignee: Yi Liu (was: Zhe Zhang) > Erasure Coding Support inside HDFS >

[jira] [Updated] (HDFS-7285) Erasure Coding Support inside HDFS

[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-7285: - Assignee: Zhe Zhang (was: Yi Liu) > Erasure Coding Support inside HDFS > -- > >

[jira] [Updated] (HDFS-8733) Keep server related definition in hdfs.proto on server side

[ https://issues.apache.org/jira/browse/HDFS-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8733: - Assignee: Mingliang Liu (was: Yi Liu) > Keep server related definition in hdfs.proto on server side >

[jira] [Reopened] (HDFS-8740) Move DistributedFileSystem to hadoop-hdfs-client

[ https://issues.apache.org/jira/browse/HDFS-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu reopened HDFS-8740: -- Yes, we should work on this. This JIRA should be the last step, it should be done after moving all client side

[jira] [Commented] (HDFS-9063) Correctly handle snapshot path for getContentSummary

[ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14860285#comment-14860285 ] Yi Liu commented on HDFS-9063: -- BTW, you can cleanup the checkstyle while committing if no other people have

[jira] [Commented] (HDFS-9063) Correctly handle snapshot path for getContentSummary

[ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791678#comment-14791678 ] Yi Liu commented on HDFS-9063: -- Thanks [~jingzhao] for working on this. Jenkins' report has issue,

[jira] [Issue Comment Deleted] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Comment: was deleted (was: \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime ||

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14802729#comment-14802729 ] Yi Liu commented on HDFS-9053: -- The failed one test is not related to this patch. > Support large directories

[jira] [Commented] (HDFS-9063) Correctly handle snapshot path for getContentSummary

[ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804930#comment-14804930 ] Yi Liu commented on HDFS-9063: -- +1, thanks Jing. I check the patch, now if {{getContentSummary}} on snapshot

[jira] [Commented] (HDFS-9063) Correctly handle snapshot path for getContentSummary

[ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14804918#comment-14804918 ] Yi Liu commented on HDFS-9063: -- Thanks Jing for the explanation, I think I missed the fact that

[jira] [Commented] (HDFS-8873) throttle directoryScanner

[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14805021#comment-14805021 ] Yi Liu commented on HDFS-8873: -- {quote} We've seen this before and never managed to track it down. It seems to

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Status: Patch Available (was: Open) > Support large directories efficiently using B-Tree >

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14769053#comment-14769053 ] Yi Liu commented on HDFS-9053: -- Upload the patch and trigger Jenkins. The patch looks large, but it's not

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053.001.patch > Support large directories efficiently using B-Tree >

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791373#comment-14791373 ] Yi Liu commented on HDFS-9053: -- Seems there is issue for Jenkins, re-trigger... > Support large directories

[jira] [Comment Edited] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740842#comment-14740842 ] Yi Liu edited comment on HDFS-9053 at 9/15/15 12:35 AM: To integrate B-Tree with

[jira] [Commented] (HDFS-8899) Erasure Coding: use threadpool for EC recovery tasks

[ https://issues.apache.org/jira/browse/HDFS-8899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744594#comment-14744594 ] Yi Liu commented on HDFS-8899: -- Thanks [~rakeshr] for the work and [~zhz] to ping me. Overall looks good. My

[jira] [Commented] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740370#comment-14740370 ] Yi Liu commented on HDFS-9053: -- Current patch only includes my implementation of B-Tree, I will post the patch

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Priority: Critical (was: Major) > Support large directories efficiently using B-Tree >

[jira] [Updated] (HDFS-9053) Support large directories efficiently using B-Tree

[ https://issues.apache.org/jira/browse/HDFS-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-9053: - Attachment: HDFS-9053 (BTree).patch > Support large directories efficiently using B-Tree >

[jira] [Created] (HDFS-9053) Support large directories efficiently using B-Tree

Yi Liu created HDFS-9053: Summary: Support large directories efficiently using B-Tree Key: HDFS-9053 URL: https://issues.apache.org/jira/browse/HDFS-9053 Project: Hadoop HDFS Issue Type: Improvement

<    1   2   3   4   5   6   7   8   9   10   >