[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041850#comment-15041850 ] Zhe Zhang commented on HDFS-9314: - Thanks Xiao! +1 on the 2.7 and 2.6 patches. I just committed them. >

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041970#comment-15041970 ] Vinayakumar B commented on HDFS-9314: - branch-2.6 compilation is broken. Please update the code to

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041889#comment-15041889 ] Xiao Chen commented on HDFS-9314: - Thank you Zhe, really appreciate it! > Improve

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041991#comment-15041991 ] Zhe Zhang commented on HDFS-9314: - Thanks Vinay for catching this! I just committed an addendum fix. >

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038629#comment-15038629 ] Xiao Chen commented on HDFS-9314: - As [discussed in

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025869#comment-15025869 ] Hudson commented on HDFS-9314: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2576 (See

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025326#comment-15025326 ] Hudson commented on HDFS-9314: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #638 (See

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025245#comment-15025245 ] Hudson commented on HDFS-9314: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #715 (See

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025302#comment-15025302 ] Hudson commented on HDFS-9314: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1448 (See

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15024912#comment-15024912 ] Xiao Chen commented on HDFS-9314: - Thanks [~djp] and [~mingma]. The failed tests in patch 5 are not

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025081#comment-15025081 ] Hudson commented on HDFS-9314: -- FAILURE: Integrated in Hadoop-trunk-Commit #8876 (See

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025088#comment-15025088 ] Xiao Chen commented on HDFS-9314: - Thanks very much [~mingma] for the reviews and commit! Thanks Zhe,

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025098#comment-15025098 ] Hudson commented on HDFS-9314: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #725 (See

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15024868#comment-15024868 ] Ming Ma commented on HDFS-9314: --- +1 on patch 5 given [~djp]'s comments. [~xiaochen], can you please confirm

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025129#comment-15025129 ] Hudson commented on HDFS-9314: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2656 (See

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15024385#comment-15024385 ] Junping Du commented on HDFS-9314: -- Let's keep BlockPlacementPolicyWithNodeGroup#pickupReplicaSet there,

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15024715#comment-15024715 ] Junping Du commented on HDFS-9314: -- Filed HDFS-9456. > Improve BlockPlacementPolicyDefault's picking of

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021832#comment-15021832 ] Hadoop QA commented on HDFS-9314: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15023225#comment-15023225 ] Xiao Chen commented on HDFS-9314: - Thanks [~mingma], good point on {{verifyBlockPlacement}}! It's true

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15023081#comment-15023081 ] Ming Ma commented on HDFS-9314: --- The approach of the jira is to relax the policy regarding which replica can

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021568#comment-15021568 ] Xiao Chen commented on HDFS-9314: - Thanks Walter. >From patch 3, the implementation is no longer as a

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021569#comment-15021569 ] Xiao Chen commented on HDFS-9314: - Sorry typo above, attached is patch 6, not 7. > Improve

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021350#comment-15021350 ] Walter Su commented on HDFS-9314: - Suppose {{first}}, {{second}} are not empty. Before changes to default

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021326#comment-15021326 ] Hadoop QA commented on HDFS-9314: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021195#comment-15021195 ] Xiao Chen commented on HDFS-9314: - Thanks for the review Ming and Walter. Patch 5 addressed both points in

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020408#comment-15020408 ] Walter Su commented on HDFS-9314: - Except the 2 replicas scenario [~mingma] mentioned, the changes to

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15019008#comment-15019008 ] Hadoop QA commented on HDFS-9314: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020186#comment-15020186 ] Ming Ma commented on HDFS-9314: --- Thanks. * If there are two replicas node1(rack 1) and node2(rack 2),

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014849#comment-15014849 ] Ming Ma commented on HDFS-9314: --- [~xiaochen], AFAIK, we can assume there is no much compatibility requirement

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014801#comment-15014801 ] Xiao Chen commented on HDFS-9314: - Thanks [~mingma] for the review, I'll be working on the next rev. 1

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014734#comment-15014734 ] Ming Ma commented on HDFS-9314: --- Thanks [~xiaochen]. This should work. * The rack-to-hosts mapping

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012152#comment-15012152 ] Ming Ma commented on HDFS-9314: --- Thanks @Xiao Chen for working on this. I agree with [~zhz] and

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012809#comment-15012809 ] Hadoop QA commented on HDFS-9314: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007005#comment-15007005 ] Xiao Chen commented on HDFS-9314: - Thanks [~mingma] for creating this, and nice work on HDFS-9313! Looks

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007524#comment-15007524 ] Zhe Zhang commented on HDFS-9314: - The patch LGTM overall. Only ask is to add a short Javadoc for the new

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007439#comment-15007439 ] Zhe Zhang commented on HDFS-9314: - Thanks for the patch Xiao. Quick comment before I finish reviewing:

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007683#comment-15007683 ] Hadoop QA commented on HDFS-9314: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007597#comment-15007597 ] Zhe Zhang commented on HDFS-9314: - Thanks Xiao! +1 pending Jenkins. My concern about other BPPs is that

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007689#comment-15007689 ] Xiao Chen commented on HDFS-9314: - The test failures all seems unrelated. > Improve

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007778#comment-15007778 ] Hadoop QA commented on HDFS-9314: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas

[ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15007848#comment-15007848 ] Walter Su commented on HDFS-9314: - Shall we consider not to violate the placement policy when we decide to