[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544432#comment-13544432 ] Hudson commented on HBASE-5258: --- Integrated in HBase-0.94-security-on-Hadoop-23 #10 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/10/]) HBASE-5258 Move coprocessors set out of RegionLoad (Sergey) (Revision 1420482) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HServerLoad.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0, 0.94.4 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13537930#comment-13537930 ] Hudson commented on HBASE-5258: --- Integrated in HBase-0.94-security #87 (See [https://builds.apache.org/job/HBase-0.94-security/87/]) HBASE-5258 Move coprocessors set out of RegionLoad (Sergey) (Revision 1420482) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HServerLoad.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0, 0.94.4 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529880#comment-13529880 ] Hudson commented on HBASE-5258: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #293 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/293/]) HBASE-5258 Move coprocessors set out of RegionLoad - Addendum (Sergey) (Revision 1420521) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0, 0.94.4 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529219#comment-13529219 ] Sergey Shelukhin commented on HBASE-5258: - Tests pass for me (for 0.94 patch), at least small and medium so far. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529389#comment-13529389 ] Ted Yu commented on HBASE-5258: --- Reviewed 0.94 patch. Run test suite with patch and didn't find reproducible test failure. Will integrate later today. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529423#comment-13529423 ] Lars Hofhansl commented on HBASE-5258: -- So this patch does not break backwards compatibility? Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529466#comment-13529466 ] Ted Yu commented on HBASE-5258: --- 0.94 patch doesn't break backwards compatibility. Handling of coprocessors is kept. It is just always reporting 0 coprocessors in RegionLoad. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529470#comment-13529470 ] Lars Hofhansl commented on HBASE-5258: -- Thanks Ted. Just making sure. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529513#comment-13529513 ] Hudson commented on HBASE-5258: --- Integrated in HBase-0.94 #621 (See [https://builds.apache.org/job/HBase-0.94/621/]) HBASE-5258 Move coprocessors set out of RegionLoad (Sergey) (Revision 1420482) Result = FAILURE tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HServerLoad.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0, 0.94.4 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529575#comment-13529575 ] Sergey Shelukhin commented on HBASE-5258: - [~ted_yu] Ted, can you please also commit HBASE-5258-fix-on-top-of-v1.patch to trunk? 0.94 patch already includes these changes. Thanks. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0, 0.94.4 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529621#comment-13529621 ] Ted Yu commented on HBASE-5258: --- HBASE-5258-fix-on-top-of-v1.patch integrated to trunk. Thanks for the reminder, Sergey. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0, 0.94.4 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13529645#comment-13529645 ] Hudson commented on HBASE-5258: --- Integrated in HBase-TRUNK #3613 (See [https://builds.apache.org/job/HBase-TRUNK/3613/]) HBASE-5258 Move coprocessors set out of RegionLoad - Addendum (Sergey) (Revision 1420521) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0, 0.94.4 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13528452#comment-13528452 ] Sergey Shelukhin commented on HBASE-5258: - Wouldn't it break backward compat on 0.94? We can do a solution where ServerLoad always has 0 CPs reported, so all is good with wire compat. Would it be non-hacky enough? I can produce a patch. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Attachments: HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13528573#comment-13528573 ] Ted Yu commented on HBASE-5258: --- Running test suite for 0.94 patch. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Fix For: 0.96.0 Attachments: HBASE-5258-094.patch, HBASE-5258-fix-on-top-of-v1.patch, HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13527114#comment-13527114 ] Hudson commented on HBASE-5258: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #287 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/287/]) HBASE-5258 Move coprocessors set out of RegionLoad (Sergey Shelukhin) (Revision 1418561) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/hbase.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/RegionLoad.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ServerLoad.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Attachments: HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13526665#comment-13526665 ] Sergey Shelukhin commented on HBASE-5258: - TestAtomicOperation passes locally Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Attachments: HBASE-5258-v0.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[
https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13526793#comment-13526793
]
Ted Yu commented on HBASE-5258:
---
For hbase.proto:
{code}
- repeated Coprocessor coprocessors = 15;
+ /** Field ID 15 is reserved */
{code}
We should preserve ordinal in 0.94 But for trunk, I think we can bump up
ordinal for completeSequenceId field
For TestClassLoading.java:
{code}
private static final String[] regionServerSystemCoprocessors =
new String[]{
- regionCoprocessor1.getSimpleName(),
- regionServerCoprocessor.getSimpleName()
- };
-
- private static final String[] regionServerSystemAndUserCoprocessors =
- new String[] {
- regionCoprocessor1.getSimpleName(),
- regionCoprocessor2.getSimpleName(),
regionServerCoprocessor.getSimpleName()
{code}
Originally regionCoprocessor1 was classified as system coprocessor. What was
the reason for changing that ?
Thanks
Move coprocessors set out of RegionLoad
---
Key: HBASE-5258
URL: https://issues.apache.org/jira/browse/HBASE-5258
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Sergey Shelukhin
Priority: Critical
Attachments: HBASE-5258-v0.patch
When I worked on HBASE-5256, I revisited the code related to Ser/De of
coprocessors set in RegionLoad.
I think the rationale for embedding coprocessors set is for maximum
flexibility where each region can load different coprocessors.
This flexibility is causing extra cost in the region server to Master
communication and increasing the footprint of Master heap.
Would HServerLoad be a better place for this set ?
If required, region server should calculate disparity of loaded coprocessors
among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13526813#comment-13526813 ] Sergey Shelukhin commented on HBASE-5258: - bq. Originally regionCoprocessor1 was classified as system coprocessor. What was the reason for changing that regionCoprocessor1 is gone. In proto - yeah, discussed this with Enis. Given that there were no protobuf releases, it should be acceptable for now to just remove the field. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Attachments: HBASE-5258-v0.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13526816#comment-13526816 ] stack commented on HBASE-5258: -- I'm +1 on commit. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Attachments: HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13526896#comment-13526896 ] Ted Yu commented on HBASE-5258: --- Integrated patch v2 to trunk. Thanks for the patch, Sergey. Thanks for the review. @Lars: Is it Okay for this to go into 0.94 ? Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Attachments: HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13526970#comment-13526970 ] Hudson commented on HBASE-5258: --- Integrated in HBase-TRUNK #3599 (See [https://builds.apache.org/job/HBase-TRUNK/3599/]) HBASE-5258 Move coprocessors set out of RegionLoad (Sergey Shelukhin) (Revision 1418561) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/hbase-protocol/src/main/protobuf/hbase.proto * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/RegionLoad.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ServerLoad.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Attachments: HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13527040#comment-13527040 ] Lars Hofhansl commented on HBASE-5258: -- This seems a good addition to 0.94 as well. Thanks for pinging Ted. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Attachments: HBASE-5258-v0.patch, HBASE-5258-v1.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13526108#comment-13526108 ] Sergey Shelukhin commented on HBASE-5258: - Added patch Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Sergey Shelukhin Priority: Critical Attachments: HBASE-5258-v0.patch When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[
https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13526152#comment-13526152
]
Hadoop QA commented on HBASE-5258:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12556482/HBASE-5258-v0.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
101 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 23 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3455//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3455//console
This message is automatically generated.
Move coprocessors set out of RegionLoad
---
Key: HBASE-5258
URL: https://issues.apache.org/jira/browse/HBASE-5258
Project: HBase
Issue Type: Task
Reporter: Ted Yu
Assignee: Sergey Shelukhin
Priority: Critical
Attachments: HBASE-5258-v0.patch
When I worked on HBASE-5256, I revisited the code related to Ser/De of
coprocessors set in RegionLoad.
I think the rationale for embedding coprocessors set is for maximum
flexibility where each region can load different coprocessors.
This flexibility is causing extra cost in the region server to Master
communication and increasing the footprint of Master heap.
Would HServerLoad be a better place for this set ?
If required, region server should calculate disparity of loaded coprocessors
among regions and send report through HServerLoad
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510958#comment-13510958 ] Sergey Shelukhin commented on HBASE-5258: - Hi. Is there any objections to removing them for now? I don't see getting coprocessors from RegionLoad actually being used anywhere. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Priority: Critical When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13511077#comment-13511077 ] Andrew Purtell commented on HBASE-5258: --- bq. I don't see getting coprocessors from RegionLoad actually being used anywhere. +1 I searched for users of RegionLoad#getCoprocessors and didn't find any either. Originally we put this in so the loaded coprocessors on a region would show up next to other region information in the master UI. Unfortunately the result was not that useful, just clutter. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Priority: Critical When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493441#comment-13493441 ] Sergey Shelukhin commented on HBASE-5258: - There may be potential for using info about region server load that is more realtime/critical than metrics channel, in master, for balancing/etc. Probably not important short term. W.r.t. zk - why cannot master read zk nodes and apply rules too? ZK nodes may need to be made smarter for this purpose. Assuming RSs will have to talk to ZK anyway, for large clusters it will avoid extra all-to-one communication. Although master reading ZK to make decisions may introduce additional delay, especially if master just watches the nodes and builds up internal state, instead of querying entire state periodically. Regardless, does having different coprocessors on the regions of the same table make sense? If user has logic relying on coprocessors for some data querying or correctness it seems dangerous. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Priority: Critical When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13492669#comment-13492669 ] stack commented on HBASE-5258: -- [~sershe] Currently the RegionLoad in ServerLoad is badly overloaded carrying both load and attributes. Historically it was just easiest adding attributes to the message the RS was passing the Master on a heartbeat. Now we are pb'ing everywhere, we can entertain breaking load and attributes apart. Or, as below outlines, we'd not put load nor attributes on the heartbeat at all. Load is currently sent by the regionservers to the master on the old heartbeat channel that has been around since the early days. The only reason the master gets the load is so its UI can give a bit of a clue as to what is going on clusterwise. The load the regionservers put on the heartbeat is mostly a duplicate of what the regionserver emits over its metrics channel. It would be good if we did not duplicate this load composition. It would be good if instead the master listened to the regionserver metrics emissions rather than get load off the heartbeat (or, RS wrote their load/metrics to a table and master read that whenever it wanted to show a page on cluster state). That would make it so the heartbeat now only carries attributes. But maybe it should not carry this even. Most of the time the attributes do not change so it is almost as though they should be written once and then the master would consult them when it needs them. The master could, say, write them to zk into the body of the RS ephemeral znode. Master can read them when it joins the cluster or when they are updated by the RS. So, then the heartbeat would seem to have no purpose given zk client in the RS is already heartbeating so we could drop it. But maybe we shouldn't and rather, we should rethread it. FB are starting to go down the path where they are starting to rely on this RS heartbeat rather than the ephemeral node in zk and letting master make the call on whether a RS should be let go or not. They are tending this way because the zk ephemeral node is dumb -- no logic other than timeout -- and because if master is allowed say what is dead and what is alive, it has the opportunity of applying rules; e.g. if all regionservers in the cluster are unresponsive all at the same time, its probably a network issue so hold tight till the network comes back (as opposed to zk which would just expire all sessions and all of the cluster would go down). Hope this helps. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Ted Yu Priority: Critical When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? If required, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13192336#comment-13192336 ] Eugene Koontz commented on HBASE-5258: -- @Zhihong, I think it should be ok to raise coprocessor reporting out of HServerLoad.HRegionLoad to just HServerLoad. How do you see this as it relates to HBASE-4660 ? Does this fix make it (HBASE-4660) easier? Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13192338#comment-13192338 ] Zhihong Yu commented on HBASE-5258: --- I was looking for HBASE-4660 yesterday, thanks for finding it. Please work on this JIRA before tackling webuiport. Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191443#comment-13191443 ] Eugene Koontz commented on HBASE-5258: -- Hi Ted, Do you have an estimate for how much network traffic or heap footprint that this would save? Just curious, not an objection. -Eugene Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191461#comment-13191461 ] Zhihong Yu commented on HBASE-5258: --- Since each coprocessor is represented by a string, the potential savings can be considerable, especially if many regions are hosted on each region server. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191589#comment-13191589 ] Andrew Purtell commented on HBASE-5258: --- bq. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. No doubt it is redundant to have each region report a coprocessor given how the framework currently works: All regions for a table will have an identical set of coprocessors loaded, or there is something bad happening. bq. Would HServerLoad be a better place for this set ? I have no major objection. However, maybe we want a way to know if something bad happened on a region and a coprocessor on it went away? One could comb logs but that is hardly a convenient way to get online state. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191594#comment-13191594 ] Zhihong Yu commented on HBASE-5258: --- To my knowledge, for a mis-behaving coprocessor we either remove the buggy coprocessor or abort. I wonder what scenario would lead to imbalanced coprocessors on a region. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191639#comment-13191639 ] Andrew Purtell commented on HBASE-5258: --- bq. for a mis-behaving coprocessor we either remove the buggy coprocessor ... from the coprocessor host for the given region (in the case of RegionCoprocessorHost) only... Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191651#comment-13191651 ] Zhihong Yu commented on HBASE-5258: --- Since the combination of coprocessors on a region server is limited, I was suggesting that the report of uneven coprocessor presence be embedded in HServerLoad. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191646#comment-13191646 ] Zhihong Yu commented on HBASE-5258: --- I agree with the last comment @ 23/Jan/12 22:58 My understanding of the feature is that user should validate coprocessor by choosing the Abort policy for buggy coprocessor in pre-deployment stage. In production, the chance of buggy coprocessor dropping from individual region(s) should be low. The ability of querying imbalanced coprocessors on a region server should be on-demand feature. Move coprocessors set out of RegionLoad --- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5258) Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad
[ https://issues.apache.org/jira/browse/HBASE-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13191862#comment-13191862 ] Zhihong Yu commented on HBASE-5258: --- @Andrew, @Eugene: Do you agree with the proposal as outlined in the title of this JIRA ? Move coprocessors set out of RegionLoad, region server should calculate disparity of loaded coprocessors among regions and send report through HServerLoad -- Key: HBASE-5258 URL: https://issues.apache.org/jira/browse/HBASE-5258 Project: HBase Issue Type: Task Reporter: Zhihong Yu When I worked on HBASE-5256, I revisited the code related to Ser/De of coprocessors set in RegionLoad. I think the rationale for embedding coprocessors set is for maximum flexibility where each region can load different coprocessors. This flexibility is causing extra cost in the region server to Master communication and increasing the footprint of Master heap. Would HServerLoad be a better place for this set ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
