[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485225#comment-13485225 ] Hudson commented on HBASE-5360: --- Integrated in HBase-TRUNK #3489 (See [https://builds.apache.org/job/HBase-TRUNK/3489/]) HBASE-6223 Document hbck improvements: HBASE-6173, HBASE-5360 (Revision 1402650) Result = FAILURE jxiang : Files : * /hbase/trunk/src/docbkx/book.xml [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485299#comment-13485299 ] Hudson commented on HBASE-5360: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #240 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/240/]) HBASE-6223 Document hbck improvements: HBASE-6173, HBASE-5360 (Revision 1402650) Result = FAILURE jxiang : Files : * /hbase/trunk/src/docbkx/book.xml [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.94.1, 0.96.0 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13405074#comment-13405074 ] ramkrishna.s.vasudevan commented on HBASE-5360: --- @Jimmy I have a question here. Currently the LINGERING_PARENT is added back to META but still the region is not deployed. In the testcase added we try to do doFsck(conf, true) which does the assignment also. What happens is when we have a lingering parent and we say fixSplitParent we fix the META but do not assign it. So when we run hbck it says Region Not Deployed. Do we need to do assignment also here? Correct me if am wrong . [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13405115#comment-13405115 ] Jimmy Xiang commented on HBASE-5360: @Ram, good question. For lingering parent, the idea is to clear the split flag and let the rest to be taken care by hbck, which will handle assignment too. I considered to fix it in one shot. But it is some duplicate effort already in hbck. It is much clean and safe to use hbck to handle the assignment, region overlaps, and so on. [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13405180#comment-13405180 ] ramkrishna.s.vasudevan commented on HBASE-5360: --- @Jimmy Thanks for your reply. :) [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403274#comment-13403274 ] Hudson commented on HBASE-5360: --- Integrated in HBase-0.94-security #38 (See [https://builds.apache.org/job/HBase-0.94-security/38/]) HBASE-5360 [uberhbck] Add options for how to handle offline split parents, addendum (Revision 1353659) Result = FAILURE jxiang : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[
https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400381#comment-13400381
]
ramkrishna.s.vasudevan commented on HBASE-5360:
---
Small nit
{code}
p.add(HConstants.CATALOG_FAMILY, HConstants.SPLITA_QUALIFIER,
Writables.getBytes(a));
p.add(HConstants.CATALOG_FAMILY, HConstants.SPLITA_QUALIFIER,
Writables.getBytes(b));
{code}
In the testcase, the second put should be for SPLITB?
[uberhbck] Add options for how to handle offline split parents.
Key: HBASE-5360
URL: https://issues.apache.org/jira/browse/HBASE-5360
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch,
5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path
In a recent case, we attempted to repair a cluster that suffered from
HBASE-4238 that had about 6-7 generations of leftover split data. The hbck
repair options in an development version of HBASE-5128 treat HDFS as ground
truth but didn't check SPLIT and OFFLINE flags only found in meta. The net
effect was that it essentially attempted to merge many regions back into its
eldest geneneration's parent's range.
More safe guards to prevent mega-merges are being added on HBASE-5128.
This issue would automate the handling of the mega-merge avoiding cases
such as lingering grandparents. The strategy here would be to add more
checks against .META., and perform part of the catalog janitor's
responsibilities for lingering grandparents. This would potentially include
options to sideline regions, deleting grandparent regions, min size for
sidelining, and mechanisms for cleaning .META..
Note: There already exists an mechanism to reload these regions -- the bulk
loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents
(automatically splitting them if necessary) to HBase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400592#comment-13400592 ] Jimmy Xiang commented on HBASE-5360: Good catch. I will fix it in a separate jira. Thanks. [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401047#comment-13401047 ] Hudson commented on HBASE-5360: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #68 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/68/]) HBASE-5360 [uberhbck] Add options for how to handle offline split parents, addendum (Revision 1353658) Result = FAILURE jxiang : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13400010#comment-13400010 ] Hudson commented on HBASE-5360: --- Integrated in HBase-0.94-security #37 (See [https://builds.apache.org/job/HBase-0.94-security/37/]) HBASE-5360 [uberhbck] Add options for how to handle offline split parents. (Revision 1351182) Result = SUCCESS jxiang : Files : * /hbase/branches/0.94/CHANGES.txt * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13393473#comment-13393473 ] Jonathan Hsieh commented on HBASE-5360: --- I've +1'd on reviewboard with a few minor nits. Feel free to update and commit if you see fit. [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Attachments: hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13393640#comment-13393640 ] Hadoop QA commented on HBASE-5360: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532353/5360_hbase_v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2178//console This message is automatically generated. [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1 Attachments: 5360-0.90-hbase.patch, 5360-0.92-hbase.patch, 5360-0.94-hbase.patch, 5360_hbase_v4.patch, hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291186#comment-13291186 ] Jimmy Xiang commented on HBASE-5360: Sure. I posted the code review here: https://review.cloudera.org/r/2122/ [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Attachments: hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[
https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290300#comment-13290300
]
Zhihong Ted Yu commented on HBASE-5360:
---
{code}
- Path referencePath = getReferredToFile(p);
+ Path referencePath = StoreFileUtil.getReferredToFile(p);
{code}
Can you outline the rationale behind moving methods to StoreFileUtil ?
For sidelineSplitParent():
{code}
+if (needReassign) {
+ toBeReassigned.add(child);
+}
{code}
The above code is inside inner loop. Would a child be added to toBeReassigned
multiple times ?
{code}
+ // scenario (2)
+ errors.reportError(ERROR_CODE.FAILED_SPLIT_PARENT, Region
++ descriptiveName + , key= + key + , on HDFS, failed split parent);
+ if (shouldFixSplitParents()) {
+resetSplitParent(hbi);
{code}
Do we need to do something about the children in above scenario ?
I think we should provide two flags to user corresponding to the two scenarios
so that they can decide which scenario(s) to fix.
{code}
+ * Daughters do refer to parent.
+ */
+ @Test
+ public void testLingeringSplitParent2() throws Exception {
{code}
Please give the two test cases meaningful names that are consistent with
javadoc.
[uberhbck] Add options for how to handle offline split parents.
Key: HBASE-5360
URL: https://issues.apache.org/jira/browse/HBASE-5360
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Attachments: hbase-5360.path
In a recent case, we attempted to repair a cluster that suffered from
HBASE-4238 that had about 6-7 generations of leftover split data. The hbck
repair options in an development version of HBASE-5128 treat HDFS as ground
truth but didn't check SPLIT and OFFLINE flags only found in meta. The net
effect was that it essentially attempted to merge many regions back into its
eldest geneneration's parent's range.
More safe guards to prevent mega-merges are being added on HBASE-5128.
This issue would automate the handling of the mega-merge avoiding cases
such as lingering grandparents. The strategy here would be to add more
checks against .META., and perform part of the catalog janitor's
responsibilities for lingering grandparents. This would potentially include
options to sideline regions, deleting grandparent regions, min size for
sidelining, and mechanisms for cleaning .META..
Note: There already exists an mechanism to reload these regions -- the bulk
loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents
(automatically splitting them if necessary) to HBase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290422#comment-13290422 ] Jimmy Xiang commented on HBASE-5360: Those helper functions are in two places: StoreFile and LoadIncrementalHFiles. I need to add some more utils. I think it is better to put them in one class. You are right, a child could be added multiple times. I will move it outside. I will address the test cases too. Thanks. [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Attachments: hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290448#comment-13290448 ] Jimmy Xiang commented on HBASE-5360: For the children, they will be handled by other options like fixHdfsHoles and fixHdfsOverlaps. Not sure if we need flags for the two scenarios now. We can add if there many such requests. [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Attachments: hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13289848#comment-13289848 ] Hadoop QA commented on HBASE-5360: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531032/hbase-5360.path against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager org.apache.hadoop.hbase.client.TestFromClientSide Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2112//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2112//console This message is automatically generated. [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Attachments: hbase-5360.path In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253154#comment-13253154 ] Anoop Sam John commented on HBASE-5360: --- @Jon HBASE-5719 Handling this issue? [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[ https://issues.apache.org/jira/browse/HBASE-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253187#comment-13253187 ] Jonathan Hsieh commented on HBASE-5360: --- @Anoop. Partially. HBASE-5719 sidelines potential overlapping regions but doesn't take .META.'s info about if the region is offline into account. It could sideline a live region while it could have been more efficient to sideline an offline region. However, HBASE-5719 was sufficient for the case we were dealing with. Let's make this a placeholder for ways to improve it. Some ideas include: * Taking a region's offline/splitparent state into account if .META. entry is present * Making decisions about sidelining vs merging based on region size and max region size (instead of range) xml properties. * Improving the heuristic used to decide which region are sidelined. [uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.1, 0.94.0 Reporter: Jonathan Hsieh In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
