[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469905#comment-13469905 ] Hudson commented on HBASE-5631: --- Integrated in HBase-0.94-security-on-Hadoop-23 #8 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/8/]) HBASE-5631 ADDENDUM (extra comments) (Revision 1382628) HBASE-5631 hbck should handle case where .tableinfo file is missing (Jie Huang) (Revision 1382530) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java jmhsieh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.92.3, 0.94.2, 0.96.0 Attachments: hbase-5631-addendum.patch, hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469981#comment-13469981 ] Hudson commented on HBASE-5631: --- Integrated in HBase-0.92-security #143 (See [https://builds.apache.org/job/HBase-0.92-security/143/]) HBASE-5631 ADDENDUM (extra comments) (Revision 1382626) HBASE-5631 hbck should handle case where .tableinfo file is missing (Jie Huang) (Revision 1382531) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.92.3, 0.94.2, 0.96.0 Attachments: hbase-5631-addendum.patch, hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13455032#comment-13455032 ] Hudson commented on HBASE-5631: --- Integrated in HBase-0.94-security #52 (See [https://builds.apache.org/job/HBase-0.94-security/52/]) HBASE-5631 ADDENDUM (extra comments) (Revision 1382628) HBASE-5631 hbck should handle case where .tableinfo file is missing (Jie Huang) (Revision 1382530) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java jmhsieh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: hbase-5631-addendum.patch, hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451913#comment-13451913 ] Hudson commented on HBASE-5631: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #168 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/168/]) HBASE-5631 ADDENDUM (extra comments) (Revision 1382627) Result = FAILURE jmhsieh : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: hbase-5631-addendum.patch, hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451586#comment-13451586 ] Jie Huang commented on HBASE-5631: -- Thanks. Nice to have that FamilyDirFilter() already. bq. Jie Huang with regards to the second cut – I was suggesting that if you want tackle it, file a new separate jira after this one is handled. No need for creep scope on this jira. Yes. Sure. I will take the new issue for the second cut as well. Thanks. hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Attachments: hbase-5631.patch, hbase-5631-v1.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451595#comment-13451595 ] Hadoop QA commented on HBASE-5631: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544400/hbase-5631-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause mvn compile goal to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2834//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2834//console This message is automatically generated. hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Attachments: hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451620#comment-13451620 ] Jonathan Hsieh commented on HBASE-5631: --- I noticed one thing as I was committing -- we don't actually test the non-cached tableinfo fabrication path. File a follow-on issue for that? hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451639#comment-13451639 ] Hudson commented on HBASE-5631: --- Integrated in HBase-0.94 #459 (See [https://builds.apache.org/job/HBase-0.94/459/]) HBASE-5631 hbck should handle case where .tableinfo file is missing (Jie Huang) (Revision 1382530) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451680#comment-13451680 ] Hudson commented on HBASE-5631: --- Integrated in HBase-0.92 #572 (See [https://builds.apache.org/job/HBase-0.92/572/]) HBASE-5631 hbck should handle case where .tableinfo file is missing (Jie Huang) (Revision 1382531) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451715#comment-13451715 ] Hudson commented on HBASE-5631: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #167 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/167/]) HBASE-5631 hbck should handle case where .tableinfo file is missing (Jie Huang) (Revision 1382529) Result = FAILURE jmhsieh : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/HbckTestingUtil.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[
https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451721#comment-13451721
]
Jie Huang commented on HBASE-5631:
--
bq. I noticed one thing as I was committing – we don't actually test the
non-cached tableinfo fabrication path. File a follow-on issue for that?
I have tested the non-cached case as well. see following lines
{code}
+
+ // fix OrphanTable with default .tableinfo non-cached case here
+ hbck = doFsck(conf, true);
+ assertNoErrors(hbck);
+ status = null;
+ status = FSTableDescriptors.getTableInfoPath(fs, hbaseTableDir);
+ assertNotNull(status);
+
+ HTableDescriptor htd = admin.getTableDescriptor(table.getBytes());
+ htd.setValue(NOT_DEFAULT, true);
+ admin.disableTable(table);
+ admin.modifyTable(table.getBytes(), htd);
+ admin.enableTable(table);
+ fs.delete(status.getPath(), true);
+
+ // fix OrphanTable with cache cached case here
+ htd = admin.getTableDescriptor(table.getBytes());
+ hbck = doFsck(conf, true);
+ assertNoErrors(hbck);
+ status = null;
+ status = FSTableDescriptors.getTableInfoPath(fs, hbaseTableDir);
+ assertNotNull(status);
+ htd = admin.getTableDescriptor(table.getBytes());
+ assertEquals(htd.getValue(NOT_DEFAULT), true);
{code}
hbck should handle case where .tableinfo file is missing.
-
Key: HBASE-5631
URL: https://issues.apache.org/jira/browse/HBASE-5631
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.92.2, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jie Huang
Fix For: 0.96.0, 0.92.3, 0.94.2
Attachments: hbase-5631.patch, hbase-5631-v1.patch,
hbase-5631-v2.patch
0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck
should be able to detect and repair this properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[
https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451741#comment-13451741
]
Jonathan Hsieh commented on HBASE-5631:
---
Got it. I'm going to add a comment about this line warming up the cache on the
master side (took a little digging to remember this).
{code}
+ htd = admin.getTableDescriptor(table.getBytes());
{code}
hbck should handle case where .tableinfo file is missing.
-
Key: HBASE-5631
URL: https://issues.apache.org/jira/browse/HBASE-5631
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.92.2, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jie Huang
Fix For: 0.96.0, 0.92.3, 0.94.2
Attachments: hbase-5631.patch, hbase-5631-v1.patch,
hbase-5631-v2.patch
0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck
should be able to detect and repair this properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451749#comment-13451749 ] Hudson commented on HBASE-5631: --- Integrated in HBase-TRUNK #3319 (See [https://builds.apache.org/job/HBase-TRUNK/3319/]) HBASE-5631 ADDENDUM (extra comments) (Revision 1382627) Result = FAILURE jmhsieh : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: hbase-5631-addendum.patch, hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451754#comment-13451754 ] Hudson commented on HBASE-5631: --- Integrated in HBase-0.94 #460 (See [https://builds.apache.org/job/HBase-0.94/460/]) HBASE-5631 ADDENDUM (extra comments) (Revision 1382628) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: hbase-5631-addendum.patch, hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451768#comment-13451768 ] Hudson commented on HBASE-5631: --- Integrated in HBase-0.92 #573 (See [https://builds.apache.org/job/HBase-0.92/573/]) HBASE-5631 ADDENDUM (extra comments) (Revision 1382626) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: hbase-5631-addendum.patch, hbase-5631.patch, hbase-5631-v1.patch, hbase-5631-v2.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13450814#comment-13450814 ] Jonathan Hsieh commented on HBASE-5631: --- [~grace.huang] with regards to the second cut -- I was suggesting that if you want tackle it, file a new separate jira after this one is handled. No need for creep scope on this jira. hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Attachments: hbase-5631.patch, hbase-5631-v1.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[
https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13450826#comment-13450826
]
Jonathan Hsieh commented on HBASE-5631:
---
For this:
{code}
+ private SetString getColumnFamilyList(SetString columns, HbckInfo hbi)
throws IOException {
+Path regionDir = hbi.getHdfsRegionDir();
+FileSystem fs = regionDir.getFileSystem(conf);
+FileStatus[] subDirs = fs.listStatus(regionDir);
+for (FileStatus subdir : subDirs) {
+ String columnfamily = subdir.getPath().getName();
+ if(subdir.isDir() !columnfamily.startsWith(.)){
+columns.add(columnfamily);
+ }
+}
+return columns;
+ }
+
{code}
Let's do this:
{code}
+ private SetString getColumnFamilyList(SetString columns, HbckInfo hbi)
throws IOException {
+Path regionDir = hbi.getHdfsRegionDir();
+FileSystem fs = regionDir.getFileSystem(conf);
// FileStatus[] subDirs = fs.listStatus(regionDir, new FamilyDirFilter())
+for (FileStatus subdir : subDirs) {
+ String columnfamily = subdir.getPath().getName();
// columns.add(columnfamily);
+}
+return columns;
+ }
+
{code}
It is relatively new but it actually uses the exact bounds for valid column
family names.
hbck should handle case where .tableinfo file is missing.
-
Key: HBASE-5631
URL: https://issues.apache.org/jira/browse/HBASE-5631
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.92.2, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jie Huang
Attachments: hbase-5631.patch, hbase-5631-v1.patch
0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck
should be able to detect and repair this properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13450831#comment-13450831 ] Jonathan Hsieh commented on HBASE-5631: --- Otherwise, patch looks great and the test is nice! hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Attachments: hbase-5631.patch, hbase-5631-v1.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449540#comment-13449540 ] Jonathan Hsieh commented on HBASE-5631: --- I haven't encountered this situation but for a first cut we should do the bare minimum to get a functional table again. I agree that at a bare minimum we can infer the table name and the colfam names from the dir structure. Once those are up, a user could alter the table to fix the other parameters. A second cut (follow on jira issue) could probably look at the newest hfile in each colfam and get some of the parameters (compression settings, encodings, etc). hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Attachments: hbase-5631.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449604#comment-13449604 ] Jie Huang commented on HBASE-5631: -- Oops. Sorry to miss your latest comment. For the second cut, I haven't done in this patch version. We may try to figure out all those properties we can get from the HFile. Thanks Jonathan. hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Attachments: hbase-5631.patch, hbase-5631-v1.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13448773#comment-13448773 ] Hadoop QA commented on HBASE-5631: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543845/hbase-5631.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. -1 javadoc. The javadoc tool appears to have generated 108 warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2788//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2788//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2788//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2788//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2788//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2788//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2788//console This message is automatically generated. hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Attachments: hbase-5631.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[
https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13448823#comment-13448823
]
Jonathan Hsieh commented on HBASE-5631:
---
Have you tried shutting down the cluster and then restarting it? I have a
suspicion that this may not work if the HTD isn't cached. Could you modify the
test (add a few lines) from HBASE-6516 to verify that this patch fixes the
table?
{code}
+ HTableDescriptor[] htds = getHTableDescriptors(tmpList); // this goes to
master which goes to the filesystem
{code}
Nits:
instead of this:
{code}
+ Path hbaseRoot = new Path(conf.get(HConstants.HBASE_DIR));
{code}
use this:
{code}
FSUtils.getRootDir(conf);
{code}
Are we purposely updating the passed in array? could we just use tmpList?
{code}
+ ListString tmpList = new ArrayListString();
+ tmpList.addAll(orphanTableDirs);
+ HTableDescriptor[] htds = getHTableDescriptors(tmpList);
+ Iterator iter = orphanTableDirs.iterator();
+ int j = 0;
+ while (iter.hasNext()) {
+String tableName = (String) iter.next();
+
{code}
I wasn't consistent with error.print vs log. I think I prefer log. Any reason
you picked this vs the other?
{code}
+errors.print(Try to fix orphan table: + tableName);
..
+errors.print(fixing table: + tableName);
..
+ errors.report(Failed to fix orphan table: + tableName);
{code}
typo/reword: hfsck - hbck, It is strongly recommended that you re-run hbck
manually since orphan table dirs have been fixed
{code}
+LOG.warn(Strongly recommend to re-run manually hfsck after all
orphanTableDirs being fixed);
{code}
hbck should handle case where .tableinfo file is missing.
-
Key: HBASE-5631
URL: https://issues.apache.org/jira/browse/HBASE-5631
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.92.2, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jie Huang
Attachments: hbase-5631.patch
0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck
should be able to detect and repair this properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449351#comment-13449351 ] Jie Huang commented on HBASE-5631: -- Thanks Jonathan. bq. Have you tried shutting down the cluster and then restarting it? I have a suspicion that this may not work if the HTD isn't cached. Yes. you are right. This proposal only fixes .tableinfo files which have been already cached. From my understanding, we have no way to figure out the entire table information without cache. Another thought is that we can try to recover parts of the information(like. table name, column-family name with some default properties). Meanwhile some warnings are printed out since part of the information(what we guessed) may not be as accurate as before. Is there any other suggestion? Please let me know. bq. I wasn't consistent with error.print vs log. I think I prefer log. Any reason you picked this vs the other? I have hesitated between log and error.print before. I will change that in the next version. thanks. hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jie Huang Attachments: hbase-5631.patch 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
[ https://issues.apache.org/jira/browse/HBASE-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401288#comment-13401288 ] Anoop Sam John commented on HBASE-5631: --- One point is with out the .tableinfo file in HDFS, HBCK can not fix HDFS integrity issues. For recreating the .tableinfo file in HDFS we need the HTD instance of the table. Well we can try getting this from the Master or RSs. In RS side the HRegion will have HTD instances. Also in Master, it might be already cached in FSTableDescriptors before the file actually got missed. We can try getting from any where possible (Hope from some where we will get HTD) and recreate the .tableinfo file. One point is this can work only in online mode. And if the .tableinfo file is missing the offline mode fixes wont work also. Pls validate my analysis hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
