[jira] [Created] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
Andrew Purtell created HBASE-8232:
-
Summary: TestAccessController occasionally fails with
IndexOutOfBoundsException
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
We are not waiting for the test table to be deployed before getting a reference
to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8226) HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions
[ https://issues.apache.org/jira/browse/HBASE-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618258#comment-13618258 ] Andrew Purtell commented on HBASE-8226: --- On trunk, found HBASE-8232, will circle back here after that. HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions Key: HBASE-8226 URL: https://issues.apache.org/jira/browse/HBASE-8226 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0, 0.96.0, 0.94.7 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: 8226-branch-0.94.patch, 8226.patch, 8226.patch HBaseTestingUtility#waitUntilAllRegionsAssigned does not check if the region it is counting belongs to the table created by the test and will not return if it accidentally counts too many regions, for example the regions of the ACL table when security is enabled. Made an attempt at fixing this as part of HBASE-8209 but broke TestMasterTransitions. Try again here. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Attachment: 8232.patch
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Status: Patch Available (was: Open)
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Attachment: 8232-branch-0.94.patch
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Attachment: (was: 8232-branch-0.94.patch)
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Attachment: (was: 8232.patch)
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Attachment: 8232-branch-0.94.patch
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Attachment: 8232.patch
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Affects Version/s: 0.94.7
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0, 0.94.7
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Trivial
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Priority: Trivial (was: Major)
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Trivial
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8218) pass HTable as a parameter to method of AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-8218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618266#comment-13618266 ] cuijianwei commented on HBASE-8218: --- As HTablePool is being deprecated, is there alternative HTable pool to replace the old one? pass HTable as a parameter to method of AggregationClient - Key: HBASE-8218 URL: https://issues.apache.org/jira/browse/HBASE-8218 Project: HBase Issue Type: Improvement Components: Client, Coprocessors Affects Versions: 0.94.3 Reporter: cuijianwei In AggregationClient, methods such as max(...), min(...) pass 'tableName' as a parameter, then a HTable will be created in the method, before the method return, the created HTable will be closed. The process above may be heavy because each call must create and close a HTable. The situation becomes worse when there is only one thread access HBase using AggregationClient. The underly HConnection of created HTable will also be created and then closed each time when we invoke these method because no other HTables using the HConnection. This operation is heavy. Therefore, can we add another group of methods which pass HTable or HTablePool as a parameter to methods defined in AggregationClient? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618272#comment-13618272
]
Hadoop QA commented on HBASE-8232:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576275/8232.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5071//console
This message is automatically generated.
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0, 0.94.7
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Trivial
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8152) Avoid creating empty reference file when splitkey is outside the key range of a store file
[ https://issues.apache.org/jira/browse/HBASE-8152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618276#comment-13618276 ] Hudson commented on HBASE-8152: --- Integrated in HBase-0.94 #930 (See [https://builds.apache.org/job/HBase-0.94/930/]) HBASE-8152 Avoid creating empty reference file when splitkey is outside the key range of a store file (clockfly) (Revision 1462880) Result = ABORTED tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java Avoid creating empty reference file when splitkey is outside the key range of a store file -- Key: HBASE-8152 URL: https://issues.apache.org/jira/browse/HBASE-8152 Project: HBase Issue Type: Improvement Components: Filesystem Integration, HFile Affects Versions: 0.94.5 Reporter: clockfly Assignee: clockfly Priority: Minor Fix For: 0.95.0, 0.98.0, 0.94.7 Attachments: 8152-0.96v2.patch, hbase-8152.0.94patch.v2, HBASE-8152-0.94v3.patch, HBASE-8152-0.96v1.patch, hbase-8152.patch0.94 When splitting a store file, if the split key is before the first key, or greater than the last key, then only one reference file should be created. Currently, two reference file will be created. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8220) can we record the count opened HTable for HTablePool
[ https://issues.apache.org/jira/browse/HBASE-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cuijianwei updated HBASE-8220: -- Attachment: HBASE-8220-0.94.3.txt patch for HBASE-8220.0.94.3 can we record the count opened HTable for HTablePool Key: HBASE-8220 URL: https://issues.apache.org/jira/browse/HBASE-8220 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.94.3 Reporter: cuijianwei Attachments: HBASE-8220-0.94.3.txt In HTablePool, we have a method getCurrentPoolSize(...) to get how many opened HTable has been pooled. However, we don't know ConcurrentOpenedHTable which means the count of HTable get from HTablePool.getTable(...) and don't return to HTablePool by PooledTable.close(). The ConcurrentOpenedHTable may be meaningful because it indicates how many HTables should be opened for the application which may help us set the appropriate MaxSize of HTablePool. Therefore, we can and a ConcurrentOpenedHTable as a counter in HTablePool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8220) can we record the count opened HTable for HTablePool
[ https://issues.apache.org/jira/browse/HBASE-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618278#comment-13618278 ] cuijianwei commented on HBASE-8220: --- ok, I make a path for this issue, could you please help to review? can we record the count opened HTable for HTablePool Key: HBASE-8220 URL: https://issues.apache.org/jira/browse/HBASE-8220 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.94.3 Reporter: cuijianwei Attachments: HBASE-8220-0.94.3.txt In HTablePool, we have a method getCurrentPoolSize(...) to get how many opened HTable has been pooled. However, we don't know ConcurrentOpenedHTable which means the count of HTable get from HTablePool.getTable(...) and don't return to HTablePool by PooledTable.close(). The ConcurrentOpenedHTable may be meaningful because it indicates how many HTables should be opened for the application which may help us set the appropriate MaxSize of HTablePool. Therefore, we can and a ConcurrentOpenedHTable as a counter in HTablePool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8220) can we record the count opened HTable for HTablePool
[ https://issues.apache.org/jira/browse/HBASE-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cuijianwei updated HBASE-8220: -- Attachment: HBASE-8220-0.94.3.txt can we record the count opened HTable for HTablePool Key: HBASE-8220 URL: https://issues.apache.org/jira/browse/HBASE-8220 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.94.3 Reporter: cuijianwei Attachments: HBASE-8220-0.94.3.txt, HBASE-8220-0.94.3.txt In HTablePool, we have a method getCurrentPoolSize(...) to get how many opened HTable has been pooled. However, we don't know ConcurrentOpenedHTable which means the count of HTable get from HTablePool.getTable(...) and don't return to HTablePool by PooledTable.close(). The ConcurrentOpenedHTable may be meaningful because it indicates how many HTables should be opened for the application which may help us set the appropriate MaxSize of HTablePool. Therefore, we can and a ConcurrentOpenedHTable as a counter in HTablePool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8164: -- Status: Open (was: Patch Available) TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8164: -- Attachment: HBASE-8164_addendum_3.patch Pls check this patch. I can think of rewriting the logic if this still fails. TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8164: -- Status: Patch Available (was: Open) TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8220) can we record the count opened HTable for HTablePool
[
https://issues.apache.org/jira/browse/HBASE-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618310#comment-13618310
]
Jean-Marc Spaggiari commented on HBASE-8220:
Sound good to me.
Few comments.
- Remove the extra space at the end of this.concurrentUsingTableCounts = new
ConcurrentHashMapString, AtomicInteger();
- Remove the extra space at the end of +if ((previousCount =
concurrentUsingTableCounts.putIfAbsent(tableName, initCount)) == null) {
- In returnTable, I'm not sure we should reduce the counter after
releaseHTableInterface and not before. The table you are removing will already
be removed from the tables field even if releaseHTableInterface fails. I
think we should keep the 2 in sync and therefore make sure decrementAndGet is
called before releaseHTableInterface. Might need to have a 2nd opinion on that.
- concurrentUsingTableCounts field (and subsequent methods) might be renamed
concurrentUsedTableCounts or something similar (concurrentTableCount, etc.)
- Make sure the patch is build over the 0.94 branch and not a specific 0.94.x
tag.
Have you run the test suite locally? Does it pass?
can we record the count opened HTable for HTablePool
Key: HBASE-8220
URL: https://issues.apache.org/jira/browse/HBASE-8220
Project: HBase
Issue Type: Improvement
Components: Client
Affects Versions: 0.94.3
Reporter: cuijianwei
Attachments: HBASE-8220-0.94.3.txt, HBASE-8220-0.94.3.txt
In HTablePool, we have a method getCurrentPoolSize(...) to get how many
opened HTable has been pooled. However, we don't know ConcurrentOpenedHTable
which means the count of HTable get from HTablePool.getTable(...) and don't
return to HTablePool by PooledTable.close(). The ConcurrentOpenedHTable may
be meaningful because it indicates how many HTables should be opened for the
application which may help us set the appropriate MaxSize of HTablePool.
Therefore, we can and a ConcurrentOpenedHTable as a counter in HTablePool.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7868) HFile performance regression between 0.92 and 0.94
[ https://issues.apache.org/jira/browse/HBASE-7868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618311#comment-13618311 ] Jean-Marc Spaggiari commented on HBASE-7868: I still have on my ToDo to build more granular tests to figure where the performances are impacted between 0.92 and 0.94, but I'm lacking some free time to complete that :( Based on Matteo's tests, it seems to be metrics related. Were the metrics there in 0.92? BTW, I'm fine to remove this JIRA from 0.94 until we have a plan for it. HFile performance regression between 0.92 and 0.94 -- Key: HBASE-7868 URL: https://issues.apache.org/jira/browse/HBASE-7868 Project: HBase Issue Type: Bug Components: io Affects Versions: 0.94.5 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Attachments: FilteredScan.png, hfileperf-graphs.png, HFilePerformanceEvaluation.txt, performances.pdf, performances.pdf By HFilePerformanceEvaluation seems that 0.94 is slower then 0.92 Looking at the profiler for the Scan path, seems that most of the time, compared to 92, is spent in the metrics dictionary lookup. [~eclark] pointed out the new per family/block metrics. By commenting the metrics call in HFileReaderV2, the performance seems to get better, but maybe metrics is not the only problem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6881) All regionservers are marked offline even there is still one up
[
https://issues.apache.org/jira/browse/HBASE-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618312#comment-13618312
]
Jean-Marc Spaggiari commented on HBASE-6881:
Sorry for beeing late on that:
{code}
RegionPlan newPlan = plan;
-if (!regionAlreadyInTransitionException) {
+if (!regionAlreadyInTransitionException !serverNotRunningYet) {
// Force a new plan and reassign. Will return null if no servers.
- newPlan = getRegionPlan(state, plan.getDestination(), true);
+ // The new plan could be the same as the existing plan since we don't
+ // exclude the server of the original plan, which should not be
+ // excluded since it could be the only server up now.
+ newPlan = getRegionPlan(state, true);
}
if (newPlan == null) {
{code}
If there is a possibility for newPlan to be null, than mean there is a
possibility for plan to be null, and therefore for plan.getDestination() to
fail with NPE? Or will plan never be null if regionAlreadyInTransitionException
and serverNotRunningYet are false?
All regionservers are marked offline even there is still one up
---
Key: HBASE-6881
URL: https://issues.apache.org/jira/browse/HBASE-6881
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.95.0
Attachments: trunk-6881.patch, trunk-6881_v3.patch
{noformat}
+RegionPlan newPlan = plan;
+if (!regionAlreadyInTransitionException) {
+ // Force a new plan and reassign. Will return null if no servers.
+ newPlan = getRegionPlan(state, plan.getDestination(), true);
+}
+if (newPlan == null) {
this.timeoutMonitor.setAllRegionServersOffline(true);
LOG.warn(Unable to find a viable location to assign region +
state.getRegion().getRegionNameAsString());
{noformat}
Here, when newPlan is null, plan.getDestination() could be up actually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8229) Replication code logs like crazy if a target table cannot be found.
[ https://issues.apache.org/jira/browse/HBASE-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618313#comment-13618313 ] Jean-Marc Spaggiari commented on HBASE-8229: [~jeason], I should have put a question mark at the end ;) Was more a question than an affirmation. I was thinking like if the node which is trying to create the table on the other side fails (power suply, etc.), is another one going to take is place? anyway, adding the delay will fix the crazy loggin issue. Replication code logs like crazy if a target table cannot be found. --- Key: HBASE-8229 URL: https://issues.apache.org/jira/browse/HBASE-8229 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.95.0, 0.98.0, 0.94.7 One of our RS/DN machines ran out of diskspace on the partition to which we write the log files. It turns out we still had a table in our source cluster with REPLICATION_SCOPE=1 that did not have a matching table in the remote cluster. In then logged a long stack trace every 50ms or so, over a few days that filled up our log partition. Since ReplicationSource cannot make any progress in this case anyway, it should probably sleep a bit before retrying (or at least limit the rate at which it spews out these exceptions to the log). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[
https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618320#comment-13618320
]
Hadoop QA commented on HBASE-8164:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576281/HBASE-8164_addendum_3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.master.TestTableLockManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5072//console
This message is automatically generated.
TestTableLockManager fails intermittently in trunk builds
-
Key: HBASE-8164
URL: https://issues.apache.org/jira/browse/HBASE-8164
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.95.0, 0.98.0
Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt,
8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch
In build #3979:
testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test
timed out after 60 milliseconds
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8220) can we record the count opened HTable for HTablePool
[ https://issues.apache.org/jira/browse/HBASE-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618344#comment-13618344 ] cuijianwei commented on HBASE-8220: --- Thanks for your review. Comments#3 looks a little complex. As you describe, if we put decrementAndGet after releaseHTableInterface, the HTable has been removed from HTable.tables while concurrentUsingTableCounts won't decrease if releaseHTableInterface throws an exception. On the other hand, if we put decrementAndGet before releaseHTableInterface, we may depend on what the users do when they getting an IOException from PoolTable.close(...). For example, if the users retries some times after this exception, concurrentUsingTableCounts will decrease many times. From this aspect, it may be a little better to put decrementAndGet after releaseHTableInterface. If HTable close fail, users should handle the exception, decide to retry until success(concurrentUsingTableCounts will decrease once eventually) or tolerate a close-fail HTable in their program(The HTable seems 'Using' by user from the view of HTablePool in this situation). On the other hand, removing the HTable from HTablePool.tables many times may not bring negative affect. can we record the count opened HTable for HTablePool Key: HBASE-8220 URL: https://issues.apache.org/jira/browse/HBASE-8220 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.94.3 Reporter: cuijianwei Attachments: HBASE-8220-0.94.3.txt, HBASE-8220-0.94.3.txt In HTablePool, we have a method getCurrentPoolSize(...) to get how many opened HTable has been pooled. However, we don't know ConcurrentOpenedHTable which means the count of HTable get from HTablePool.getTable(...) and don't return to HTablePool by PooledTable.close(). The ConcurrentOpenedHTable may be meaningful because it indicates how many HTables should be opened for the application which may help us set the appropriate MaxSize of HTablePool. Therefore, we can and a ConcurrentOpenedHTable as a counter in HTablePool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6881) All regionservers are marked offline even there is still one up
[
https://issues.apache.org/jira/browse/HBASE-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618357#comment-13618357
]
Jimmy Xiang commented on HBASE-6881:
That line is removed actually. Is the latest code still this way?
All regionservers are marked offline even there is still one up
---
Key: HBASE-6881
URL: https://issues.apache.org/jira/browse/HBASE-6881
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.95.0
Attachments: trunk-6881.patch, trunk-6881_v3.patch
{noformat}
+RegionPlan newPlan = plan;
+if (!regionAlreadyInTransitionException) {
+ // Force a new plan and reassign. Will return null if no servers.
+ newPlan = getRegionPlan(state, plan.getDestination(), true);
+}
+if (newPlan == null) {
this.timeoutMonitor.setAllRegionServersOffline(true);
LOG.warn(Unable to find a viable location to assign region +
state.getRegion().getRegionNameAsString());
{noformat}
Here, when newPlan is null, plan.getDestination() could be up actually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8218) pass HTable as a parameter to method of AggregationClient
[ https://issues.apache.org/jira/browse/HBASE-8218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618360#comment-13618360 ] cuijianwei commented on HBASE-8218: --- Will we remove HTablePool in 0.95 as discussed in [HBASE-6651|https://issues.apache.org/jira/browse/HBASE-6651]? pass HTable as a parameter to method of AggregationClient - Key: HBASE-8218 URL: https://issues.apache.org/jira/browse/HBASE-8218 Project: HBase Issue Type: Improvement Components: Client, Coprocessors Affects Versions: 0.94.3 Reporter: cuijianwei In AggregationClient, methods such as max(...), min(...) pass 'tableName' as a parameter, then a HTable will be created in the method, before the method return, the created HTable will be closed. The process above may be heavy because each call must create and close a HTable. The situation becomes worse when there is only one thread access HBase using AggregationClient. The underly HConnection of created HTable will also be created and then closed each time when we invoke these method because no other HTables using the HConnection. This operation is heavy. Therefore, can we add another group of methods which pass HTable or HTablePool as a parameter to methods defined in AggregationClient? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6881) All regionservers are marked offline even there is still one up
[
https://issues.apache.org/jira/browse/HBASE-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618363#comment-13618363
]
Jimmy Xiang commented on HBASE-6881:
That's inside the assign method. The plan over that should not be null.
Checked the latest code, it is enhanced.
All regionservers are marked offline even there is still one up
---
Key: HBASE-6881
URL: https://issues.apache.org/jira/browse/HBASE-6881
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.95.0
Attachments: trunk-6881.patch, trunk-6881_v3.patch
{noformat}
+RegionPlan newPlan = plan;
+if (!regionAlreadyInTransitionException) {
+ // Force a new plan and reassign. Will return null if no servers.
+ newPlan = getRegionPlan(state, plan.getDestination(), true);
+}
+if (newPlan == null) {
this.timeoutMonitor.setAllRegionServersOffline(true);
LOG.warn(Unable to find a viable location to assign region +
state.getRegion().getRegionNameAsString());
{noformat}
Here, when newPlan is null, plan.getDestination() could be up actually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8192) wrong logic in HRegion.bulkLoadHFiles(List)
[
https://issues.apache.org/jira/browse/HBASE-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618364#comment-13618364
]
Ted Yu commented on HBASE-8192:
---
TestTableLockManager failure is tracked in HBASE-8164
{code}
assertTrue(false);
{code}
Please add a message saying why we fail the test.
{code}
+} catch (Exception e) {
+ e.printStackTrace();
{code}
Stack printing is not needed above.
{code}
+ assertTrue(e instanceof IOException);
{code}
Add message for the case where assertion fails.
{code}
+HTable table = new HTable(util.getConfiguration(), TABLE);
{code}
Looks like the table is not closed.
wrong logic in HRegion.bulkLoadHFiles(List)
---
Key: HBASE-8192
URL: https://issues.apache.org/jira/browse/HBASE-8192
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.1, 0.94.2, 0.94.3, 0.94.4, 0.94.5
Reporter: Chenghao Jiang
Labels: patch
Fix For: 0.94.7
Attachments: 8192.txt, 8192-v2-with-a-test-case.txt,
8192-v3-with-a-test-case.txt, 8192-v4-with-a-test-case.txt,
8192-v5-with-a-test-case.txt, 8192-v6-with-a-test-case.txt
the wrong logic is here:
when a ColumnFamily does not exist, it gets a null store object, then
ioes.add(ioe); failures.add(p)
but the code below, if (failures.size() != 0), it prints a warn log and
return false, so it will never go into the code if (ioes.size() != 0) below,
and IOException will not be thrown, then the client will keep retry forever.
there is the same situation when doing store.assertBulkLoadHFileOk, if any
WrongRegionException is caught and failures.add(p), then all the other
IOException thrown by assertBulkLoadHFileOk will be ignored.
so i think if (failures.size() != 0) {} should be dealt with after if
(ioes.size() !=0) {}
{code}
for (Pairbyte[], String p : familyPaths) {
byte[] familyName = p.getFirst();
String path = p.getSecond();
Store store = getStore(familyName);
if (store == null) {
IOException ioe = new DoNotRetryIOException(
No such column family + Bytes.toStringBinary(familyName));
ioes.add(ioe);
failures.add(p);
} else {
try {
store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
// recoverable (file doesn't fit in region)
failures.add(p);
} catch (IOException ioe) {
// unrecoverable (hdfs problem)
ioes.add(ioe);
}
}
}
// validation failed, bail out before doing anything permanent.
if (failures.size() != 0) {
StringBuilder list = new StringBuilder();
for (Pairbyte[], String p : failures) {
list.append(\n).append(Bytes.toString(p.getFirst())).append( : )
.append(p.getSecond());
}
// problem when validating
LOG.warn(There was a recoverable bulk load failure likely due to a +
split. These (family, HFile) pairs were not loaded: + list);
return false;
}
// validation failed because of some sort of IO problem.
if (ioes.size() != 0) {
LOG.error(There were IO errors when checking if bulk load is ok. +
throwing exception!);
throw MultipleIOException.createIOException(ioes);
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8232:
--
Resolution: Fixed
Fix Version/s: 0.94.7
0.96.0
0.95.0
Status: Resolved (was: Patch Available)
Green HadoopQA result. Green 0.94 build with this change:
http://54.241.6.143/job/HBase-0.94-apurtell/2/console
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0, 0.94.7
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Trivial
Fix For: 0.95.0, 0.96.0, 0.94.7
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8192) wrong logic in HRegion.bulkLoadHFiles(List)
[
https://issues.apache.org/jira/browse/HBASE-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8192:
--
Attachment: 8192-v7-with-a-test-case.txt
Patch v7 addresses the above comments.
wrong logic in HRegion.bulkLoadHFiles(List)
---
Key: HBASE-8192
URL: https://issues.apache.org/jira/browse/HBASE-8192
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.1, 0.94.2, 0.94.3, 0.94.4, 0.94.5
Reporter: Chenghao Jiang
Labels: patch
Fix For: 0.94.7
Attachments: 8192.txt, 8192-v2-with-a-test-case.txt,
8192-v3-with-a-test-case.txt, 8192-v4-with-a-test-case.txt,
8192-v5-with-a-test-case.txt, 8192-v6-with-a-test-case.txt,
8192-v7-with-a-test-case.txt
the wrong logic is here:
when a ColumnFamily does not exist, it gets a null store object, then
ioes.add(ioe); failures.add(p)
but the code below, if (failures.size() != 0), it prints a warn log and
return false, so it will never go into the code if (ioes.size() != 0) below,
and IOException will not be thrown, then the client will keep retry forever.
there is the same situation when doing store.assertBulkLoadHFileOk, if any
WrongRegionException is caught and failures.add(p), then all the other
IOException thrown by assertBulkLoadHFileOk will be ignored.
so i think if (failures.size() != 0) {} should be dealt with after if
(ioes.size() !=0) {}
{code}
for (Pairbyte[], String p : familyPaths) {
byte[] familyName = p.getFirst();
String path = p.getSecond();
Store store = getStore(familyName);
if (store == null) {
IOException ioe = new DoNotRetryIOException(
No such column family + Bytes.toStringBinary(familyName));
ioes.add(ioe);
failures.add(p);
} else {
try {
store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
// recoverable (file doesn't fit in region)
failures.add(p);
} catch (IOException ioe) {
// unrecoverable (hdfs problem)
ioes.add(ioe);
}
}
}
// validation failed, bail out before doing anything permanent.
if (failures.size() != 0) {
StringBuilder list = new StringBuilder();
for (Pairbyte[], String p : failures) {
list.append(\n).append(Bytes.toString(p.getFirst())).append( : )
.append(p.getSecond());
}
// problem when validating
LOG.warn(There was a recoverable bulk load failure likely due to a +
split. These (family, HFile) pairs were not loaded: + list);
return false;
}
// validation failed because of some sort of IO problem.
if (ioes.size() != 0) {
LOG.error(There were IO errors when checking if bulk load is ok. +
throwing exception!);
throw MultipleIOException.createIOException(ioes);
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7925) Back port HBASE-6881 into 0.94
[ https://issues.apache.org/jira/browse/HBASE-7925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618371#comment-13618371 ] Jimmy Xiang commented on HBASE-7925: This issue comes up most likely in some unit test cases since real cluster rarely has just one region server up. I am fine with not doing this in 0.94 too. However, the patch looks good to me. Ram, what do you think? [~rajesh23], any special consideration you'd like it to be fixed in 0.94? Back port HBASE-6881 into 0.94 -- Key: HBASE-7925 URL: https://issues.apache.org/jira/browse/HBASE-7925 Project: HBase Issue Type: Bug Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.94.7 Attachments: HBASE-7925_94.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618372#comment-13618372 ] ramkrishna.s.vasudevan commented on HBASE-8164: --- Wow..this same test even fails. :). Will try out a diff approach. TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8192) wrong logic in HRegion.bulkLoadHFiles(List)
[
https://issues.apache.org/jira/browse/HBASE-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HBASE-8192:
-
Assignee: Chenghao Jiang
wrong logic in HRegion.bulkLoadHFiles(List)
---
Key: HBASE-8192
URL: https://issues.apache.org/jira/browse/HBASE-8192
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.1, 0.94.2, 0.94.3, 0.94.4, 0.94.5
Reporter: Chenghao Jiang
Assignee: Chenghao Jiang
Labels: patch
Fix For: 0.94.7
Attachments: 8192.txt, 8192-v2-with-a-test-case.txt,
8192-v3-with-a-test-case.txt, 8192-v4-with-a-test-case.txt,
8192-v5-with-a-test-case.txt, 8192-v6-with-a-test-case.txt,
8192-v7-with-a-test-case.txt
the wrong logic is here:
when a ColumnFamily does not exist, it gets a null store object, then
ioes.add(ioe); failures.add(p)
but the code below, if (failures.size() != 0), it prints a warn log and
return false, so it will never go into the code if (ioes.size() != 0) below,
and IOException will not be thrown, then the client will keep retry forever.
there is the same situation when doing store.assertBulkLoadHFileOk, if any
WrongRegionException is caught and failures.add(p), then all the other
IOException thrown by assertBulkLoadHFileOk will be ignored.
so i think if (failures.size() != 0) {} should be dealt with after if
(ioes.size() !=0) {}
{code}
for (Pairbyte[], String p : familyPaths) {
byte[] familyName = p.getFirst();
String path = p.getSecond();
Store store = getStore(familyName);
if (store == null) {
IOException ioe = new DoNotRetryIOException(
No such column family + Bytes.toStringBinary(familyName));
ioes.add(ioe);
failures.add(p);
} else {
try {
store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
// recoverable (file doesn't fit in region)
failures.add(p);
} catch (IOException ioe) {
// unrecoverable (hdfs problem)
ioes.add(ioe);
}
}
}
// validation failed, bail out before doing anything permanent.
if (failures.size() != 0) {
StringBuilder list = new StringBuilder();
for (Pairbyte[], String p : failures) {
list.append(\n).append(Bytes.toString(p.getFirst())).append( : )
.append(p.getSecond());
}
// problem when validating
LOG.warn(There was a recoverable bulk load failure likely due to a +
split. These (family, HFile) pairs were not loaded: + list);
return false;
}
// validation failed because of some sort of IO problem.
if (ioes.size() != 0) {
LOG.error(There were IO errors when checking if bulk load is ok. +
throwing exception!);
throw MultipleIOException.createIOException(ioes);
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618377#comment-13618377 ] ramkrishna.s.vasudevan commented on HBASE-8164: --- Am not able to see the logs for this test failure. TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618388#comment-13618388 ] Ted Yu commented on HBASE-8164: --- @Ram: Here it is: https://builds.apache.org/job/PreCommit-HBASE-Build/5072//artifact/trunk/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestTableLockManager-output.txt TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8202) MultiTableOutputFormat should support writing to another HBase cluster
[
https://issues.apache.org/jira/browse/HBASE-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618389#comment-13618389
]
David Koch commented on HBASE-8202:
---
Hello,
I asked the original question on the mailing list. Here is a minimalist example
to illustrate the behavior. Run with $quorum != $output_quorum for maximum
effect ;-).
HBase version was 0.92.1-cdh4.1.1.
{code:title=Example.java}
package org.hbase.example;
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
import org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil;
import org.apache.hadoop.hbase.mapreduce.TableMapper;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;
/**
* Test to show how hbase.mapred.output.quorum setting is ignored with {@link
MultiTableOutputFormat}.
*
* @author davidkoch
*
* See: https://issues.apache.org/jira/browse/HBASE-8202
*
* Hadoop/HBase configurations are read from command line. Replace environment
variables below.
*
* 1. Test with {@link TableOutputFormat} (Ok):
*
* hadoop jar $jar_name org.hbase.example.Example \
* -D hbase.zookeeper.quorum=$quorum \
* -D hbase.zookeeper.property.clientPort=2181 \
* -D hbase.mapreduce.inputtable=$input_table \
* -D hbase.mapreduce.scan.column.family=$colfam \
* -D hbase.mapred.outputtable=$output_table \
* -D
mapreduce.outputformat.class=org.apache.hadoop.hbase.mapreduce.TableOutputFormat
\
* -D hbase.mapred.output.quorum=$output_quorum:2181:/hbase
*
* 2. Test with {@link MultiTableOutputFormat} (Fails):
*
* hadoop jar $jar_name org.hbase.example.Example \
* -D hbase.zookeeper.quorum=$quorum \
* -D hbase.zookeeper.property.clientPort=2181 \
* -D hbase.mapreduce.inputtable=$input_table \
* -D hbase.mapreduce.scan.column.family=$colfam \
* -D hbase.mapred.outputtable=$output_table \
* -D
mapreduce.outputformat.class=org.apache.hadoop.hbase.mapreduce.MultiTableOutputFormat
\
* -D hbase.mapred.output.quorum=$output_quorum:2181:/hbase
*
* In the second example, the job itself will not fail if $output_table exists
on $quorum but $output_quorum will
* be ignored.
*/
public class Example extends Configured implements Tool {
public static class ExampleMapper extends
TableMapperImmutableBytesWritable, Put {
ImmutableBytesWritable tableName;
@Override
public void setup(Context context) {
tableName = new
ImmutableBytesWritable(context.getConfiguration().get(hbase.mapred.outputtable)
.getBytes());
}
public void map(ImmutableBytesWritable row, Result value, Context
context)
throws IOException, InterruptedException {
Put put = new Put(row.get());
for (KeyValue kv : value.raw()) {
put.add(kv);
}
context.write(tableName, put);
}
}
public int run(String[] args) throws Exception {
Configuration conf = getConf();
Scan scan = new Scan();
scan.addFamily(conf.get(hbase.mapreduce.scan.column.family).getBytes());
String inTable = conf.get(hbase.mapreduce.inputtable);
Job job = new Job(conf);
job.setJobName(Example-HBASE-8202);
TableMapReduceUtil.initTableMapperJob(inTable, scan,
ExampleMapper.class, null, null, job);
job.setJarByClass(Example.class);
job.setNumReduceTasks(0);
return job.waitForCompletion(true) ? 0 : 1;
}
public static void main(String[] args) throws Exception {
int res = ToolRunner.run(new Example(), args);
System.exit(res);
}
}
{code}
MultiTableOutputFormat should support writing to another HBase cluster
--
Key: HBASE-8202
URL: https://issues.apache.org/jira/browse/HBASE-8202
Project: HBase
Issue Type: Improvement
Components: mapreduce
Reporter: Ted Yu
This was brought up by David Koch in thread 'hbase.mapred.output.quorum
ignored in Mapper job with HDFS source and HBase sink' where he wanted to
import a file on HDFS from one cluster A (source) into HBase
tables on a different cluster B (destination) using a Mapper job with an
HBase sink.
Here is my analysis:
MultiTableOutputFormat doesn't extend TableOutputFormat:
{code}
public class MultiTableOutputFormat extends
OutputFormatImmutableBytesWritable, Mutation {
[jira] [Commented] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[
https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618390#comment-13618390
]
ramkrishna.s.vasudevan commented on HBASE-8164:
---
{code}
2013-03-31 12:51:05,877 DEBUG
[RS_CLOSE_REGION-asf000.sp2.ygridcore.net,38041,1364734244359-0]
zookeeper.ZKAssign(721): regionserver:38041-0x13dc07ff3b50002 Attempting to
transition node 7512a3d74f54292162072ab564904679 from M_ZK_REGION_CLOSING to
RS_ZK_REGION_CLOSED
2013-03-31 12:51:05,879 WARN [Split thread]
master.TestTableLockManager$6(366): Caught exception
java.lang.IllegalArgumentException: n must be positive
at java.util.Random.nextInt(Random.java:250)
at
org.apache.hadoop.hbase.HBaseTestingUtility.getSplittableRegion(HBaseTestingUtility.java:1946)
at
org.apache.hadoop.hbase.master.TestTableLockManager$6.chore(TestTableLockManager.java:357)
at org.apache.hadoop.hbase.Chore.run(Chore.java:81)
at java.lang.Thread.run(Thread.java:662)
{code}
Tomorrow will check more on this.
TestTableLockManager fails intermittently in trunk builds
-
Key: HBASE-8164
URL: https://issues.apache.org/jira/browse/HBASE-8164
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.95.0, 0.98.0
Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt,
8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch
In build #3979:
testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test
timed out after 60 milliseconds
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7925) Back port HBASE-6881 into 0.94
[ https://issues.apache.org/jira/browse/HBASE-7925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618391#comment-13618391 ] ramkrishna.s.vasudevan commented on HBASE-7925: --- Patch looks good to me. [~rajesh23] May be you can tell what ever Jimmy feels. Back port HBASE-6881 into 0.94 -- Key: HBASE-7925 URL: https://issues.apache.org/jira/browse/HBASE-7925 Project: HBase Issue Type: Bug Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.94.7 Attachments: HBASE-7925_94.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8192) wrong logic in HRegion.bulkLoadHFiles(List)
[
https://issues.apache.org/jira/browse/HBASE-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618394#comment-13618394
]
Hadoop QA commented on HBASE-8192:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576287/8192-v7-with-a-test-case.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testNonexistentColumnFamilyLoad(TestLoadIncrementalHFiles.java:200)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5073//console
This message is automatically generated.
wrong logic in HRegion.bulkLoadHFiles(List)
---
Key: HBASE-8192
URL: https://issues.apache.org/jira/browse/HBASE-8192
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.1, 0.94.2, 0.94.3, 0.94.4, 0.94.5
Reporter: Chenghao Jiang
Assignee: Chenghao Jiang
Labels: patch
Fix For: 0.94.7
Attachments: 8192.txt, 8192-v2-with-a-test-case.txt,
8192-v3-with-a-test-case.txt, 8192-v4-with-a-test-case.txt,
8192-v5-with-a-test-case.txt, 8192-v6-with-a-test-case.txt,
8192-v7-with-a-test-case.txt
the wrong logic is here:
when a ColumnFamily does not exist, it gets a null store object, then
ioes.add(ioe); failures.add(p)
but the code below, if (failures.size() != 0), it prints a warn log and
return false, so it will never go into the code if (ioes.size() != 0) below,
and IOException will not be thrown, then the client will keep retry forever.
there is the same situation when doing store.assertBulkLoadHFileOk, if any
WrongRegionException is caught and failures.add(p), then all the other
IOException thrown by assertBulkLoadHFileOk will be ignored.
so i think if (failures.size() != 0) {} should be dealt with after if
(ioes.size() !=0) {}
{code}
for (Pairbyte[], String p : familyPaths) {
byte[] familyName = p.getFirst();
String path = p.getSecond();
Store store = getStore(familyName);
if (store == null) {
IOException ioe = new DoNotRetryIOException(
No such column family + Bytes.toStringBinary(familyName));
ioes.add(ioe);
failures.add(p);
} else {
try {
store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
// recoverable (file doesn't fit
[jira] [Commented] (HBASE-7925) Back port HBASE-6881 into 0.94
[ https://issues.apache.org/jira/browse/HBASE-7925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618395#comment-13618395 ] rajeshbabu commented on HBASE-7925: --- [~jxiang] When I raised this issue ROOT region is hanged because of SSH and TM race because of this issue. Didnt remember exact scenario(I think just restarted the on RS available). Back port HBASE-6881 into 0.94 -- Key: HBASE-7925 URL: https://issues.apache.org/jira/browse/HBASE-7925 Project: HBase Issue Type: Bug Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.94.7 Attachments: HBASE-7925_94.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6881) All regionservers are marked offline even there is still one up
[
https://issues.apache.org/jira/browse/HBASE-6881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618421#comment-13618421
]
Jean-Marc Spaggiari commented on HBASE-6881:
I was looking at the patch only. Looking at the code directly in trunk, as you
said, it's not there anymore. It changed a lot already. So my comment is not
relevent anymore for the trunk code.
All regionservers are marked offline even there is still one up
---
Key: HBASE-6881
URL: https://issues.apache.org/jira/browse/HBASE-6881
Project: HBase
Issue Type: Bug
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Fix For: 0.95.0
Attachments: trunk-6881.patch, trunk-6881_v3.patch
{noformat}
+RegionPlan newPlan = plan;
+if (!regionAlreadyInTransitionException) {
+ // Force a new plan and reassign. Will return null if no servers.
+ newPlan = getRegionPlan(state, plan.getDestination(), true);
+}
+if (newPlan == null) {
this.timeoutMonitor.setAllRegionServersOffline(true);
LOG.warn(Unable to find a viable location to assign region +
state.getRegion().getRegionNameAsString());
{noformat}
Here, when newPlan is null, plan.getDestination() could be up actually.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8220) can we record the count opened HTable for HTablePool
[ https://issues.apache.org/jira/browse/HBASE-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618424#comment-13618424 ] Jean-Marc Spaggiari commented on HBASE-8220: I see your point. Both options are fine for me as long as we understand what's going on. can we record the count opened HTable for HTablePool Key: HBASE-8220 URL: https://issues.apache.org/jira/browse/HBASE-8220 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.94.3 Reporter: cuijianwei Attachments: HBASE-8220-0.94.3.txt, HBASE-8220-0.94.3.txt In HTablePool, we have a method getCurrentPoolSize(...) to get how many opened HTable has been pooled. However, we don't know ConcurrentOpenedHTable which means the count of HTable get from HTablePool.getTable(...) and don't return to HTablePool by PooledTable.close(). The ConcurrentOpenedHTable may be meaningful because it indicates how many HTables should be opened for the application which may help us set the appropriate MaxSize of HTablePool. Therefore, we can and a ConcurrentOpenedHTable as a counter in HTablePool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8192) wrong logic in HRegion.bulkLoadHFiles(List)
[
https://issues.apache.org/jira/browse/HBASE-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chenghao Jiang updated HBASE-8192:
--
Attachment: 8192-v8-with-a-test-case.txt
Do testNonexistentColumnFamilyLoad with useSecure=false, since doSecureBulkLoad
won't stop retry when bulkloading nonexistent family fails (it catchs the
exception)
wrong logic in HRegion.bulkLoadHFiles(List)
---
Key: HBASE-8192
URL: https://issues.apache.org/jira/browse/HBASE-8192
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.1, 0.94.2, 0.94.3, 0.94.4, 0.94.5
Reporter: Chenghao Jiang
Assignee: Chenghao Jiang
Labels: patch
Fix For: 0.94.7
Attachments: 8192.txt, 8192-v2-with-a-test-case.txt,
8192-v3-with-a-test-case.txt, 8192-v4-with-a-test-case.txt,
8192-v5-with-a-test-case.txt, 8192-v6-with-a-test-case.txt,
8192-v7-with-a-test-case.txt, 8192-v8-with-a-test-case.txt
the wrong logic is here:
when a ColumnFamily does not exist, it gets a null store object, then
ioes.add(ioe); failures.add(p)
but the code below, if (failures.size() != 0), it prints a warn log and
return false, so it will never go into the code if (ioes.size() != 0) below,
and IOException will not be thrown, then the client will keep retry forever.
there is the same situation when doing store.assertBulkLoadHFileOk, if any
WrongRegionException is caught and failures.add(p), then all the other
IOException thrown by assertBulkLoadHFileOk will be ignored.
so i think if (failures.size() != 0) {} should be dealt with after if
(ioes.size() !=0) {}
{code}
for (Pairbyte[], String p : familyPaths) {
byte[] familyName = p.getFirst();
String path = p.getSecond();
Store store = getStore(familyName);
if (store == null) {
IOException ioe = new DoNotRetryIOException(
No such column family + Bytes.toStringBinary(familyName));
ioes.add(ioe);
failures.add(p);
} else {
try {
store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
// recoverable (file doesn't fit in region)
failures.add(p);
} catch (IOException ioe) {
// unrecoverable (hdfs problem)
ioes.add(ioe);
}
}
}
// validation failed, bail out before doing anything permanent.
if (failures.size() != 0) {
StringBuilder list = new StringBuilder();
for (Pairbyte[], String p : failures) {
list.append(\n).append(Bytes.toString(p.getFirst())).append( : )
.append(p.getSecond());
}
// problem when validating
LOG.warn(There was a recoverable bulk load failure likely due to a +
split. These (family, HFile) pairs were not loaded: + list);
return false;
}
// validation failed because of some sort of IO problem.
if (ioes.size() != 0) {
LOG.error(There were IO errors when checking if bulk load is ok. +
throwing exception!);
throw MultipleIOException.createIOException(ioes);
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-6469:
--
Attachment: HBASE-6469_retry_enable_or_disable.patch
We can handle re attempts of enable/disable with with small changes while
preparing EnableTableHandler/DisableTableHandler as in patch. Just for an idea
uploading it.
Exceptions used in the patch may not. People may not accept this idea also.
One more idea is retry internally in EnableTableHandler until all regions
become online even in failures or timeouts then set ENABLED(Samething happening
in DisableTableHandler. It will retry until all regions become offline and then
set to table state to DISABLED).
Failure on enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restarted
---
Key: HBASE-6469
URL: https://issues.apache.org/jira/browse/HBASE-6469
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.0, 0.94.6
Reporter: Enis Soztutar
Assignee: rajeshbabu
Fix For: 0.95.0, 0.94.7
Attachments: 6469-expose-force-r3.patch, HBASE-6469_2.patch,
HBASE-6469_3.patch, HBASE-6469_4.patch, HBASE-6469.patch,
HBASE-6469_retry_enable_or_disable.patch
In Enable/DisableTableHandler code, if something goes wrong in handling, the
table state in zk is left as ENABLING / DISABLING. After that we cannot force
any more action from the API or CLI, and the only recovery path is restarting
the master.
{code}
if (done) {
// Flip the table to enabled.
this.assignmentManager.getZKTable().setEnabledTable(
this.tableNameStr);
LOG.info(Table ' + this.tableNameStr
+ ' was successfully enabled. Status: done= + done);
} else {
LOG.warn(Table ' + this.tableNameStr
+ ' wasn't successfully enabled. Status: done= + done);
}
{code}
Here, if done is false, the table state is not changed. There is also no way
to set skipTableStateCheck from cli / api.
We have run into this issue a couple of times before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8192) wrong logic in HRegion.bulkLoadHFiles(List)
[
https://issues.apache.org/jira/browse/HBASE-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618462#comment-13618462
]
Hadoop QA commented on HBASE-8192:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576297/8192-v8-with-a-test-case.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestTableLockManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5074//console
This message is automatically generated.
wrong logic in HRegion.bulkLoadHFiles(List)
---
Key: HBASE-8192
URL: https://issues.apache.org/jira/browse/HBASE-8192
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.1, 0.94.2, 0.94.3, 0.94.4, 0.94.5
Reporter: Chenghao Jiang
Assignee: Chenghao Jiang
Labels: patch
Fix For: 0.94.7
Attachments: 8192.txt, 8192-v2-with-a-test-case.txt,
8192-v3-with-a-test-case.txt, 8192-v4-with-a-test-case.txt,
8192-v5-with-a-test-case.txt, 8192-v6-with-a-test-case.txt,
8192-v7-with-a-test-case.txt, 8192-v8-with-a-test-case.txt
the wrong logic is here:
when a ColumnFamily does not exist, it gets a null store object, then
ioes.add(ioe); failures.add(p)
but the code below, if (failures.size() != 0), it prints a warn log and
return false, so it will never go into the code if (ioes.size() != 0) below,
and IOException will not be thrown, then the client will keep retry forever.
there is the same situation when doing store.assertBulkLoadHFileOk, if any
WrongRegionException is caught and failures.add(p), then all the other
IOException thrown by assertBulkLoadHFileOk will be ignored.
so i think if (failures.size() != 0) {} should be dealt with after if
(ioes.size() !=0) {}
{code}
for (Pairbyte[], String p : familyPaths) {
byte[] familyName = p.getFirst();
String path = p.getSecond();
Store store = getStore(familyName);
if (store == null) {
IOException ioe = new DoNotRetryIOException(
No such column family + Bytes.toStringBinary(familyName));
ioes.add(ioe);
failures.add(p);
} else {
try {
store.assertBulkLoadHFileOk(new Path(path));
} catch (WrongRegionException wre) {
// recoverable (file doesn't fit in region)
failures.add(p);
} catch (IOException ioe) {
// unrecoverable (hdfs
[jira] [Commented] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618465#comment-13618465
]
Hadoop QA commented on HBASE-6469:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576298/HBASE-6469_retry_enable_or_disable.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.util.TestMergeTable
org.apache.hadoop.hbase.util.TestHBaseFsck
org.apache.hadoop.hbase.procedure.TestZKProcedure
org.apache.hadoop.hbase.backup.TestHFileArchiving
org.apache.hadoop.hbase.master.handler.TestTableDeleteFamilyHandler
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient
org.apache.hadoop.hbase.client.TestSnapshotMetadata
org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
org.apache.hadoop.hbase.client.TestSnapshotFromClient
org.apache.hadoop.hbase.snapshot.TestExportSnapshot
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5075//console
This message is automatically generated.
Failure on enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restarted
---
Key: HBASE-6469
URL: https://issues.apache.org/jira/browse/HBASE-6469
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.0, 0.94.6
Reporter: Enis Soztutar
Assignee: rajeshbabu
Fix For: 0.95.0, 0.94.7
Attachments: 6469-expose-force-r3.patch, HBASE-6469_2.patch,
HBASE-6469_3.patch, HBASE-6469_4.patch, HBASE-6469.patch,
HBASE-6469_retry_enable_or_disable.patch
In Enable/DisableTableHandler code, if something goes wrong in handling, the
table state in zk is left as ENABLING / DISABLING. After that we cannot force
[jira] [Commented] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618471#comment-13618471
]
Hudson commented on HBASE-8232:
---
Integrated in HBase-0.94 #931 (See
[https://builds.apache.org/job/HBase-0.94/931/])
HBASE-8232. TestAccessController occasionally fails with
IndexOutOfBoundsException (Revision 1462998)
Result = SUCCESS
apurtell :
Files :
*
/hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0, 0.94.7
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Trivial
Fix For: 0.95.0, 0.96.0, 0.94.7
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618473#comment-13618473
]
Hudson commented on HBASE-8232:
---
Integrated in hbase-0.95 #116 (See
[https://builds.apache.org/job/hbase-0.95/116/])
HBASE-8232. TestAccessController occasionally fails with
IndexOutOfBoundsException (Revision 1462997)
Result = FAILURE
apurtell :
Files :
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0, 0.94.7
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Trivial
Fix For: 0.95.0, 0.96.0, 0.94.7
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[ https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-7836: - Attachment: hbase-7836_v1.patch New Replay Command Implementation. The patch is built on top of the patch of hbase-7835. The replay command belongs to Admin.proto not client so we don't expose it to clients. Introducing the command, we can better control replay command priority, add metrics and better monitor the traffic from this command. Create a new replay command so that recovered edits won't mess up normal coprocessing metrics - Key: HBASE-7836 URL: https://issues.apache.org/jira/browse/HBASE-7836 Project: HBase Issue Type: Sub-task Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.95.0 Attachments: hbase-7836_v1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[ https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-7836 started by Jeffrey Zhong. Create a new replay command so that recovered edits won't mess up normal coprocessing metrics - Key: HBASE-7836 URL: https://issues.apache.org/jira/browse/HBASE-7836 Project: HBase Issue Type: Sub-task Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.95.0 Attachments: hbase-7836_v1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7835) Implementation of the log splitting without creating intermediate files
[ https://issues.apache.org/jira/browse/HBASE-7835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-7835: - Attachment: hbase-7006_13.patch Rebase the patch. Implementation of the log splitting without creating intermediate files Key: HBASE-7835 URL: https://issues.apache.org/jira/browse/HBASE-7835 Project: HBase Issue Type: Sub-task Components: regionserver Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.95.0 Attachments: hbase-7006_12.patch, hbase-7006_13.patch, hbase-7006_1.patch, hbase-7006_2.patch, hbase-7006_3.patch, hbase-7006_3.patch, hbase-7006_6.patch, hbase-7006_8.patch The sub task is to check in a separate patch for major implementation for the proposal. More checkins will include a new replay command and more metrics support. Thanks, -Jeffrey -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7904) Make mapreduce jobs pass based on 2.0.4-alpha
[ https://issues.apache.org/jira/browse/HBASE-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618532#comment-13618532 ] Hudson commented on HBASE-7904: --- Integrated in hbase-0.95-on-hadoop2 #49 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/49/]) HBASE-7904 Make mapreduce jobs pass based on 2.0.4-alpha (Ted Yu) (Revision 1462873) Result = FAILURE tedyu : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java * /hbase/branches/0.95/pom.xml Make mapreduce jobs pass based on 2.0.4-alpha - Key: HBASE-7904 URL: https://issues.apache.org/jira/browse/HBASE-7904 Project: HBase Issue Type: Task Reporter: Ted Yu Assignee: Ted Yu Priority: Critical Fix For: 0.95.0, 0.98.0 Attachments: 7904-addendum.txt, 7904.txt, 7904-v2-hadoop-2.0.txt, 7904-v2.txt, 7904-v4-hadoop-2.0.txt, 7904-v4.txt, 7904-v4.txt, 7904-v5-hadoop-2.0.txt, 7904-v5.txt, 7904-v6-hadoop-2.0.txt, 7904-v7-hadoop-2.0.txt, 7904-v8-hadoop-2.0.txt, 7904-v8.txt, 7904-v9-hadoop-2.0.txt, 7904-v9.txt, hbase-7904-v3.txt 2.0.3-alpha has been released. We should upgrade the dependency. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8232) TestAccessController occasionally fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/HBASE-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618531#comment-13618531
]
Hudson commented on HBASE-8232:
---
Integrated in hbase-0.95-on-hadoop2 #49 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/49/])
HBASE-8232. TestAccessController occasionally fails with
IndexOutOfBoundsException (Revision 1462997)
Result = FAILURE
apurtell :
Files :
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
TestAccessController occasionally fails with IndexOutOfBoundsException
--
Key: HBASE-8232
URL: https://issues.apache.org/jira/browse/HBASE-8232
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.95.0, 0.96.0, 0.94.7
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Trivial
Fix For: 0.95.0, 0.96.0, 0.94.7
Attachments: 8232-branch-0.94.patch, 8232.patch
We are not waiting for the test table to be deployed before getting a
reference to one of its regions.
{noformat}
testShutdown(org.apache.hadoop.hbase.security.access.TestAccessController)
Time elapsed: 2.593 sec\
ERROR!
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.hadoop.hbase.security.access.TestAccessController.setUp(TestAccessController.\
java:172)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8226) HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions
[ https://issues.apache.org/jira/browse/HBASE-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618534#comment-13618534 ] Andrew Purtell commented on HBASE-8226: --- TestHBaseFsck hangs in teardown, otherwise trunk tests are ok: http://54.241.6.143/job/HBase-TRUNK-apurtell/10 . This is unrelated. HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions Key: HBASE-8226 URL: https://issues.apache.org/jira/browse/HBASE-8226 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0, 0.96.0, 0.94.7 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: 8226-branch-0.94.patch, 8226.patch, 8226.patch HBaseTestingUtility#waitUntilAllRegionsAssigned does not check if the region it is counting belongs to the table created by the test and will not return if it accidentally counts too many regions, for example the regions of the ACL table when security is enabled. Made an attempt at fixing this as part of HBASE-8209 but broke TestMasterTransitions. Try again here. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8226) HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions
[ https://issues.apache.org/jira/browse/HBASE-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-8226: -- Resolution: Fixed Fix Version/s: 0.94.7 0.96.0 0.95.0 Status: Resolved (was: Patch Available) HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions Key: HBASE-8226 URL: https://issues.apache.org/jira/browse/HBASE-8226 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0, 0.96.0, 0.94.7 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.95.0, 0.96.0, 0.94.7 Attachments: 8226-branch-0.94.patch, 8226.patch, 8226.patch HBaseTestingUtility#waitUntilAllRegionsAssigned does not check if the region it is counting belongs to the table created by the test and will not return if it accidentally counts too many regions, for example the regions of the ACL table when security is enabled. Made an attempt at fixing this as part of HBASE-8209 but broke TestMasterTransitions. Try again here. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[
https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618543#comment-13618543
]
Ted Yu commented on HBASE-7836:
---
I got the following from running test suite for combined patch (7835 + 7836):
{code}
Failed tests:
testWorkerAbort(org.apache.hadoop.hbase.master.TestDistributedLogSplitting):
none of the following counters went up in 8 milliseconds -
tot_wkr_task_resigned, tot_wkr_task_err, tot_wkr_final_transition_failed,
tot_wkr_task_done, tot_wkr_preempt_task
Tests in error:
testZKClosingNodeVersionMismatch(org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler)
testCloseRegion(org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler)
testFailedUpdateMeta(org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler)
testYankingRegionFromUnderIt(org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler)
{code}
Create a new replay command so that recovered edits won't mess up normal
coprocessing metrics
-
Key: HBASE-7836
URL: https://issues.apache.org/jira/browse/HBASE-7836
Project: HBase
Issue Type: Sub-task
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.95.0
Attachments: hbase-7836_v1.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7925) Back port HBASE-6881 into 0.94
[ https://issues.apache.org/jira/browse/HBASE-7925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618547#comment-13618547 ] Jimmy Xiang commented on HBASE-7925: I see. That makes sense. If all region servers are dead except just one alive, this could happen. I am ok to get it in 0.94. It is a safe one to me. Back port HBASE-6881 into 0.94 -- Key: HBASE-7925 URL: https://issues.apache.org/jira/browse/HBASE-7925 Project: HBase Issue Type: Bug Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.94.7 Attachments: HBASE-7925_94.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8230) Possible NPE on regionserver abort if replication service has not been started
[
https://issues.apache.org/jira/browse/HBASE-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618549#comment-13618549
]
Jieshan Bean commented on HBASE-8230:
-
bq.Did the failure happen when region server restarted ?
Yes.
bq.If this was repeatable, I would suggest finding the root cause.
The root cause in our env was NameNode was in safemode:
{noformat}
2013-03-29 10:32:42,260 FATAL [regionserver26003] ABORTING region server
om-host2,26003,1364524173470: Unhandled exception: cannot get log writer
org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1737)
java.io.IOException: cannot get log writer
at
org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:757)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.createWriterInstance(HLog.java:701)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:637)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.rollWriter(HLog.java:582)
at org.apache.hadoop.hbase.regionserver.wal.HLog.init(HLog.java:436)
at org.apache.hadoop.hbase.regionserver.wal.HLog.init(HLog.java:362)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateHLog(HRegionServer.java:1327)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1316)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1030)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:706)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException:
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create
file/hbase/.logs/om-host2,26003,1364524173470/om-host2%2C26003%2C1364524173470.1364524361366.
Name node is in safe mode.
The reported blocks 14 has reached the threshold 0.9990 of total blocks 14.
Safe mode will be turned off automatically in 21 seconds.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1601)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1547)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:412)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:204)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:43664)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:924)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1704)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter.init(SequenceFileLogWriter.java:209)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.createWriter(HLog.java:754)
... 10 more
{noformat}
Possible NPE on regionserver abort if replication service has not been started
--
Key: HBASE-8230
URL: https://issues.apache.org/jira/browse/HBASE-8230
Project: HBase
Issue Type: Bug
Components: regionserver, Replication
Affects Versions: 0.94.6
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Attachments: HBASE-8230-94.patch
RegionServer got Exception on calling setupWALAndReplication, so entered
abort flow. Since replicationSink had not been inialized yet, we got below
exception:
{noformat}
Exception in thread regionserver26003 java.lang.NullPointerException
at
org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:129)
at
org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:120)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.join(HRegionServer.java:1803)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:834)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8229) Replication code logs like crazy if a target table cannot be found.
[ https://issues.apache.org/jira/browse/HBASE-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jieshan Bean updated HBASE-8229: Component/s: Replication Replication code logs like crazy if a target table cannot be found. --- Key: HBASE-8229 URL: https://issues.apache.org/jira/browse/HBASE-8229 Project: HBase Issue Type: Bug Components: Replication Reporter: Lars Hofhansl Fix For: 0.95.0, 0.98.0, 0.94.7 One of our RS/DN machines ran out of diskspace on the partition to which we write the log files. It turns out we still had a table in our source cluster with REPLICATION_SCOPE=1 that did not have a matching table in the remote cluster. In then logged a long stack trace every 50ms or so, over a few days that filled up our log partition. Since ReplicationSource cannot make any progress in this case anyway, it should probably sleep a bit before retrying (or at least limit the rate at which it spews out these exceptions to the log). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7925) Back port HBASE-6881 into 0.94
[ https://issues.apache.org/jira/browse/HBASE-7925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618551#comment-13618551 ] Lars Hofhansl commented on HBASE-7925: -- Let's do it.:) Back port HBASE-6881 into 0.94 -- Key: HBASE-7925 URL: https://issues.apache.org/jira/browse/HBASE-7925 Project: HBase Issue Type: Bug Reporter: rajeshbabu Assignee: rajeshbabu Fix For: 0.94.7 Attachments: HBASE-7925_94.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7462) TestDrainingServer is an integration test. It should be a unit test instead
[ https://issues.apache.org/jira/browse/HBASE-7462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618555#comment-13618555 ] Gustavo Anatoly commented on HBASE-7462: Hi, Nicolas. After a long time... I was able to get back. Could you please verify this unit test: [https://gist.github.com/gustavoanatoly/5282776] this approach it works, but mocking I'm facing some problems to finalize: [https://gist.github.com/gustavoanatoly/5282781] if you have a tip with this mocking test, I would be greatful. TestDrainingServer is an integration test. It should be a unit test instead --- Key: HBASE-7462 URL: https://issues.apache.org/jira/browse/HBASE-7462 Project: HBase Issue Type: Wish Components: test Affects Versions: 0.96.0 Reporter: Nicolas Liochon Priority: Trivial Labels: noob TestDrainingServer tests the function that allows to say that a regionserver should not get new regions. As it is written today, it's an integration test: it starts stops a cluster. The test would be more efficient if it would just check that the AssignmentManager does not use the drained region server; whatever the circumstances (bulk assign or not for example). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8233) Poke the cache and let the system reload cached information
Jimmy Xiang created HBASE-8233: -- Summary: Poke the cache and let the system reload cached information Key: HBASE-8233 URL: https://issues.apache.org/jira/browse/HBASE-8233 Project: HBase Issue Type: New Feature Reporter: Jimmy Xiang Sometimes, cached information is stale and we don't have a way to get rid of it except restarting the server. If we can poke the cache and let the system reloads, it will much easier and quicker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restarted
[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618558#comment-13618558
]
Jimmy Xiang commented on HBASE-6469:
I think it is because the ZKTable states are cached. Otherwise, we can
manually reset the table state in ZK, then retry will be able to move on.
If there is a feature to poke the cache, it will be great, so we don't have to
restart the master. I just filed HBASE-8233, which should be helpful in this
case.
Failure on enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restarted
---
Key: HBASE-6469
URL: https://issues.apache.org/jira/browse/HBASE-6469
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.0, 0.94.6
Reporter: Enis Soztutar
Assignee: rajeshbabu
Fix For: 0.95.0, 0.94.7
Attachments: 6469-expose-force-r3.patch, HBASE-6469_2.patch,
HBASE-6469_3.patch, HBASE-6469_4.patch, HBASE-6469.patch,
HBASE-6469_retry_enable_or_disable.patch
In Enable/DisableTableHandler code, if something goes wrong in handling, the
table state in zk is left as ENABLING / DISABLING. After that we cannot force
any more action from the API or CLI, and the only recovery path is restarting
the master.
{code}
if (done) {
// Flip the table to enabled.
this.assignmentManager.getZKTable().setEnabledTable(
this.tableNameStr);
LOG.info(Table ' + this.tableNameStr
+ ' was successfully enabled. Status: done= + done);
} else {
LOG.warn(Table ' + this.tableNameStr
+ ' wasn't successfully enabled. Status: done= + done);
}
{code}
Here, if done is false, the table state is not changed. There is also no way
to set skipTableStateCheck from cli / api.
We have run into this issue a couple of times before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8229) Replication code logs like crazy if a target table cannot be found.
[ https://issues.apache.org/jira/browse/HBASE-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618559#comment-13618559 ] Jieshan Bean commented on HBASE-8229: - bq.For this issue, I'll just add the same waiting we do when the peer is down (which is the same logical behavior we currently have, but without the insane busy retrying). +1 Replication code logs like crazy if a target table cannot be found. --- Key: HBASE-8229 URL: https://issues.apache.org/jira/browse/HBASE-8229 Project: HBase Issue Type: Bug Components: Replication Reporter: Lars Hofhansl Fix For: 0.95.0, 0.98.0, 0.94.7 One of our RS/DN machines ran out of diskspace on the partition to which we write the log files. It turns out we still had a table in our source cluster with REPLICATION_SCOPE=1 that did not have a matching table in the remote cluster. In then logged a long stack trace every 50ms or so, over a few days that filled up our log partition. Since ReplicationSource cannot make any progress in this case anyway, it should probably sleep a bit before retrying (or at least limit the rate at which it spews out these exceptions to the log). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8211) Support for NN HA for 0.94
[ https://issues.apache.org/jira/browse/HBASE-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618564#comment-13618564 ] Fengdong Yu commented on HBASE-8211: Himanshu, Thanks for comments. blockquoteThe idea of HA support is that a client doesn't need to change any url whatsoever, otherwise its not really a failover. /blockquote yes. I read the code again. that's it. blockquoteIf you mean a change in their value because of failover, then it is not required./blockquote for HDFS HA, fs.defaultFS should be configured such as valuehdfs://mycluster/value, so for hbase.rootdir, can we configured such as: valuehdfs://mycluster/hbase/value currently, we just using a specified name node address for hbase.rootdit. Support for NN HA for 0.94 -- Key: HBASE-8211 URL: https://issues.apache.org/jira/browse/HBASE-8211 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 Attachments: HBase-8211-v1.patch HBase-8156 is for adding support for retrying non-idempotent operations. This is useful in case NN is suffering from n/w hiccups, etc. This jira is to add similar support for 0.94.x branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618565#comment-13618565 ] James Taylor commented on HBASE-1936: - Yes, this is definitely important for Phoenix. If we could run without requiring any server-side jar installation, that would definitely remove one hurdle. It's not that big a deal for a dev environment, but as soon as it's production, getting a new jar installed on all region servers is like rolling a 1 ton rock up hill for a few miles. ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Labels: noob Attachments: cp_from_hdfs.patch, HBASE-1936-trunk(forReview).patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8233) Poke the cache and let the system reload cached information
[ https://issues.apache.org/jira/browse/HBASE-8233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618574#comment-13618574 ] Lars Hofhansl commented on HBASE-8233: -- What do mean with cache in this context? I first thought you meant the block cache, but then I saw that HBASE-6469 is linked to this. Poke the cache and let the system reload cached information --- Key: HBASE-8233 URL: https://issues.apache.org/jira/browse/HBASE-8233 Project: HBase Issue Type: New Feature Reporter: Jimmy Xiang Sometimes, cached information is stale and we don't have a way to get rid of it except restarting the server. If we can poke the cache and let the system reloads, it will much easier and quicker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8233) Poke the cache and let the system reload cached information
[ https://issues.apache.org/jira/browse/HBASE-8233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618577#comment-13618577 ] Jimmy Xiang commented on HBASE-8233: Not block cache, ;) I meant the cached information like ZKTable states, sometimes, configuration as well, for example, a coprocessor is removed/added. This may be useful for semi-dynamical configuration changes. Poke the cache and let the system reload cached information --- Key: HBASE-8233 URL: https://issues.apache.org/jira/browse/HBASE-8233 Project: HBase Issue Type: New Feature Reporter: Jimmy Xiang Sometimes, cached information is stale and we don't have a way to get rid of it except restarting the server. If we can poke the cache and let the system reloads, it will much easier and quicker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8211) Support for NN HA for 0.94
[ https://issues.apache.org/jira/browse/HBASE-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618579#comment-13618579 ] Himanshu Vashishtha commented on HBASE-8211: bq. s.defaultFS should be configured such as valuehdfs://mycluster/value, so for hbase.rootdir, can we configured such as: valuehdfs://mycluster/hbase/value Yes, it can support such path. Support for NN HA for 0.94 -- Key: HBASE-8211 URL: https://issues.apache.org/jira/browse/HBASE-8211 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 Attachments: HBase-8211-v1.patch HBase-8156 is for adding support for retrying non-idempotent operations. This is useful in case NN is suffering from n/w hiccups, etc. This jira is to add similar support for 0.94.x branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8211) Support for NN HA for 0.94
[ https://issues.apache.org/jira/browse/HBASE-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618589#comment-13618589 ] Fengdong Yu commented on HBASE-8211: Thanks，I will try. Support for NN HA for 0.94 -- Key: HBASE-8211 URL: https://issues.apache.org/jira/browse/HBASE-8211 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.94.6 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.94.7 Attachments: HBase-8211-v1.patch HBase-8156 is for adding support for retrying non-idempotent operations. This is useful in case NN is suffering from n/w hiccups, etc. This jira is to add similar support for 0.94.x branch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8219) Align Offline Merge with Online Merge
[ https://issues.apache.org/jira/browse/HBASE-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618597#comment-13618597 ] chunhui shen commented on HBASE-8219: - bq.The second and third calls wouldn't have effect because server is null.So if we expose createMergedRegion() as public method, we can call it directly. merged.openHRegion(reporter) is also called in RegionMergeTransaction#openMergedRegion. bq.Maybe we can also implement hbase shell bindings for online one. Shell command of online region is already done(HBASE-8189) IMHO, 1.We should fix the snapshot break caused by offline merge now 2.Do some easy alignment about the code of online merge and offline merge, since we plan to remove offline merge in a futuer moment Align Offline Merge with Online Merge - Key: HBASE-8219 URL: https://issues.apache.org/jira/browse/HBASE-8219 Project: HBase Issue Type: Task Components: regionserver Affects Versions: 0.95.0 Reporter: Matteo Bertozzi Assignee: chunhui shen Attachments: hbase-8219v1.patch, hbase-8219v2.patch After HBASE-7403 we now have two different tools for online and offline merge, and the result produced by the two are different. (the online one works with snapshots, the offline not) We should remove the offline one, or align it to the online code. Most of the offline code in HRegion.merge() can be replaced with the one in RegionMergeTransaction, used by the online version. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8219) Align Offline Merge with Online Merge
[ https://issues.apache.org/jira/browse/HBASE-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-8219: Attachment: hbase-8219v2.patch Addressing Ted's comment Align Offline Merge with Online Merge - Key: HBASE-8219 URL: https://issues.apache.org/jira/browse/HBASE-8219 Project: HBase Issue Type: Task Components: regionserver Affects Versions: 0.95.0 Reporter: Matteo Bertozzi Assignee: chunhui shen Attachments: hbase-8219v1.patch, hbase-8219v2.patch After HBASE-7403 we now have two different tools for online and offline merge, and the result produced by the two are different. (the online one works with snapshots, the offline not) We should remove the offline one, or align it to the online code. Most of the offline code in HRegion.merge() can be replaced with the one in RegionMergeTransaction, used by the online version. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8226) HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions
[ https://issues.apache.org/jira/browse/HBASE-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618605#comment-13618605 ] Hudson commented on HBASE-8226: --- Integrated in HBase-TRUNK #4003 (See [https://builds.apache.org/job/HBase-TRUNK/4003/]) HBASE-8226. HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts 'too many' regions (Revision 1463070) Result = SUCCESS apurtell : Files : * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHTableMultiplexer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorEndpoint.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithRemove.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterTransitions.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogFiltering.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadSequential.java HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions Key: HBASE-8226 URL: https://issues.apache.org/jira/browse/HBASE-8226 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0, 0.96.0, 0.94.7 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.95.0, 0.96.0, 0.94.7 Attachments: 8226-branch-0.94.patch, 8226.patch, 8226.patch HBaseTestingUtility#waitUntilAllRegionsAssigned does not check if the region it is counting belongs to the table created by the test and will not return if it accidentally counts too many regions, for example the regions of the ACL table when security is enabled. Made an attempt at fixing this as part of HBASE-8209 but broke TestMasterTransitions. Try again here. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8213) global authorization may lose efficacy
[
https://issues.apache.org/jira/browse/HBASE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8213:
--
Attachment: HBASE-8213-trunk.patch
+1 on patch 0.94, new test fails if change in TableAuthManager is not applied.
Attached a patch for trunk. TestAccessController passes locally for both 0.94
and trunk builds with patch applied.
global authorization may lose efficacy
---
Key: HBASE-8213
URL: https://issues.apache.org/jira/browse/HBASE-8213
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.0, 0.96.0, 0.94.7
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Priority: Critical
Attachments: HBASE-8213-94.patch, HBASE-8213-trunk.patch
It depends on the order of which region be opened first.
Suppose we have one 1 regionserver and only 1 user region REGION-A on this
server, _acl_ region was on another regionserver. _acl_ was opened a few
seconds before REGION-A.
The global authorization data read from Zookeeper was overwritten by the data
read from configuration.
{code}
private TableAuthManager(ZooKeeperWatcher watcher, Configuration conf)
throws IOException {
this.conf = conf;
this.zkperms = new ZKPermissionWatcher(watcher, this, conf);
try {
// Read global authorization data from zookeeper.
this.zkperms.start();
} catch (KeeperException ke) {
LOG.error(ZooKeeper initialization failed, ke);
}
// It will overwrite globalCache.
// initialize global permissions based on configuration
globalCache = initGlobal(conf);
}
{code}
This issue can be easily reproduced by below steps:
1. Start a cluster with 3 regionservers.
2. Create a new table T1.
3. grant a new user USER-A with global authorization.
4. Kill 1 regionserver RS3 and switch balance off.
5. Start regionserver RS3.
6. Assign region T1 to RS3.
7. Put data with user USER-A.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8213) global authorization may lose efficacy
[
https://issues.apache.org/jira/browse/HBASE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8213:
--
Status: Patch Available (was: Open)
global authorization may lose efficacy
---
Key: HBASE-8213
URL: https://issues.apache.org/jira/browse/HBASE-8213
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.0, 0.96.0, 0.94.7
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Priority: Critical
Attachments: HBASE-8213-94.patch, HBASE-8213-trunk.patch
It depends on the order of which region be opened first.
Suppose we have one 1 regionserver and only 1 user region REGION-A on this
server, _acl_ region was on another regionserver. _acl_ was opened a few
seconds before REGION-A.
The global authorization data read from Zookeeper was overwritten by the data
read from configuration.
{code}
private TableAuthManager(ZooKeeperWatcher watcher, Configuration conf)
throws IOException {
this.conf = conf;
this.zkperms = new ZKPermissionWatcher(watcher, this, conf);
try {
// Read global authorization data from zookeeper.
this.zkperms.start();
} catch (KeeperException ke) {
LOG.error(ZooKeeper initialization failed, ke);
}
// It will overwrite globalCache.
// initialize global permissions based on configuration
globalCache = initGlobal(conf);
}
{code}
This issue can be easily reproduced by below steps:
1. Start a cluster with 3 regionservers.
2. Create a new table T1.
3. grant a new user USER-A with global authorization.
4. Kill 1 regionserver RS3 and switch balance off.
5. Start regionserver RS3.
6. Assign region T1 to RS3.
7. Put data with user USER-A.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8234) Introducing recovering region state to mark a region in recovering status used in distributedLogReplay
Jeffrey Zhong created HBASE-8234: Summary: Introducing recovering region state to mark a region in recovering status used in distributedLogReplay Key: HBASE-8234 URL: https://issues.apache.org/jira/browse/HBASE-8234 Project: HBase Issue Type: Sub-task Reporter: Jeffrey Zhong There are two advantages to have this new recovering state for a region: 1) Instead of mark a region recovering in ZK, we can consolidate all region states in one place and be aware by assignment manager 2) When handing disabling table, we have to have this new state so that regions of a disabling table can be transitioned into this state for recovering. Notes: In the initial release of distributed log replay, we may not do this subtask for simplifications. Without the new state, we still need to create recovered edits files for regions of a disabling table. With the new state, we can retire recover edits files creation business totally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8219) Align Offline Merge with Online Merge
[
https://issues.apache.org/jira/browse/HBASE-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618616#comment-13618616
]
Hadoop QA commented on HBASE-8219:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576331/hbase-8219v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5077//console
This message is automatically generated.
Align Offline Merge with Online Merge
-
Key: HBASE-8219
URL: https://issues.apache.org/jira/browse/HBASE-8219
Project: HBase
Issue Type: Task
Components: regionserver
Affects Versions: 0.95.0
Reporter: Matteo Bertozzi
Assignee: chunhui shen
Attachments: hbase-8219v1.patch, hbase-8219v2.patch
After HBASE-7403 we now have two different tools for online and offline
merge, and the result produced by the two are different. (the online one
works with snapshots, the offline not)
We should remove the offline one, or align it to the online code.
Most of the offline code in HRegion.merge() can be replaced with the one in
RegionMergeTransaction, used by the online version.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618620#comment-13618620
]
Jeffrey Zhong commented on HBASE-8127:
--
[~ram_krish] Have you got a chance to revisit this JIRA? It'd be better if you
can offer some insights.
[~rajesh23] I integrated your current batch with another fix into hbase-7824
patch, so far everything looks good in tests. Let me know when you'll update
your patch. Thanks!
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_feedback.patch,
HBASE-8127.patch, hbase-8127_v1.patch, reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8127) Region of a disabling or disabled table could be stuck in transition state when RS dies during Master initialization
[
https://issues.apache.org/jira/browse/HBASE-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618621#comment-13618621
]
rajeshbabu commented on HBASE-8127:
---
Thanks [~jeffreyz]. Today I will update it.
Region of a disabling or disabled table could be stuck in transition state
when RS dies during Master initialization
Key: HBASE-8127
URL: https://issues.apache.org/jira/browse/HBASE-8127
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.5
Reporter: Jeffrey Zhong
Assignee: rajeshbabu
Fix For: 0.94.7
Attachments: HBASE-8127_94_2.patch, HBASE-8127_feedback.patch,
HBASE-8127.patch, hbase-8127_v1.patch, reproduce-hang.patch
The issue happens when a RS dies during a master starts up. After the RS
reports open to the new master instance and dies immediately thereafter, the
RITs of disabling tables(or disabled table) on the died RS will be in RIT
state forever.
I attached a patch to simulate the situation and you can run the following
command to reproduce the issue:
{code}mvn test -PlocalTests
-Dtest=TestMasterFailover#testMasterFailoverWithMockedRITOnDeadRS{code}
Basically, we skip regions of a dead server inside
AM.processDeadServersAndRecoverLostRegions as the following code and relies
on SSH to process those skipped regions:
{code}
for (PairHRegionInfo, Result deadRegion : deadServer.getValue()) {
nodes.remove(deadRegion.getFirst().getEncodedName());
}
{code}
While in SSH, we skip regions of disabling(or disabled table) again by
function processDeadRegion. Finally comes to the issue that RITs of
disabling(or disabled table) stuck there forever.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8226) HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions
[ https://issues.apache.org/jira/browse/HBASE-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618626#comment-13618626 ] Hudson commented on HBASE-8226: --- Integrated in HBase-0.94 #932 (See [https://builds.apache.org/job/HBase-0.94/932/]) HBASE-8226. HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts 'too many' regions (Revision 1463072) Result = ABORTED apurtell : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithAbort.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithRemove.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterTransitions.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadSequential.java HBaseTestingUtility#waitUntilAllRegionsAssigned won't return if it counts too many regions Key: HBASE-8226 URL: https://issues.apache.org/jira/browse/HBASE-8226 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.95.0, 0.96.0, 0.94.7 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.95.0, 0.96.0, 0.94.7 Attachments: 8226-branch-0.94.patch, 8226.patch, 8226.patch HBaseTestingUtility#waitUntilAllRegionsAssigned does not check if the region it is counting belongs to the table created by the test and will not return if it accidentally counts too many regions, for example the regions of the ACL table when security is enabled. Made an attempt at fixing this as part of HBASE-8209 but broke TestMasterTransitions. Try again here. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8213) global authorization may lose efficacy
[
https://issues.apache.org/jira/browse/HBASE-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618628#comment-13618628
]
Hadoop QA commented on HBASE-8213:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576337/HBASE-8213-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5078//console
This message is automatically generated.
global authorization may lose efficacy
---
Key: HBASE-8213
URL: https://issues.apache.org/jira/browse/HBASE-8213
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.95.0, 0.96.0, 0.94.7
Reporter: Jieshan Bean
Assignee: Jieshan Bean
Priority: Critical
Attachments: HBASE-8213-94.patch, HBASE-8213-trunk.patch
It depends on the order of which region be opened first.
Suppose we have one 1 regionserver and only 1 user region REGION-A on this
server, _acl_ region was on another regionserver. _acl_ was opened a few
seconds before REGION-A.
The global authorization data read from Zookeeper was overwritten by the data
read from configuration.
{code}
private TableAuthManager(ZooKeeperWatcher watcher, Configuration conf)
throws IOException {
this.conf = conf;
this.zkperms = new ZKPermissionWatcher(watcher, this, conf);
try {
// Read global authorization data from zookeeper.
this.zkperms.start();
} catch (KeeperException ke) {
LOG.error(ZooKeeper initialization failed, ke);
}
// It will overwrite globalCache.
// initialize global permissions based on configuration
globalCache = initGlobal(conf);
}
{code}
This issue can be easily reproduced by below steps:
1. Start a cluster with 3 regionservers.
2. Create a new table T1.
3. grant a new user USER-A with global authorization.
4. Kill 1 regionserver RS3 and switch balance off.
5. Start regionserver RS3.
6. Assign region T1 to RS3.
7. Put data with user USER-A.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8164: -- Status: Open (was: Patch Available) TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8164: -- Attachment: HBASE-8164_addendum_4.patch The reason for getting 0 regions for the table is that, as online schema change is enabled, just when we try to get the region for split the region has been trying to reopen after the modifytable was completed. This closure of region removed the region from online regions list in the RS. So we get 0 regions. My latest addendum avoids split to proceed in such cases. TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch, HBASE-8164_addendum_4.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8164) TestTableLockManager fails intermittently in trunk builds
[ https://issues.apache.org/jira/browse/HBASE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8164: -- Status: Patch Available (was: Open) TestTableLockManager fails intermittently in trunk builds - Key: HBASE-8164 URL: https://issues.apache.org/jira/browse/HBASE-8164 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.95.0, 0.98.0 Attachments: 8164-addendum-2.txt, 8164-addendum.txt, 8164-v2.txt, 8164-v3.txt, 8164-v4.txt, HBASE-8164_addendum_3.patch, HBASE-8164_addendum_4.patch In build #3979: testTableReadLock(org.apache.hadoop.hbase.master.TestTableLockManager): test timed out after 60 milliseconds -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
