[jira] [Commented] (HBASE-5007) HBaseAdmin.stopRegionServer do not stop the region server
[
https://issues.apache.org/jira/browse/HBASE-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167061#comment-13167061
]
Hadoop QA commented on HBASE-5007:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12506882/HBase_0.92_HBASE-5007.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
-1 javadoc. The javadoc tool appears to have generated -160 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 75 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/486//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/486//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/486//console
This message is automatically generated.
HBaseAdmin.stopRegionServer do not stop the region server
-
Key: HBASE-5007
URL: https://issues.apache.org/jira/browse/HBASE-5007
Project: HBase
Issue Type: Bug
Components: ipc
Affects Versions: 0.92.0
Environment: all
Reporter: honghua zhu
Fix For: 0.92.0
Attachments: HBase_0.92_HBASE-5007.patch
Please running this example:
public class Test {
public static void main(String[] args) throws Exception {
HBaseAdmin admin = new HBaseAdmin(HBaseConfiguration.create());
admin.stopRegionServer(your.rs.hostname:60020);
}
}
then, you can see:
Exception in thread main java.lang.RuntimeException: The interface
org.apache.hadoop.hbase.Stoppable
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:61)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:151)
at $Proxy2.stop(Unknown Source)
at
org.apache.hadoop.hbase.client.HBaseAdmin.stopRegionServer(HBaseAdmin.java:1492)
at Test.main(Test.java:7)
Caused by: java.lang.NoSuchFieldException: VERSION
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:57)
... 4 more
When invoking the HBaseAdmin.stopRegionServer method，
we obtain a proxy for org.apache.hadoop.hbase.ipc.HRegionInterface，
(HRegionInterface extends org.apache.hadoop.hbase.Stoppable)
but the stop method declared in Stoppable.
In the constructor of org.apache.hadoop.hbase.ipc.Invocation，
the method argument is public abstract void
org.apache.hadoop.hbase.Stoppable.stop(java.lang.String)，
so, method.getDeclaringClass() is org.apache.hadoop.hbase.Stoppable，
but, the Stoppable interface no VERSION field.
[fix suggestion]:
Override the stop method in org.apache.hadoop.hbase.ipc.HRegionInterface as
follows:
==
@Override
public void stop(String why);
of courese, another attempt is ok.
(e.g. declare VERSION field in Stoppable interface，
then modify some code fragment of Invocation and
org.apache.hadoop.hbase.ipc.WritableRpcEngine.Server)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4974) Remove some resources leaks on the tests
[ https://issues.apache.org/jira/browse/HBASE-4974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167065#comment-13167065 ] Hudson commented on HBASE-4974: --- Integrated in HBase-0.92-security #35 (See [https://builds.apache.org/job/HBase-0.92-security/35/]) HBASE-4974 Remove some resources leaks on the tests stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/rest/TestScannersWithFilters.java Remove some resources leaks on the tests Key: HBASE-4974 URL: https://issues.apache.org/jira/browse/HBASE-4974 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.92.0 Attachments: 4974_all.patch, 4974_all.v2.patch Cf. title and HBASE-4965 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4994) TestHeapSize broke in trunk
[
https://issues.apache.org/jira/browse/HBASE-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167062#comment-13167062
]
Hudson commented on HBASE-4994:
---
Integrated in HBase-0.92-security #35 (See
[https://builds.apache.org/job/HBase-0.92-security/35/])
HBASE-4994 TestHeapSize broke in trunk
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
TestHeapSize broke in trunk
---
Key: HBASE-4994
URL: https://issues.apache.org/jira/browse/HBASE-4994
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.92.0
Attachments: heapsize.txt
This commit added Map to HRegion
{code}
commit 888d73a9f5fe907f7c616211322fff339eeaa446
Author: Zhihong Yu te...@apache.org
Date: Fri Dec 9 06:01:58 2011 +
HBASE-4946 HTable.coprocessorExec (and possibly coprocessorProxy) does
not work with
dynamically loaded coprocessors (Andrei Dragomir)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4995) Increase zk maxClientCnxns to give us some head room
[ https://issues.apache.org/jira/browse/HBASE-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167063#comment-13167063 ] Hudson commented on HBASE-4995: --- Integrated in HBase-0.92-security #35 (See [https://builds.apache.org/job/HBase-0.92-security/35/]) HBASE-4995 Increase zk maxClientCnxns to give us some head room stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.92/src/main/resources/hbase-default.xml Increase zk maxClientCnxns to give us some head room Key: HBASE-4995 URL: https://issues.apache.org/jira/browse/HBASE-4995 Project: HBase Issue Type: Improvement Affects Versions: 0.90.4 Reporter: Jean-Daniel Cryans Assignee: stack Priority: Blocker Fix For: 0.92.0, 0.94.0 Attachments: 4995-v2.txt, 4995.txt It's pretty easy to run out of zk connections on a single host if it's running a master, region server, and a TT with a few slots. Just to make it easier for our users, we should set it to something like 100 by default. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4859) Correctly PreWarm HBCK ThreadPool
[ https://issues.apache.org/jira/browse/HBASE-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167064#comment-13167064 ] Hudson commented on HBASE-4859: --- Integrated in HBase-0.92-security #35 (See [https://builds.apache.org/job/HBase-0.92-security/35/]) HBASE-4859 Correctly PreWarm HBCK ThreadPool stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java Correctly PreWarm HBCK ThreadPool - Key: HBASE-4859 URL: https://issues.apache.org/jira/browse/HBASE-4859 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Fix For: 0.92.0 Attachments: HBASE-4859.patch, HBASE-4859.patch See description at HBASE-3553. We had a patch ready for this in HBASE-3620 but never applied it publicly. Testing showed massive speedup in HBCK, especially when RegionServers were down or had long response times. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4922) [packaging] Assembly tars up hbase in a subdir; i.e. after untar hbase-0.92.0 has a subdir named 0.92.0
[ https://issues.apache.org/jira/browse/HBASE-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167066#comment-13167066 ] Hudson commented on HBASE-4922: --- Integrated in HBase-0.92-security #35 (See [https://builds.apache.org/job/HBase-0.92-security/35/]) HBASE-4922 [packaging] Assembly tars up hbase in a subdir; i.e. after untar hbase-0.92.0 has a subdir named 0.92.0 stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/pom.xml [packaging] Assembly tars up hbase in a subdir; i.e. after untar hbase-0.92.0 has a subdir named 0.92.0 --- Key: HBASE-4922 URL: https://issues.apache.org/jira/browse/HBASE-4922 Project: HBase Issue Type: Bug Reporter: stack Assignee: Roman Shaposhnik Priority: Blocker Fix For: 0.92.0 Attachments: HBASE-4922.patch.txt Reported by Roman. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5000) Speed up simultaneous reads of a block when block caching is turned off
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin reassigned HBASE-5000: - Assignee: Mikhail Bautin Speed up simultaneous reads of a block when block caching is turned off --- Key: HBASE-5000 URL: https://issues.apache.org/jira/browse/HBASE-5000 Project: HBase Issue Type: Bug Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a sparse lock primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it. Setting priority as minor, as it is very unusual to turn off block caching. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5008) The clusters can't provide services because Region can't flush.
The clusters can't provide services because Region can't flush. Key: HBASE-5008 URL: https://issues.apache.org/jira/browse/HBASE-5008 Project: HBase Issue Type: Bug Components: regionserver Reporter: gaojinchao Priority: Blocker Fix For: 0.90.6 Hbase version 0.90.4 + patches My analysis is as follows: //Started splitting region b24d8ccb852ff742f2a27d01b7f5853e and closed region. 2011-12-10 17:32:48,653 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:32:49,759 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: disabling compactions flushes 2011-12-10 17:32:49,759 INFO org.apache.hadoop.hbase.regionserver.HRegion: Running close preflush of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. //Processed a flush request and skipped , But flushRequested had set to true 2011-12-10 17:33:06,963 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e., current region memstore size 12.6m 2011-12-10 17:33:17,277 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Skipping flush on Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. because closing //split region b24d8ccb852ff742f2a27d01b7f5853 failed and rolled back, flushRequested flag was true, So all handle was blocked 2011-12-10 17:34:01,293 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Cleaned up old failed split transaction detritus: hdfs://193.195.18.121:9000/hbase/Htable_UFDR_004/b24d8ccb852ff742f2a27d01b7f5853e/splits 2011-12-10 17:34:01,294 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.; next sequenceid=15494173 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size // All handles had been blocked. The clusters could not provide services 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,192 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 34 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,193 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 51 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,196 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 85 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,199 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 88 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,202 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 44 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:11,663 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 2 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:11,665 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 10 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:11,670 INFO
[jira] [Updated] (HBASE-5008) The clusters can't provide services because Region can't flush.
[ https://issues.apache.org/jira/browse/HBASE-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaojinchao updated HBASE-5008: -- Attachment: HBASE-5008_Branch90.patch I made a patch, Please review The clusters can't provide services because Region can't flush. Key: HBASE-5008 URL: https://issues.apache.org/jira/browse/HBASE-5008 Project: HBase Issue Type: Bug Components: regionserver Reporter: gaojinchao Priority: Blocker Fix For: 0.90.6 Attachments: HBASE-5008_Branch90.patch Hbase version 0.90.4 + patches My analysis is as follows: //Started splitting region b24d8ccb852ff742f2a27d01b7f5853e and closed region. 2011-12-10 17:32:48,653 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:32:49,759 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: disabling compactions flushes 2011-12-10 17:32:49,759 INFO org.apache.hadoop.hbase.regionserver.HRegion: Running close preflush of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. //Processed a flush request and skipped , But flushRequested had set to true 2011-12-10 17:33:06,963 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e., current region memstore size 12.6m 2011-12-10 17:33:17,277 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Skipping flush on Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. because closing //split region b24d8ccb852ff742f2a27d01b7f5853 failed and rolled back, flushRequested flag was true, So all handle was blocked 2011-12-10 17:34:01,293 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Cleaned up old failed split transaction detritus: hdfs://193.195.18.121:9000/hbase/Htable_UFDR_004/b24d8ccb852ff742f2a27d01b7f5853e/splits 2011-12-10 17:34:01,294 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.; next sequenceid=15494173 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size // All handles had been blocked. The clusters could not provide services 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,192 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 34 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,193 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 51 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,196 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 85 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,199 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 88 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,202 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 44 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:11,663 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 2 on 20020' on region
[jira] [Commented] (HBASE-5008) The clusters can't provide services because Region can't flush.
[ https://issues.apache.org/jira/browse/HBASE-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167086#comment-13167086 ] gaojinchao commented on HBASE-5008: --- TestSplitTransactionOnCluster and TestSplitTransaction have passed. All test cases are running and will give a result tomorrow. The clusters can't provide services because Region can't flush. Key: HBASE-5008 URL: https://issues.apache.org/jira/browse/HBASE-5008 Project: HBase Issue Type: Bug Components: regionserver Reporter: gaojinchao Priority: Blocker Fix For: 0.90.6 Attachments: HBASE-5008_Branch90.patch Hbase version 0.90.4 + patches My analysis is as follows: //Started splitting region b24d8ccb852ff742f2a27d01b7f5853e and closed region. 2011-12-10 17:32:48,653 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:32:49,759 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: disabling compactions flushes 2011-12-10 17:32:49,759 INFO org.apache.hadoop.hbase.regionserver.HRegion: Running close preflush of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. //Processed a flush request and skipped , But flushRequested had set to true 2011-12-10 17:33:06,963 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e., current region memstore size 12.6m 2011-12-10 17:33:17,277 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Skipping flush on Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. because closing //split region b24d8ccb852ff742f2a27d01b7f5853 failed and rolled back, flushRequested flag was true, So all handle was blocked 2011-12-10 17:34:01,293 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Cleaned up old failed split transaction detritus: hdfs://193.195.18.121:9000/hbase/Htable_UFDR_004/b24d8ccb852ff742f2a27d01b7f5853e/splits 2011-12-10 17:34:01,294 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.; next sequenceid=15494173 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size // All handles had been blocked. The clusters could not provide services 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,192 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 34 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,193 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 51 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,196 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 85 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,199 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 88 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,202 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 44 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:11,663 INFO
[jira] [Resolved] (HBASE-4705) HBase won't initialize if /hbase is not present
[
https://issues.apache.org/jira/browse/HBASE-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J resolved HBASE-4705.
Resolution: Later
Not a problem right now. Triggering patch was reverted.
HBase won't initialize if /hbase is not present
---
Key: HBASE-4705
URL: https://issues.apache.org/jira/browse/HBASE-4705
Project: HBase
Issue Type: Bug
Reporter: Harsh J
{code}
2011-10-31 00:09:09,549 FATAL org.apache.hadoop.hbase.master.HMaster:
Unhandled exception. Starting shutdown.
java.io.FileNotFoundException: File does not exist: hdfs://C3S31:9000/hbase
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:731)
at org.apache.hadoop.hbase.util.FSUtils.isInSafeMode(FSUtils.java:163)
at
org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:458)
at
org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:301)
at
org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127)
at
org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:426)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)
at java.lang.Thread.run(Thread.java:662)
2011-10-31 00:09:09,551 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
2011-10-31 00:09:09,551 DEBUG org.apache.hadoop.hbase.master.HMaster:
Stopping service threads
{code}
Trunk won't start HBase unless /hbase is already present, after HBASE-4680
(and the silly error I made in HBASE-4510).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4680) FSUtils.isInSafeMode() checks should operate on HBase root dir, where we have permissions
[
https://issues.apache.org/jira/browse/HBASE-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J resolved HBASE-4680.
Resolution: Later
Not a problem right now.
FSUtils.isInSafeMode() checks should operate on HBase root dir, where we have
permissions
-
Key: HBASE-4680
URL: https://issues.apache.org/jira/browse/HBASE-4680
Project: HBase
Issue Type: Bug
Components: util
Affects Versions: 0.92.0, 0.94.0
Reporter: Gary Helmling
Assignee: Gary Helmling
Attachments: HBASE-4680.patch
The HDFS safe mode check workaround introduced by HBASE-4510 performs a
{{FileSystem.setPermission()}} operation on the root directory (/) when
attempting to trigger a {{SafeModeException}}. As a result, it requires
superuser privileges when running with DFS permission checking enabled.
Changing the operations to act on the HBase root directory should be safe,
since the master process must have write access to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5007) HBaseAdmin.stopRegionServer do not stop the region server
[
https://issues.apache.org/jira/browse/HBASE-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5007:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed branch and trunk. Thanks for patch Honghua (The failed TestAdmin was
because of too many open files).
HBaseAdmin.stopRegionServer do not stop the region server
-
Key: HBASE-5007
URL: https://issues.apache.org/jira/browse/HBASE-5007
Project: HBase
Issue Type: Bug
Components: ipc
Affects Versions: 0.92.0
Environment: all
Reporter: honghua zhu
Fix For: 0.92.0
Attachments: HBase_0.92_HBASE-5007.patch
Please running this example:
public class Test {
public static void main(String[] args) throws Exception {
HBaseAdmin admin = new HBaseAdmin(HBaseConfiguration.create());
admin.stopRegionServer(your.rs.hostname:60020);
}
}
then, you can see:
Exception in thread main java.lang.RuntimeException: The interface
org.apache.hadoop.hbase.Stoppable
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:61)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:151)
at $Proxy2.stop(Unknown Source)
at
org.apache.hadoop.hbase.client.HBaseAdmin.stopRegionServer(HBaseAdmin.java:1492)
at Test.main(Test.java:7)
Caused by: java.lang.NoSuchFieldException: VERSION
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:57)
... 4 more
When invoking the HBaseAdmin.stopRegionServer method，
we obtain a proxy for org.apache.hadoop.hbase.ipc.HRegionInterface，
(HRegionInterface extends org.apache.hadoop.hbase.Stoppable)
but the stop method declared in Stoppable.
In the constructor of org.apache.hadoop.hbase.ipc.Invocation，
the method argument is public abstract void
org.apache.hadoop.hbase.Stoppable.stop(java.lang.String)，
so, method.getDeclaringClass() is org.apache.hadoop.hbase.Stoppable，
but, the Stoppable interface no VERSION field.
[fix suggestion]:
Override the stop method in org.apache.hadoop.hbase.ipc.HRegionInterface as
follows:
==
@Override
public void stop(String why);
of courese, another attempt is ok.
(e.g. declare VERSION field in Stoppable interface，
then modify some code fragment of Invocation and
org.apache.hadoop.hbase.ipc.WritableRpcEngine.Server)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5001) Improve the performance of block cache keys
[
https://issues.apache.org/jira/browse/HBASE-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167237#comment-13167237
]
stack commented on HBASE-5001:
--
What happens if you hfile name is bytes only and we make key by doing
Bytes.add, adding the hfilename as bytes + offset as bytes (Would have to
include the Bytes.toBytes(long).
We'd have to be doing a lot of key making for the above to show, agreed.
Improve the performance of block cache keys
---
Key: HBASE-5001
URL: https://issues.apache.org/jira/browse/HBASE-5001
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Priority: Minor
Fix For: 0.94.0
Doing a pure random read test on data that's 100% block cache, I see that we
are spending quite some time in getBlockCacheKey:
{quote}
IPC Server handler 19 on 62023 daemon prio=10 tid=0x7fe0501ff800
nid=0x6c87 runnable [0x7fe0577f6000]
java.lang.Thread.State: RUNNABLE
at java.util.Arrays.copyOf(Arrays.java:2882)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuilder.append(StringBuilder.java:119)
at
org.apache.hadoop.hbase.io.hfile.HFile.getBlockCacheKey(HFile.java:457)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:249)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:521)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:536)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:178)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekExactly(StoreFileScanner.java:219)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.init(StoreScanner.java:80)
at
org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1689)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.init(HRegion.java:2857)
{quote}
Since the HFile name size is known and the offset is a long, it should be
possible to allocate exactly what we need. Maybe use byte[] as the key and
drop the separator too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5008) The clusters can't provide services because Region can't flush.
[ https://issues.apache.org/jira/browse/HBASE-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167239#comment-13167239 ] Lars Hofhansl commented on HBASE-5008: -- This if I understand this correctly: # a requested flush was canceled (because of a split?), we never unset flushRequested # from this point on every new flush request is ignored because flushRequested is already true Change seems sensible, although I do not know this part of the code very well. Can flushRequested is never be legitimately true at this point? The clusters can't provide services because Region can't flush. Key: HBASE-5008 URL: https://issues.apache.org/jira/browse/HBASE-5008 Project: HBase Issue Type: Bug Components: regionserver Reporter: gaojinchao Priority: Blocker Fix For: 0.90.6 Attachments: HBASE-5008_Branch90.patch Hbase version 0.90.4 + patches My analysis is as follows: //Started splitting region b24d8ccb852ff742f2a27d01b7f5853e and closed region. 2011-12-10 17:32:48,653 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:32:49,759 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: disabling compactions flushes 2011-12-10 17:32:49,759 INFO org.apache.hadoop.hbase.regionserver.HRegion: Running close preflush of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. //Processed a flush request and skipped , But flushRequested had set to true 2011-12-10 17:33:06,963 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e., current region memstore size 12.6m 2011-12-10 17:33:17,277 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Skipping flush on Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. because closing //split region b24d8ccb852ff742f2a27d01b7f5853 failed and rolled back, flushRequested flag was true, So all handle was blocked 2011-12-10 17:34:01,293 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Cleaned up old failed split transaction detritus: hdfs://193.195.18.121:9000/hbase/Htable_UFDR_004/b24d8ccb852ff742f2a27d01b7f5853e/splits 2011-12-10 17:34:01,294 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.; next sequenceid=15494173 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size // All handles had been blocked. The clusters could not provide services 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,192 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 34 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,193 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 51 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,196 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 85 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,199 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 88 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,202 INFO org.apache.hadoop.hbase.regionserver.HRegion:
[jira] [Updated] (HBASE-4971) Useless sleeps in TestTimestampsFilter and TestMultipleTimestamps
[ https://issues.apache.org/jira/browse/HBASE-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4971: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Applied to trunk. Thanks for patch N. Useless sleeps in TestTimestampsFilter and TestMultipleTimestamps - Key: HBASE-4971 URL: https://issues.apache.org/jira/browse/HBASE-4971 Project: HBase Issue Type: Improvement Components: test Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.94.0 Attachments: 4971.patch, 4971_all.v2.patch Comment says Flush tables. Since flushing is asynchronous, sleep for a bit., but the function is synchronous. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5001) Improve the performance of block cache keys
[
https://issues.apache.org/jira/browse/HBASE-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167284#comment-13167284
]
Lars Hofhansl commented on HBASE-5001:
--
* Bytes.add(hfileNameInBytes, Bytes.toBytes(offset)) - 0.07us
But byte[] cannot be directly use as key in a map, no? Would need to wrap in
HashBytes, so:
* new HashedBytes(Bytes.add(x, Bytes.toBytes(offser))); - 0.08us
Which brought me to a new idea, what if we have a CacheKey Object that takes a
String and a long:
* new CacheKey(hfileName, offset) - 0.01us
That would be the cleanest design anyway. Cachkey would implement the proper
equals and hashCode methods.
The LruCache could just take CacheKey (or even just java.lang.Object) as cache
key, that way we can pass whatever.
Improve the performance of block cache keys
---
Key: HBASE-5001
URL: https://issues.apache.org/jira/browse/HBASE-5001
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Priority: Minor
Fix For: 0.94.0
Doing a pure random read test on data that's 100% block cache, I see that we
are spending quite some time in getBlockCacheKey:
{quote}
IPC Server handler 19 on 62023 daemon prio=10 tid=0x7fe0501ff800
nid=0x6c87 runnable [0x7fe0577f6000]
java.lang.Thread.State: RUNNABLE
at java.util.Arrays.copyOf(Arrays.java:2882)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuilder.append(StringBuilder.java:119)
at
org.apache.hadoop.hbase.io.hfile.HFile.getBlockCacheKey(HFile.java:457)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:249)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:521)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:536)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:178)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekExactly(StoreFileScanner.java:219)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.init(StoreScanner.java:80)
at
org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1689)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.init(HRegion.java:2857)
{quote}
Since the HFile name size is known and the offset is a long, it should be
possible to allocate exactly what we need. Maybe use byte[] as the key and
drop the separator too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5007) HBaseAdmin.stopRegionServer do not stop the region server
[
https://issues.apache.org/jira/browse/HBASE-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167300#comment-13167300
]
Hudson commented on HBASE-5007:
---
Integrated in HBase-TRUNK #2536 (See
[https://builds.apache.org/job/HBase-TRUNK/2536/])
HBASE-5007 HBaseAdmin.stopRegionServer do not stop the region server
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
HBaseAdmin.stopRegionServer do not stop the region server
-
Key: HBASE-5007
URL: https://issues.apache.org/jira/browse/HBASE-5007
Project: HBase
Issue Type: Bug
Components: ipc
Affects Versions: 0.92.0
Environment: all
Reporter: honghua zhu
Fix For: 0.92.0
Attachments: HBase_0.92_HBASE-5007.patch
Please running this example:
public class Test {
public static void main(String[] args) throws Exception {
HBaseAdmin admin = new HBaseAdmin(HBaseConfiguration.create());
admin.stopRegionServer(your.rs.hostname:60020);
}
}
then, you can see:
Exception in thread main java.lang.RuntimeException: The interface
org.apache.hadoop.hbase.Stoppable
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:61)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:151)
at $Proxy2.stop(Unknown Source)
at
org.apache.hadoop.hbase.client.HBaseAdmin.stopRegionServer(HBaseAdmin.java:1492)
at Test.main(Test.java:7)
Caused by: java.lang.NoSuchFieldException: VERSION
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:57)
... 4 more
When invoking the HBaseAdmin.stopRegionServer method，
we obtain a proxy for org.apache.hadoop.hbase.ipc.HRegionInterface，
(HRegionInterface extends org.apache.hadoop.hbase.Stoppable)
but the stop method declared in Stoppable.
In the constructor of org.apache.hadoop.hbase.ipc.Invocation，
the method argument is public abstract void
org.apache.hadoop.hbase.Stoppable.stop(java.lang.String)，
so, method.getDeclaringClass() is org.apache.hadoop.hbase.Stoppable，
but, the Stoppable interface no VERSION field.
[fix suggestion]:
Override the stop method in org.apache.hadoop.hbase.ipc.HRegionInterface as
follows:
==
@Override
public void stop(String why);
of courese, another attempt is ok.
(e.g. declare VERSION field in Stoppable interface，
then modify some code fragment of Invocation and
org.apache.hadoop.hbase.ipc.WritableRpcEngine.Server)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4970) Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch)
[ https://issues.apache.org/jira/browse/HBASE-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167308#comment-13167308 ] ramkrishna.s.vasudevan commented on HBASE-4970: --- +1 on patch.. Allow better control of resource consumption in HTable (backport HBASE-4805 to 0.90 branch) --- Key: HBASE-4970 URL: https://issues.apache.org/jira/browse/HBASE-4970 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.90.4 Reporter: gaojinchao Assignee: gaojinchao Priority: Trivial Fix For: 0.90.6 Attachments: HBASE-4970_Branch90.patch, HBASE-4970_Branch90_V1_trial.patch, HBASE-4970_Branch90_V2.patch In my cluster, I changed keepAliveTime from 60 s to 3600 s. Increasing RES is slowed down. Why increasing keepAliveTime of HBase thread pool is slowing down our problem occurance [RES value increase]? You can go through the source of sun.nio.ch.Util. Every thread hold 3 softreference of direct buffer(mustangsrc) for reusage. The code names the 3 softreferences buffercache. If the buffer was all occupied or none was suitable in size, and new request comes, new direct buffer is allocated. After the service, the bigger one replaces the smaller one in buffercache. The replaced buffer is released. So I think we can add a parameter to change keepAliveTime of Htable thread pool. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5008) The clusters can't provide services because Region can't flush.
[ https://issues.apache.org/jira/browse/HBASE-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167311#comment-13167311 ] ramkrishna.s.vasudevan commented on HBASE-5008: --- +1 on patch.. Good catch and good analysis. May be we can add for trunk if the problem is found in trunk. The clusters can't provide services because Region can't flush. Key: HBASE-5008 URL: https://issues.apache.org/jira/browse/HBASE-5008 Project: HBase Issue Type: Bug Components: regionserver Reporter: gaojinchao Priority: Blocker Fix For: 0.90.6 Attachments: HBASE-5008_Branch90.patch Hbase version 0.90.4 + patches My analysis is as follows: //Started splitting region b24d8ccb852ff742f2a27d01b7f5853e and closed region. 2011-12-10 17:32:48,653 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:32:49,759 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: disabling compactions flushes 2011-12-10 17:32:49,759 INFO org.apache.hadoop.hbase.regionserver.HRegion: Running close preflush of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. //Processed a flush request and skipped , But flushRequested had set to true 2011-12-10 17:33:06,963 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e., current region memstore size 12.6m 2011-12-10 17:33:17,277 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Skipping flush on Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. because closing //split region b24d8ccb852ff742f2a27d01b7f5853 failed and rolled back, flushRequested flag was true, So all handle was blocked 2011-12-10 17:34:01,293 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Cleaned up old failed split transaction detritus: hdfs://193.195.18.121:9000/hbase/Htable_UFDR_004/b24d8ccb852ff742f2a27d01b7f5853e/splits 2011-12-10 17:34:01,294 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.; next sequenceid=15494173 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size // All handles had been blocked. The clusters could not provide services 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,192 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 34 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,193 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 51 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,196 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 85 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,199 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 88 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:10,202 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 44 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is = than blocking 384.0m size 2011-12-10 17:43:11,663 INFO
[jira] [Commented] (HBASE-4923) [packaging] Assembly should make only executables executable (docs should not be executable!)
[ https://issues.apache.org/jira/browse/HBASE-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167321#comment-13167321 ] Hudson commented on HBASE-4923: --- Integrated in HBase-0.92 #182 (See [https://builds.apache.org/job/HBase-0.92/182/]) HBASE-4923 [packaging] Assembly should make only executables executable (docs should not be executable) stack : Files : * /hbase/branches/0.92/src/assembly/all.xml [packaging] Assembly should make only executables executable (docs should not be executable!) - Key: HBASE-4923 URL: https://issues.apache.org/jira/browse/HBASE-4923 Project: HBase Issue Type: Bug Reporter: stack Reported by Roman. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5007) HBaseAdmin.stopRegionServer do not stop the region server
[
https://issues.apache.org/jira/browse/HBASE-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167320#comment-13167320
]
Hudson commented on HBASE-5007:
---
Integrated in HBase-0.92 #182 (See
[https://builds.apache.org/job/HBase-0.92/182/])
HBASE-5007 HBaseAdmin.stopRegionServer do not stop the region server
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
HBaseAdmin.stopRegionServer do not stop the region server
-
Key: HBASE-5007
URL: https://issues.apache.org/jira/browse/HBASE-5007
Project: HBase
Issue Type: Bug
Components: ipc
Affects Versions: 0.92.0
Environment: all
Reporter: honghua zhu
Fix For: 0.92.0
Attachments: HBase_0.92_HBASE-5007.patch
Please running this example:
public class Test {
public static void main(String[] args) throws Exception {
HBaseAdmin admin = new HBaseAdmin(HBaseConfiguration.create());
admin.stopRegionServer(your.rs.hostname:60020);
}
}
then, you can see:
Exception in thread main java.lang.RuntimeException: The interface
org.apache.hadoop.hbase.Stoppable
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:61)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:151)
at $Proxy2.stop(Unknown Source)
at
org.apache.hadoop.hbase.client.HBaseAdmin.stopRegionServer(HBaseAdmin.java:1492)
at Test.main(Test.java:7)
Caused by: java.lang.NoSuchFieldException: VERSION
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:57)
... 4 more
When invoking the HBaseAdmin.stopRegionServer method，
we obtain a proxy for org.apache.hadoop.hbase.ipc.HRegionInterface，
(HRegionInterface extends org.apache.hadoop.hbase.Stoppable)
but the stop method declared in Stoppable.
In the constructor of org.apache.hadoop.hbase.ipc.Invocation，
the method argument is public abstract void
org.apache.hadoop.hbase.Stoppable.stop(java.lang.String)，
so, method.getDeclaringClass() is org.apache.hadoop.hbase.Stoppable，
but, the Stoppable interface no VERSION field.
[fix suggestion]:
Override the stop method in org.apache.hadoop.hbase.ipc.HRegionInterface as
follows:
==
@Override
public void stop(String why);
of courese, another attempt is ok.
(e.g. declare VERSION field in Stoppable interface，
then modify some code fragment of Invocation and
org.apache.hadoop.hbase.ipc.WritableRpcEngine.Server)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4746) Use a random ZK client port in unit tests so we can run them in parallel
[ https://issues.apache.org/jira/browse/HBASE-4746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167331#comment-13167331 ] Phabricator commented on HBASE-4746: mbautin has abandoned the revision [jira] [HBASE-4746] [89-fb] Use a random ZK client port in unit tests so we can run them in parallel. This was committed into 89-fb, abandoning revision. REVISION DETAIL https://reviews.facebook.net/D255 Use a random ZK client port in unit tests so we can run them in parallel Key: HBASE-4746 URL: https://issues.apache.org/jira/browse/HBASE-4746 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: 4746-trunk-v2.txt, D255.1.patch, D255.2.patch, D279-trunk-v5.txt, D279-trunk-v7.txt, D279.1.patch, D279.2.patch, D279.3.patch, D279.4.patch, D279.5.patch, D279.6.patch, D279.7.patch, D279.92 The hard-coded ZK client port has long been a problem for running HBase test suite in parallel. The mini ZK cluster should run on a random free port, and that port should be passed to all parts of the unit tests that need to talk to the mini cluster. In fact, randomizing the port exposes a lot of places in the code where a new configuration is instantiated, and as a result the client tries to talk to the default ZK client port and times out. The initial fix is for 0.89-fb, where it already allows to run unit tests in parallel in 10 minutes. A fix for the trunk will follow. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167335#comment-13167335 ] jirapos...@reviews.apache.org commented on HBASE-4120: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1421/ --- (Updated 2011-12-12 03:43:32.270550) Review request for hbase. Changes --- Move all Operation.getPriority() methods to PriorityJobQueue.ActionPriorities. Move all Qos related codes out of HRegionServer. Add coprocessor hook to HRegionServer.ScannerListener.leaseExpired(). Improve the TestTablePriority class to test scan and put operations by default. Summary --- Patch used for table priority alone,In this patch, not only tables can have different priorities but also the different actions like get,scan,put and delete can have priorities. This addresses bug HBase-4120. https://issues.apache.org/jira/browse/HBase-4120 Diffs (updated) - http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java 1213130 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java 1213130 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/PriorityFunction.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/PriorityHBaseServer.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/PriorityJobQueue.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/QosRegionObserver.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1213130 http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/ipc/TestActionPriority.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/ipc/TestPriorityJobQueue.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/ipc/TestTablePriority.java PRE-CREATION Diff: https://reviews.apache.org/r/1421/diff Testing --- Tested with test cases in TestCase_For_TablePriority_trunk_v1.patch please apply the patch of HBASE-4181 first,in some circumstances this bug will affect the performance of client. Thanks, Jia isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is
[jira] [Commented] (HBASE-5001) Improve the performance of block cache keys
[
https://issues.apache.org/jira/browse/HBASE-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167343#comment-13167343
]
stack commented on HBASE-5001:
--
I like 0.01us. Better than 0.34us. +1 on your idea Lars. Could compare
offsets first. That should be fast. Then filename.
Improve the performance of block cache keys
---
Key: HBASE-5001
URL: https://issues.apache.org/jira/browse/HBASE-5001
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Priority: Minor
Fix For: 0.94.0
Doing a pure random read test on data that's 100% block cache, I see that we
are spending quite some time in getBlockCacheKey:
{quote}
IPC Server handler 19 on 62023 daemon prio=10 tid=0x7fe0501ff800
nid=0x6c87 runnable [0x7fe0577f6000]
java.lang.Thread.State: RUNNABLE
at java.util.Arrays.copyOf(Arrays.java:2882)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuilder.append(StringBuilder.java:119)
at
org.apache.hadoop.hbase.io.hfile.HFile.getBlockCacheKey(HFile.java:457)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:249)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:521)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:536)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:178)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekExactly(StoreFileScanner.java:219)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.init(StoreScanner.java:80)
at
org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1689)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.init(HRegion.java:2857)
{quote}
Since the HFile name size is known and the offset is a long, it should be
possible to allocate exactly what we need. Maybe use byte[] as the key and
drop the separator too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5001) Improve the performance of block cache keys
[
https://issues.apache.org/jira/browse/HBASE-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167344#comment-13167344
]
stack commented on HBASE-5001:
--
Hmm... maybe filename compare first where filename is byte array, not
String?
Improve the performance of block cache keys
---
Key: HBASE-5001
URL: https://issues.apache.org/jira/browse/HBASE-5001
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Priority: Minor
Fix For: 0.94.0
Doing a pure random read test on data that's 100% block cache, I see that we
are spending quite some time in getBlockCacheKey:
{quote}
IPC Server handler 19 on 62023 daemon prio=10 tid=0x7fe0501ff800
nid=0x6c87 runnable [0x7fe0577f6000]
java.lang.Thread.State: RUNNABLE
at java.util.Arrays.copyOf(Arrays.java:2882)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuilder.append(StringBuilder.java:119)
at
org.apache.hadoop.hbase.io.hfile.HFile.getBlockCacheKey(HFile.java:457)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:249)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:521)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:536)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:178)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekExactly(StoreFileScanner.java:219)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.init(StoreScanner.java:80)
at
org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1689)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.init(HRegion.java:2857)
{quote}
Since the HFile name size is known and the offset is a long, it should be
possible to allocate exactly what we need. Maybe use byte[] as the key and
drop the separator too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4923) [packaging] Assembly should make only executables executable (docs should not be executable!)
[ https://issues.apache.org/jira/browse/HBASE-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4923: - Attachment: 4923.txt Patch I committed (while JIRA was down). [packaging] Assembly should make only executables executable (docs should not be executable!) - Key: HBASE-4923 URL: https://issues.apache.org/jira/browse/HBASE-4923 Project: HBase Issue Type: Bug Reporter: stack Attachments: 4923.txt Reported by Roman. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4989) Metrics to measure sequential reads and random reads separately
[ https://issues.apache.org/jira/browse/HBASE-4989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HBASE-4989: Attachment: metrics1.txt Patchfor trunk. Metrics to measure sequential reads and random reads separately --- Key: HBASE-4989 URL: https://issues.apache.org/jira/browse/HBASE-4989 Project: HBase Issue Type: Improvement Components: regionserver Reporter: dhruba borthakur Assignee: dhruba borthakur Priority: Minor Attachments: metrics1.txt HBase does sequential reads for compactions and positional random reads for satisfying user's queries. It would be nice if we can measure their latencies separately. It is mostly the random reads that dominate a transactional workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4989) Metrics to measure sequential reads and random reads separately
[ https://issues.apache.org/jira/browse/HBASE-4989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HBASE-4989: Release Note: The metric fsReadLatency records the number of sequential reads. The metric fsPreadLatency records the number of random reads. Hadoop Flags: Incompatible change Status: Patch Available (was: Open) Metrics to measure sequential reads and random reads separately --- Key: HBASE-4989 URL: https://issues.apache.org/jira/browse/HBASE-4989 Project: HBase Issue Type: Improvement Components: regionserver Reporter: dhruba borthakur Assignee: dhruba borthakur Priority: Minor Attachments: metrics1.txt HBase does sequential reads for compactions and positional random reads for satisfying user's queries. It would be nice if we can measure their latencies separately. It is mostly the random reads that dominate a transactional workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4971) Useless sleeps in TestTimestampsFilter and TestMultipleTimestamps
[ https://issues.apache.org/jira/browse/HBASE-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167353#comment-13167353 ] Hudson commented on HBASE-4971: --- Integrated in HBase-TRUNK #2537 (See [https://builds.apache.org/job/HBase-TRUNK/2537/]) HBASE-4971 Useless sleeps in TestTimestampsFilter and TestMultipleTimestamps stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultipleTimestamps.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestTimestampsFilter.java Useless sleeps in TestTimestampsFilter and TestMultipleTimestamps - Key: HBASE-4971 URL: https://issues.apache.org/jira/browse/HBASE-4971 Project: HBase Issue Type: Improvement Components: test Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.94.0 Attachments: 4971.patch, 4971_all.v2.patch Comment says Flush tables. Since flushing is asynchronous, sleep for a bit., but the function is synchronous. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4923) [packaging] Assembly should make only executables executable (docs should not be executable!)
[ https://issues.apache.org/jira/browse/HBASE-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167354#comment-13167354 ] Hudson commented on HBASE-4923: --- Integrated in HBase-TRUNK #2537 (See [https://builds.apache.org/job/HBase-TRUNK/2537/]) HBASE-4923 [packaging] Assembly should make only executables executable (docs should not be executable) stack : Files : * /hbase/trunk/src/assembly/all.xml [packaging] Assembly should make only executables executable (docs should not be executable!) - Key: HBASE-4923 URL: https://issues.apache.org/jira/browse/HBASE-4923 Project: HBase Issue Type: Bug Reporter: stack Attachments: 4923.txt Reported by Roman. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5001) Improve the performance of block cache keys
[
https://issues.apache.org/jira/browse/HBASE-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167358#comment-13167358
]
Lars Hofhansl commented on HBASE-5001:
--
Was thinking something as simple as this:
{code}
public class Key {
private String file;
private long offset;
public Key(String file, long offset) {
this.file = file;
this.offset = offset;
}
@Override
public int hashCode() {
return file.hashCode()*127+(int)(offset ^ (offset 32));
}
@Override
public boolean equals(Object o) {
if (o instanceof Key) {
Key k = (Key)o;
return offset == k.offset (file == null ? k.file == null :
file.equals(k.file));
} else {
return false;
}
}
}
{code}
Improve the performance of block cache keys
---
Key: HBASE-5001
URL: https://issues.apache.org/jira/browse/HBASE-5001
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.4
Reporter: Jean-Daniel Cryans
Priority: Minor
Fix For: 0.94.0
Doing a pure random read test on data that's 100% block cache, I see that we
are spending quite some time in getBlockCacheKey:
{quote}
IPC Server handler 19 on 62023 daemon prio=10 tid=0x7fe0501ff800
nid=0x6c87 runnable [0x7fe0577f6000]
java.lang.Thread.State: RUNNABLE
at java.util.Arrays.copyOf(Arrays.java:2882)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuilder.append(StringBuilder.java:119)
at
org.apache.hadoop.hbase.io.hfile.HFile.getBlockCacheKey(HFile.java:457)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:249)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:209)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:521)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:536)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:178)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:111)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekExactly(StoreFileScanner.java:219)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.init(StoreScanner.java:80)
at
org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:1689)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.init(HRegion.java:2857)
{quote}
Since the HFile name size is known and the offset is a long, it should be
possible to allocate exactly what we need. Maybe use byte[] as the key and
drop the separator too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5000) Speed up simultaneous reads of a block when block caching is turned off
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167360#comment-13167360 ] Lars Hofhansl commented on HBASE-5000: -- +1 on documenting. Could state there, that block caching can be disabled on a per scan bases, and in fact should be disable if the data scanned is known (or likely to) not to fit in the cache. Generally shouldn't aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded... Speed up simultaneous reads of a block when block caching is turned off --- Key: HBASE-5000 URL: https://issues.apache.org/jira/browse/HBASE-5000 Project: HBase Issue Type: Bug Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a sparse lock primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it. Setting priority as minor, as it is very unusual to turn off block caching. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5000) Speed up simultaneous reads of a block when block caching is turned off
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167360#comment-13167360 ] Lars Hofhansl edited comment on HBASE-5000 at 12/12/11 5:39 AM: +1 on documenting. Could state there that block caching can be disabled on a per scan bases, and in fact should be disabled if the data scanned is known (or likely to) not to fit in the cache. Generally shouldn't aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded... was (Author: lhofhansl): +1 on documenting. Could state there, that block caching can be disabled on a per scan bases, and in fact should be disable if the data scanned is known (or likely to) not to fit in the cache. Generally shouldn't aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded... Speed up simultaneous reads of a block when block caching is turned off --- Key: HBASE-5000 URL: https://issues.apache.org/jira/browse/HBASE-5000 Project: HBase Issue Type: Bug Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a sparse lock primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it. Setting priority as minor, as it is very unusual to turn off block caching. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5000) Speed up simultaneous reads of a block when block caching is turned off
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167360#comment-13167360 ] Lars Hofhansl edited comment on HBASE-5000 at 12/12/11 5:40 AM: +1 on documenting. Could state there that block caching can be disabled on a per scan bases, and in fact should be disabled if the data scanned is known (or likely to) not to fit in the cache. Generally shouldn't we aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded... was (Author: lhofhansl): +1 on documenting. Could state there that block caching can be disabled on a per scan bases, and in fact should be disabled if the data scanned is known (or likely to) not to fit in the cache. Generally shouldn't aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded... Speed up simultaneous reads of a block when block caching is turned off --- Key: HBASE-5000 URL: https://issues.apache.org/jira/browse/HBASE-5000 Project: HBase Issue Type: Bug Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a sparse lock primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it. Setting priority as minor, as it is very unusual to turn off block caching. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4989) Metrics to measure sequential reads and random reads separately
[ https://issues.apache.org/jira/browse/HBASE-4989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167381#comment-13167381 ] Hadoop QA commented on HBASE-4989: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506971/metrics1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -160 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestInstantSchemaChange org.apache.hadoop.hbase.client.TestAdmin Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/487//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/487//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/487//console This message is automatically generated. Metrics to measure sequential reads and random reads separately --- Key: HBASE-4989 URL: https://issues.apache.org/jira/browse/HBASE-4989 Project: HBase Issue Type: Improvement Components: regionserver Reporter: dhruba borthakur Assignee: dhruba borthakur Priority: Minor Attachments: metrics1.txt HBase does sequential reads for compactions and positional random reads for satisfying user's queries. It would be nice if we can measure their latencies separately. It is mostly the random reads that dominate a transactional workload. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4923) [packaging] Assembly should make only executables executable (docs should not be executable!)
[ https://issues.apache.org/jira/browse/HBASE-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167384#comment-13167384 ] Hudson commented on HBASE-4923: --- Integrated in HBase-0.92-security #36 (See [https://builds.apache.org/job/HBase-0.92-security/36/]) HBASE-4923 [packaging] Assembly should make only executables executable (docs should not be executable) stack : Files : * /hbase/branches/0.92/src/assembly/all.xml [packaging] Assembly should make only executables executable (docs should not be executable!) - Key: HBASE-4923 URL: https://issues.apache.org/jira/browse/HBASE-4923 Project: HBase Issue Type: Bug Reporter: stack Attachments: 4923.txt Reported by Roman. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5007) HBaseAdmin.stopRegionServer do not stop the region server
[
https://issues.apache.org/jira/browse/HBASE-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167383#comment-13167383
]
Hudson commented on HBASE-5007:
---
Integrated in HBase-0.92-security #36 (See
[https://builds.apache.org/job/HBase-0.92-security/36/])
HBASE-5007 HBaseAdmin.stopRegionServer do not stop the region server
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
HBaseAdmin.stopRegionServer do not stop the region server
-
Key: HBASE-5007
URL: https://issues.apache.org/jira/browse/HBASE-5007
Project: HBase
Issue Type: Bug
Components: ipc
Affects Versions: 0.92.0
Environment: all
Reporter: honghua zhu
Fix For: 0.92.0
Attachments: HBase_0.92_HBASE-5007.patch
Please running this example:
public class Test {
public static void main(String[] args) throws Exception {
HBaseAdmin admin = new HBaseAdmin(HBaseConfiguration.create());
admin.stopRegionServer(your.rs.hostname:60020);
}
}
then, you can see:
Exception in thread main java.lang.RuntimeException: The interface
org.apache.hadoop.hbase.Stoppable
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:61)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:151)
at $Proxy2.stop(Unknown Source)
at
org.apache.hadoop.hbase.client.HBaseAdmin.stopRegionServer(HBaseAdmin.java:1492)
at Test.main(Test.java:7)
Caused by: java.lang.NoSuchFieldException: VERSION
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:57)
... 4 more
When invoking the HBaseAdmin.stopRegionServer method，
we obtain a proxy for org.apache.hadoop.hbase.ipc.HRegionInterface，
(HRegionInterface extends org.apache.hadoop.hbase.Stoppable)
but the stop method declared in Stoppable.
In the constructor of org.apache.hadoop.hbase.ipc.Invocation，
the method argument is public abstract void
org.apache.hadoop.hbase.Stoppable.stop(java.lang.String)，
so, method.getDeclaringClass() is org.apache.hadoop.hbase.Stoppable，
but, the Stoppable interface no VERSION field.
[fix suggestion]:
Override the stop method in org.apache.hadoop.hbase.ipc.HRegionInterface as
follows:
==
@Override
public void stop(String why);
of courese, another attempt is ok.
(e.g. declare VERSION field in Stoppable interface，
then modify some code fragment of Invocation and
org.apache.hadoop.hbase.ipc.WritableRpcEngine.Server)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5007) HBaseAdmin.stopRegionServer do not stop the region server
[
https://issues.apache.org/jira/browse/HBASE-5007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167399#comment-13167399
]
Hudson commented on HBASE-5007:
---
Integrated in HBase-TRUNK-security #28 (See
[https://builds.apache.org/job/HBase-TRUNK-security/28/])
HBASE-5007 HBaseAdmin.stopRegionServer do not stop the region server
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java
HBaseAdmin.stopRegionServer do not stop the region server
-
Key: HBASE-5007
URL: https://issues.apache.org/jira/browse/HBASE-5007
Project: HBase
Issue Type: Bug
Components: ipc
Affects Versions: 0.92.0
Environment: all
Reporter: honghua zhu
Fix For: 0.92.0
Attachments: HBase_0.92_HBASE-5007.patch
Please running this example:
public class Test {
public static void main(String[] args) throws Exception {
HBaseAdmin admin = new HBaseAdmin(HBaseConfiguration.create());
admin.stopRegionServer(your.rs.hostname:60020);
}
}
then, you can see:
Exception in thread main java.lang.RuntimeException: The interface
org.apache.hadoop.hbase.Stoppable
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:61)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:151)
at $Proxy2.stop(Unknown Source)
at
org.apache.hadoop.hbase.client.HBaseAdmin.stopRegionServer(HBaseAdmin.java:1492)
at Test.main(Test.java:7)
Caused by: java.lang.NoSuchFieldException: VERSION
at java.lang.Class.getField(Class.java:1520)
at org.apache.hadoop.hbase.ipc.Invocation.init(Invocation.java:57)
... 4 more
When invoking the HBaseAdmin.stopRegionServer method，
we obtain a proxy for org.apache.hadoop.hbase.ipc.HRegionInterface，
(HRegionInterface extends org.apache.hadoop.hbase.Stoppable)
but the stop method declared in Stoppable.
In the constructor of org.apache.hadoop.hbase.ipc.Invocation，
the method argument is public abstract void
org.apache.hadoop.hbase.Stoppable.stop(java.lang.String)，
so, method.getDeclaringClass() is org.apache.hadoop.hbase.Stoppable，
but, the Stoppable interface no VERSION field.
[fix suggestion]:
Override the stop method in org.apache.hadoop.hbase.ipc.HRegionInterface as
follows:
==
@Override
public void stop(String why);
of courese, another attempt is ok.
(e.g. declare VERSION field in Stoppable interface，
then modify some code fragment of Invocation and
org.apache.hadoop.hbase.ipc.WritableRpcEngine.Server)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4923) [packaging] Assembly should make only executables executable (docs should not be executable!)
[ https://issues.apache.org/jira/browse/HBASE-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167401#comment-13167401 ] Hudson commented on HBASE-4923: --- Integrated in HBase-TRUNK-security #28 (See [https://builds.apache.org/job/HBase-TRUNK-security/28/]) HBASE-4923 [packaging] Assembly should make only executables executable (docs should not be executable) stack : Files : * /hbase/trunk/src/assembly/all.xml [packaging] Assembly should make only executables executable (docs should not be executable!) - Key: HBASE-4923 URL: https://issues.apache.org/jira/browse/HBASE-4923 Project: HBase Issue Type: Bug Reporter: stack Attachments: 4923.txt Reported by Roman. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4971) Useless sleeps in TestTimestampsFilter and TestMultipleTimestamps
[ https://issues.apache.org/jira/browse/HBASE-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167400#comment-13167400 ] Hudson commented on HBASE-4971: --- Integrated in HBase-TRUNK-security #28 (See [https://builds.apache.org/job/HBase-TRUNK-security/28/]) HBASE-4971 Useless sleeps in TestTimestampsFilter and TestMultipleTimestamps stack : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultipleTimestamps.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestTimestampsFilter.java Useless sleeps in TestTimestampsFilter and TestMultipleTimestamps - Key: HBASE-4971 URL: https://issues.apache.org/jira/browse/HBASE-4971 Project: HBase Issue Type: Improvement Components: test Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.94.0 Attachments: 4971.patch, 4971_all.v2.patch Comment says Flush tables. Since flushing is asynchronous, sleep for a bit., but the function is synchronous. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
