[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268911#comment-13268911 ] Hudson commented on HBASE-5889: --- Integrated in HBase-TRUNK #2850 (See [https://builds.apache.org/job/HBase-TRUNK/2850/]) HBASE-5889 Remove HRegionInterface (Revision 1334314) Result = FAILURE stack : Files : * /hbase/trunk/conf/hbase-policy.xml * /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/HBasePolicyProvider.java * /hbase/trunk/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRpcMetrics.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java * /hbase/trunk/src/main/protobuf/Admin.proto * /hbase/trunk/src/main/protobuf/RPC.proto * /hbase/trunk/src/main/resources/hbase-default.xml * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestGlobalMemStoreSize.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSStatusServlet.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly,
[jira] [Commented] (HBASE-5844) Delete the region servers znode after a regions server crash
[ https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268906#comment-13268906 ] Hudson commented on HBASE-5844: --- Integrated in HBase-TRUNK-security #192 (See [https://builds.apache.org/job/HBase-TRUNK-security/192/]) HBASE-5844 Delete the region servers znode after a regions server crash (Revision 1334028) Result = SUCCESS stack : Files : * /hbase/trunk/bin/hbase-daemon.sh * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > Delete the region servers znode after a regions server crash > > > Key: HBASE-5844 > URL: https://issues.apache.org/jira/browse/HBASE-5844 > Project: HBase > Issue Type: Improvement > Components: regionserver, scripts >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal > Fix For: 0.96.0 > > Attachments: 5844.v1.patch, 5844.v2.patch, 5844.v3.patch, > 5844.v3.patch, 5844.v4.patch > > > today, if the regions server crashes, its znode is not deleted in ZooKeeper. > So the recovery process will stop only after a timeout, usually 30s. > By deleting the znode in start script, we remove this delay and the recovery > starts immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268907#comment-13268907 ] Hudson commented on HBASE-5889: --- Integrated in HBase-TRUNK-security #192 (See [https://builds.apache.org/job/HBase-TRUNK-security/192/]) HBASE-5889 Remove HRegionInterface (Revision 1334314) Result = SUCCESS stack : Files : * /hbase/trunk/conf/hbase-policy.xml * /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/security/HBasePolicyProvider.java * /hbase/trunk/src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRpcMetrics.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java * /hbase/trunk/src/main/protobuf/Admin.proto * /hbase/trunk/src/main/protobuf/RPC.proto * /hbase/trunk/src/main/resources/hbase-default.xml * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestGlobalMemStoreSize.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSStatusServlet.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was se
[jira] [Commented] (HBASE-5902) Some scripts are not executable
[ https://issues.apache.org/jira/browse/HBASE-5902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268909#comment-13268909 ] Hudson commented on HBASE-5902: --- Integrated in HBase-TRUNK-security #192 (See [https://builds.apache.org/job/HBase-TRUNK-security/192/]) HBASE-5902 Some scripts are not executable (Revision 1334019) Result = SUCCESS stack : Files : * /hbase/trunk/bin/graceful_stop.sh * /hbase/trunk/bin/local-master-backup.sh * /hbase/trunk/bin/local-regionservers.sh > Some scripts are not executable > --- > > Key: HBASE-5902 > URL: https://issues.apache.org/jira/browse/HBASE-5902 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Trivial > Fix For: 0.96.0, 0.94.1 > > Attachments: 5902.v1.patch, 5902v2.txt > > > -rw-rw-r-- graceful_stop.sh > -rw-rw-r-- hbase-config.sh > -rw-rw-r-- local-master-backup.sh > -rw-rw-r-- local-regionservers.sh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5887) Make TestAcidGuarantees usable for system testing.
[ https://issues.apache.org/jira/browse/HBASE-5887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268910#comment-13268910 ] Hudson commented on HBASE-5887: --- Integrated in HBase-TRUNK-security #192 (See [https://builds.apache.org/job/HBase-TRUNK-security/192/]) HBASE-5887 Make TestAcidGuarantees usable for system testing (Revision 1333785) Result = SUCCESS jmhsieh : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java > Make TestAcidGuarantees usable for system testing. > -- > > Key: HBASE-5887 > URL: https://issues.apache.org/jira/browse/HBASE-5887 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.0, 0.92.1, 0.94.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: hbase-5887-92.patch, hbase-5887.patch > > > Currently, the TestAcidGuarantees run via main() will always abort with an > NPE because it digs into a non-existant HBaseTestingUtility for a flusher > thread. We should tool this up so that it works properly from the command > line. This would be a very useful long running test when used in conjunction > with fault injections to verify row acid properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5876) TestImportExport has been failing against hadoop 0.23 profile
[ https://issues.apache.org/jira/browse/HBASE-5876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268908#comment-13268908 ] Hudson commented on HBASE-5876: --- Integrated in HBase-TRUNK-security #192 (See [https://builds.apache.org/job/HBase-TRUNK-security/192/]) HBASE-5876 TestImportExport has been failing against hadoop 0.23 profile (Revision 1333778) Result = SUCCESS jmhsieh : Files : * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/MapreduceTestingShim.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java > TestImportExport has been failing against hadoop 0.23 profile > - > > Key: HBASE-5876 > URL: https://issues.apache.org/jira/browse/HBASE-5876 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0, 0.96.0 >Reporter: Zhihong Yu >Assignee: Jonathan Hsieh > Fix For: 0.96.0, 0.94.1 > > Attachments: hbase-5876-94.patch, hbase-5876-v2.patch, > hbase-5876.patch > > > TestImportExport has been failing against hadoop 0.23 profile -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5907) enhance HLog pretty printer to print additional useful stats
[ https://issues.apache.org/jira/browse/HBASE-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268900#comment-13268900 ] Kannan Muthukkaruppan commented on HBASE-5907: -- @Stack: Sure, if you can help commit to trunk, please do. Thanks. > enhance HLog pretty printer to print additional useful stats > > > Key: HBASE-5907 > URL: https://issues.apache.org/jira/browse/HBASE-5907 > Project: HBase > Issue Type: Improvement >Reporter: Kannan Muthukkaruppan >Priority: Minor > Attachments: D2979.1.patch, D2979.2.patch > > > It would be useful for analysis purposes to enhance the HLog pretty printer > to optionally print a bunch of additional stats such as: > 1) # of txns > 2) # of KVs updated > 3) avg size of txns > 4) avg size of KVs > 5) avg # of KVs written per txn > 5) unique CF signatures involved in put/delete operatons; and breakdown of > some of the above metrics by these signatures, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5295) Improve the Thrift API to switch on/off writing to wal for Mutations
[ https://issues.apache.org/jira/browse/HBASE-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268898#comment-13268898 ] Phabricator commented on HBASE-5295: Kannan has commented on the revision "[jira] [HBASE-5295] [89-fb] Improve the Thrift API to switch on/off writing to wal for Mutations". looks good! nice work as usual Mikhail. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2370 good catch! REVISION DETAIL https://reviews.facebook.net/D3015 > Improve the Thrift API to switch on/off writing to wal for Mutations > - > > Key: HBASE-5295 > URL: https://issues.apache.org/jira/browse/HBASE-5295 > Project: HBase > Issue Type: Improvement > Components: thrift >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: D1515.1.patch, D1515.1.patch, D1515.1.patch, > D1515.1.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, > D3015.1.patch, D3015.2.patch, D3015.3.patch, D3015.4.patch > > > The thrift api currently does not support switching off updating wal for > Puts/Deletes. Support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5889: - Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change) Status: Resolved (was: Patch Available) Committed TRUNK. Good stuff Jimmy. > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268895#comment-13268895 ] stack commented on HBASE-5889: -- I think this not the best soln. but I'm not going to argue for a better patch in here when this one is already fat and its gotten a +1 from hadoopqa. Lets get this in and work on polish in other issues. Good stuff Jimmy. > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5914) Bulk assign regions in the process of ServerShutdownHandler
[ https://issues.apache.org/jira/browse/HBASE-5914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268890#comment-13268890 ] Hadoop QA commented on HBASE-5914: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525380/HBASE-5914v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster org.apache.hadoop.hbase.TestDrainingServer org.apache.hadoop.hbase.master.TestAssignmentManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1778//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1778//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1778//console This message is automatically generated. > Bulk assign regions in the process of ServerShutdownHandler > --- > > Key: HBASE-5914 > URL: https://issues.apache.org/jira/browse/HBASE-5914 > Project: HBase > Issue Type: Improvement >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-5914.patch, HBASE-5914v2.patch > > > In the process of ServerShutdownHandler, we currently assign regions singly. > In the large cluster, one regionserver always carried many regions, this > action is quite slow. > What about using bulk assign regions like cluster start up. > In current logic, if we failed assigning many regions to one destination > server, we will wait unitl timeout, > however in the process of ServerShutdownHandler, we should retry it to > another server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5295) Improve the Thrift API to switch on/off writing to wal for Mutations
[ https://issues.apache.org/jira/browse/HBASE-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5295: --- Attachment: D3015.4.patch mbautin updated the revision "[jira] [HBASE-5295] [89-fb] Improve the Thrift API to switch on/off writing to wal for Mutations". Reviewers: Kannan, dhruba, sc, JIRA Simplifying a couple of lines REVISION DETAIL https://reviews.facebook.net/D3015 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/KeyValue.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java src/main/java/org/apache/hadoop/hbase/thrift/ThriftUtilities.java src/main/java/org/apache/hadoop/hbase/thrift/generated/Mutation.java src/main/java/org/apache/hadoop/hbase/util/Bytes.java src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift src/test/java/org/apache/hadoop/hbase/HTestConst.java src/test/java/org/apache/hadoop/hbase/thrift/TestMutationWriteToWAL.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerLegacy.java > Improve the Thrift API to switch on/off writing to wal for Mutations > - > > Key: HBASE-5295 > URL: https://issues.apache.org/jira/browse/HBASE-5295 > Project: HBase > Issue Type: Improvement > Components: thrift >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: D1515.1.patch, D1515.1.patch, D1515.1.patch, > D1515.1.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, > D3015.1.patch, D3015.2.patch, D3015.3.patch, D3015.4.patch > > > The thrift api currently does not support switching off updating wal for > Puts/Deletes. Support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5295) Improve the Thrift API to switch on/off writing to wal for Mutations
[ https://issues.apache.org/jira/browse/HBASE-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5295: --- Attachment: D3015.3.patch mbautin updated the revision "[jira] [HBASE-5295] [89-fb] Improve the Thrift API to switch on/off writing to wal for Mutations". Reviewers: Kannan, dhruba, sc, JIRA Testing deletes. Fixed a bug: we were always writing deletes to WAL (this is fixed in trunk). REVISION DETAIL https://reviews.facebook.net/D3015 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/KeyValue.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java src/main/java/org/apache/hadoop/hbase/thrift/ThriftUtilities.java src/main/java/org/apache/hadoop/hbase/thrift/generated/Mutation.java src/main/java/org/apache/hadoop/hbase/util/Bytes.java src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift src/test/java/org/apache/hadoop/hbase/HTestConst.java src/test/java/org/apache/hadoop/hbase/thrift/TestMutationWriteToWAL.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerLegacy.java > Improve the Thrift API to switch on/off writing to wal for Mutations > - > > Key: HBASE-5295 > URL: https://issues.apache.org/jira/browse/HBASE-5295 > Project: HBase > Issue Type: Improvement > Components: thrift >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: D1515.1.patch, D1515.1.patch, D1515.1.patch, > D1515.1.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, > D3015.1.patch, D3015.2.patch, D3015.3.patch > > > The thrift api currently does not support switching off updating wal for > Puts/Deletes. Support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5914) Bulk assign regions in the process of ServerShutdownHandler
[ https://issues.apache.org/jira/browse/HBASE-5914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5914: -- Status: Patch Available (was: Open) > Bulk assign regions in the process of ServerShutdownHandler > --- > > Key: HBASE-5914 > URL: https://issues.apache.org/jira/browse/HBASE-5914 > Project: HBase > Issue Type: Improvement >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0 > > Attachments: HBASE-5914.patch, HBASE-5914v2.patch > > > In the process of ServerShutdownHandler, we currently assign regions singly. > In the large cluster, one regionserver always carried many regions, this > action is quite slow. > What about using bulk assign regions like cluster start up. > In current logic, if we failed assigning many regions to one destination > server, we will wait unitl timeout, > however in the process of ServerShutdownHandler, we should retry it to > another server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available
[ https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268877#comment-13268877 ] Hudson commented on HBASE-5849: --- Integrated in HBase-0.92-security #106 (See [https://builds.apache.org/job/HBase-0.92-security/106/]) HBASE-5849 On first cluster startup, RS aborts if root znode is not available; REAPPLY (Revision 1330119) HBASE-5849 On first cluster startup, RS aborts if root znode is not available; REAPPLY (Revision 1330118) HBASE-5849 On first cluster startup, RS aborts if root znode is not available; REVERT (Revision 1329562) HBASE-5849 On first cluster startup, RS aborts if root znode is not available (Revision 1329548) HBASE-5849 On first cluster startup, RS aborts if root znode is not available (Revision 1329530) Result = SUCCESS stack : Files : * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java stack : Files : * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestClusterBootOrder.java > On first cluster startup, RS aborts if root znode is not available > -- > > Key: HBASE-5849 > URL: https://issues.apache.org/jira/browse/HBASE-5849 > Project: HBase > Issue Type: Bug > Components: master, regionserver, zookeeper >Affects Versions: 0.92.2, 0.96.0, 0.94.1 >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.92.2, 0.94.0 > > Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch, > HBASE-5849_v4-0.92.patch, HBASE-5849_v4.patch, HBASE-5849_v4.patch, > HBASE-5849_v4.patch > > > When launching a fresh new cluster, the master has to be started first, which > might create race conditions for starting master and rs at the same time. > Master startup code is smt like this: > - establish zk connection > - create root znodes in zk (/hbase) > - create ephemeral node for master /hbase/master, > Region server start up code is smt like this: > - establish zk connection > - check whether the root znode (/hbase) is there. If not, shutdown. > - wait for the master to create znodes /hbase/master > So, the problem is on the very first launch of the cluster, RS aborts to > start since /hbase znode might not have been created yet (only the master > creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on > subsequent cluster starts, it does not matter which order the servers are > started. So this affects only first launchs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5863) Improve the graceful_stop.sh CLI help (especially about reloads)
[
https://issues.apache.org/jira/browse/HBASE-5863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268876#comment-13268876
]
Hudson commented on HBASE-5863:
---
Integrated in HBase-0.92-security #106 (See
[https://builds.apache.org/job/HBase-0.92-security/106/])
HBASE-5863 Improve the graceful_stop.sh CLI help (especially about reloads)
(Revision 1329795)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/bin/graceful_stop.sh
> Improve the graceful_stop.sh CLI help (especially about reloads)
>
>
> Key: HBASE-5863
> URL: https://issues.apache.org/jira/browse/HBASE-5863
> Project: HBase
> Issue Type: Improvement
> Components: scripts
>Affects Versions: 0.94.0
>Reporter: Harsh J
>Assignee: Harsh J
>Priority: Minor
> Fix For: 0.92.2, 0.94.0
>
> Attachments: HBASE-5863.patch
>
>
> Right now, graceful_stop.sh prints:
> {code}
> Usage: graceful_stop.sh [--config ] [--restart] [--reload]
> [--thrift] [--rest]
> thrift If we should stop/start thrift before/after the hbase stop/start
> restIf we should stop/start rest before/after the hbase stop/start
> restart If we should restart after graceful stop
> reload Move offloaded regions back on to the stopped server
> debug Move offloaded regions back on to the stopped server
> hostnameHostname of server we are to stop
> {code}
> This does not help us specify that "reload" is actually a sub/additive-option
> to "restart".
> Also, the "debug" line seems to still have an old copy/paste mistake.
> I've updated these two in the patch here.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5883) Backup master is going down due to connection refused exception
[
https://issues.apache.org/jira/browse/HBASE-5883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268871#comment-13268871
]
Hudson commented on HBASE-5883:
---
Integrated in HBase-0.92-security #106 (See
[https://builds.apache.org/job/HBase-0.92-security/106/])
HBASE-5883 Backup master is going down due to connection refused exception
(Jieshan) (Revision 1333537)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
> Backup master is going down due to connection refused exception
> ---
>
> Key: HBASE-5883
> URL: https://issues.apache.org/jira/browse/HBASE-5883
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.6, 0.92.1, 0.94.0
>Reporter: Gopinathan A
>Assignee: Jieshan Bean
> Fix For: 0.96.0, 0.94.1
>
> Attachments: HBASE-5883-90-addendum.patch, HBASE-5883-90.patch,
> HBASE-5883-92-addendum.patch, HBASE-5883-92.patch,
> HBASE-5883-94-addendum.patch, HBASE-5883-94.patch,
> HBASE-5883-trunk-addendum.patch, HBASE-5883-trunk.patch
>
>
> The active master node network was down for some time (This node contains
> Master,DN,ZK,RS). Here backup node got
> notification, and started to became active. Immedietly backup node got
> aborted with the below exception.
> {noformat}
> 2012-04-09 10:42:24,270 INFO org.apache.hadoop.hbase.master.SplitLogManager:
> finished splitting (more than or equal to) 861248320 bytes in 4 log files in
> [hdfs://192.168.47.205:9000/hbase/.logs/HOST-192-168-47-202,60020,1333715537172-splitting]
> in 26374ms
> 2012-04-09 10:42:24,316 FATAL org.apache.hadoop.hbase.master.HMaster: Master
> server abort: loaded coprocessors are: []
> 2012-04-09 10:42:24,333 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unhandled exception. Starting shutdown.
> java.io.IOException: java.net.ConnectException: Connection refused
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:375)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
> at $Proxy13.getProtocolVersion(Unknown Source)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1276)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1233)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1220)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.getCachedConnection(CatalogTracker.java:569)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.getRootServerConnection(CatalogTracker.java:369)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.waitForRootServerConnection(CatalogTracker.java:353)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.verifyRootRegionLocation(CatalogTracker.java:660)
> at
> org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:616)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:540)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:488)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
> ... 20 more
> 2012-04-09 10:42:24,336 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
> 2012-04-09 10:42:24,336 DEBUG org.apache.hadoop.hbase.m
[jira] [Commented] (HBASE-5857) RIT map in RS not getting cleared while region opening
[
https://issues.apache.org/jira/browse/HBASE-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268878#comment-13268878
]
Hudson commented on HBASE-5857:
---
Integrated in HBase-0.92-security #106 (See
[https://builds.apache.org/job/HBase-0.92-security/106/])
HBASE-5857 RIT map in RS not getting cleared while region opening (Chinna
Rao) (Revision 1329510)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java
> RIT map in RS not getting cleared while region opening
> --
>
> Key: HBASE-5857
> URL: https://issues.apache.org/jira/browse/HBASE-5857
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.6
>Reporter: Chinna Rao Lalam
>Assignee: Chinna Rao Lalam
> Fix For: 0.92.2, 0.94.0, 0.96.0
>
> Attachments: HBASE-5857_0.92.1.patch, HBASE-5857_0.92.patch,
> HBASE-5857_0.94.1.patch, HBASE-5857_94.patch, HBASE-5857_trunk.1.patch,
> HBASE-5857_trunk.1.patch, HBASE-5857_trunk.2.patch, HBASE-5857_trunk.patch
>
>
> While opening the region in RS after adding the region to
> regionsInTransitionInRS if tableDescriptors.get() throws exception the region
> wont be cleared from regionsInTransitionInRS. So next time if it tries to
> open the region in the same RS it will throw the
> RegionAlreadyInTransitionException.
> if swap the below statement this issue wont come.
> {code}
> this.regionsInTransitionInRS.putIfAbsent(region.getEncodedNameAsBytes(),true);
> HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5871) Usability regression, we don't parse compression algos anymore
[
https://issues.apache.org/jira/browse/HBASE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268873#comment-13268873
]
Hudson commented on HBASE-5871:
---
Integrated in HBase-0.92-security #106 (See
[https://builds.apache.org/job/HBase-0.92-security/106/])
HBASE-5871 Usability regression, we don't parse compression algos anymore
(Revision 1330122)
Result = SUCCESS
larsh :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/ruby/hbase/admin.rb
> Usability regression, we don't parse compression algos anymore
> --
>
> Key: HBASE-5871
> URL: https://issues.apache.org/jira/browse/HBASE-5871
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1
>Reporter: Jean-Daniel Cryans
>Assignee: Lars Hofhansl
>Priority: Critical
> Fix For: 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5871-0.92.txt, 5871-0.94.txt, 5871-trunk.txt
>
>
> It seems that string with 0.92.0 we can't create tables in the shell by
> specifying "lzo" anymore. I remember we used to do better parsing than that,
> but right now if you follow the wiki doing this:
> bq. create 'mytable', {NAME=>'colfam:', COMPRESSION=>'lzo'}
> You'll get:
> bq. ERROR: java.lang.IllegalArgumentException: No enum const class
> org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.lzo
> Bad for usability.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5712) Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
[ https://issues.apache.org/jira/browse/HBASE-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268875#comment-13268875 ] Hudson commented on HBASE-5712: --- Integrated in HBase-0.92-security #106 (See [https://builds.apache.org/job/HBase-0.92-security/106/]) HBASE-5712 Parallelize load of .regioninfo files in diagnostic/repair portion of hbck (Revision 1332070) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java > Parallelize load of .regioninfo files in diagnostic/repair portion of hbck. > --- > > Key: HBASE-5712 > URL: https://issues.apache.org/jira/browse/HBASE-5712 > Project: HBase > Issue Type: Improvement > Components: hbck >Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > > Attachments: hbase-5712-90-v2.patch, hbase-5712-90.patch, > hbase-5712-v2.patch, hbase-5712.patch > > > On heavily loaded hdfs's some dfs nodes may not respond quickly and backs off > for 60s before attempting to read data from another datanode. Portions of > the information gathered from hdfs (.regioninfo files) are loaded serially. > With HBase with clusters with 100's, or 1000's, or 1's regions > encountering these 60s delay blocks progress and can be very painful. > There is already some parallelization of portions of the hdfs information > load operations and the goal here is move the reading of .regioninfos into > the parallelized sections.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268874#comment-13268874
]
Hudson commented on HBASE-5873:
---
Integrated in HBase-0.92-security #106 (See
[https://builds.apache.org/job/HBase-0.92-security/106/])
HBASE-5873 TimeOut Monitor thread should be started after atleast one
region server registers. (Revision 1330558)
Result = SUCCESS
larsh :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> TimeOut Monitor thread should be started after atleast one region server
> registers.
> ---
>
> Key: HBASE-5873
> URL: https://issues.apache.org/jira/browse/HBASE-5873
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.6
>Reporter: ramkrishna.s.vasudevan
>Assignee: rajeshbabu
>Priority: Minor
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5873-trunk.txt, HBASE-5873.patch
>
>
> Currently timeout monitor thread is started even before the region server has
> registered with the master.
> In timeout monitor we depend on the region server to be online
> {code}
> boolean allRSsOffline = this.serverManager.getOnlineServersList().
> isEmpty();
> {code}
> Now when the master starts up it sees there are no online servers and hence
> sets
> allRSsOffline to true.
> {code}
> setAllRegionServersOffline(allRSsOffline);
> {code}
> So this.allRegionServersOffline is also true.
> By this time an RS has come up,
> Now timeout comes up again (after 10secs) in the next cycle he sees
> allRSsOffline as false.
> Hence
> {code}
> else if (this.allRegionServersOffline && !allRSsOffline) {
> // if some RSs just came back online, we can start the
> // the assignment right away
> actOnTimeOut(regionState);
> {code}
> This condition makes him to take action based on timeout.
> Because of this even if one Region assignment of ROOT is going on, this piece
> of code triggers another assignment and thus we get RegionAlreadyinTransition
> Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5893) Allow spaces in coprocessor conf (aka trim() className)
[
https://issues.apache.org/jira/browse/HBASE-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268872#comment-13268872
]
Hudson commented on HBASE-5893:
---
Integrated in HBase-0.92-security #106 (See
[https://builds.apache.org/job/HBase-0.92-security/106/])
HBASE-5893 Allow spaces in coprocessor conf (aka trim() className)
(Revision 1331641)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
> Allow spaces in coprocessor conf (aka trim() className)
> ---
>
> Key: HBASE-5893
> URL: https://issues.apache.org/jira/browse/HBASE-5893
> Project: HBase
> Issue Type: Bug
> Components: coprocessors
>Affects Versions: 0.92.1, 0.94.0, 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
>Priority: Minor
> Labels: conf, coprocessors
> Fix For: 0.92.2, 0.94.0
>
> Attachments: HBASE-5893-v0.patch
>
>
> This is annoying especially for coprocessors where you've long class name.
> but maybe is a bug of Configuration.getStrings() that doesn't trim each
> string.
> When you've comma separated values like in the coprocessors case, you've to
> pack together your values without spaces ("v1,v2,v3,...") otherwise the
> coprocessor is not loaded because the class name with spaces is not found.
> {code}
>
>hbase.coprocessor.master.classes
>
> org.apache.hadoop.hbase.security.token.TokenProvider,
> org.apache.hadoop.hbase.security.access.AccessController
>
>
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268866#comment-13268866
]
Hudson commented on HBASE-5611:
---
Integrated in HBase-0.92-security #106 (See
[https://builds.apache.org/job/HBase-0.92-security/106/])
HBASE-5611 Replayed edits from regions that failed to open during recovery
aren't removed from the global MemStore size (Jieshan) (Revision 1331684)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerAccounting.java
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java
> Replayed edits from regions that failed to open during recovery aren't
> removed from the global MemStore size
>
>
> Key: HBASE-5611
> URL: https://issues.apache.org/jira/browse/HBASE-5611
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.90.6
>Reporter: Jean-Daniel Cryans
>Assignee: Jieshan Bean
>Priority: Critical
> Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
>
> Attachments: 5611-94-v2.txt, 5611-94.addendum, HBASE-5611-92.patch,
> HBASE-5611-94-minorchange.patch, HBASE-5611-trunk-v2-minorchange.patch
>
>
> This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
> it's still possible to hit it if a region fails to open for more obscure
> reasons like HDFS errors.
> Consider a region that just went through distributed splitting and that's now
> being opened by a new RS. The first thing it does is to read the recovery
> files and put the edits in the {{MemStores}}. If this process takes a long
> time, the master will move that region away. At that point the edits are
> still accounted for in the global {{MemStore}} size but they are dropped when
> the {{HRegion}} gets cleaned up. It's completely invisible until the
> {{MemStoreFlusher}} needs to force flush a region and that none of them have
> edits:
> {noformat}
> 2012-03-21 00:33:39,303 DEBUG
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
> because memory above low water=5.9g
> 2012-03-21 00:33:39,303 ERROR
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
> for entry null
> java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:129)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> The {{null}} here is a region. In my case I had so many edits in the
> {{MemStore}} during recovery that I'm over the low barrier although in fact
> I'm at 0. It happened yesterday and it still printing this out.
> To fix this we need to be able to decrease the global {{MemStore}} size when
> the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5897) prePut coprocessor hook causing substantial CPU usage
[ https://issues.apache.org/jira/browse/HBASE-5897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268870#comment-13268870 ] Hudson commented on HBASE-5897: --- Integrated in HBase-0.92-security #106 (See [https://builds.apache.org/job/HBase-0.92-security/106/]) HBASE-5897 prePut coprocessor hook causing substantial CPU usage (Revision 1332816) Result = SUCCESS larsh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > prePut coprocessor hook causing substantial CPU usage > - > > Key: HBASE-5897 > URL: https://issues.apache.org/jira/browse/HBASE-5897 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Critical > Fix For: 0.92.2, 0.94.0, 0.96.0 > > Attachments: 5897-simple.txt, 5897-v2.txt, 5897-v3-0.92.txt, > 5897-v3-0.94.txt, 5897-v3.txt, hbase-5897.txt, > testRegionServerCoprocessorExceptionWithRemove.stack > > > I was running an insert workload against trunk under oprofile and saw that a > significant portion of CPU usage was going to calling the "prePut" > coprocessor hook inside doMiniBatchPut, even though I don't have any > coprocessors installed. I ran a million-row insert and collected CPU time > spent in the RS after commenting out the preput hook, and found CPU usage > reduced by 33%. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5887) Make TestAcidGuarantees usable for system testing.
[ https://issues.apache.org/jira/browse/HBASE-5887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268869#comment-13268869 ] Hudson commented on HBASE-5887: --- Integrated in HBase-0.92-security #106 (See [https://builds.apache.org/job/HBase-0.92-security/106/]) HBASE-5887 Make TestAcidGuarantees usable for system testing (Revision 1333783) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java > Make TestAcidGuarantees usable for system testing. > -- > > Key: HBASE-5887 > URL: https://issues.apache.org/jira/browse/HBASE-5887 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.0, 0.92.1, 0.94.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: hbase-5887-92.patch, hbase-5887.patch > > > Currently, the TestAcidGuarantees run via main() will always abort with an > NPE because it digs into a non-existant HBaseTestingUtility for a flusher > thread. We should tool this up so that it works properly from the command > line. This would be a very useful long running test when used in conjunction > with fault injections to verify row acid properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5908) TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog
[ https://issues.apache.org/jira/browse/HBASE-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268867#comment-13268867 ] Hudson commented on HBASE-5908: --- Integrated in HBase-0.92-security #106 (See [https://builds.apache.org/job/HBase-0.92-security/106/]) HBASE-5908. TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog. Contributed by Gregory Chanan. (Revision 1332493) Result = SUCCESS todd : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java > TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use > append to corrupt the HLog > > > Key: HBASE-5908 > URL: https://issues.apache.org/jira/browse/HBASE-5908 > Project: HBase > Issue Type: Bug > Components: test, wal >Affects Versions: 0.96.0 >Reporter: Gregory Chanan >Assignee: Gregory Chanan >Priority: Minor > Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 > > Attachments: HBASE-5908-trunk.patch > > > TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses fails against > a version of hadoop with https://issues.apache.org/jira/browse/HADOOP-8230 > The failure: > "java.io.IOException: Append is not supported. Please see the > dfs.support.append configuration parameter." > Instead of using append, we can probably just: > - copy over the contents to a new file > - append the garbage to the new file > - copy back to the old file -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5928) Hbck shouldn't npe when there are no tables.
[ https://issues.apache.org/jira/browse/HBASE-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268868#comment-13268868 ] Hudson commented on HBASE-5928: --- Integrated in HBase-0.92-security #106 (See [https://builds.apache.org/job/HBase-0.92-security/106/]) HBASE-5928 Hbck shouldn't npe when there are no tables (Elliott Clark) (Revision 1333693) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java > Hbck shouldn't npe when there are no tables. > > > Key: HBASE-5928 > URL: https://issues.apache.org/jira/browse/HBASE-5928 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Minor > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-5928-0.patch, HBASE-5928-1.patch, > hbase-5928-94.patch > > > hbase fsck errors out when there are no tables. > Exception in thread "main" java.lang.NullPointerException > at > org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:560) > at > org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:346) > at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:382) > at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3120) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5945) Reduce buffer copies in IPC server response path
[
https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268863#comment-13268863
]
Zhihong Yu commented on HBASE-5945:
---
Interesting finding.
Can I assume that the number of occurrences of Message's is higher than that of
Result's ?
If so, the following check can be lifted above that for Result.class:
{code}
+if (instance instanceof Message) {
{code}
> Reduce buffer copies in IPC server response path
>
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
>Affects Versions: 0.96.0
>Reporter: Todd Lipcon
>Assignee: Todd Lipcon
>Priority: Minor
> Attachments: buffer-copies.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies.
> This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5295) Improve the Thrift API to switch on/off writing to wal for Mutations
[ https://issues.apache.org/jira/browse/HBASE-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5295: --- Attachment: D3015.2.patch mbautin updated the revision "[jira] [HBASE-5295] [89-fb] Improve the Thrift API to switch on/off writing to wal for Mutations". Reviewers: Kannan, dhruba, sc, JIRA Addressing Kannan's comment. Testing both API methods in the unit test. REVISION DETAIL https://reviews.facebook.net/D3015 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/KeyValue.java src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java src/main/java/org/apache/hadoop/hbase/thrift/ThriftUtilities.java src/main/java/org/apache/hadoop/hbase/thrift/generated/Mutation.java src/main/java/org/apache/hadoop/hbase/util/Bytes.java src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift src/test/java/org/apache/hadoop/hbase/HTestConst.java src/test/java/org/apache/hadoop/hbase/thrift/TestMutationWriteToWAL.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServer.java src/test/java/org/apache/hadoop/hbase/thrift/TestThriftServerLegacy.java > Improve the Thrift API to switch on/off writing to wal for Mutations > - > > Key: HBASE-5295 > URL: https://issues.apache.org/jira/browse/HBASE-5295 > Project: HBase > Issue Type: Improvement > Components: thrift >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: D1515.1.patch, D1515.1.patch, D1515.1.patch, > D1515.1.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, > D3015.1.patch, D3015.2.patch > > > The thrift api currently does not support switching off updating wal for > Puts/Deletes. Support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5945) Reduce buffer copies in IPC server response path
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HBASE-5945: --- Attachment: buffer-copies.txt Here's a first pass. I don't have great benchmark results, but it looked from my "jstat -gcutil" output like this cut the amount of allocation by a factor of 2.5x or so for a benchmark that returned large responses. Can probably get rid of at least one more copy in there with a bit more trickery. > Reduce buffer copies in IPC server response path > > > Key: HBASE-5945 > URL: https://issues.apache.org/jira/browse/HBASE-5945 > Project: HBase > Issue Type: Improvement > Components: ipc >Affects Versions: 0.96.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Attachments: buffer-copies.txt > > > The new PB code is sloppy with buffers and makes several needless copies. > This increases GC time a lot. A few simple changes can cut this back down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5945) Reduce buffer copies in IPC server response path
Todd Lipcon created HBASE-5945: -- Summary: Reduce buffer copies in IPC server response path Key: HBASE-5945 URL: https://issues.apache.org/jira/browse/HBASE-5945 Project: HBase Issue Type: Improvement Components: ipc Affects Versions: 0.96.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5883) Backup master is going down due to connection refused exception
[
https://issues.apache.org/jira/browse/HBASE-5883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268839#comment-13268839
]
Hudson commented on HBASE-5883:
---
Integrated in HBase-0.94-security #26 (See
[https://builds.apache.org/job/HBase-0.94-security/26/])
HBASE-5883 Backup master is going down due to connection refused exception
(Jieshan) (Revision 1333533)
Result = SUCCESS
tedyu :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
> Backup master is going down due to connection refused exception
> ---
>
> Key: HBASE-5883
> URL: https://issues.apache.org/jira/browse/HBASE-5883
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.90.6, 0.92.1, 0.94.0
>Reporter: Gopinathan A
>Assignee: Jieshan Bean
> Fix For: 0.96.0, 0.94.1
>
> Attachments: HBASE-5883-90-addendum.patch, HBASE-5883-90.patch,
> HBASE-5883-92-addendum.patch, HBASE-5883-92.patch,
> HBASE-5883-94-addendum.patch, HBASE-5883-94.patch,
> HBASE-5883-trunk-addendum.patch, HBASE-5883-trunk.patch
>
>
> The active master node network was down for some time (This node contains
> Master,DN,ZK,RS). Here backup node got
> notification, and started to became active. Immedietly backup node got
> aborted with the below exception.
> {noformat}
> 2012-04-09 10:42:24,270 INFO org.apache.hadoop.hbase.master.SplitLogManager:
> finished splitting (more than or equal to) 861248320 bytes in 4 log files in
> [hdfs://192.168.47.205:9000/hbase/.logs/HOST-192-168-47-202,60020,1333715537172-splitting]
> in 26374ms
> 2012-04-09 10:42:24,316 FATAL org.apache.hadoop.hbase.master.HMaster: Master
> server abort: loaded coprocessors are: []
> 2012-04-09 10:42:24,333 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unhandled exception. Starting shutdown.
> java.io.IOException: java.net.ConnectException: Connection refused
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:375)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
> at $Proxy13.getProtocolVersion(Unknown Source)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1276)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1233)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1220)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.getCachedConnection(CatalogTracker.java:569)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.getRootServerConnection(CatalogTracker.java:369)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.waitForRootServerConnection(CatalogTracker.java:353)
> at
> org.apache.hadoop.hbase.catalog.CatalogTracker.verifyRootRegionLocation(CatalogTracker.java:660)
> at
> org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:616)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:540)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:488)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
> ... 20 more
> 2012-04-09 10:42:24,336 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
> 2012-04-09 10:42:24,336 DEBUG org.apache.hadoop.hbase.master.HMaster:
> Stopping service thr
[jira] [Commented] (HBASE-5887) Make TestAcidGuarantees usable for system testing.
[ https://issues.apache.org/jira/browse/HBASE-5887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268837#comment-13268837 ] Hudson commented on HBASE-5887: --- Integrated in HBase-0.94-security #26 (See [https://builds.apache.org/job/HBase-0.94-security/26/]) HBASE-5887 Make TestAcidGuarantees usable for system testing (Revision 1333784) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java > Make TestAcidGuarantees usable for system testing. > -- > > Key: HBASE-5887 > URL: https://issues.apache.org/jira/browse/HBASE-5887 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.92.0, 0.92.1, 0.94.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: hbase-5887-92.patch, hbase-5887.patch > > > Currently, the TestAcidGuarantees run via main() will always abort with an > NPE because it digs into a non-existant HBaseTestingUtility for a flusher > thread. We should tool this up so that it works properly from the command > line. This would be a very useful long running test when used in conjunction > with fault injections to verify row acid properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2214) Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly
[ https://issues.apache.org/jira/browse/HBASE-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268838#comment-13268838 ] Hudson commented on HBASE-2214: --- Integrated in HBase-0.94-security #26 (See [https://builds.apache.org/job/HBase-0.94-security/26/]) HBASE-2214 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly (Ferdy Galema) (Revision 1333157) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java > Do HBASE-1996 -- setting size to return in scan rather than count of rows -- > properly > - > > Key: HBASE-2214 > URL: https://issues.apache.org/jira/browse/HBASE-2214 > Project: HBase > Issue Type: New Feature >Reporter: stack >Assignee: Ferdy Galema > Fix For: 0.96.0, 0.94.1 > > Attachments: HBASE-2214-0.94-v2.txt, HBASE-2214-0.94-v3.txt, > HBASE-2214-0.94.txt, HBASE-2214-v4.txt, HBASE-2214-v5.txt, HBASE-2214-v6.txt, > HBASE-2214-v7.txt, HBASE-2214_with_broken_TestShell.txt > > > The notion that you set size rather than row count specifying how many rows a > scanner should return in each cycle was raised over in hbase-1966. Its a > good one making hbase "regular" though the data under it may vary. > HBase-1966 was committed but the patch was constrained by the fact that it > needed to not change RPC interface. This issue is about doing hbase-1966 for > 0.21 in a clean, unconstrained way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5913) Speed up the full scan of META
[ https://issues.apache.org/jira/browse/HBASE-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268836#comment-13268836 ] Hudson commented on HBASE-5913: --- Integrated in HBase-0.94-security #26 (See [https://builds.apache.org/job/HBase-0.94-security/26/]) HBASE-5913 Speed up the full scan of META (Revision 115) Result = SUCCESS larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java > Speed up the full scan of META > -- > > Key: HBASE-5913 > URL: https://issues.apache.org/jira/browse/HBASE-5913 > Project: HBase > Issue Type: Improvement >Reporter: chunhui shen >Assignee: chunhui shen > Fix For: 0.96.0, 0.94.1 > > Attachments: 5913-v2.txt, HBASE-5913.patch > > > In the master, we will do the full scan of META in some situations > for example, > 1.master start up > 2.CatalogJanitor do the full scan per 5 mins > 3.ServerShutdownHandler, getServerUserRegions for dead server. > For the online applications, we should try the best to reduce the process > time of ServerShutdownHandler in the situation 3. > However, we found MetaReader#getServerUserRegions take 14mins for 10w regions > in our production environment. > And it is caused by two reasons: > The first, we don't use cache and get one row per next() when fully scan > .META. > The second, "hbase.ipc.client.tcpnodelay" is false as default, and in our > environment it take 40ms for per next() (It is related to the length of row > in the .META. , if someone also found, could try to set it true) > For this issue, I think we could set the caching when do the full scan of META -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5902) Some scripts are not executable
[ https://issues.apache.org/jira/browse/HBASE-5902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268834#comment-13268834 ] Hudson commented on HBASE-5902: --- Integrated in HBase-0.94-security #26 (See [https://builds.apache.org/job/HBase-0.94-security/26/]) HBASE-5902 Some scripts are not executable (Revision 1334020) Result = SUCCESS stack : Files : * /hbase/branches/0.94/bin/graceful_stop.sh * /hbase/branches/0.94/bin/local-master-backup.sh * /hbase/branches/0.94/bin/local-regionservers.sh > Some scripts are not executable > --- > > Key: HBASE-5902 > URL: https://issues.apache.org/jira/browse/HBASE-5902 > Project: HBase > Issue Type: Bug > Components: scripts >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Trivial > Fix For: 0.96.0, 0.94.1 > > Attachments: 5902.v1.patch, 5902v2.txt > > > -rw-rw-r-- graceful_stop.sh > -rw-rw-r-- hbase-config.sh > -rw-rw-r-- local-master-backup.sh > -rw-rw-r-- local-regionservers.sh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5876) TestImportExport has been failing against hadoop 0.23 profile
[ https://issues.apache.org/jira/browse/HBASE-5876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268833#comment-13268833 ] Hudson commented on HBASE-5876: --- Integrated in HBase-0.94-security #26 (See [https://builds.apache.org/job/HBase-0.94-security/26/]) HBASE-5876 TestImportExport has been failing against hadoop 0.23 profile (Revision 1333777) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java > TestImportExport has been failing against hadoop 0.23 profile > - > > Key: HBASE-5876 > URL: https://issues.apache.org/jira/browse/HBASE-5876 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0, 0.96.0 >Reporter: Zhihong Yu >Assignee: Jonathan Hsieh > Fix For: 0.96.0, 0.94.1 > > Attachments: hbase-5876-94.patch, hbase-5876-v2.patch, > hbase-5876.patch > > > TestImportExport has been failing against hadoop 0.23 profile -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5928) Hbck shouldn't npe when there are no tables.
[ https://issues.apache.org/jira/browse/HBASE-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268832#comment-13268832 ] Hudson commented on HBASE-5928: --- Integrated in HBase-0.94-security #26 (See [https://builds.apache.org/job/HBase-0.94-security/26/]) HBASE-5928 Hbck shouldn't npe when there are no tables (Elliott Clark) (Revision 1333692) Result = SUCCESS jmhsieh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java > Hbck shouldn't npe when there are no tables. > > > Key: HBASE-5928 > URL: https://issues.apache.org/jira/browse/HBASE-5928 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.1, 0.94.0, 0.96.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Minor > Fix For: 0.92.2, 0.96.0, 0.94.1 > > Attachments: HBASE-5928-0.patch, HBASE-5928-1.patch, > hbase-5928-94.patch > > > hbase fsck errors out when there are no tables. > Exception in thread "main" java.lang.NullPointerException > at > org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:560) > at > org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:346) > at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:382) > at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3120) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5886) Add new metric for possible data loss due to puts without WAL
[ https://issues.apache.org/jira/browse/HBASE-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268835#comment-13268835 ] Hudson commented on HBASE-5886: --- Integrated in HBase-0.94-security #26 (See [https://builds.apache.org/job/HBase-0.94-security/26/]) HBASE-5886 Add new metric for possible data loss due to puts without WAL (Revision 1333689) Result = SUCCESS larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java > Add new metric for possible data loss due to puts without WAL > -- > > Key: HBASE-5886 > URL: https://issues.apache.org/jira/browse/HBASE-5886 > Project: HBase > Issue Type: New Feature > Components: metrics, regionserver >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Labels: metrics > Fix For: 0.96.0, 0.94.1 > > Attachments: 5886-0.94.patch, HBASE-5886-v0.patch, > HBASE-5886-v1.patch, HBASE-5886-v2.patch, HBASE-5886-v3.patch, > HBASE-5886-v4.patch > > > Add a metrics to keep track of puts without WAL and possible data loss size. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1996) Configure scanner buffer in bytes instead of number of rows
[ https://issues.apache.org/jira/browse/HBASE-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268831#comment-13268831 ] Hudson commented on HBASE-1996: --- Integrated in HBase-0.94-security #26 (See [https://builds.apache.org/job/HBase-0.94-security/26/]) HBASE-2214 Do HBASE-1996 -- setting size to return in scan rather than count of rows -- properly (Ferdy Galema) (Revision 1333157) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/Scan.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionScanner.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java > Configure scanner buffer in bytes instead of number of rows > --- > > Key: HBASE-1996 > URL: https://issues.apache.org/jira/browse/HBASE-1996 > Project: HBase > Issue Type: Improvement >Reporter: Dave Latham >Assignee: Dave Latham > Fix For: 0.90.0 > > Attachments: 1966.patch, 1996-0.20.3-v2.patch, 1996-0.20.3-v3.patch, > 1996-0.20.3.patch > > > Currently, the default scanner fetches a single row at a time. This makes > for very slow scans on tables where the rows are not large. You can change > the setting for an HTable instance or for each Scan. > It would be better to have a default that performs reasonably well so that > people stop running into slow scans because they are evaluating HBase, aren't > familiar with the setting, or simply forgot. Unfortunately, if we increase > the value of the current setting, then we run the risk of running OOM for > tables with large rows. Let's change the setting so that it works with a > size in bytes, rather than in rows. This will allow us to set a reasonable > default so that tables with small rows will scan performantly and tables with > large rows will not run OOM. > Note that the case is very similar to table writes as well. When disabling > auto flush, we buffer a list of Put's to commit at once. That buffer is > measured in bytes, so that a small number of large Puts or a lot of small > Puts can each fit in a single flush. If that buffer were measured in number > of Put's it would have the same problem that we have for the scan buffer, and > we wouldn't be able to set a good default value for tables with different > size rows. Changing the scan buffer to be configured like the write buffer > will make it more consistent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268816#comment-13268816 ] Jimmy Xiang commented on HBASE-5889: :) I thought to create new util class for client side. I used protobuf util instead since it is for the client side. For the sever side case you referred to (openRegion returns a OpenRegionState), I moved it out of the util to the original place since it's not shared. I changed the shared one so that it doesn't refer to region server directly. > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5944) Convert rs, shutdown, and table dir content to pb
stack created HBASE-5944: Summary: Convert rs, shutdown, and table dir content to pb Key: HBASE-5944 URL: https://issues.apache.org/jira/browse/HBASE-5944 Project: HBase Issue Type: Task Reporter: stack -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5446) Replace serialization for client-facing ZK data with PBs
[
https://issues.apache.org/jira/browse/HBASE-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268810#comment-13268810
]
stack commented on HBASE-5446:
--
Here's whats up in zk as of trunk today:
{code}
[splitlog, unassigned, root-region-server, rs, backup-masters, draining, table,
master, shutdown, hbaseid]
{code}
The 'rs' has nothing in it. Might add something. Leaves shutdown and content
of table dir to do.
> Replace serialization for client-facing ZK data with PBs
>
>
> Key: HBASE-5446
> URL: https://issues.apache.org/jira/browse/HBASE-5446
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
>Reporter: Todd Lipcon
>Assignee: stack
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268806#comment-13268806 ] stack commented on HBASE-5889: -- I can't tell highlevel what you've done Jimmy? I see RegionServerUtil is missing. Where do you spread its content? I don't see obvious new util classes. Thanks boss. > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268805#comment-13268805 ] Hadoop QA commented on HBASE-5889: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525693/hbase_5889_v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 60 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1777//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1777//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1777//console This message is automatically generated. > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5943) Set tcpnodelay as default
[
https://issues.apache.org/jira/browse/HBASE-5943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268792#comment-13268792
]
Hadoop QA commented on HBASE-5943:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12525685/nodelay.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 hadoop23. The patch compiles against the hadoop 0.23.x profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.master.TestMasterNoCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1776//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1776//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1776//console
This message is automatically generated.
> Set tcpnodelay as default
> -
>
> Key: HBASE-5943
> URL: https://issues.apache.org/jira/browse/HBASE-5943
> Project: HBase
> Issue Type: Improvement
>Reporter: stack
>Assignee: stack
> Attachments: nodelay.txt
>
>
> From IRC:
> {code}
> 14:46 < tlipcon> i turned off the FirstKeyOnlyFilter on the shell count
> command
> 14:46 < tlipcon> thgouhtput dropped like carzy... looked at strace, and saw
> 40ms delays
> 14:46 < tlipcon> turned on ipc.server.nodelay or whaever and it got fast again
> 14:46 < tlipcon> we should really ship that by default on
> 14:46 < tlipcon> there are probably people getting screwed by it
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5699) Run with > 1 WAL in HRegionServer
[ https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268789#comment-13268789 ] Zhihong Yu commented on HBASE-5699: --- bq. If you have multiple hlogs do you use a different hlog in different regions? Correct. I have to go through legal procedure at my employer before disclosing my patch. > Run with > 1 WAL in HRegionServer > - > > Key: HBASE-5699 > URL: https://issues.apache.org/jira/browse/HBASE-5699 > Project: HBase > Issue Type: Improvement >Reporter: binlijin >Assignee: Li Pi > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5876) TestImportExport has been failing against hadoop 0.23 profile
[ https://issues.apache.org/jira/browse/HBASE-5876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reopened HBASE-5876: --- Ted noticed that the test is still failing. Reopening and reinvestigating. > TestImportExport has been failing against hadoop 0.23 profile > - > > Key: HBASE-5876 > URL: https://issues.apache.org/jira/browse/HBASE-5876 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0, 0.96.0 >Reporter: Zhihong Yu >Assignee: Jonathan Hsieh > Fix For: 0.96.0, 0.94.1 > > Attachments: hbase-5876-94.patch, hbase-5876-v2.patch, > hbase-5876.patch > > > TestImportExport has been failing against hadoop 0.23 profile -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5699) Run with > 1 WAL in HRegionServer
[ https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268786#comment-13268786 ] Jonathan Hsieh commented on HBASE-5699: --- Zhihong, I'm curious to learn about the approach you have taken in the prototype that you have. Is it on github somewhere perhaps? If you have multiple hlogs do you use a different hlog in different regions? Do you have a shim that looks like an hlog but has two hlogs inside it (as opposed to hdfs file handles)? > Run with > 1 WAL in HRegionServer > - > > Key: HBASE-5699 > URL: https://issues.apache.org/jira/browse/HBASE-5699 > Project: HBase > Issue Type: Improvement >Reporter: binlijin >Assignee: Li Pi > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268785#comment-13268785 ] jirapos...@reviews.apache.org commented on HBASE-5889: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4993/ --- (Updated 2012-05-04 22:54:46.398933) Review request for hbase. Summary --- Removed HRegionInterface, and cleaned up the HRegionServer, moved pb code from RegionServer back to HRegionServer. The goal is to avoid two copies of region server code to maintain, and make it possible to avoid data type conversion in the sever side. Fixed some unit tests. Now all region server unit tests test the new pb functions. Enhanced getServerInfo so that it returns the webui port too. This addresses bug HBASE-5889. https://issues.apache.org/jira/browse/HBASE-5889 Diffs (updated) - conf/hbase-policy.xml e45f23c security/src/main/java/org/apache/hadoop/hbase/security/HBasePolicyProvider.java fda40cc src/main/jamon/org/apache/hadoop/hbase/tmpl/regionserver/RSStatusTmpl.jamon 87f04f4 src/main/java/org/apache/hadoop/hbase/HConstants.java a9d80a0 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java e3912c2 src/main/java/org/apache/hadoop/hbase/ipc/HBaseRpcMetrics.java fc9176d src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 757f98e src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java cd9b528 src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java 79d5fdd src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 6ba8ab0 src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 212ee3e src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java d1e0993 src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java 81603af src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java fbf0127 src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java db1333b src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java ae2094d src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java 8b45f03 src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java 827fb23 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8c8381b src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 4cb070e src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java c2c89ea src/main/protobuf/Admin.proto 2ad6fb0 src/main/protobuf/RPC.proto 105fb3f src/main/resources/hbase-default.xml f54b345 src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java a1992c3 src/test/java/org/apache/hadoop/hbase/TestGlobalMemStoreSize.java ad77e0a src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 5574b7f src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 3dfc94e src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java 42092b7 src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java c270e28 src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java c36272f src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java bdec3ee src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java 7dbba66 src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java 3acb988 src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java eb546a5 src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java ceba5cd src/test/java/org/apache/hadoop/hbase/master/TestMasterRestartAfterDisablingTable.java ec08b17 src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 30c6cf1 src/test/java/org/apache/hadoop/hbase/master/TestZKBasedOpenCloseRegion.java 8c3f67e src/test/java/org/apache/hadoop/hbase/regionserver/TestEndToEndSplitTransaction.java 7bfe4cd src/test/java/org/apache/hadoop/hbase/regionserver/TestRSStatusServlet.java ffce7e8 src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java aa5ca37 src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 65fa948 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java fc4a0a7 Diff: https://reviews.apache.org/r/4993/diff Testing --- All regular and security profile tests are green before I rebased to the latest today. Thanks, Jimmy > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issu
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268781#comment-13268781 ] Jimmy Xiang commented on HBASE-5889: After a second thought, I used protobuf util instead since it is for the client side. For the sever side case you referred to (openRegion returns a OpenRegionState), I moved it out of the util. > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5889: --- Status: Patch Available (was: Open) > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5813) Retry immediately after a NotServingRegionException in a multiput
[ https://issues.apache.org/jira/browse/HBASE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268780#comment-13268780 ] Phabricator commented on HBASE-5813: Kannan has commented on the revision "[jira] [HBASE-5813] [89-fb] Retry immediately after a NotServingRegionException in a multiput". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java:1707 for efficiency, let's populate this map only if msWaitOnSameRegionLoc > 0. That for most common cases, i.e. the 1 attempt case, we avoid building this map. src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java:1765 could you explain this change: why don't we add these failed requests to the "failed" list? REVISION DETAIL https://reviews.facebook.net/D2847 > Retry immediately after a NotServingRegionException in a multiput > - > > Key: HBASE-5813 > URL: https://issues.apache.org/jira/browse/HBASE-5813 > Project: HBase > Issue Type: Improvement >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin > Attachments: D2847.1.patch, D2847.10.patch, D2847.11.patch, > D2847.12.patch, D2847.2.patch, D2847.3.patch, D2847.4.patch, D2847.5.patch, > D2847.6.patch, D2847.7.patch, D2847.8.patch, D2847.9.patch > > > After we get some errors in a multiput we invalidate the region location > cache and wait for the configured time interval according to the backoff > policy. However, if all "errors" in multiput processing were > NotServingRegionExceptions, we don't really need to wait. We can retry > immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client
[ https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268778#comment-13268778 ] Hadoop QA commented on HBASE-5877: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525684/5877-v17.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1775//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1775//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1775//console This message is automatically generated. > When a query fails because the region has moved, let the regionserver return > the new address to the client > -- > > Key: HBASE-5877 > URL: https://issues.apache.org/jira/browse/HBASE-5877 > Project: HBase > Issue Type: Improvement > Components: client, master, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5877-v16.txt, 5877-v17.txt, 5877.v1.patch, > 5877.v12.patch, 5877.v15.patch, 5877.v6.patch > > > This is mainly useful when we do a rolling restart. This will decrease the > load on the master and the network load. > Note that a region is not immediately opened after a close. So: > - it seems preferable to wait before retrying on the other server. An > optimisation would be to have an heuristic depending on when the region was > closed. > - during a rolling restart, the server moves the regions then stops. So we > may have failures when the server is stopped, and this patch won't help. > The implementation in the first patch does: > - on the region move, there is an added parameter on the regionserver#close > to say where we are sending the region > - the regionserver keeps a list of what was moved. Each entry is kept 100 > seconds. > - the regionserver sends a specific exception when it receives a query on a > moved region. This exception contains the new address. > - the client analyses the exeptions and update its cache accordingly... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5889: --- Attachment: hbase_5889_v4.patch > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5889: --- Status: Open (was: Patch Available) > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch, hbase_5889_v4.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5876) TestImportExport has been failing against hadoop 0.23 profile
[ https://issues.apache.org/jira/browse/HBASE-5876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268773#comment-13268773 ] Zhihong Yu commented on HBASE-5876: --- Strange, I saw this after the fix went in: https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-23/167/testReport/junit/org.apache.hadoop.hbase.mapreduce/TestImportExport/testSimpleCase/ > TestImportExport has been failing against hadoop 0.23 profile > - > > Key: HBASE-5876 > URL: https://issues.apache.org/jira/browse/HBASE-5876 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.0, 0.96.0 >Reporter: Zhihong Yu >Assignee: Jonathan Hsieh > Fix For: 0.96.0, 0.94.1 > > Attachments: hbase-5876-94.patch, hbase-5876-v2.patch, > hbase-5876.patch > > > TestImportExport has been failing against hadoop 0.23 profile -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5943) Set tcpnodelay as default
[
https://issues.apache.org/jira/browse/HBASE-5943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268747#comment-13268747
]
Todd Lipcon commented on HBASE-5943:
before doing this we should run a wireshark and make sure we're not sending too
many small packets, etc. See the discussion in HADOOP-8069
> Set tcpnodelay as default
> -
>
> Key: HBASE-5943
> URL: https://issues.apache.org/jira/browse/HBASE-5943
> Project: HBase
> Issue Type: Improvement
>Reporter: stack
>Assignee: stack
> Attachments: nodelay.txt
>
>
> From IRC:
> {code}
> 14:46 < tlipcon> i turned off the FirstKeyOnlyFilter on the shell count
> command
> 14:46 < tlipcon> thgouhtput dropped like carzy... looked at strace, and saw
> 40ms delays
> 14:46 < tlipcon> turned on ipc.server.nodelay or whaever and it got fast again
> 14:46 < tlipcon> we should really ship that by default on
> 14:46 < tlipcon> there are probably people getting screwed by it
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5943) Set tcpnodelay as default
stack created HBASE-5943:
Summary: Set tcpnodelay as default
Key: HBASE-5943
URL: https://issues.apache.org/jira/browse/HBASE-5943
Project: HBase
Issue Type: Improvement
Reporter: stack
Attachments: nodelay.txt
>From IRC:
{code}
14:46 < tlipcon> i turned off the FirstKeyOnlyFilter on the shell count command
14:46 < tlipcon> thgouhtput dropped like carzy... looked at strace, and saw
40ms delays
14:46 < tlipcon> turned on ipc.server.nodelay or whaever and it got fast again
14:46 < tlipcon> we should really ship that by default on
14:46 < tlipcon> there are probably people getting screwed by it
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5943) Set tcpnodelay as default
[
https://issues.apache.org/jira/browse/HBASE-5943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5943:
-
Attachment: nodelay.txt
> Set tcpnodelay as default
> -
>
> Key: HBASE-5943
> URL: https://issues.apache.org/jira/browse/HBASE-5943
> Project: HBase
> Issue Type: Improvement
>Reporter: stack
> Attachments: nodelay.txt
>
>
> From IRC:
> {code}
> 14:46 < tlipcon> i turned off the FirstKeyOnlyFilter on the shell count
> command
> 14:46 < tlipcon> thgouhtput dropped like carzy... looked at strace, and saw
> 40ms delays
> 14:46 < tlipcon> turned on ipc.server.nodelay or whaever and it got fast again
> 14:46 < tlipcon> we should really ship that by default on
> 14:46 < tlipcon> there are probably people getting screwed by it
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5943) Set tcpnodelay as default
[
https://issues.apache.org/jira/browse/HBASE-5943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5943:
-
Assignee: stack
Status: Patch Available (was: Open)
> Set tcpnodelay as default
> -
>
> Key: HBASE-5943
> URL: https://issues.apache.org/jira/browse/HBASE-5943
> Project: HBase
> Issue Type: Improvement
>Reporter: stack
>Assignee: stack
> Attachments: nodelay.txt
>
>
> From IRC:
> {code}
> 14:46 < tlipcon> i turned off the FirstKeyOnlyFilter on the shell count
> command
> 14:46 < tlipcon> thgouhtput dropped like carzy... looked at strace, and saw
> 40ms delays
> 14:46 < tlipcon> turned on ipc.server.nodelay or whaever and it got fast again
> 14:46 < tlipcon> we should really ship that by default on
> 14:46 < tlipcon> there are probably people getting screwed by it
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5922) HalfStoreFileReader seekBefore causes StackOverflowError
[ https://issues.apache.org/jira/browse/HBASE-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268739#comment-13268739 ] Hadoop QA commented on HBASE-5922: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525680/HBASE-5922.v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1774//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1774//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1774//console This message is automatically generated. > HalfStoreFileReader seekBefore causes StackOverflowError > > > Key: HBASE-5922 > URL: https://issues.apache.org/jira/browse/HBASE-5922 > Project: HBase > Issue Type: Bug > Components: client, io >Affects Versions: 0.90.0 > Environment: HBase 0.90.4 >Reporter: Nate Putnam >Assignee: Nate Putnam >Priority: Critical > Fix For: 0.90.0 > > Attachments: HBASE-5922.patch, HBASE-5922.patch, HBASE-5922.v2.patch, > HBASE-5922.v3.patch > > > Calling HRegionServer.getClosestRowBefore() can cause a stack overflow if the > underlying store file is a reference and the row key is in the bottom. > java.io.IOException: java.io.IOException: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:990) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:978) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1651) > at sun.reflect.GeneratedMethodAccessor174.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) > Caused by: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:147) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client
[ https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5877: -- Attachment: 5877-v17.txt I forgot to add the check in one place. Trying v17. > When a query fails because the region has moved, let the regionserver return > the new address to the client > -- > > Key: HBASE-5877 > URL: https://issues.apache.org/jira/browse/HBASE-5877 > Project: HBase > Issue Type: Improvement > Components: client, master, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5877-v16.txt, 5877-v17.txt, 5877.v1.patch, > 5877.v12.patch, 5877.v15.patch, 5877.v6.patch > > > This is mainly useful when we do a rolling restart. This will decrease the > load on the master and the network load. > Note that a region is not immediately opened after a close. So: > - it seems preferable to wait before retrying on the other server. An > optimisation would be to have an heuristic depending on when the region was > closed. > - during a rolling restart, the server moves the regions then stops. So we > may have failures when the server is stopped, and this patch won't help. > The implementation in the first patch does: > - on the region move, there is an added parameter on the regionserver#close > to say where we are sending the region > - the regionserver keeps a list of what was moved. Each entry is kept 100 > seconds. > - the regionserver sends a specific exception when it receives a query on a > moved region. This exception contains the new address. > - the client analyses the exeptions and update its cache accordingly... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client
[ https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268734#comment-13268734 ] Hadoop QA commented on HBASE-5877: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525676/5877-v16.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestMultiParallel Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1773//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1773//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1773//console This message is automatically generated. > When a query fails because the region has moved, let the regionserver return > the new address to the client > -- > > Key: HBASE-5877 > URL: https://issues.apache.org/jira/browse/HBASE-5877 > Project: HBase > Issue Type: Improvement > Components: client, master, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5877-v16.txt, 5877.v1.patch, 5877.v12.patch, > 5877.v15.patch, 5877.v6.patch > > > This is mainly useful when we do a rolling restart. This will decrease the > load on the master and the network load. > Note that a region is not immediately opened after a close. So: > - it seems preferable to wait before retrying on the other server. An > optimisation would be to have an heuristic depending on when the region was > closed. > - during a rolling restart, the server moves the regions then stops. So we > may have failures when the server is stopped, and this patch won't help. > The implementation in the first patch does: > - on the region move, there is an added parameter on the regionserver#close > to say where we are sending the region > - the regionserver keeps a list of what was moved. Each entry is kept 100 > seconds. > - the regionserver sends a specific exception when it receives a query on a > moved region. This exception contains the new address. > - the client analyses the exeptions and update its cache accordingly... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5922) HalfStoreFileReader seekBefore causes StackOverflowError
[ https://issues.apache.org/jira/browse/HBASE-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268731#comment-13268731 ] Todd Johnson commented on HBASE-5922: - @Anoop, Thanks for the help understanding what was going on here. > HalfStoreFileReader seekBefore causes StackOverflowError > > > Key: HBASE-5922 > URL: https://issues.apache.org/jira/browse/HBASE-5922 > Project: HBase > Issue Type: Bug > Components: client, io >Affects Versions: 0.90.0 > Environment: HBase 0.90.4 >Reporter: Nate Putnam >Assignee: Nate Putnam >Priority: Critical > Fix For: 0.90.0 > > Attachments: HBASE-5922.patch, HBASE-5922.patch, HBASE-5922.v2.patch, > HBASE-5922.v3.patch > > > Calling HRegionServer.getClosestRowBefore() can cause a stack overflow if the > underlying store file is a reference and the row key is in the bottom. > java.io.IOException: java.io.IOException: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:990) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:978) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1651) > at sun.reflect.GeneratedMethodAccessor174.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) > Caused by: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:147) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268732#comment-13268732 ] stack commented on HBASE-5889: -- The util is for serverside though? If not, if for client side, then yes, belongs in client package. Good on you Jimmy. > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5295) Improve the Thrift API to switch on/off writing to wal for Mutations
[ https://issues.apache.org/jira/browse/HBASE-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268724#comment-13268724 ] Phabricator commented on HBASE-5295: Kannan has commented on the revision "[jira] [HBASE-5295] [89-fb] Improve the Thrift API to switch on/off writing to wal for Mutations". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java:989 @mikhail: the tighter check you added to make sure all mutations have the same writeToWAL setting should be added for this case too. Otherwise, rest of the diff looks really good. And nice test! REVISION DETAIL https://reviews.facebook.net/D3015 > Improve the Thrift API to switch on/off writing to wal for Mutations > - > > Key: HBASE-5295 > URL: https://issues.apache.org/jira/browse/HBASE-5295 > Project: HBase > Issue Type: Improvement > Components: thrift >Reporter: dhruba borthakur >Assignee: dhruba borthakur > Attachments: D1515.1.patch, D1515.1.patch, D1515.1.patch, > D1515.1.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, D1515.2.patch, > D3015.1.patch > > > The thrift api currently does not support switching off updating wal for > Puts/Deletes. Support it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5494) Introduce a zk hosted table-wide read/write lock so only one table operation at a time
[ https://issues.apache.org/jira/browse/HBASE-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268722#comment-13268722 ] Phabricator commented on HBASE-5494: mbautin has commented on the revision "[jira] [HBASE-5494] [89-fb] Table-level locks for schema changing operations.". Discussed offline with Alex. Changes to createTable (double-checked locking) look good to me. REVISION DETAIL https://reviews.facebook.net/D2997 BRANCH table_level_ddl_locks > Introduce a zk hosted table-wide read/write lock so only one table operation > at a time > -- > > Key: HBASE-5494 > URL: https://issues.apache.org/jira/browse/HBASE-5494 > Project: HBase > Issue Type: Improvement >Reporter: stack > Attachments: D2997.3.patch, D2997.4.patch, D2997.5.patch, > D2997.6.patch > > > I saw this facility over in the accumulo code base. > Currently we just try to sort out the mess when splits come in during an > online schema edit; somehow we figure we can figure all possible region > transition combinations and make the right call. > We could try and narrow the number of combinations by taking out a zk table > lock when doing table operations. > For example, on split or merge, we could take a read-only lock meaning the > table can't be disabled while these are running. > We could then take a write only lock if we want to ensure the table doesn't > change while disabling or enabling process is happening. > Shouldn't be too hard to add. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268719#comment-13268719 ] Phabricator commented on HBASE-5045: mbautin has commented on the revision "[jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info". INLINE COMMENTS src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java:101 I think in general we don't put more than one class in a file. Either use a static inner class or put this class in its own file. src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java:131 Make a static inner class or move to a separate file. src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java:147 Make a static inner class or move to a separate file. REVISION DETAIL https://reviews.facebook.net/D3045 > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch, D3045.1.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268718#comment-13268718 ] Phabricator commented on HBASE-5045: tedyu has commented on the revision "[jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info". INLINE COMMENTS src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java:30 Please add @Category(SmallTests.class) src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java:42 Two extra empty lines should be removed. src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java:77 Please insert space between catch and ( src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java:131 This class and MyFailureCaseServer2 can be made private, right ? REVISION DETAIL https://reviews.facebook.net/D3045 > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch, D3045.1.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268714#comment-13268714 ] Phabricator commented on HBASE-5045: amirshim has commented on the revision "[jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info". Mikhail: Most of the comments in the other diff were addressed with comments in the other diff, but I added some more comments where needed in this diff. REVISION DETAIL https://reviews.facebook.net/D3045 > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch, D3045.1.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268706#comment-13268706 ] Phabricator commented on HBASE-5045: mbautin has commented on the revision "[jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info". Amir: does this version address comments from the other diff? REVISION DETAIL https://reviews.facebook.net/D3045 > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch, D3045.1.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268707#comment-13268707 ] Phabricator commented on HBASE-5045: mbautin has abandoned the revision "[jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info". REVISION DETAIL https://reviews.facebook.net/D2913 > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch, D3045.1.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268703#comment-13268703 ] Hadoop QA commented on HBASE-5045: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525679/D3045.1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.TestCheckTestClasses Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1772//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1772//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1772//console This message is automatically generated. > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch, D3045.1.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5922) HalfStoreFileReader seekBefore causes StackOverflowError
[ https://issues.apache.org/jira/browse/HBASE-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268702#comment-13268702 ] Nate Putnam commented on HBASE-5922: @Anoop happy to back port the fix to 92.x,94.x and 90.x once we agree on the solution. I've submitted another patch (v3) witch changes the return false to a delegate.seekBefore. I'm fairly confident that this is the final solution. If you have any concerns/comments let me know. > HalfStoreFileReader seekBefore causes StackOverflowError > > > Key: HBASE-5922 > URL: https://issues.apache.org/jira/browse/HBASE-5922 > Project: HBase > Issue Type: Bug > Components: client, io >Affects Versions: 0.90.0 > Environment: HBase 0.90.4 >Reporter: Nate Putnam >Assignee: Nate Putnam >Priority: Critical > Fix For: 0.90.0 > > Attachments: HBASE-5922.patch, HBASE-5922.patch, HBASE-5922.v2.patch, > HBASE-5922.v3.patch > > > Calling HRegionServer.getClosestRowBefore() can cause a stack overflow if the > underlying store file is a reference and the row key is in the bottom. > java.io.IOException: java.io.IOException: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:990) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:978) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1651) > at sun.reflect.GeneratedMethodAccessor174.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) > Caused by: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:147) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5922) HalfStoreFileReader seekBefore causes StackOverflowError
[ https://issues.apache.org/jira/browse/HBASE-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nate Putnam updated HBASE-5922: --- Attachment: HBASE-5922.v3.patch > HalfStoreFileReader seekBefore causes StackOverflowError > > > Key: HBASE-5922 > URL: https://issues.apache.org/jira/browse/HBASE-5922 > Project: HBase > Issue Type: Bug > Components: client, io >Affects Versions: 0.90.0 > Environment: HBase 0.90.4 >Reporter: Nate Putnam >Assignee: Nate Putnam >Priority: Critical > Fix For: 0.90.0 > > Attachments: HBASE-5922.patch, HBASE-5922.patch, HBASE-5922.v2.patch, > HBASE-5922.v3.patch > > > Calling HRegionServer.getClosestRowBefore() can cause a stack overflow if the > underlying store file is a reference and the row key is in the bottom. > java.io.IOException: java.io.IOException: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:990) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:978) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1651) > at sun.reflect.GeneratedMethodAccessor174.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) > Caused by: java.lang.StackOverflowError > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:147) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) > at > org.apache.hadoop.hbase.io.HalfStoreFileReader$1.seekBefore(HalfStoreFileReader.java:149) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client
[ https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268701#comment-13268701 ] nkeywal commented on HBASE-5877: thank you Ted. It's strange, it though it worked locally. May be this test was hung and was not reported as an error. Thank you anyway. > When a query fails because the region has moved, let the regionserver return > the new address to the client > -- > > Key: HBASE-5877 > URL: https://issues.apache.org/jira/browse/HBASE-5877 > Project: HBase > Issue Type: Improvement > Components: client, master, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5877-v16.txt, 5877.v1.patch, 5877.v12.patch, > 5877.v15.patch, 5877.v6.patch > > > This is mainly useful when we do a rolling restart. This will decrease the > load on the master and the network load. > Note that a region is not immediately opened after a close. So: > - it seems preferable to wait before retrying on the other server. An > optimisation would be to have an heuristic depending on when the region was > closed. > - during a rolling restart, the server moves the regions then stops. So we > may have failures when the server is stopped, and this patch won't help. > The implementation in the first patch does: > - on the region move, there is an added parameter on the regionserver#close > to say where we are sending the region > - the regionserver keeps a list of what was moved. Each entry is kept 100 > seconds. > - the regionserver sends a specific exception when it receives a query on a > moved region. This exception contains the new address. > - the client analyses the exeptions and update its cache accordingly... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268699#comment-13268699 ] Phabricator commented on HBASE-5045: amirshim has commented on the revision "[jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info". Can't override Mikhail diff, so created a new diff: https://reviews.facebook.net/D3045 REVISION DETAIL https://reviews.facebook.net/D2913 BRANCH add_the_table_name_and_cf_name_for_the_next_HBASE-5045_v3 > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch, D3045.1.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268700#comment-13268700 ] Phabricator commented on HBASE-5045: amirshim has commented on the revision "[jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info". See https://reviews.facebook.net/D2913 for comments. REVISION DETAIL https://reviews.facebook.net/D3045 > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch, D3045.1.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5699) Run with > 1 WAL in HRegionServer
[ https://issues.apache.org/jira/browse/HBASE-5699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268697#comment-13268697 ] Zhihong Yu commented on HBASE-5699: --- bq. there is still a single "HLog" class, but underneath it would be multiple "SequenceFileLogWriters" My approach is different from the above. The new interface should be general enough that multi-HLog can be implemented without the requirement that HLog have multiple writers. > Run with > 1 WAL in HRegionServer > - > > Key: HBASE-5699 > URL: https://issues.apache.org/jira/browse/HBASE-5699 > Project: HBase > Issue Type: Improvement >Reporter: binlijin >Assignee: Li Pi > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5045: --- Attachment: D3045.1.patch amirshim requested code review of "[jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info". Reviewers: mbautin, Liyin, tedyu, stack, JIRA, nspiegelberg Porting Amir's fix from 89-fb. His original summary below. A method for associating pretty print classes with method calls. These allow you to get information about a method call given the params it was called with and what instance it was called on. The first use case is for getting info about a next() RPC call. TEST PLAN Original test plan from Amir: run a script that stresses a regionserver with scan and next() scans, and check that the information is show in the JSON view of the TaskMonitor REVISION DETAIL https://reviews.facebook.net/D3045 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredRPCHandler.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredRPCHandlerImpl.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredTask.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredTaskImpl.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/main/java/org/apache/hadoop/hbase/util/ParamFormat.java src/main/java/org/apache/hadoop/hbase/util/ParamFormatHelper.java src/main/java/org/apache/hadoop/hbase/util/ParamFormatter.java src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6915/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch, D3045.1.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client
[ https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5877: -- Attachment: 5877-v16.txt Patch v16 is able to pass TestMultiParallel. See the following link for the reason of infinite recursion: http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera.hadoop/hadoop-core/0.20.2-737/org/apache/hadoop/ipc/RemoteException.java#RemoteException.unwrapRemoteException%28%29 > When a query fails because the region has moved, let the regionserver return > the new address to the client > -- > > Key: HBASE-5877 > URL: https://issues.apache.org/jira/browse/HBASE-5877 > Project: HBase > Issue Type: Improvement > Components: client, master, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5877-v16.txt, 5877.v1.patch, 5877.v12.patch, > 5877.v15.patch, 5877.v6.patch > > > This is mainly useful when we do a rolling restart. This will decrease the > load on the master and the network load. > Note that a region is not immediately opened after a close. So: > - it seems preferable to wait before retrying on the other server. An > optimisation would be to have an heuristic depending on when the region was > closed. > - during a rolling restart, the server moves the regions then stops. So we > may have failures when the server is stopped, and this patch won't help. > The implementation in the first patch does: > - on the region move, there is an added parameter on the regionserver#close > to say where we are sending the region > - the regionserver keeps a list of what was moved. Each entry is kept 100 > seconds. > - the regionserver sends a specific exception when it receives a query on a > moved region. This exception contains the new address. > - the client analyses the exeptions and update its cache accordingly... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268691#comment-13268691 ] Phabricator commented on HBASE-5045: amirshim has commented on the revision "[jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java:319 status.setRPC a couple of lines away now takes method as a param. src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredRPCHandler.java:45 The "real" method returned by the Java reflection API... see MonitoredRPCHandlerImpl for more info. src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:3489 This object is stored across RPC calls and we store the original scan, so that we can easily print the information about the scan in the task monitor. It needs to be accessible from ScanParamsFormatter src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2210 It doesn't have to be in here, but if not, then we have to expose some internals of HRS. This class needs to understand the internals, since it provides introspection about what it's method(s) do. i.e. If we change how we do scanner lookups, this class needs to change too. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2221 It's supposed to be strongly bound since it provides introspection about it's methods. src/main/java/org/apache/hadoop/hbase/util/ParamFormat.java:27 It's possible, but should be done in another diff. src/main/java/org/apache/hadoop/hbase/util/ParamFormatHelper.java:63 It's introspection on a method we control, since we force it to derive from ParamFormatter<>. I called it getMap() to be consistent with the getMap() used in the TaskMonitor. src/main/java/org/apache/hadoop/hbase/util/ParamFormatHelper.java:102 See above. I called it getMap to be consistent with the TaskMonitor functions. src/main/java/org/apache/hadoop/hbase/util/ParamFormatter.java:24 It formats information about a method call given the params it takes. PrettyPrint makes me think that it actually outputs the info somewhere, as opposed to a formatter that arranges the data for later pretty printing. I shouldn't have put pretty print in so many comments. REVISION DETAIL https://reviews.facebook.net/D2913 BRANCH add_the_table_name_and_cf_name_for_the_next_HBASE-5045_v3 > Add the table name and cf name for the next call int the task monitor > - > > Key: HBASE-5045 > URL: https://issues.apache.org/jira/browse/HBASE-5045 > Project: HBase > Issue Type: Improvement >Reporter: Liyin Tang >Assignee: Amir Shimoni > Attachments: D2913.1.patch, D2913.2.patch > > > In the task monitor, we don't have much information about the next call > compared to other operations. > It would be nice to add the table name and cf name for each next call in the > task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268690#comment-13268690 ] Jimmy Xiang commented on HBASE-5889: The purpose is to re-use some shared code. I was thinking to separate it into two util classes: ClientUtil and AdminUtil, in the client package. It is not for region server only. How is that? > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5942) HConnnectionManager.getRegionServerWithRetries doesn't call afterCall properly
Zhihong Yu created HBASE-5942:
-
Summary: HConnnectionManager.getRegionServerWithRetries doesn't
call afterCall properly
Key: HBASE-5942
URL: https://issues.apache.org/jira/browse/HBASE-5942
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Zhihong Yu
HConnnectionManager.getRegionServerWithRetries:
{code}
return callable.call();
} catch (Throwable t) {
callable.shouldRetry(t);
{code}
shouldRetry relies on the proper startTime and endTime to calculate the timeout
value. However, callable.afterCall() is called in finally block. Thus
callable.callTimeout will be set to negative value in callable.shouldRetry().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[
https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268682#comment-13268682
]
stack commented on HBASE-5889:
--
I took a look. Its good. I tried changing RegionServerUtil so it was package
private but I got these failures which are odd given the class is named
RegionServerUtil. What you reckon?
{code}
[INFO] Compilation failure
/Users/Stack/checkouts/trunk/src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java:[43,43]
org.apache.hadoop.hbase.regionserver.RegionServerUtil is not public in
org.apache.hadoop.hbase.regionserver; cannot be accessed from outside package
/Users/Stack/checkouts/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java:[79,43]
org.apache.hadoop.hbase.regionserver.RegionServerUtil is not public in
org.apache.hadoop.hbase.regionserver; cannot be accessed from outside package
/Users/Stack/checkouts/trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java:[72,43]
org.apache.hadoop.hbase.regionserver.RegionServerUtil is not public in
org.apache.hadoop.hbase.regionserver; cannot be accessed from outside package
/Users/Stack/checkouts/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java:[70,43]
org.apache.hadoop.hbase.regionserver.RegionServerUtil is not public in
org.apache.hadoop.hbase.regionserver; cannot be accessed from outside package
/Users/Stack/checkouts/trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java:[55,43]
org.apache.hadoop.hbase.regionserver.RegionServerUtil is not public in
org.apache.hadoop.hbase.regionserver; cannot be accessed from outside package
{code}
What are these classes making use of RegionServerUtil?
> Remove HRegionInterface
> ---
>
> Key: HBASE-5889
> URL: https://issues.apache.org/jira/browse/HBASE-5889
> Project: HBase
> Issue Type: Improvement
> Components: client, ipc, regionserver
>Affects Versions: 0.96.0
>Reporter: Jimmy Xiang
>Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: hbase-5889_v3.patch, hbase_5889.patch,
> hbase_5889_v2.patch
>
>
> As a step to move internals to PB, so as to avoid the conversion for
> performance reason, we should remove the HRegionInterface.
> Therefore region server only supports ClientProtocol and AdminProtocol.
> Later on, HRegion can work with PB messages directly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client
[
https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268678#comment-13268678
]
Zhihong Yu commented on HBASE-5877:
---
I can reproduce the TestMultiParallel failure:
{code}
testFlushCommitsNoAbort(org.apache.hadoop.hbase.client.TestMultiParallel) Time
elapsed: 1.36 sec <<< ERROR!
java.lang.StackOverflowError
at
org.apache.hadoop.hbase.RegionMovedException.find(RegionMovedException.java:91)
at
org.apache.hadoop.hbase.RegionMovedException.find(RegionMovedException.java:108)
at
org.apache.hadoop.hbase.RegionMovedException.find(RegionMovedException.java:108)
at
org.apache.hadoop.hbase.RegionMovedException.find(RegionMovedException.java:108)
at
org.apache.hadoop.hbase.RegionMovedException.find(RegionMovedException.java:108)
at
org.apache.hadoop.hbase.RegionMovedException.find(RegionMovedException.java:108)
at
org.apache.hadoop.hbase.RegionMovedException.find(RegionMovedException.java:108)
{code}
> When a query fails because the region has moved, let the regionserver return
> the new address to the client
> --
>
> Key: HBASE-5877
> URL: https://issues.apache.org/jira/browse/HBASE-5877
> Project: HBase
> Issue Type: Improvement
> Components: client, master, regionserver
>Affects Versions: 0.96.0
>Reporter: nkeywal
>Assignee: nkeywal
>Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 5877.v1.patch, 5877.v12.patch, 5877.v15.patch,
> 5877.v6.patch
>
>
> This is mainly useful when we do a rolling restart. This will decrease the
> load on the master and the network load.
> Note that a region is not immediately opened after a close. So:
> - it seems preferable to wait before retrying on the other server. An
> optimisation would be to have an heuristic depending on when the region was
> closed.
> - during a rolling restart, the server moves the regions then stops. So we
> may have failures when the server is stopped, and this patch won't help.
> The implementation in the first patch does:
> - on the region move, there is an added parameter on the regionserver#close
> to say where we are sending the region
> - the regionserver keeps a list of what was moved. Each entry is kept 100
> seconds.
> - the regionserver sends a specific exception when it receives a query on a
> moved region. This exception contains the new address.
> - the client analyses the exeptions and update its cache accordingly...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268670#comment-13268670 ] Jimmy Xiang commented on HBASE-5889: Yes, I think it is good to go. I moved those utils like openRegion from ProtobufUtil to RegionServerUtil. > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268666#comment-13268666 ] stack commented on HBASE-5889: -- Is this good to go in Jimmy? What comments did you address? > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5916) RS restart just before master intialization we make the cluster non operative
[
https://issues.apache.org/jira/browse/HBASE-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-5916:
--
Status: Patch Available (was: Open)
> RS restart just before master intialization we make the cluster non operative
> -
>
> Key: HBASE-5916
> URL: https://issues.apache.org/jira/browse/HBASE-5916
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.94.1
>
> Attachments: HBASE-5916_trunk.patch, HBASE-5916_trunk_1.patch
>
>
> Consider a case where my master is getting restarted. RS that was alive when
> the master restart started, gets restarted before the master initializes the
> ServerShutDownHandler.
> {code}
> serverShutdownHandlerEnabled = true;
> {code}
> In this case when the RS tries to register with the master, the master will
> try to expire the server but the server cannot be expired as still the
> serverShutdownHandler is not enabled.
> This case may happen when i have only one RS gets restarted or all the RS
> gets restarted at the same time.(before assignRootandMeta).
> {code}
> LOG.info(message);
> if (existingServer.getStartcode() < serverName.getStartcode()) {
> LOG.info("Triggering server recovery; existingServer " +
> existingServer + " looks stale, new server:" + serverName);
> expireServer(existingServer);
> }
> {code}
> If another RS is brought up then the cluster comes back to normalcy.
> May be a very corner case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5916) RS restart just before master intialization we make the cluster non operative
[
https://issues.apache.org/jira/browse/HBASE-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-5916:
--
Attachment: HBASE-5916_trunk_1.patch
Updated patch.
> RS restart just before master intialization we make the cluster non operative
> -
>
> Key: HBASE-5916
> URL: https://issues.apache.org/jira/browse/HBASE-5916
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.94.1
>
> Attachments: HBASE-5916_trunk.patch, HBASE-5916_trunk_1.patch
>
>
> Consider a case where my master is getting restarted. RS that was alive when
> the master restart started, gets restarted before the master initializes the
> ServerShutDownHandler.
> {code}
> serverShutdownHandlerEnabled = true;
> {code}
> In this case when the RS tries to register with the master, the master will
> try to expire the server but the server cannot be expired as still the
> serverShutdownHandler is not enabled.
> This case may happen when i have only one RS gets restarted or all the RS
> gets restarted at the same time.(before assignRootandMeta).
> {code}
> LOG.info(message);
> if (existingServer.getStartcode() < serverName.getStartcode()) {
> LOG.info("Triggering server recovery; existingServer " +
> existingServer + " looks stale, new server:" + serverName);
> expireServer(existingServer);
> }
> {code}
> If another RS is brought up then the cluster comes back to normalcy.
> May be a very corner case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5916) RS restart just before master intialization we make the cluster non operative
[
https://issues.apache.org/jira/browse/HBASE-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-5916:
--
Status: Open (was: Patch Available)
Will make use of hadoopqa to run the testcases, as am at home.
> RS restart just before master intialization we make the cluster non operative
> -
>
> Key: HBASE-5916
> URL: https://issues.apache.org/jira/browse/HBASE-5916
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.1, 0.94.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.94.1
>
> Attachments: HBASE-5916_trunk.patch, HBASE-5916_trunk_1.patch
>
>
> Consider a case where my master is getting restarted. RS that was alive when
> the master restart started, gets restarted before the master initializes the
> ServerShutDownHandler.
> {code}
> serverShutdownHandlerEnabled = true;
> {code}
> In this case when the RS tries to register with the master, the master will
> try to expire the server but the server cannot be expired as still the
> serverShutdownHandler is not enabled.
> This case may happen when i have only one RS gets restarted or all the RS
> gets restarted at the same time.(before assignRootandMeta).
> {code}
> LOG.info(message);
> if (existingServer.getStartcode() < serverName.getStartcode()) {
> LOG.info("Triggering server recovery; existingServer " +
> existingServer + " looks stale, new server:" + serverName);
> expireServer(existingServer);
> }
> {code}
> If another RS is brought up then the cluster comes back to normalcy.
> May be a very corner case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client
[ https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268649#comment-13268649 ] Hadoop QA commented on HBASE-5877: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525660/5877.v15.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestMultiParallel Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1770//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1770//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1770//console This message is automatically generated. > When a query fails because the region has moved, let the regionserver return > the new address to the client > -- > > Key: HBASE-5877 > URL: https://issues.apache.org/jira/browse/HBASE-5877 > Project: HBase > Issue Type: Improvement > Components: client, master, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5877.v1.patch, 5877.v12.patch, 5877.v15.patch, > 5877.v6.patch > > > This is mainly useful when we do a rolling restart. This will decrease the > load on the master and the network load. > Note that a region is not immediately opened after a close. So: > - it seems preferable to wait before retrying on the other server. An > optimisation would be to have an heuristic depending on when the region was > closed. > - during a rolling restart, the server moves the regions then stops. So we > may have failures when the server is stopped, and this patch won't help. > The implementation in the first patch does: > - on the region move, there is an added parameter on the regionserver#close > to say where we are sending the region > - the regionserver keeps a list of what was moved. Each entry is kept 100 > seconds. > - the regionserver sends a specific exception when it receives a query on a > moved region. This exception contains the new address. > - the client analyses the exeptions and update its cache accordingly... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5941) improve multiDelete performance by grabbing locks ahead of time
Amitanand Aiyer created HBASE-5941: -- Summary: improve multiDelete performance by grabbing locks ahead of time Key: HBASE-5941 URL: https://issues.apache.org/jira/browse/HBASE-5941 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.89.20100924, 0.89-fb, 0.94.1 Reporter: Amitanand Aiyer Assignee: Amitanand Aiyer Priority: Minor Ning reported that the performance of deletes is slower than the performance of Puts. This should not be the case. On digging up, it turns out that there is a difference between multiPut and multiDelete in the way we grab locks. multiPut grabs all the locks optimistically and processes the puts one by one. multiDelete grabs locks and releases them one at a time, for each delete operation, as if it were done separately. This may be causing a performance slow down for deletes. Trying to improve it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268628#comment-13268628 ] Hadoop QA commented on HBASE-5889: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525658/hbase-5889_v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 63 new or modified tests. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1769//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1769//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1769//console This message is automatically generated. > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5494) Introduce a zk hosted table-wide read/write lock so only one table operation at a time
[ https://issues.apache.org/jira/browse/HBASE-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268625#comment-13268625 ] Phabricator commented on HBASE-5494: Kannan has commented on the revision "[jira] [HBASE-5494] [89-fb] Table-level locks for schema changing operations.". lgtm! REVISION DETAIL https://reviews.facebook.net/D2997 BRANCH table_level_ddl_locks > Introduce a zk hosted table-wide read/write lock so only one table operation > at a time > -- > > Key: HBASE-5494 > URL: https://issues.apache.org/jira/browse/HBASE-5494 > Project: HBase > Issue Type: Improvement >Reporter: stack > Attachments: D2997.3.patch, D2997.4.patch, D2997.5.patch, > D2997.6.patch > > > I saw this facility over in the accumulo code base. > Currently we just try to sort out the mess when splits come in during an > online schema edit; somehow we figure we can figure all possible region > transition combinations and make the right call. > We could try and narrow the number of combinations by taking out a zk table > lock when doing table operations. > For example, on split or merge, we could take a read-only lock meaning the > table can't be disabled while these are running. > We could then take a write only lock if we want to ensure the table doesn't > change while disabling or enabling process is happening. > Shouldn't be too hard to add. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5494) Introduce a zk hosted table-wide read/write lock so only one table operation at a time
[ https://issues.apache.org/jira/browse/HBASE-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5494: --- Attachment: D2997.6.patch avf updated the revision "[jira] [HBASE-5494] [89-fb] Table-level locks for schema changing operations.". Reviewers: Kannan, mbautin, Liyin, JIRA Move InjectionEventHandler.processEvent() call after lock acquisition in createTable(), explain the way "node is deleted before watch is set" race condition is handled in more detail. REVISION DETAIL https://reviews.facebook.net/D2997 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HConstants.java src/main/java/org/apache/hadoop/hbase/TableLockTimeoutException.java src/main/java/org/apache/hadoop/hbase/master/HMaster.java src/main/java/org/apache/hadoop/hbase/master/TableLockManager.java src/main/java/org/apache/hadoop/hbase/util/InjectionEvent.java src/main/java/org/apache/hadoop/hbase/util/InjectionHandler.java src/main/java/org/apache/hadoop/hbase/zookeeper/DistributedLock.java src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWrapper.java src/test/java/org/apache/hadoop/hbase/master/TestSchemaModificationLocks.java src/test/java/org/apache/hadoop/hbase/util/DelayInducingInjectionHandler.java src/test/java/org/apache/hadoop/hbase/zookeeper/TestDistributedLock.java > Introduce a zk hosted table-wide read/write lock so only one table operation > at a time > -- > > Key: HBASE-5494 > URL: https://issues.apache.org/jira/browse/HBASE-5494 > Project: HBase > Issue Type: Improvement >Reporter: stack > Attachments: D2997.3.patch, D2997.4.patch, D2997.5.patch, > D2997.6.patch > > > I saw this facility over in the accumulo code base. > Currently we just try to sort out the mess when splits come in during an > online schema edit; somehow we figure we can figure all possible region > transition combinations and make the right call. > We could try and narrow the number of combinations by taking out a zk table > lock when doing table operations. > For example, on split or merge, we could take a read-only lock meaning the > table can't be disabled while these are running. > We could then take a write only lock if we want to ensure the table doesn't > change while disabling or enabling process is happening. > Shouldn't be too hard to add. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client
[ https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5877: --- Status: Patch Available (was: Open) > When a query fails because the region has moved, let the regionserver return > the new address to the client > -- > > Key: HBASE-5877 > URL: https://issues.apache.org/jira/browse/HBASE-5877 > Project: HBase > Issue Type: Improvement > Components: client, master, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5877.v1.patch, 5877.v12.patch, 5877.v15.patch, > 5877.v6.patch > > > This is mainly useful when we do a rolling restart. This will decrease the > load on the master and the network load. > Note that a region is not immediately opened after a close. So: > - it seems preferable to wait before retrying on the other server. An > optimisation would be to have an heuristic depending on when the region was > closed. > - during a rolling restart, the server moves the regions then stops. So we > may have failures when the server is stopped, and this patch won't help. > The implementation in the first patch does: > - on the region move, there is an added parameter on the regionserver#close > to say where we are sending the region > - the regionserver keeps a list of what was moved. Each entry is kept 100 > seconds. > - the regionserver sends a specific exception when it receives a query on a > moved region. This exception contains the new address. > - the client analyses the exeptions and update its cache accordingly... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5877) When a query fails because the region has moved, let the regionserver return the new address to the client
[ https://issues.apache.org/jira/browse/HBASE-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5877: --- Attachment: 5877.v15.patch > When a query fails because the region has moved, let the regionserver return > the new address to the client > -- > > Key: HBASE-5877 > URL: https://issues.apache.org/jira/browse/HBASE-5877 > Project: HBase > Issue Type: Improvement > Components: client, master, regionserver >Affects Versions: 0.96.0 >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Fix For: 0.96.0 > > Attachments: 5877.v1.patch, 5877.v12.patch, 5877.v15.patch, > 5877.v6.patch > > > This is mainly useful when we do a rolling restart. This will decrease the > load on the master and the network load. > Note that a region is not immediately opened after a close. So: > - it seems preferable to wait before retrying on the other server. An > optimisation would be to have an heuristic depending on when the region was > closed. > - during a rolling restart, the server moves the regions then stops. So we > may have failures when the server is stopped, and this patch won't help. > The implementation in the first patch does: > - on the region move, there is an added parameter on the regionserver#close > to say where we are sending the region > - the regionserver keeps a list of what was moved. Each entry is kept 100 > seconds. > - the regionserver sends a specific exception when it receives a query on a > moved region. This exception contains the new address. > - the client analyses the exeptions and update its cache accordingly... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5889) Remove HRegionInterface
[ https://issues.apache.org/jira/browse/HBASE-5889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5889: --- Status: Patch Available (was: Open) > Remove HRegionInterface > --- > > Key: HBASE-5889 > URL: https://issues.apache.org/jira/browse/HBASE-5889 > Project: HBase > Issue Type: Improvement > Components: client, ipc, regionserver >Affects Versions: 0.96.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.96.0 > > Attachments: hbase-5889_v3.patch, hbase_5889.patch, > hbase_5889_v2.patch > > > As a step to move internals to PB, so as to avoid the conversion for > performance reason, we should remove the HRegionInterface. > Therefore region server only supports ClientProtocol and AdminProtocol. > Later on, HRegion can work with PB messages directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
