[
https://issues.apache.org/jira/browse/HBASE-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HBASE-4908:
-------------------------------
Attachment: D549.6.patch
mbautin updated the revision "[jira] [HBASE-4908] HBase cluster test tool (port
from 0.89-fb)".
Reviewers: stack, Karthik, Kannan, nspiegelberg, JIRA
The unit tests run fine (except TestReplication.queueFailover, which is
flaky), and cluster testing works well. I rewrote the writer/reader
synchronization mechanism after talking to Kannan, so that the reader gets the
precise knowledge of what is the last key in the contiguous sequence of keys
starting from startKey that the writer has written. This required implementing
a producer/consumer pattern with a special InsertedKeyTracker thread sorting
inserted keys and advancing the "last contiguous key" marker. Also clarified
and improved a bunch of command-line stuff in LoadTest.
REVISION DETAIL
https://reviews.facebook.net/D549
AFFECTED FILES
pom.xml
src/main/java/org/apache/hadoop/hbase/EmptyWatcher.java
src/test/java/org/apache/hadoop/hbase/EmptyWatcher.java
src/main/java/org/apache/hadoop/hbase/HConstants.java
src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java
src/main/java/org/apache/hadoop/hbase/util/Bytes.java
src/main/java/org/apache/hadoop/hbase/util/HMerge.java
src/main/java/org/apache/hadoop/hbase/util/Keying.java
src/main/java/org/apache/hadoop/hbase/util/RegionSplitter.java
src/main/java/org/apache/hadoop/hbase/zookeeper/ZKConfig.java
src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java
src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java
src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java
src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionSplitPolicy.java
src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
src/test/java/org/apache/hadoop/hbase/util/IntegrationTestTool.java
src/test/java/org/apache/hadoop/hbase/util/LoadTest.java
src/test/java/org/apache/hadoop/hbase/util/LoadTestKVGenerator.java
src/test/java/org/apache/hadoop/hbase/util/MultiThreadedAction.java
src/test/java/org/apache/hadoop/hbase/util/MultiThreadedReader.java
src/test/java/org/apache/hadoop/hbase/util/MultiThreadedWriter.java
src/test/java/org/apache/hadoop/hbase/util/ProcessBasedLocalHBaseCluster.java
src/test/java/org/apache/hadoop/hbase/util/RestartMetaTest.java
src/test/java/org/apache/hadoop/hbase/util/TestBytes.java
src/test/java/org/apache/hadoop/hbase/util/TestLoadTestKVGenerator.java
src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java
src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadParallel.java
src/test/java/org/apache/hadoop/hbase/util/TestMiniClusterLoadSequential.java
> HBase cluster test tool (port from 0.89-fb)
> -------------------------------------------
>
> Key: HBASE-4908
> URL: https://issues.apache.org/jira/browse/HBASE-4908
> Project: HBase
> Issue Type: Sub-task
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Attachments: D549.1.patch, D549.2.patch, D549.3.patch, D549.4.patch,
> D549.5.patch, D549.6.patch
>
>
> Porting one of our HBase cluster test tools (a single-process multi-threaded
> load generator and verifier) from 0.89-fb to trunk.
> I cleaned up the code a bit compared to what's in 0.89-fb, and discovered
> that it has some features that I have not tried yet (some kind of a kill
> test, and some way to run HBase as multiple processes on one machine).
> The main utility of this piece of code for us has been the HBaseClusterTest
> command-line tool (called HBaseTest in 0.89-fb), which we usually invoke as a
> load test in our five-node dev cluster testing, e.g.:
> hbase org.apache.hadoop.hbase.manual.HBaseTest -load 1000000000:50:100:20 -tn
> load_test -read 1:1000000000:50:20 -zk <zk_quorum> -bloom ROWCOL -compression
> GZIP
> I will be using this code to load-test the delta encoding patch and making
> fixes, but I am submitting the patch for early feedback. I will probably try
> out its other functionality and comment on how it works.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
