I've filed an issue for this and will post more results there. (against 0.96.0 etc).
https://issues.apache.org/jira/browse/HBASE-10079 On Wed, Dec 4, 2013 at 8:10 AM, Jonathan Hsieh <[email protected]> wrote: > -1 > > I have a new test I tried on 0.96.1rc1. (polishing to share). Run #2 with > flushes is a correctness regression (HBASE-6195 fixed this in 94, and I had > verified there but it is broken here).. I'm not sure if #3 (kill) or #4 > (kill -9) are a regression against 0.94. Here's a link to the testcode > (its got all kinds of caveats at the moment and will likely get turn into > integration test). > > https://github.com/jmhsieh/hbase/tree/increval > > 5 nodes > hadoop 2.1.0 (a pre cdh5b1 hdfs). > hbase 0.96.1 rc1 > > Test: 250000 increments on a single row an single col with various number > of client threads (IncrementBlaster). Verify we have a count of 250000 > after the run (IncrementVerifier). > > Run 1: No fault injection. 5 runs. count = 250000. on multiple runs. > Correctness verified. 1638 inc/s throughput. > Run 2: flushes table with incrementing row. count = 246875 !=250000. > correctness failed. 1517 inc/s throughput. > Run 3: kill of rs hosting incremented row. count = 243750 != 250000. > Correctness failed. 1451 inc/s throughput. > Run 4: one kill -9 of rs hosting incremented row. 246878.!= 250000. > Correctness failed. 1395 inc/s (including recovery) > > > > On Wed, Nov 27, 2013 at 7:57 PM, Stack <[email protected]> wrote: > >> The second hbase-0.96.1 release candidate is available for download: >> >> http://people.apache.org/~stack/hbase-0.96.1RC1/ >> >> The maven artifacts are available here: >> >> https://repository.apache.org/content/repositories/orgapachehbase-016/ >> >> The first RC was sunk by: >> >> HBASE-10053 ProtobufUtil#toMutationNoData interface change causes >> downstream >> >> We broke hive building against HBase. >> >> 137 issues [1] have been fixed since 0.96.0, mostly performance >> improvement >> and bug fixes. >> >> Shall we release this as hbase-0.96.1? Please vote by December 3rd. >> >> Thanks, >> Your HBase Team >> >> 1. http://goo.gl/ZgYBW2 >> >> >> On Tue, Nov 26, 2013 at 4:48 PM, Stack <[email protected]> wrote: >> >> > The first hbase-0.96.1 release candidate is available for download: >> > >> > http://people.apache.org/~stack/hbase-0.96.1RC0/ >> > >> > The staged mvn artifacts can be found here: >> > >> > >> https://repository.apache.org/content/repositories/orgapachehbase-012/ >> > >> > 127 issues [1] have been fixed since 0.96.0, mostly performance >> improvement >> > and bug fixes. See below for the full list. >> > >> > Shall we release this as hbase-0.96.1? Please vote by December 3rd. >> > >> > Thanks, >> > Your HBase Team >> > >> > 1. http://goo.gl/ZgYBW2 >> > >> > Issues fixed in this RC: >> > >> > HBASE-8143 HBase on Hadoop 2 with local short circuit reads (ssr) causes >> > OOM >> > HBASE-10015 Replace intrinsic locking with explicit locks in >> StoreScanner >> > HBASE-10026 HBaseAdmin#createTable could fail if region splits too fast >> > HBASE-10024 Add an interface to create put with immutable arrays >> > HBASE-8465 Auto-drop rollback snapshot for snapshot restore >> > HBASE-10011 Fix some findbugs in the client >> > HBASE-9865 Reused WALEdits in replication may cause RegionServers to go >> OOM >> > HBASE-9890 MR jobs are not working if started by a delegated user >> > HBASE-9975 Not starting ReplicationSink when using custom implementation >> > for the ReplicationSink. >> > HBASE-9831 hbasefsck.numthreads' property isn't passed to hbck via >> cmdline >> > -D option >> > HBASE-9963 Remove the ReentrantReadWriteLock in the MemStore >> > HBASE-9956 Remove keyLength cache from KeyValue >> > HBASE-9165 HBASE-8084 Improvements to addDependencyJars >> > HBASE-8084 Improvements to addDependencyJars >> > HBASE-9138 getHaseIntegrationTestingUtility() is misspelled >> > HBASE-9970 HBase BulkLoad, table is creating with the timestamp key also >> > as a column to the table. >> > HBASE-9952 Snapshot restore may fail due to NullPointerException >> > HBASE-9906 Restore snapshot fails to restore the meta edits sporadically >> > HBASE-8438 Extend bin/hbase to print a "mapreduce classpath" >> > HBASE-9915 Performance: isSeeked() in EncodedScannerV2 always returns >> false >> > HBASE-8556 fix coverage org.apache.hadoop.hbase.metrics.histogram >> > HBASE-9872 ModifyTable does not modify the attributes of a newly >> > modified/changed ColumnDescriptor >> > HBASE-8559 increase unit-test coverage of package >> > org.apache.hadoop.hbase.coprocessor >> > HBASE-8543 fix coverage org.apache.hadoop.hbase.rest.client >> > HBASE-9902 Region Server is starting normally even if clock skew is more >> > than default 30 seconds(or any configured). -> Regionserver node time is >> > greater than master node time >> > HBASE-9849 [REST] Forbidden schema delete in read only mode >> > HBASE-8552 fix coverage org.apache.hadoop.hbase.rest.filter >> > HBASE-9850 Issues with UI for table compact/split operation completion. >> > After split/compaction operation using UI, the page is not automatically >> > redirecting back using IE8/Firefox. >> > HBASE-4654 [replication] Add a check to make sure we don't replicate to >> > ourselves >> > HBASE-10022 Fix the HE_EQUALS_USE_HASHCODE warning under >> > hbase-hadoop1-compat >> > HBASE-9976 Don't create duplicated TableName objects >> > HBASE-9992 [hbck] Refactor so that arbitrary -D cmdline options are >> > included >> > HBASE-5945 Reduce buffer copies in IPC server response path >> > HBASE-9322 Contention on the SoftValueSortedMap with multiple clients >> > HBASE-10007 PerformanceEvaluation: Add sampling and latency collection >> to >> > randomRead test >> > HBASE-9989 Add a test on get in TestClientNoCluster >> > HBASE-10014 HRegion#doMiniBatchMutation rollbacks the memstore even if >> > there is nothing to rollback. >> > HBASE-10013 Enhance the message about ZK when there is an exception >> > HBASE-10001 Add a coprocessor to help testing the performances without >> > taking into account the i/o >> > HBASE-10012 Hide ServerName constructor >> > HBASE-9961 [WINDOWS] Multicast should bind to local address >> > HBASE-9893 Incorrect assert condition in OrderedBytes decoding >> > HBASE-9998 Fix javadoc warnings induced by commits >> > HBASE-9959 Remove some array copy - server side >> > HBASE-9988 DOn't use HRI#getEncodedName in the client >> > HBASE-9995 Not stopping ReplicationSink when using custom implementation >> > for the ReplicationSink >> > HBASE-9924 Avoid potential filename conflict in region_mover.rb >> > HBASE-9982 TestClientNoCluster should use random numbers >> > HBASE-9973 [ACL]: Users with 'Admin' ACL permission will lose >> permissions >> > after upgrade to 0.96.x from 0.94.x or 0.92.x >> > HBASE-9987 Remove some synchronisation points in HConnectionManager >> > HBASE-9985 Add HBase shell command to start/stop tracing >> > HBASE-9983 Lower the memory footprint of HRegionLocation >> > HBASE-9869 Optimize HConnectionManager#getCachedLocation >> > HBASE-9954 Incorporate HTTPS support for HBase >> > HBASE-9908 [WINDOWS] Fix filesystem / classloader related unit tests >> > HBASE-9112 Custom TableInputFormat in initTableMapperJob throws >> > ClassNoFoundException on TableMapper >> > HBASE-9947 Add CM action for online compression algorithm change >> > HBASE-9958 Remove some array copy, change lock scope in locateRegion >> > HBASE-9880 client.TestAsyncProcess.testWithNoClearOnFail broke on 0.96 >> by >> > HBASE-9867 >> > HBASE-9810 Global memstore size will be calculated wrongly if replaying >> > recovered edits throws exception >> > HBASE-9907 HBASE-9775 Rig to fake a cluster so can profile client >> behaviors >> > HBASE-9870 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format >> > HBASE-9757 Reenable fast region move in SlowDeterministicMonkey >> > HBASE-9710 Use the region name, not the encoded name, when region is not >> > on current server >> > HBASE-9926 Scanner doesn't check if a region is available >> > HBASE-9918 MasterAddressTracker & ZKNamespaceManager ZK listeners are >> > missed after master recovery >> > HBASE-9917 HBASE-9775 Fix it so Default Connection Pool does not spin up >> > max threads even when not needed >> > HBASE-9895 0.96 Import utility can't import an exported file from 0.94 >> > HBASE-9957 [WINDOWS] TestNamespaceUpgrade fails on windows >> > HBASE-9885 Avoid some Result creation in protobuf conversions >> > HBASE-9563 Autorestart doesn't work if zkcleaner fails >> > HBASE-9792 Region states should update last assignments when a region is >> > opened. >> > HBASE-9813 Log splitting doesn't prevent RS creating new hlog file >> > HBASE-9900 Fix unintended byte[].toString in AccessController >> > HBASE-9909 TestHFilePerformance should not be a unit test, but a tool >> > HBASE-9737 Corrupt HFile cause resource leak leading to Region Server >> OOM >> > HBASE-9751 Excessive readpoints checks in StoreFileScanner >> > HBASE-9716 LoadTestTool should provide default min and max settings to >> the >> > data generator >> > HBASE-7600 TestAdmin.testCreateBadTables is failing occasionally >> > HBASE-9807 block encoder unnecessarily copies the key for each reseek >> > HBASE-9753 Excessive readpoint checks in MemstoreScanner >> > HBASE-9745 HBASE_CLASSPATH to end of Java classpath and use another env >> > var for prefix >> > HBASE-9851 TestHBaseFsck.testQuarantineMissingHFile is flaky >> > HBASE-9747 PrefixFilter with OR condition gives wrong results >> > HBASE-9727 HBase Rest Server - DELETE scanner operation is a no-op >> > HBASE-9221 Provide interface for getting a User in the client >> > HBASE-9886 Optimize ServerName#compareTo >> > HBASE-9837 HBASE-9080 'Retain assignment should be used when re-enabling >> > table(s)' >> > HBASE-9901 Add a toString in HTable, fix a log in AssignmentManager >> > HBASE-9855 evictBlocksByHfileName improvement for bucket cache >> > HBASE-9836 Intermittent >> > >> TestRegionObserverScannerOpenHook#testRegionObserverCompactionTimeStacking >> > failure >> > HBASE-9859 Canary Shouldn't go off if the table being read from is >> disabled >> > HBASE-9867 Save on array copies with a subclass of LiteralByteString >> > HBASE-9868 Remove some array copy, especially around protobuf >> > HBASE-9871 PB issue : Increment attributes are not getting passed to >> server >> > HBASE-9593 Region server left in online servers list forever if it went >> > down after registering to master and before creating ephemeral node >> > HBASE-9862 manage error per server and per region in the protobuffed >> client >> > HBASE-9861 Location does not have to be refreshed on regionTooBusy >> > HBASE-9659 some integration tests can no longer be run using maven >> > HBASE-9822 IntegrationTestLazyCfLoading failed occasionally in a secure >> > enviroment >> > HBASE-9843 Various fixes in client code >> > HBASE-9750 Add retries around Action server stop/start >> > HBASE-9759 Prevent random number collision in IntegrationTestBulkLoad >> > HBASE-9848 Make regionserver Canary exit with different exit codes >> > HBASE-9765 TestImportTsv.testBulkOutputWithTsvImporterTextMapper() may >> not >> > be working correctly >> > HBASE-7525 A canary monitoring program specifically for regionserver >> > HBASE-9748 HBASE-9696 >> > HBASE-9827 Intermittent TestLogRollingNoCluster#testContendedLogRolling >> > failure >> > HBASE-9785 Fix heap size reporting in HRegion >> > HBASE-9821 Scanner id could collide >> > HBASE-9694 HBaseTestingUtility.createTable infinite loop >> > HBASE-9772 Normalize new client default values >> > HBASE-9768 Two issues in AsyncProcess >> > HBASE-9784 Switch to Hadoop-2.2 >> > HBASE-9793 Offline a region before it's closed could cause double >> > assignment >> > HBASE-9798 Include dependency hamcrest-core >> > HBASE-9796 npe in RegionServerCallable >> > HBASE-9776 Test Load And Verify Fails with TableNotEnabledException >> > HBASE-9788 [WINDOWS] Update rest server class name >> > HBASE-9755 Cannot run classes in hbase-server tests jar from command >> line >> > HBASE-9791 MR initializes scanner twice >> > HBASE-9773 Master aborted when hbck asked the master to assign a region >> > that was already online >> > HBASE-9720 >> > TestSplitTransactionOnCluster#testShutdownFixupWhenDaughterHasSplit >> > occasionally times out >> > HBASE-9771 [WebUI] Humanize store and blockcache statistics on RS >> > HBASE-9570 With AccessDeniedException, HBase shell would be better to >> just >> > display the error message to be user friendly >> > HBASE-9758 Log missing rows and their regions if ITBLL fails >> > HBASE-9692 TestOrderedBytes doesn't display the arrays correctly >> > >> > >> > > > > -- > // Jonathan Hsieh (shay) > // Software Engineer, Cloudera > // [email protected] > > -- // Jonathan Hsieh (shay) // Software Engineer, Cloudera // [email protected]
