I modified the test so that the unit test runs against branch-1.4 : + Configuration hbaseConfig = TEST_UTIL.getConfiguration(); + //hbaseConfig.set(HConstants.ZOOKEEPER_QUORUM, "hbase123.docker");
However the test runs to completion without assertion - meaning the assertion wouldn't be triggered in homogeneous cluster. Here were the last few lines in the test output: 2017-12-31 07:33:36,865 INFO [Time-limited test] client.HBaseAdmin$CreateTableFuture(793): Created AssertionTest table created 2017-12-31 07:33:36,871 DEBUG [Time-limited test] client.ClientScanner(248): Advancing internal scanner to startKey at '\x01\x02\x03\x04\x05\x06\x07', inclusive 2017-12-31 07:33:36,873 DEBUG [hconnection-0x6e0982ab-shared--pool24-t3] ipc.RpcConnection(126): Use SIMPLE authentication for service ClientService, sasl=false 2017-12-31 07:33:36,873 DEBUG [hconnection-0x6e0982ab-shared--pool24-t3] ipc.BlockingRpcConnection(431): Connecting to tyus-mbp.attlocal.net/192.168.1.64:61585 2017-12-31 07:33:36,875 DEBUG [RpcServer.listener,port=61585] ipc.RpcServer$Listener(920): RpcServer.listener,port=61585: connection from 192.168.1.64:61633; # active connections: 3 2017-12-31 07:33:36,875 INFO [RpcServer.reader=0,bindAddress= tyus-mbp.attlocal.net,port=61585] ipc.RpcServer$Connection(1809): Auth successful for tyu (auth:SIMPLE) 2017-12-31 07:33:36,875 INFO [RpcServer.reader=0,bindAddress= tyus-mbp.attlocal.net,port=61585] ipc.RpcServer$Connection(1839): Connection from 192.168.1.64 port: 61633 with version info: version: "1.4.1-SNAPSHOT" url: "git://TYus-MBP.attlocal.net/Users/tyu/1.4-hbase" revision: "a7383851b9e62f23eb6f06eb23f09967f636c985" user: "tyu" date: "Sun Dec 31 07:32:58 PST 2017" src_checksum: "39bdbbe40374e2094f53ce77af539671" version_major: 1 version_minor: 4 2017-12-31 07:33:36,917 INFO [main] hbase.ResourceChecker(171): after: client.TestHCM#test Thread=325 (was 305) I assume you pointed the test at a 1.2.3 cluster which triggered the assertion. It seems the patch which introduced the assertion didn't take into account client running against older release cluster deployment. FYI On Sun, Dec 31, 2017 at 6:09 AM, Clément Guillaume <[email protected]> wrote: > I have consistent failure with this code (a table with pre-splits): > > @Test >> public void test() throws IOException{ >> Configuration hbaseConfig = HBaseConfiguration.create(); >> hbaseConfig.set(HConstants.ZOOKEEPER_QUORUM, "hbase123.docker"); >> Connection connection = ConnectionFactory.createConnection(hbaseConfig); >> > > > TableName tableName = TableName.valueOf("AssertionTest"); >> Admin admin = connection.getAdmin(); >> >> if(!admin.tableExists(tableName)){ >> HTableDescriptor htable = new HTableDescriptor(tableName); >> htable.addFamily(new HColumnDescriptor(new byte[]{(byte)'a'})); >> byte[][] splitPoints = {{1, 2, 3, 4, 5, 6, 7}}; >> admin.createTable(htable, splitPoints); >> System.out.println("table created"); >> } >> >> Table table = connection.getTable(tableName); >> ResultScanner scanner = table.getScanner(new Scan()); >> scanner.iterator().hasNext(); // Exception thrown here >> } > > > 2017-12-30 5:04 GMT-08:00 Ted Yu <[email protected]>: > >> bq. a table has been created with pre-splits >> >> Unless you record the initial number of regions somewhere, there is no >> built-in mechanism to provide the answer. >> >> Even if you cannot reproduce the assertion consistently, as long as the >> chance of reproduction is high, the unit test would still be useful. >> >> Cheers >> >> On Sat, Dec 30, 2017 at 3:31 AM, Clément Guillaume <[email protected] >> > wrote: >> >>> I dropped all the tables in my test cluster. Now I'm inconstantly able >>> to reproduce the issue. I'm digging more. >>> >>> I have a related question: is it possible to check if a table has been >>> created with pre-splits? >>> >>> I forgot to mention that I'm connecting to a version 1.2.3 cluster, but >>> I think this should be ok according to this post >>> http://mail-archives.apache.org/mod_mbox/hbase-dev/2017 >>> 11.mbox/%3C5BAAC90F-31D8-4A5F-B9E4-BA61FF4CD40E%40gmail.com%3E >>> >>> 2017-12-25 17:10 GMT-08:00 Ted Yu <[email protected]>: >>> >>>> I got the following if I use your code directly: >>>> >>>> testAssert(org.apache.hadoop.hbase.client.TestHCM) Time elapsed: >>>> 0.096 sec <<< ERROR! >>>> java.io.UncheckedIOException: >>>> org.apache.hadoop.hbase.TableNotFoundException: >>>> AssertionTest >>>> at org.apache.hadoop.hbase.client.TestHCM.testAssert(TestHCM.java:243) >>>> >>>> If I create the table at the beginning of the test, the test passes >>>> without triggering assertion. >>>> >>>> I tried both master branch and branch-1.4 >>>> >>>> As I mentioned previously, the assertion in your case was probably >>>> related to the formation of row keys in hbase:meta table. >>>> See if you can reproduce the assertion by presplitting the table, etc. >>>> >>>> Here is the diff I used based on branch-1.4 >>>> >>>> diff --git >>>> a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java >>>> b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/ >>>> TestHCM.java >>>> index a5e3a65..2944c8e 100644 >>>> --- a/hbase-server/src/test/java/org/apache/hadoop/hbase/client/ >>>> TestHCM.java >>>> +++ b/hbase-server/src/test/java/org/apache/hadoop/hbase/client/ >>>> TestHCM.java >>>> @@ -246,6 +246,18 @@ public class TestHCM { >>>> TEST_UTIL.shutdownMiniCluster(); >>>> } >>>> >>>> + @Test >>>> + public void testAssert() throws IOException{ >>>> + Configuration hbaseConfig = TEST_UTIL.getConfiguration(); >>>> + final TableName tableName = TableName.valueOf("assert"); >>>> + TEST_UTIL.createTable(tableName, FAM_NAM).close(); >>>> + String zkQuorum = "hbase123.docker"; >>>> + //hbaseConfig.set(HConstants.ZOOKEEPER_QUORUM, zkQuorum); >>>> + Connection connection = ConnectionFactory.createConnec >>>> tion(hbaseConfig); >>>> + Table table = connection.getTable(tableName); >>>> + ResultScanner scanner = table.getScanner(new Scan()); >>>> + scanner.iterator().hasNext(); // Exception thrown here >>>> + } >>>> >>>> private static int getHConnectionManagerCacheSize(){ >>>> return HConnectionTestingUtility.getConnectionCount(); >>>> >>>> On Mon, Dec 25, 2017 at 4:16 PM, Clément Guillaume < >>>> [email protected]> wrote: >>>> >>>>> Here is a unit test, note that the AssertionTest table can exit or not >>>>> exit, assertion will fail un both case. >>>>> >>>>> > @Test >>>>> > public void test() throws IOException{ >>>>> > Configuration hbaseConfig = HBaseConfiguration.create(); >>>>> > String zkQuorum = "hbase123.docker"; >>>>> > hbaseConfig.set(HConstants.ZOOKEEPER_QUORUM, zkQuorum); >>>>> > Connection connection = ConnectionFactory.createConnec >>>>> tion(hbaseConfig); >>>>> > Table table = connection.getTable(TableName. >>>>> valueOf("AssertionTest")); >>>>> > ResultScanner scanner = table.getScanner(new Scan()); >>>>> > scanner.iterator().hasNext(); // Exception thrown here >>>>> > } >>>>> > >>>>> >>>>> PS: I'm not receiving email when a response is posted here. I'm not >>>>> subscribed at the hbase-user mailing list but I would have hope that >>>>> answer >>>>> get sent to me as direct email. >>>>> >>>>> 2017-12-21 12:28 GMT-08:00 Clément Guillaume <[email protected]>: >>>>> >>>>> > I'm upgrading my projects from 1.2.6 to 1.4, and I have a system >>>>> assertion >>>>> > failure (in a failsafe test that run with system assertion enable) >>>>> > >>>>> > java.lang.AssertionError >>>>> > at org.apache.hadoop.hbase.client.ClientScanner. >>>>> > loadCache(ClientScanner.java:484) ~[hbase-shaded-client-1.4.0.ja >>>>> r:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScan >>>>> ner.java:312) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.ConnectionManager$ >>>>> > HConnectionImplementation.locateRegionInMeta(ConnectionManag >>>>> er.java:1324) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.ConnectionManager$ >>>>> > HConnectionImplementation.locateRegion(ConnectionManager.java:1221) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutu >>>>> reImpl. >>>>> > findAllLocationsOrFail(AsyncProcess.java:1063) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutu >>>>> reImpl. >>>>> > groupAndSendMultiAction(AsyncProcess.java:980) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.AsyncProcess$ >>>>> > AsyncRequestFutureImpl.access$200(AsyncProcess.java:667) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncP >>>>> rocess.java:649) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncP >>>>> rocess.java:612) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.ConnectionManager$ >>>>> > HConnectionImplementation.processBatchCallback(ConnectionMan >>>>> ager.java:2392) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > at org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.j >>>>> ava:950) >>>>> > ~[hbase-shaded-client-1.4.0.jar:1.4.0] >>>>> > >>>>> > I have one Put action that look like that : {"totalColumns":1,"row":" >>>>> > a7ttxdQY9euS-KEaPIMvKsRUWANCmigsaiT8_-V0Sgc\\x00visited\\x00\\xFF\\ >>>>> > xFF\\xFE\\x9F\\x85Dh\\x19","families":{"a":[{"qualifier":" >>>>> > value","vlen":17,"tag":[],"timestamp":9223372036854775807}]}} >>>>> > The cluster is a local standalone instance. >>>>> > >>>>> > I haven't dig into it yet, but it looks like the data is correctly >>>>> written >>>>> > in hbase if I disable assertion. >>>>> > Any idea of what's causing the issue? >>>>> > >>>>> > Thank you. >>>>> > >>>>> >>>> >>>> >>> >> >
