Sanjeet Malhotra created PHOENIX-7184:
-----------------------------------------
Summary: Teardown logic is failing for tests
MutableIndexSplit*ScanIT
Key: PHOENIX-7184
URL: https://issues.apache.org/jira/browse/PHOENIX-7184
Project: Phoenix
Issue Type: Bug
Affects Versions: 5.2.0
Environment: JDK 8 with HBase 2.5
Reporter: Sanjeet Malhotra
If ITs: MutableIndexSplitReverseScanIT and MutableIndexSplitForwardScanIT, were
run and table count threshold defined in BaseTest class is reached for the
minicluster then teardown logic is invoked. The teardown logic deregisters the
Phoenix driver first followed by deleting the table. As the Phoenix Driver got
deregistered so regions were about to be opened seemed to be stuck in
transition and causing deletion of tables to fail.
Steps to reproduce error using MutableIndexSplitReverseScanIT:
# Set `TEARDOWN_THRESHOLD` to 1 in BaseTest.
# Run the IT and once it completes, check for following lines in failsafe
`*-output.txt` file.
##
{quote}2024-01-17T08:40:58,966 INFO [Listener at localhost/64544]
query.BaseTest(811): Shutting down mini cluster because number of tables on
this mini cluster is likely greater than 1{quote}
##
{quote}2024-01-17T08:41:01,408 ERROR
[RS_OPEN_REGION-regionserver/localhost:0-1] regionserver.HRegion(1155): Could
not initialize all stores for the
region=TBL_N000007,m,1705509658950.de4279eace097e5b32211e520b666b2c.
2024-01-17T08:41:01,408 ERROR [RS_OPEN_REGION-regionserver/localhost:0-0]
regionserver.HRegion(1155): Could not initialize all stores for the
region=TBL_N000007,e,1705509658950.aeadc0dbcdd260980ef8cffa1463c3f2.{quote}
Exception stacktrace:
{quote}java.io.IOException: java.io.IOException: java.io.IOException:
java.sql.SQLException: No suitable driver found for
jdbc:phoenix+zk:127.0.0.1\:52437::/hbase;test=true;
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1150)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeStores(HRegion.java:1093)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:996)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:943)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7230)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:7185)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7161)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7120)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7076)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at
org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:149)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_345]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_345]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_345]
Caused by: java.io.IOException: java.io.IOException: java.sql.SQLException: No
suitable driver found for jdbc:phoenix+zk:127.0.0.1\:52437::/hbase;test=true;
at
org.apache.hadoop.hbase.regionserver.StoreEngine.openStoreFiles(StoreEngine.java:288)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at
org.apache.hadoop.hbase.regionserver.StoreEngine.initialize(StoreEngine.java:338)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at org.apache.hadoop.hbase.regionserver.HStore.<init>(HStore.java:297)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6361)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1116)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1113)
~[hbase-server-2.5.5-hadoop3.jar:2.5.5-hadoop3]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_345]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_345]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_345]
... 3 more{quote}
If we call deregister Driver after deleting all the tables then we run into
cyclic dependency where implementation of CQSI for test setup calls
`clearCache` and that requires SYSCAT to be present. As this is just for
test/IT env so, in teardown logic we can delete non system tables first
followed by deregistering driver and lastly delete system tables.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
