Lars Hofhansl created PHOENIX-5364:
--------------------------------------
Summary: Tests fail "Queue Full" only when run in parallel with
other tests.
Key: PHOENIX-5364
URL: https://issues.apache.org/jira/browse/PHOENIX-5364
Project: Phoenix
Issue Type: Test
Reporter: Lars Hofhansl
Spooling defaults to true. Yet, I see some tests failing (sometimes) with
exceptions like this:
{code}
org.apache.phoenix.exception.PhoenixIOException:
org.apache.hadoop.hbase.DoNotRetryIOException:
N000531,00D300000000XHP00A\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1560900587336.ca5a6d7b1f0951e30d15567c3aa909b1.:
Queue full. Consider increasing memory threshold or spooling to disk. Max
size: 1048576, Current size: 77, Number of elements:1
at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:121)
at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:87)
at
org.apache.phoenix.iterate.NonAggregateRegionScannerFactory.getTopNScanner(NonAggregateRegionScannerFactory.java:357)
at
org.apache.phoenix.iterate.NonAggregateRegionScannerFactory.getRegionScanner(NonAggregateRegionScannerFactory.java:182)
at
org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:187)
at
org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.overrideDelegate(BaseScannerRegionObserver.java:251)
at
org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:299)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2904)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3153)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36613)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2380)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277)
Caused by: java.lang.IllegalStateException: Queue full. Consider increasing
memory threshold or spooling to disk. Max size: 1048576, Current size: 77,
Number of elements:1
at org.apache.phoenix.iterate.SizeBoundQueue.add(SizeBoundQueue.java:58)
at
org.apache.phoenix.iterate.OrderedResultIterator.getResultIterator(OrderedResultIterator.java:315)
at
org.apache.phoenix.iterate.OrderedResultIterator.next(OrderedResultIterator.java:245)
at
org.apache.phoenix.iterate.NonAggregateRegionScannerFactory.getTopNScanner(NonAggregateRegionScannerFactory.java:352)
... 11 more
{code}
This should never happen.
However, I found two more tests that set
phoenix.query.server.orderBy.spooling.enabled to false:
* OrderByWithServerClientSpoolingDisabledIT, and
* OrderByWithServerMemoryLimitIT
They need to run in their own cluster.
(Also, OrderByWithServerMemoryLimitIT seems to not run at all (it does not show
in the logs of the jenkins run... Perhaps because it's a direct child of
BaseTest...?))
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
