[
https://issues.apache.org/jira/browse/PHOENIX-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399837#comment-15399837
]
Samarth Jain commented on PHOENIX-3121:
---------------------------------------
I was slightly wrong. The cache size matters even for the case when the offset
is computed on the server side. This is because hbase internally uses that
cache size as the number of records to fetch from every server side scanner.
Having said that, the change to account for limit + offset doesn't help.
{code}
int scannerCacheSize = context.getStatement().getFetchSize();
Integer perScanLimit = QueryUtil.getOffsetLimit(limit, offset);
if (perScanLimit != null && perScanLimit % scannerCacheSize == 0) {
scan.setCaching(scannerCacheSize + 1);
}
{code}
select * from PLATFORM_ENTITY.engagement_history_poc where who_id =
'00Qx0000001S2qa' and organization_id='00Dx0000000GyYS' order by activity_date
desc LIMIT 8 OFFSET 1
Record 1 Time: 15
Record 2 Time: 0
Record 3 Time: 0
Record 4 Time: 0
Record 5 Time: 44021
Record 6 Time: 1
Record 7 Time: 0
Record 8 Time: 0
The perScanLimit of 9 in this case isn't a multiple of cache size 5. So the
hack isn't even getting used.
Switching to LIMIT 8 OFFSET 2, the hack gets used. This time it is the record
6th that hits the reverse scan bug:
Record 1 Time: 18
Record 2 Time: 0
Record 3 Time: 0
Record 4 Time: 0
Record 5 Time: 0
Record 6 Time: 44375
Record 7 Time: 0
Record 8 Time: 0
I noticed that my overall scan limit was more than the cache size. So I changed
the code to do this:
{code}
scan.setCaching(Math.max(perScanLimit, scannerCacheSize) + 1);
{code}
Now, for LIMIT 8 OFFSET 2 and cache size 5 it is the fetching the first record
(technically the third on server side) that hits the bug:
Record 1 Time: 44300
Record 2 Time: 0
Record 3 Time: 0
Record 4 Time: 0
Record 5 Time: 0
Record 6 Time: 0
Record 7 Time: 0
Record 8 Time: 1
> Queries with filter and reverse scan failing when limit is a multiple of
> scanner cache size
> -------------------------------------------------------------------------------------------
>
> Key: PHOENIX-3121
> URL: https://issues.apache.org/jira/browse/PHOENIX-3121
> Project: Phoenix
> Issue Type: Bug
> Reporter: Samarth Jain
> Assignee: Samarth Jain
> Fix For: 4.8.0
>
> Attachments: PHOENIX-3121.patch, PHOENIX-3121_v2.patch
>
>
> {code}
> org.apache.hadoop.hbase.UnknownScannerException:
> org.apache.hadoop.hbase.UnknownScannerException: Unknown scanner '644116'.
> This can happen due to any of the following reasons: a) Scanner id given is
> wrong, b) Scanner lease expired because of long wait between consecutive
> client checkins, c) Server may be closing down, d) RegionServer restart
> during upgrade.
> If the issue is due to reason (b), a possible fix would be increasing the
> value of'hbase.client.scanner.timeout.period' configuration.
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3228)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32492)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2208)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
