[
https://issues.apache.org/jira/browse/PHOENIX-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanuj Khurana updated PHOENIX-8015:
-----------------------------------
Description:
When scanning an uncovered index for a query which has a limit clause but no
filter the client attaches a PageFilter to the scan. This PageFilter is handled
by the TTLRegionScanner
which sits in the scanner stack below the TTLRegionScanner.
{code:java}
RegionScannerHolder
-> UncoveredGlobalIndexRegionScanner <- join-back + verification (drops
orphans)
-> TTLRegionScanner <- strips PageFilter, counts RAW rows,
hard-stops
-> PagingRegionScanner
-> HBase raw index scanner{code}
All the rows in the uncovered index are unverified. TTLRegionScanner counts the
unverified rows towards the query limit but these rows can be orphaned rows
such that the data table row doesn't point back to the index row. In such
cases, the query returns incorrect results as the TTLRegionScanner returns no
more rows once the limit is reached but the UncoveredRegionScanner actually
drops those unverified rows.
Covered indexes don't have this problem because scans on covered indexes are
processed by GlobalIndexRegionScanner which removes the PageFilter from the
scan and then reinitializes the underlying the scanners
([GlobalIndexChecker#init()|https://github.com/apache/phoenix/blob/master/phoenix-core-server/src/main/java/org/apache/phoenix/index/GlobalIndexChecker.java#L205-L209])
Uncovered indexes should also follow the similar pattern and only count the row
towards the query limit after it has been checked and verified.
was:
When scanning an uncovered index for a query which has a limit clause but no
filter the client attaches a PageFilter to the scan. This PageFilter is handled
by the TTLRegionScanner
which sits in the scanner stack below the TTLRegionScanner.
{code:java}
RegionScannerHolder
-> UncoveredGlobalIndexRegionScanner <- join-back + verification (drops
orphans)
-> TTLRegionScanner <- strips PageFilter, counts RAW rows,
hard-stops
-> PagingRegionScanner
-> HBase raw index scanner{code}
All the rows in the uncovered index are unverified. TTLRegionScanner counts the
unverified rows towards the query limit but these rows can be orphaned rows
such that the data table row doesn't point back to the index row. In such
cases, the query returns incorrect results as the TTLRegionScanner returns no
more rows once the limit is reached but the UncoveredRegionScanner actually
drops those unverified rows.
Covered indexes don't have this problem because scans on covered indexes are
processed by GlobalIndexRegionScanner which removes the PageFilter from the
scan and then reinitializes the underlying the scanners
([GlobalIndexChecker#init()|https://github.com/apache/phoenix/blob/master/phoenix-core-server/src/main/java/org/apache/phoenix/index/GlobalIndexChecker.java#L205-L209]
Uncovered indexes should also follow the similar pattern and only count the row
towards the query limit after it has been checked and verified.
> LIMIT query on uncovered index with no filter silently truncates when a
> leading run of unverifiable index rows consumes the limit
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-8015
> URL: https://issues.apache.org/jira/browse/PHOENIX-8015
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 5.2.0, 5.2.1, 5.3.0, 5.2.2, 5.3.1, 5.3.2
> Reporter: Tanuj Khurana
> Priority: Major
>
> When scanning an uncovered index for a query which has a limit clause but no
> filter the client attaches a PageFilter to the scan. This PageFilter is
> handled by the TTLRegionScanner
> which sits in the scanner stack below the TTLRegionScanner.
> {code:java}
> RegionScannerHolder
> -> UncoveredGlobalIndexRegionScanner <- join-back + verification (drops
> orphans)
> -> TTLRegionScanner <- strips PageFilter, counts RAW
> rows, hard-stops
> -> PagingRegionScanner
> -> HBase raw index scanner{code}
> All the rows in the uncovered index are unverified. TTLRegionScanner counts
> the unverified rows towards the query limit but these rows can be orphaned
> rows such that the data table row doesn't point back to the index row. In
> such cases, the query returns incorrect results as the TTLRegionScanner
> returns no more rows once the limit is reached but the UncoveredRegionScanner
> actually drops those unverified rows.
> Covered indexes don't have this problem because scans on covered indexes are
> processed by GlobalIndexRegionScanner which removes the PageFilter from the
> scan and then reinitializes the underlying the scanners
> ([GlobalIndexChecker#init()|https://github.com/apache/phoenix/blob/master/phoenix-core-server/src/main/java/org/apache/phoenix/index/GlobalIndexChecker.java#L205-L209])
>
> Uncovered indexes should also follow the similar pattern and only count the
> row towards the query limit after it has been checked and verified.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
