Sanjeet Malhotra created PHOENIX-7961:
-----------------------------------------
Summary: Secondary index diverges from data table after TTL expiry
on partial-touch upserts
Key: PHOENIX-7961
URL: https://issues.apache.org/jira/browse/PHOENIX-7961
Project: Phoenix
Issue Type: Bug
Reporter: Sanjeet Malhotra
Assignee: Sanjeet Malhotra
On a table or view with a literal TTL and a secondary index, an UPSERT that
does not re-write any index-referenced column (a "partial touch") can resurrect
logically-expired values into the index. Phoenix rebuilds the affected index
entry from the current on-disk row, but that internal current-row read is not
TTL-masked the way an ordinary client read is. Cells that are logically expired
under the TTL but not yet physically removed by compaction get rebuilt into the
index, while the data table correctly expires them on read. The data table and
its secondary index then diverge.
There is a second facet at the exact TTL boundary. Even once the internal read
is masked, \{{TTLRegionScanner}} derives its "current time" from the scan-open
wall clock, whereas the index is rebuilt at \{{batchTimestamp}}, which
\{{getBatchTimestamp}} bumps slightly forward to avoid timestamp collisions. A
cell whose expiry boundary falls in the sub-millisecond \{{(scanOpenTime,
batchTimestamp]}} window stays visible to the masked read, is rebuilt into the
index, yet is expired on the data side -- the same divergence at the boundary.
Data-integrity / correctness bug: after TTL expiry the secondary index returns
a value the data table no longer returns. No error is raised -- the divergence
is silent and surfaces only after logical TTL expiry and before major
compaction physically purges the expired cells.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
