[
https://issues.apache.org/jira/browse/PHOENIX-6058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204189#comment-17204189
]
ASF GitHub Bot commented on PHOENIX-6058:
-----------------------------------------
gjacoby126 commented on pull request #901:
URL: https://github.com/apache/phoenix/pull/901#issuecomment-700896096
I can see two ways to handle the case where max lookback is disabled / not
supported :
1. As @kadirozde has done here, treat everything as beyond max lookback.
This will let the jobs pass, but all missing / invalid rows will go into the
beyond max lookback counters rather than the normal ones.
2. Treat nothing as beyond max lookback, because the concept is disabled /
not supported, but change the reducer to pass the job in this case. This will
put all missing / invalid rows in the normal counters.
Either of these could cause user confusion. In the first case, users might
wonder why everything's in the max lookback counters, and not know to look at
the data more closely in case there's a real failure. In the second case, users
might wonder why the job passed even though there's failed counters, and not
know that some of them might be compaction artifacts from the lack of max
lookback. (This might be alleviated with logging, assuming users read the
logs.)
Curious on what others' thoughts are, or if there's another approach I'm
missing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> When maxLookback is not enabled, IndexTool should not do deep verification
> --------------------------------------------------------------------------
>
> Key: PHOENIX-6058
> URL: https://issues.apache.org/jira/browse/PHOENIX-6058
> Project: Phoenix
> Issue Type: Bug
> Reporter: Kadir OZDEMIR
> Assignee: Kadir OZDEMIR
> Priority: Blocker
> Fix For: 5.1.0, 4.16.0
>
> Attachments: PHOENIX-6058.4.x.001.patch, PHOENIX-6058.4.x.002.patch,
> PHOENIX-6058.master.001.patch
>
>
> IndexTool does two types of verification, verifying the latest row versions,
> and verifying all row versions within the maxLookback windows. Verifying just
> the latest version can be called shallow verification whereas verifying all
> versions can be called deep verification. The maxLookback feature preserves
> all row versions including delete markers within the maxLookback window. This
> allows IndexTool to compare all versions of data rows with all versions of
> the index rows within the window. However, due to a bug, when the maxLookback
> feature is not enabled, i.e., the maxLookback window is set to zero,
> IndexTool assumes the window size is infinite and thus does deep
> verification. Since some versions of the index rows can be removed by
> compaction independently if the maxLookback feature is not enabled, IndexTool
> may report missing or invalid index rows by doing deep verification.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
