[
https://issues.apache.org/jira/browse/PHOENIX-2446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111165#comment-15111165
]
James Taylor commented on PHOENIX-2446:
---------------------------------------
Patch looks good, [~tdsilva]. Does it fix the issue? Here's some minor feedback:
- Do we really need the default sleep time to be 30seconds? Would 5 seconds be
enough?
- Don't do the catch-up UPSERT SELECT if the table is transactional (as it's
not needed).
- I don't think the min time range is being set correctly in MetaDataClient
here (shouldn't it be index.getTimeStamp()-delta):
{code}
long minTimestamp = index.getTimeStamp()-sleepTime;
{code}
- I don't think we need to add the Dependencies stuff in MutationState (unless
you really need that for your test verification). We already have a lot of
existing tests that verify that the data table and index are consistent after
mutations.
- No need to check for the index being a local index in MetaDataClient here, as
local indexes are handled in the if branch:
{code}
if (index.getIndexType() != IndexType.LOCAL && sleepTime>0) {
{code}
- We will want to run the catchup for local indexes (that aren't transactional)
too, but you'll need to involve the same code that's in the if statement (not
run the UPSERT SELECT). The idea with the PostIndexDDLCompiler is that it
abstracts this. Ideally, we could have a PostLocalIndexDDLCompiler and you
could just move the existing code into that. If that proves difficult, then
just factor out the code in the if statement to a new MetaDataClient method.
> Immutable index - Index vs base table row count does not match when index is
> created during data load
> -----------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-2446
> URL: https://issues.apache.org/jira/browse/PHOENIX-2446
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.6.0
> Reporter: Mujtaba Chohan
> Assignee: Thomas D'Silva
> Fix For: 4.7.0
>
> Attachments: PHOENIX-2446-v4.patch, PHOENIX-2446-wip-v3.patch,
> PHOENIX-2446-wip.patch, PHOENIX-2446.patch, PHOENIX-2446_partial.patch,
> PHOENIX-2446_testfix.patch, output.log, output_reads.log, output_writes.log,
> server.log
>
>
> I'll add more details later but here's the scenario that consistently
> produces wrong row count for index table vs base table for immutable async
> index.
> 1. Start data upsert
> 2. Create async index
> 3. Trigger M/R index build
> 4. Keep data upsert going in background during step 2,3 and a while after M/R
> index finishes.
> 5. End data upsert.
> Now count with index enabled vs count with hint to not use index is off by a
> large factor. Will get a cleaner repro for this issue soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
