[
https://issues.apache.org/jira/browse/PHOENIX-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954850#comment-16954850
]
Kadir OZDEMIR commented on PHOENIX-5527:
----------------------------------------
[~priyankporwal], This change does not make any change on the read repair. As
you can see my patch did not make any change on GlobalIndexChecker (except
removing a debug statement). There will be no upgrade/switch issue. I want to
make sure I am clear on this. What we are changing here for the on-disk image
of a given index table is the unverified cell timestamp with ts-1. We do not
change the timestamp of the verified rows. Unverified cells are not visible to
the clients. This does not impact the correctness of read verify.
Now let's talk about a very interesting case. Assume that a scan is done on
with on a Phoenix connection with CurrentSCN. This will set the time range for
the scan [0, CurrentSCN]. Now assume that there are some unverified cells with
ts = CurrentSCN. This unverified rows will trigger read repair. If an
unverified cell is created because of deleting a data table row, the delete
marker for that row will not visible to read repair. Instead, the previous
version of the data table row will be retrieved. If compaction happened before
this scan and the data table row is deleted, then the row will not visible to
scan, which is expected behavior. I will explain this later.
If the unverified cell is created because of overwriting a data table row,
again the read repair will rebuild the previous version of the row if the
compaction process did not removed it already, otherwise noting will be
returned. Again this is the expected behavior.
Finally, If the unverified cell is created because of writing a new data table
row (i.e., with a new row key), then the row will not be visible to scan and
should not be.
Now, let me explain why rows may not be visible to the connections with
CurrentSCN after compaction. Assume we have a data table with no index. This
data table had two full row updates on the same row at time t0 and tN. Assume
that there was a compaction at time t1 where t0 < t1 < tN and removed the
version of the row with timestamp t0. After this compaction, any connection
with CurrentSCN < tN will not see this row. This is my understanding.
[~larsh], please correct me if I am wrong here.
> Unverified index rows should not be deleted due to replication lag
> -------------------------------------------------------------------
>
> Key: PHOENIX-5527
> URL: https://issues.apache.org/jira/browse/PHOENIX-5527
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 5.0.0, 4.14.3
> Reporter: Kadir OZDEMIR
> Assignee: Kadir OZDEMIR
> Priority: Major
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-5527.master.001.patch,
> PHOENIX-5527.master.002.patch, PHOENIX-5527.master.003.patch
>
>
> The current default delete time for unverified index rows is 10 minutes. If
> an index table row is replicated before its data table row and the
> replication row is unverified at the time of replication, it can be deleted
> when it is scanned on the destination cluster. To prevent these deletes due
> to replication lag issues, we should increase the default time to 7 days.
> This value is configurable using the configuration parameter,
> phoenix.global.index.row.age.threshold.to.delete.ms.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
