[
https://issues.apache.org/jira/browse/PHOENIX-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16817420#comment-16817420
]
Kadir OZDEMIR edited comment on PHOENIX-5156 at 4/14/19 9:09 PM:
-----------------------------------------------------------------
You got it right. If preparing index mutations for a batch of mutations takes
on average less than 0.5ms (at most 1ms), then the thread for this batch sleeps
for 1ms.
Assume that two batches of mutations concurrently want to update the same row X
and plus some other rows on the data table. One of them, batch 1, acquires the
row locks for its rows and the other, batch 2, is waiting for the row lock for
X.
Batch 1 goes ahead and prepares the index updates for its rows blazingly fast
and finishes this in less than 1ms (I think this will be a very rare event) .
It releases the row locks to do the RPC calls for index updates (i.e., the
first write phase).
Batch 2 now acquires its locks in the same ms and uses the same timestamp as
batch 1 did.
As you can see, the order in which the updates happens on row X on the data
table can be different than that on an index table in this case. Since these
updates use the same timestamp, HBase cannot order them correctly.
The solution I suggested for this problem is that Mutation 1 detects that the
current ms is the same as the ms just before the index update preparation, and
thus, sleeps for 1ms before releasing the row locks so that Mutation 2 gets a
different timestamp.
was (Author: kozdemir):
You got it right. Not only two batches of mutations need to arrive on the same
millisecond for at least one common row but also the one acquiring the row
locks first needs to take on average less than 0.5ms (at most 1ms) to prepare
the index mutations.
Assume that two batches of mutations concurrently want to update the same row X
and plus some other rows on the data table. One of them, batch 1, acquires the
row locks for its rows and the other, batch 2, is waiting for the row lock for
X.
Batch 1 goes ahead and prepares the index updates for its rows blazingly fast
and finishes this in less than 1ms (I think this will be a very rare event) .
It releases the row locks to do the RPC calls for index updates (i.e., the
first write phase).
Batch 2 now acquires its locks in the same ms and uses the same timestamp as
batch 1 did.
As you can see, the order in which the updates happens on row X on the data
table can be different than that on an index table in this case. Since these
updates use the same timestamp, HBase cannot order them correctly.
The solution I suggested for this problem is that Mutation 1 detects that the
current ms is the same as the ms just before the index update preparation, and
thus, sleeps for 1ms before releasing the row locks so that Mutation 2 gets a
different timestamp.
> Consistent Mutable Global Indexes for Non-Transactional Tables
> --------------------------------------------------------------
>
> Key: PHOENIX-5156
> URL: https://issues.apache.org/jira/browse/PHOENIX-5156
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.13.0, 4.14.0, 5.0.0, 4.14.1
> Reporter: Kadir OZDEMIR
> Assignee: Kadir OZDEMIR
> Priority: Major
> Attachments: PHOENIX-5156.master.001.patch,
> PHOENIX-5156.master.002.patch, PHOENIX-5156.master.003.patch,
> PHOENIX-5156.master.004.patch, PHOENIX-5156.master.005.patch,
> PHOENIX-5156.master.006.patch, PHOENIX-5156.master.007.patch,
> PHOENIX-5156.master.008.patch
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Without transactional tables, the mutable global indexes can get easily out
> of sync with their data tables in Phoenix. Transactional tables require a
> separate transaction manager, have some restrictions and performance
> penalties. This issue is to have consistent mutable global indexes without
> the need for using transactional tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
