[
https://issues.apache.org/jira/browse/PHOENIX-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kadir OZDEMIR reassigned PHOENIX-5791:
--------------------------------------
Assignee: Kadir OZDEMIR
> Eliminate false invalid row detection due to concurrent updates
> ----------------------------------------------------------------
>
> Key: PHOENIX-5791
> URL: https://issues.apache.org/jira/browse/PHOENIX-5791
> Project: Phoenix
> Issue Type: Sub-task
> Reporter: Kadir OZDEMIR
> Assignee: Kadir OZDEMIR
> Priority: Major
>
> IndexTool verification generates an expected list of index mutations from the
> data table rows and uses this list to check if index table rows are
> consistent with the data table. To do that it follows the following steps:
> # The data table rows are scanned with a raw scan. This raw scan is
> configured to read all versions of rows.
> # For each scanned row, the cells that are scanned are grouped into two
> sets: put and delete. The put set is the set of put cells and the delete set
> is the set of delete cells.
> # The put and delete sets for a given row are further grouped based on their
> timestamps into put and delete mutations such that all the cells in a
> mutation have the timestamp.
> # The put and delete mutations are then sorted within a single list.
> Mutations in this list are sorted in ascending order of their timestamp.
> The above process assumes that for each data table update, the index table
> will be updated with the correct index row key. However, this assumption does
> not hold in the presence of concurrent updates.
> From the consistent indexing design (PHOENIX-5156) perspective, two or more
> pending updates from different batches on the same data row are concurrent if
> and only if for all of these updates the data table row state is read from
> HBase under the row lock and for none of them the row lock has been acquired
> the second time for updating the data table. In other words, all of them are
> in the first update phase concurrently. For concurrent updates, the first two
> update phases are done but the last update phase is skipped. This means the
> data table row will be updated by these updates but the corresponding index
> table rows will be left with the unverified status. Then, the read repair
> process will repair these unverified index rows during scans.
> In addition to leaving index rows unverified, the concurrent updates may
> generate index row with incorrect row keys. For example, consider that
> application issues the verify first two upserts on the same row concurrently
> and the second update does not include one or more of the indexed columns.
> When these updates arrive concurrently to IndexRegionObserver, the existing
> row state would be found null for both of these updates. This mean the index
> updates will be generated solely from the pending updates. The partial upsert
> with missing indexed columns will generate an index row by assuming missing
> indexed columns have null value, and this assumption may not true as the
> other concurrent upsert may have non-null values for indexed columns.
> Since expected index mutations are derived from the data table row after
> these concurrent mutations are applied, the expected list would not match
> with the actual list of index mutations. Please note that this does not pose
> a correctness issue as these index rows are unverified and the data table row
> key can be correctly extracted from them by the read repair process even
> though their index row keys are incorrect.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
