[
https://issues.apache.org/jira/browse/PHOENIX-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950214#comment-16950214
]
Kadir OZDEMIR commented on PHOENIX-5515:
----------------------------------------
[~larsh], The new and old design coexist and share the same index committer
(TrackingParallelWriterIndexCommitter). The old design uses the HBase
connections of type INDEX_WRITER_CONNECTION_WITH_CUSTOM_THREADS_NO_RETRIES for
index write RPCs and returns MultiIndexWriteFailureException with
disableIndexOnFailure = false to the client when an index write fails. The
Phoenix client then retries index writes by simply going through the rebuild
path where data writes are skipped but index writes are replayed. The new
design does index write retries on the server side by using the connections of
INDEX_WRITER_CONNECTION_WITH_CUSTOM_THREADS. Before this Jira, the new designed
returned the same exception back with disableIndexOnFailure = true (expecting
that this will be honored by the Phoenix client). It turned that it is ignored.
Thus, the solution here is to wrap MultiIndexWriteFailureException in
DoNotRetryIOException so that the client does see this as an index failure and
does not retry and not change index status. This actually simplified the code a
bit.
I have tested the patch on a real cluster and verified it. I am waiting for +1
to commit. [~gjacoby] has reviewed the first patch and gave some minor
comments, please see [GitHub Pull Request
#598|https://github.com/apache/phoenix/pull/598]. I have updated PR based on
his comments.
> Able to write indexed value to data table without writing to index table
> ------------------------------------------------------------------------
>
> Key: PHOENIX-5515
> URL: https://issues.apache.org/jira/browse/PHOENIX-5515
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.14.3
> Reporter: Vincent Poon
> Assignee: Kadir OZDEMIR
> Priority: Major
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-5515.master.001.patch,
> PHOENIX-5515.master.002.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Using the 4.14.3 client, it still seems the IndexFailurePolicy is still
> kicking in, which disables the index on write failure. This means that while
> the index is in 'disabled' state, writes to the data table can happen without
> any writes to the index table. While in theory this might be ok since the
> rebuilder should eventually kick in and rebuild from the disable_timestamp,
> this breaks the new indexing design invariant that there should be no data
> table rows without a corresponding index row (potentially unverified), so
> this could potentially cause some unexpected behavior.
> Steps to repro:
> 1) Create data table
> 2) Create index table
> 3) "close_region" on index region from hbase shell
> 4) Upsert to data table
> Eventually after some number of retries, the index will get disabled, which
> means any other client can write to the data table without writing to the
> index table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
