[
https://issues.apache.org/jira/browse/PHOENIX-5079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731250#comment-16731250
]
Monani Mihir edited comment on PHOENIX-5079 at 12/31/18 10:00 AM:
------------------------------------------------------------------
Checking for REPLAY_ONLY_INDEX_WRITES won't work because we are setting
REPLAY_WRITES in UngroupedAggregateRegionObserver#rebuildIndices as part of
put/delete mutations, current function implementations is correct.
In Indexer.java we set BatchMutateContext in
preBatchMuate#preBatchMutateWithExceptions and during
postBatchMutateIndispensably , we use same index updates to do final index
Mutations.
In Indexer#postBatchMutateIndispensably we are passing BatchMutateContext for
doPost() for Index Updates ant this list of muations doesn't have value set for
REPLAY_WRITES. When postBatchMutateIndispensably is called from
UngroupedAggregateRegionObserver#commitBatch , we are passing list of mutations
(miniBatchOp) which has correct value set but indexUpdates from
BatchMutateContext doesn't have that value.
{code:java}
public void
postBatchMutateIndispensably(ObserverContext<RegionCoprocessorEnvironment> c,
MiniBatchOperationInProgress<Mutation> miniBatchOp, final boolean success)
throws IOException {
...
BatchMutateContext context = getBatchMutateContext(c);
...
this.builder.batchCompleted(miniBatchOp);
LOG.info("postBatchMutateIndispensably_miniBatchOp isIndexRebuild :- " +
PhoenixIndexMetaData.isIndexRebuild(miniBatchOp.getOperation(0).getAttributesMap()));
LOG.info("postBatchMutateIndispensably_context isIndexRebuild :- " +
PhoenixIndexMetaData.isIndexRebuild(context.indexUpdates.iterator().next().getFirst().getAttributesMap()));
if (success) { // if miniBatchOp was successfully written, write index
updates
...
{code}
In above log output we would get true for miniBatchOp but false for
BatchMuateContext
How do we set BatchMutateContext in case of Partial Index Rebuilder? I am not
able to find it. Can you give some pointers?
[~vincentpoon] [~lhofhansl] [~jamestaylor]
was (Author: mihir6692):
Checking for REPLAY_ONLY_INDEX_WRITES won't work because we are setting
REPLAY_WRITES in UngroupedAggregateRegionObserver#rebuildIndices as part of
put/delete mutations, current function implementations is correct.
In Indexer.java we set BatchMutateContext in
preBatchMuate#preBatchMutateWithExceptions and during
postBatchMutateIndispensably , we use same index updates to do final index
Mutations.
In Indexer#postBatchMutateIndispensably we are passing BatchMutateContext for
doPost() for Index Updates ant this list of muations doesn't have value set for
REPLAY_WRITES. When postBatchMutateIndispensably is called from
UngroupedAggregateRegionObserver#commitBatch , we are passing list of mutations
(miniBatchOp) which has correct value set but indexUpdates from
BatchMutateContext doesn't have that value.
{code:java}
public void
postBatchMutateIndispensably(ObserverContext<RegionCoprocessorEnvironment> c,
MiniBatchOperationInProgress<Mutation> miniBatchOp, final boolean success)
throws IOException {
...
this.builder.batchCompleted(miniBatchOp);
LOG.info("postBatchMutateIndispensably_miniBatchOp isIndexRebuild :- " +
PhoenixIndexMetaData.isIndexRebuild(miniBatchOp.getOperation(0).getAttributesMap()));
LOG.info("postBatchMutateIndispensably_context isIndexRebuild :- " +
PhoenixIndexMetaData.isIndexRebuild(context.indexUpdates.iterator().next().getFirst().getAttributesMap()));
if (success) { // if miniBatchOp was successfully written, write index
updates
...
{code}
In above log output we would get true for miniBatchOp but false for
BatchMuateContext
How do we set BatchMutateContext in case of Partial Index Rebuilder? I am not
able to find it. Can you give some pointers?
[~vincentpoon] [~lhofhansl] [~jamestaylor]
> PhoenixIndexMetaData.isIndexRebuild gives wrong results in Partial Index
> Rebuilding
> -----------------------------------------------------------------------------------
>
> Key: PHOENIX-5079
> URL: https://issues.apache.org/jira/browse/PHOENIX-5079
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.14.1
> Reporter: Monani Mihir
> Priority: Minor
>
> During Partial Rebuild, If rebuilder get index failure (because of region
> move/split etc), it will call PhoenixIndexFailurePolicy.handleFailure . Here
> it checks if mutations are part of Index Rebuild or not .
> {code:java}
> if (!throwing) {
> SQLException sqlException =
> new
> SQLExceptionInfo.Builder(SQLExceptionCode.INDEX_WRITE_FAILURE)
>
> .setRootCause(cause).setMessage(cause.getLocalizedMessage()).build()
> .buildException();
> IOException ioException =
> ServerUtil.wrapInDoNotRetryIOException(null, sqlException, timestamp);
> Mutation m = attempted.entries().iterator().next().getValue();
> boolean isIndexRebuild =
> PhoenixIndexMetaData.isIndexRebuild(m.getAttributesMap());
> // Always throw if rebuilding index since the rebuilder needs to
> know if it was successful
> if (throwIndexWriteFailure || isIndexRebuild) {
> throw ioException;
> } else {
> LOG.warn("Swallowing index write failure", ioException);
> }
> }
> {code}
> Here isIndexRebuild becomes falls even though we have set attribute for
> REPLY_WRITE to
> REPLAY_ONLY_INDEX_WRITES in UngroupedAggregateRegionObserver.rebuildIndices
> {code:java}
> ...
> put.setAttribute(REPLAY_WRITES, REPLAY_ONLY_INDEX_WRITES);
> ...{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
