[
https://issues.apache.org/jira/browse/PHOENIX-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Monani Mihir updated PHOENIX-5048:
----------------------------------
Description:
After rebuilder is finished for Partial Index Rebuild, It will check if Index
state has been updated after Upper bound of the scan we use in partial index
Rebuild. If that happens then it will fail Index Rebuild as Index write failure
occured while we were rebuilding Index.
{code:java}
MetaDataEndpointImpl.java#updateIndexState()
public void updateIndexState(RpcController controller, UpdateIndexStateRequest
request,
RpcCallback<MetaDataResponse> done) {
...
// If the index status has been updated after the upper bound of the scan we use
// to partially rebuild the index, then we need to fail the rebuild because an
// index write failed before the rebuild was complete.
if (actualTimestamp > expectedTimestamp) {
builder.setReturnCode(MetaDataProtos.MutationCode.UNALLOWED_TABLE_MUTATION);
builder.setMutationTime(EnvironmentEdgeManager.currentTimeMillis());
done.run(builder.build());
return;
}
...
}{code}
After Introduction of TrackingParallelWriterIndexCommitter
[PHOENIX-3815|https://issues.apache.org/jira/browse/PHOENIX-3815], we only
disable Index which get failure . Before that , in ParallelWriterIndexCommitter
we were disabling all index even if Index failure happens for one Index only.
Suppose Data Table has 3 index and above condition becomes true for first index
, then we won't even check for remain two Index.
{code:java}
MetaDataRegionObserver.java#BuildIndexScheduleTask.java#run()
for (PTable indexPTable : indexesToPartiallyRebuild) {
String indexTableFullName = SchemaUtil.getTableName(
indexPTable.getSchemaName().getString(),
indexPTable.getTableName().getString());
if (scanEndTime == latestUpperBoundTimestamp) {
IndexUtil.updateIndexState(conn, indexTableFullName, PIndexState.ACTIVE, 0L,
latestUpperBoundTimestamp);
batchExecutedPerTableMap.remove(dataPTable.getName());
LOG.info("Making Index:" + indexPTable.getTableName() + " active after
rebuilding");
} else {
// Increment timestamp so that client sees updated disable timestamp
IndexUtil.updateIndexState(conn, indexTableFullName,
indexPTable.getIndexState(), scanEndTime * signOfDisableTimeStamp,
latestUpperBoundTimestamp);
Long noOfBatches = batchExecutedPerTableMap.get(dataPTable.getName());
if (noOfBatches == null) {
noOfBatches = 0l;
}
batchExecutedPerTableMap.put(dataPTable.getName(), ++noOfBatches);
LOG.info("During Round-robin build: Successfully updated index disabled
timestamp for "
+ indexTableFullName + " to " + scanEndTime);
}
}
{code}
was:
After rebuilder is finished for Partial Index Rebuild, It will check if Index
state has been updated after Upper bound of the scan we use in partial index
Rebuild. If that happens then it will fail Index Rebuild as Index write failure
occured while we were rebuilding Index.
{code:java}
MetaDataEndpointImpl.java#updateIndexState
public void updateIndexState(RpcController controller, UpdateIndexStateRequest
request,
RpcCallback<MetaDataResponse> done) {
...
if (actualTimestamp > expectedTimestamp) {
builder.setReturnCode(MetaDataProtos.MutationCode.UNALLOWED_TABLE_MUTATION);
builder.setMutationTime(EnvironmentEdgeManager.currentTimeMillis());
done.run(builder.build());
return;
}
...
}{code}
Suppose Data Table has 3 index and above condition becomes true for first index
, then we won't even check for remain two Index.
{code:java}
for (PTable indexPTable : indexesToPartiallyRebuild) {
String indexTableFullName = SchemaUtil.getTableName(
indexPTable.getSchemaName().getString(),
indexPTable.getTableName().getString());
if (scanEndTime == latestUpperBoundTimestamp) {
IndexUtil.updateIndexState(conn, indexTableFullName, PIndexState.ACTIVE, 0L,
latestUpperBoundTimestamp);
batchExecutedPerTableMap.remove(dataPTable.getName());
LOG.info("Making Index:" + indexPTable.getTableName() + " active after
rebuilding");
} else {
// Increment timestamp so that client sees updated disable timestamp
IndexUtil.updateIndexState(conn, indexTableFullName,
indexPTable.getIndexState(), scanEndTime * signOfDisableTimeStamp,
latestUpperBoundTimestamp);
Long noOfBatches = batchExecutedPerTableMap.get(dataPTable.getName());
if (noOfBatches == null) {
noOfBatches = 0l;
}
batchExecutedPerTableMap.put(dataPTable.getName(), ++noOfBatches);
LOG.info("During Round-robin build: Successfully updated index disabled
timestamp for "
+ indexTableFullName + " to " + scanEndTime);
}
}
{code}
> Index Rebuilder does not handle INDEX_STATE timestamp check for all index
> -------------------------------------------------------------------------
>
> Key: PHOENIX-5048
> URL: https://issues.apache.org/jira/browse/PHOENIX-5048
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 4.14.1
> Reporter: Monani Mihir
> Priority: Major
>
> After rebuilder is finished for Partial Index Rebuild, It will check if Index
> state has been updated after Upper bound of the scan we use in partial index
> Rebuild. If that happens then it will fail Index Rebuild as Index write
> failure occured while we were rebuilding Index.
> {code:java}
> MetaDataEndpointImpl.java#updateIndexState()
> public void updateIndexState(RpcController controller,
> UpdateIndexStateRequest request,
> RpcCallback<MetaDataResponse> done) {
> ...
> // If the index status has been updated after the upper bound of the scan we
> use
> // to partially rebuild the index, then we need to fail the rebuild because an
> // index write failed before the rebuild was complete.
> if (actualTimestamp > expectedTimestamp) {
> builder.setReturnCode(MetaDataProtos.MutationCode.UNALLOWED_TABLE_MUTATION);
> builder.setMutationTime(EnvironmentEdgeManager.currentTimeMillis());
> done.run(builder.build());
> return;
> }
> ...
> }{code}
> After Introduction of TrackingParallelWriterIndexCommitter
> [PHOENIX-3815|https://issues.apache.org/jira/browse/PHOENIX-3815], we only
> disable Index which get failure . Before that , in
> ParallelWriterIndexCommitter we were disabling all index even if Index
> failure happens for one Index only.
> Suppose Data Table has 3 index and above condition becomes true for first
> index , then we won't even check for remain two Index.
> {code:java}
> MetaDataRegionObserver.java#BuildIndexScheduleTask.java#run()
> for (PTable indexPTable : indexesToPartiallyRebuild) {
> String indexTableFullName = SchemaUtil.getTableName(
> indexPTable.getSchemaName().getString(),
> indexPTable.getTableName().getString());
> if (scanEndTime == latestUpperBoundTimestamp) {
> IndexUtil.updateIndexState(conn, indexTableFullName, PIndexState.ACTIVE, 0L,
> latestUpperBoundTimestamp);
> batchExecutedPerTableMap.remove(dataPTable.getName());
> LOG.info("Making Index:" + indexPTable.getTableName() + " active after
> rebuilding");
> } else {
> // Increment timestamp so that client sees updated disable timestamp
> IndexUtil.updateIndexState(conn, indexTableFullName,
> indexPTable.getIndexState(), scanEndTime * signOfDisableTimeStamp,
> latestUpperBoundTimestamp);
> Long noOfBatches = batchExecutedPerTableMap.get(dataPTable.getName());
> if (noOfBatches == null) {
> noOfBatches = 0l;
> }
> batchExecutedPerTableMap.put(dataPTable.getName(), ++noOfBatches);
> LOG.info("During Round-robin build: Successfully updated index disabled
> timestamp for "
> + indexTableFullName + " to " + scanEndTime);
> }
> }
> {code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
