[
https://issues.apache.org/jira/browse/HIVE-30265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-30265:
----------------------------------
Labels: pull-request-available (was: )
> Table rename retried by RetryingHMSHandler leaves partitions at the old
> location
> --------------------------------------------------------------------------------
>
> Key: HIVE-30265
> URL: https://issues.apache.org/jira/browse/HIVE-30265
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: dzcxzl
> Priority: Major
> Labels: pull-request-available
>
> h3. Problem
> When {{alter_table}} renames a managed partitioned table and the first
> attempt fails with a recoverable datastore error, {{RetryingHMSHandler}}
> retries the call and the retry reports success, but the table is left
> inconsistent:
> * table name: new
> * table location: new path, which does not exist on the file system
> * partition locations: still the old path
> * data: still in the old directory
> The client gets no error.
> h3. Root cause
> {{HiveAlterHandler.alterTable}} decides whether a rename has to move data by
> comparing the locations of the old and the new table object:
> {code:java}
> boolean renamedManagedTable = rename &&
> !oldt.getTableType().equals(TableType.VIRTUAL_VIEW.toString())
> && (oldt.getSd().getLocation().compareTo(newt.getSd().getLocation()) == 0
> || StringUtils.isEmpty(newt.getSd().getLocation()))
> && (!MetaStoreUtils.isExternalTable(oldt));
> {code}
> Inside that flow it rewrites the location of the caller's object:
> {code:java}
> destPath = new Path(databasePath, newTblName);
> newt.getSd().setLocation(destPath.toString());
> {code}
> {{RetryingHMSHandler}} retries with the same argument objects
> ({{{}method.invoke(baseHandler, args){}}}). If the first attempt fails after
> this point with an exception that {{DatabaseProduct.isRecoverableException}}
> accepts, the following happens:
> # The first attempt rolls back the transaction and moves the directory back
> to the old path.
> # The retry sees {{newt}} with the location already set to the renamed path,
> so {{renamedManagedTable}} is false.
> # The retry is handled as "rename with a location chosen by the caller":
> only the table name and table location are written, and the transaction
> commits.
> h3. Impact
> Queries on the renamed table keep working because partition locations still
> point at existing data, so the inconsistency goes unnoticed. The old
> directory stays in use. If another table is later created with, or renamed
> to, the old name, both tables share the same partition directories. Dropping
> the renamed managed table then deletes those directories, because its
> partition locations are outside its own table directory, and the other table
> loses its data.
> h3. How to reproduce
> # Create a managed partitioned table with at least one partition.
> # Make the first {{RawStore.alterPartitions}} call fail with a
> {{{}JDODataStoreException{}}}.
> # Rename the table through the Thrift API with
> {{hive.metastore.hmshandler.retry.attempts}} > 1.
> The call succeeds, the new table directory does not exist, and the partitions
> still point at the old directory. A unit test doing exactly this with
> {{InjectableBehaviourObjectStore}} is included in the patch.
> h3. Proposed fix
> Remember the location passed in by the caller before rewriting it, and put it
> back when the transaction does not commit, so that a retry sees the same
> input as the first attempt.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)