[
https://issues.apache.org/jira/browse/HIVE-26976?focusedWorklogId=841078&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-841078
]
ASF GitHub Bot logged work on HIVE-26976:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Jan/23 11:34
Start Date: 23/Jan/23 11:34
Worklog Time Spent: 10m
Work Description: VenuReddy2103 opened a new pull request, #3976:
URL: https://github.com/apache/hive/pull/3976
### What changes were proposed in this pull request?
Setting savepoint in transaction before direct sql processing and rollback
to savepoint before moving to JDO processing.
### Why are the changes needed?
Failure in direct SQL processing do not undo the changes made to database
before falling back to JDO based processing. Thus results in stale/dangling
entries in database forever.
For instance, during dropPartitions() direct SQL processing, after dropping
rows from few tables(like PARTITIONS, PARTITION_PARAMS, PARTITION_KEY_VALS
etc), it causes typecast exception in dropStorageDescriptors() and fallback to
JDO processing. But datanucleus JDO processing cannot delete rows from
remaining tables(i.e., from sds, serdes, sds_params, serde_params, sort_cols,
bucketing_cols, skewed cols/values/location if any for the partitions) since
the partition is already in the same transaction during direct SQL processing.
This issue is applicable in the cases where database is modified(i.e., add
partitions[https://github.com/apache/hive/pull/3905](https://github.com/apache/hive/pull/3905)
, drop partitions) in direct sql processing and fails in between.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Tested manually and also ran unittests
Issue Time Tracking
-------------------
Worklog Id: (was: 841078)
Remaining Estimate: 0h
Time Spent: 10m
> Failure in direct SQL processing do not undo the changes made to database
> before fallback to JDO resulting in stale entries in database forever
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-26976
> URL: https://issues.apache.org/jira/browse/HIVE-26976
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Venugopal Reddy K
> Priority: Major
> Attachments: image-2023-01-23-15-23-00-975.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *[Description]*
> Failure in direct SQL processing do not undo the changes made to database
> before falling back to JDO based processing. Thus results in stale/dangling
> entries in database forever.
> For instance, during dropPartitions() direct SQL processing, after dropping
> rows from few tables(like PARTITIONS, PARTITION_PARAMS, PARTITION_KEY_VALS
> etc), it causes typecast exception in dropStorageDescriptors() and fallback
> to JDO processing. But datanucleus JDO processing cannot delete rows from
> remaining tables(i.e., from sds, serdes, sds_params, serde_params, sort_cols,
> bucketing_cols, skewed cols/values/location if any for the partitions) since
> the partition is already in the same transaction during direct SQL processing.
> !image-2023-01-23-15-23-00-975.png!
>
>
> *[Steps to reproduce]*
> Reproduction steps are described in issue -
> [https://issues.apache.org/jira/browse/HIVE-26860|https://issues.apache.org/jira/browse/HIVE-26860]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
