[
https://issues.apache.org/jira/browse/HIVE-29825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manya Mehrotra updated HIVE-29825:
----------------------------------
Description:
Fix stale CM-root cache in {{ReplChangeManager}} that causes ACID replication
{{REPL LOAD}} failures after {{DROP DATABASE ... CASCADE}} and re-bootstrap.
Problem
When the CM root directory is removed from HDFS (e.g. {{.cmroot}} under the
database dir during {{{}DROP DATABASE CASCADE{}}}), the HMS JVM keeps a stale
entry in the in-memory {{{}encryptionZoneToCmrootMapping{}}}. Later MOVE
recycle operations use the cached path without checking HDFS, {{rename()}}
fails because the parent directory is missing, and the code hits {{setTimes()}}
on a non-existent destination, producing:
{{FileNotFoundException: .../.cmroot/bucket_XXXXX_<checksum> does not exist}}
Root cause
{{ReplChangeManager.getCmRoot()}} returns the cached CM root path on cache hit
without verifying the directory still exists on HDFS.
Fix
On cache hit, verify the CM root exists on HDFS; if missing, recreate it via
existing {{createCmRoot()}} (mkdirs + permissions). Preserves
per-encryption-zone behavior and uses double-checked locking consistent with
the cache-miss path.
Testing
> Fix stale CM root cache in ReplChangeManager after HDFS deletion
> ----------------------------------------------------------------
>
> Key: HIVE-29825
> URL: https://issues.apache.org/jira/browse/HIVE-29825
> Project: Hive
> Issue Type: Task
> Reporter: Manya Mehrotra
> Assignee: Manya Mehrotra
> Priority: Major
> Labels: pull-request-available
>
> Fix stale CM-root cache in {{ReplChangeManager}} that causes ACID replication
> {{REPL LOAD}} failures after {{DROP DATABASE ... CASCADE}} and re-bootstrap.
> Problem
> When the CM root directory is removed from HDFS (e.g. {{.cmroot}} under the
> database dir during {{{}DROP DATABASE CASCADE{}}}), the HMS JVM keeps a stale
> entry in the in-memory {{{}encryptionZoneToCmrootMapping{}}}. Later MOVE
> recycle operations use the cached path without checking HDFS, {{rename()}}
> fails because the parent directory is missing, and the code hits
> {{setTimes()}} on a non-existent destination, producing:
> {{FileNotFoundException: .../.cmroot/bucket_XXXXX_<checksum> does not exist}}
> Root cause
> {{ReplChangeManager.getCmRoot()}} returns the cached CM root path on cache
> hit without verifying the directory still exists on HDFS.
> Fix
> On cache hit, verify the CM root exists on HDFS; if missing, recreate it via
> existing {{createCmRoot()}} (mkdirs + permissions). Preserves
> per-encryption-zone behavior and uses double-checked locking consistent with
> the cache-miss path.
> Testing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
