[
https://issues.apache.org/jira/browse/HIVE-27328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18059283#comment-18059283
]
László Bodor commented on HIVE-27328:
-------------------------------------
merged to master, thanks a lot [~ayushtkn] for the review!
> Acid dirCache is not invalidated in TezAMs while dropping table
> ---------------------------------------------------------------
>
> Key: HIVE-27328
> URL: https://issues.apache.org/jira/browse/HIVE-27328
> Project: Hive
> Issue Type: Improvement
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.3.0
>
> Attachments: Screenshot 2026-02-09 at 13.50.43.png
>
>
> The cache introduced in HIVE-22825 is not invalidated in TezAMs which can
> eventually lead to query failures if the same table is used in a scenario
> like below:
> 1. CREATE TABLE
> 2. INSERT OVERWRITE
> 3. SELECT
> 4. DROP TABLE
> ...
> in this case if 2) wrote a file like
> year=2011/base_0000001/bucket_00000_{*}1{*} (task attempt = 1), and in the
> next iteration it wrote year=2011/base_0000001/bucket_00000_{*}0{*} (task
> attempt = 0), then acid dirCache contains an invalid value within the
> configured time range *hive.txn.acid.dir.cache.duration*
> see debug screenshot where I found the actual base bucket file in the cached
> DirInfo: !Screenshot 2026-02-09 at 13.50.43.png|width=587,height=269!
> This cache is stored in memory, and the HS2-side is taken care of by
> HIVE-26060, but for the TezAMs, we need further improvement to achieve the
> same.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
