[
https://issues.apache.org/jira/browse/HIVE-26609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis reassigned HIVE-26609:
------------------------------------------
> Cleanup external table directories created in qtests after test run
> -------------------------------------------------------------------
>
> Key: HIVE-26609
> URL: https://issues.apache.org/jira/browse/HIVE-26609
> Project: Hive
> Issue Type: Improvement
> Components: Tests
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Major
>
> In many qtests we are creating external tables by setting explicitly the
> location of the table.
> [https://github.com/apache/hive/blob/566f48d3d3fc740ef958bdf963e511e0853da402/ql/src/test/queries/clientnegative/authorization_uri_create_table_ext.q#L7]
> If the test does not remove explicitly the directory (as it happens above)
> then it remains there and may cause flakiness and unrelated test failures if
> other tests happen to use the same directory somehow.
> A recent case where this problem appeared (directory conflict between tests)
> is logged under HIVE-26584. There the solution was to add explicit rm
> commands in the qfiles.
> A more general solution would be to handle the cleanup of such directories
> inside
> [QTestUtil.clearTablesCreatedDuringTests|https://github.com/apache/hive/blob/566f48d3d3fc740ef958bdf963e511e0853da402/itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java#L342].
> The idea is to get the location for an external table from the metastore and
> remove the respective directory if it is under a known "safe" directory such
> as {{{}$\{system:test.tmp.dir{}}}}.
> As discussed under HIVE-26584 it might be risky to forcefully delete any kind
> of directory coming from an external table at the risk of corrupting the
> development environment. If we restrict the cleanup to known directories it
> should be fine though.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
