[
https://issues.apache.org/jira/browse/SPARK-60128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yibo Dong updated SPARK-60128:
------------------------------
Labels: sql (was: )
> DROP DATABASE CASCADE leaves managed-table files after database location
> changes
> --------------------------------------------------------------------------------
>
> Key: SPARK-60128
> URL: https://issues.apache.org/jira/browse/SPARK-60128
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Environment: Spark: 5.0.0-SNAPSHOT
> Commit: 38422aefaf599ffb4cfa579d98898d95b1f02081
> Java: OpenJDK 17.0.17
> OS: macOS 15.7.4
> Reporter: Yibo Dong
> Priority: Major
> Labels: sql
>
> h2. What happened
> After changing a database's location using {{ALTER DATABASE ... SET
> LOCATION}}, {{DROP DATABASE ... CASCADE}} removes the database metadata but
> leaves the data files of an existing managed table at its original location.
> h2. How to reproduce
> Save the following SQL as {{tmp.sql}}:
> {code:sql}
> CREATE DATABASE t_cascade_location
> LOCATION '/tmp/spark_t_cascade_location_20261002/old';
> CREATE TABLE t_cascade_location.t (
> id INT,
> label STRING
> )
> USING PARQUET;
> INSERT INTO t_cascade_location.t VALUES (7, 'kept');
> DESCRIBE EXTENDED t_cascade_location.t;
> ALTER DATABASE t_cascade_location
> SET LOCATION '/tmp/spark_t_cascade_location_20261002/new';
> DROP DATABASE t_cascade_location CASCADE;
> SHOW DATABASES LIKE 't_cascade_location';
> {code}
> Use a fresh base directory or change the path suffix before running the SQL.
> Run with the in-memory session catalog:
> {code:bash}
> spark-sql --master 'local[2]'
> --conf spark.ui.enabled=false
> --conf spark.sql.catalogImplementation=in-memory
> -f tmp.sql
> {code}
> Then check the original managed-table location:
> {code:bash}
> find /tmp/spark_t_cascade_location_20261002/old/t
> -type f -name '*.parquet' -print
> {code}
> h2. Expected result
> {{DROP DATABASE ... CASCADE}} should remove the database and its managed
> table, including the table's data at its original location.
> {{SHOW DATABASES LIKE 't_cascade_location'}} should return no rows.
> The filesystem check should return no Parquet files:
> {code:none}
> (no output)
> {code}
> h2. Actual result
> The database is removed from the catalog, but the original managed-table data
> remains.
> {{SHOW DATABASES LIKE 't_cascade_location'}} returns no rows.
> The filesystem check still returns a Parquet file:
> {code:none}
> /tmp/spark_t_cascade_location_20261002/old/t/part-....snappy.parquet
> {code}
> The managed table's data files remain on disk after {{DROP DATABASE ...
> CASCADE}} completes successfully.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
