RyanLin5967 opened a new issue, #20479:
URL: https://github.com/apache/hudi/issues/20479
### Bug Description
**What happened:**
Currently, a KEEP_LATEST_FILE_VERSIONS clean stores `earliestCommitToRetain`
as `""` and the HUDI-9675 fallback (#13660) in `getLastCommitRetained` runs
only on null, so `create_savepoint` accepts a commit whose files were cleaned
and restoring it deletes the files that were left. KEEP_LATEST_COMMITS refuses
the same savepoint.
**What you expected:**
`create_savepoint` on `c2` fails with "Could not savepoint commit ... as
this is beyond the lookup window", as it does after a KEEP_LATEST_COMMITS clean.
**Steps to reproduce:**
```python
from pyspark.sql import SparkSession
spark = (SparkSession.builder
.config("spark.jars", "hudi-spark3.5-bundle_2.12-1.2.1.jar")
.config("spark.serializer",
"org.apache.spark.serializer.KryoSerializer")
.config("spark.sql.extensions",
"org.apache.spark.sql.hudi.HoodieSparkSessionExtension")
.config("spark.sql.catalog.spark_catalog",
"org.apache.spark.sql.hudi.catalog.HoodieCatalog")
.getOrCreate())
spark.sql("CREATE TABLE t (id INT, v STRING, ts LONG) USING hudi
TBLPROPERTIES (primaryKey = 'id', preCombineField = 'ts')")
spark.sql("INSERT INTO t VALUES (1, 'a', 1), (2, 'b', 1)")
spark.sql("UPDATE t SET v = 'c', ts = 2 WHERE id = 1")
spark.sql("UPDATE t SET v = 'd', ts = 3 WHERE id = 2")
c2 = sorted(r.commit_time for r in spark.sql("CALL show_commits(table =>
't')").collect())[1]
print(spark.sql("CALL run_clean(table => 't', clean_policy =>
'KEEP_LATEST_FILE_VERSIONS', file_versions_retained => 1)")
.select("total_files_deleted", "earliest_commit_to_retain").collect())
# [Row(total_files_deleted=2, earliest_commit_to_retain='')]
print(spark.sql(f"CALL create_savepoint(table => 't', commit_time =>
'{c2}')").collect())
# [Row(create_savepoint_result=True)], should be refused, since c2's file
was just cleaned
spark.sql(f"CALL rollback_to_savepoint(table => 't', instant_time =>
'{c2}')")
spark.sql("REFRESH TABLE t")
print(spark.sql("SELECT id, v FROM t ORDER BY id").collect())
# [], should be [Row(id=1, v='c'), Row(id=2, v='b')]
```
While a KEEP_LATEST_FILE_VERSIONS clean is still requested and not yet run,
its plan's `earliestInstantToRetain` is null and `create_savepoint` throws a
NullPointerException, even on the latest commit.
### Environment
**Hudi version:** 1.2.1 (`hudi-spark3.5-bundle_2.12`).
`SavepointActionExecutor` is the same on master.
**Query engine:** Spark 3.5.6 (pyspark, local mode) on Java 17.0.21, macOS
26.7.1, arm64 (Darwin 25.6.0)
**Relevant configs:** none beyond the defaults; copy-on-write table,
metadata table on
### Logs and Stack Trace
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
