voonhous opened a new issue, #19722:
URL: https://github.com/apache/hudi/issues/19722
## Bug Description
**What happened:**
`ALTER TABLE ... SET TBLPROPERTIES` on a Hudi table only updates the Spark
catalog entry and
never rewrites `.hoodie/hoodie.properties`, so no Hudi table config is
changeable in-band
through SQL. `AlterTableCommand#applyPropertySet` builds a new
`CatalogTable` and calls
`catalog.alterTable(newTable)` -- nothing touches the table's own config
file:
```scala
// AlterTableCommand.scala:202-212
def applyPropertySet(sparkSession: SparkSession): Unit = {
val catalog = sparkSession.sessionState.catalog
val properties = changes.map(_.asInstanceOf[SetProperty]).map(f =>
f.property -> f.value).toMap
val newTable = table.copy(
properties = table.properties ++ properties, ...)
catalog.alterTable(newTable)
}
```
The result is a silent divergence: the catalog says one thing,
`hoodie.properties` (which the
write and read paths actually consult) says another. The user believes the
ALTER took effect,
and nothing warns them otherwise.
**Motivating case:**
#19648 added a write-time rejection whose remedy would naturally be "turn
the config off" --
but there is no in-band way to do that:
1. `ALTER TABLE t SET
TBLPROPERTIES('hoodie.datasource.write.slash.separated.date.partitioning'='false')`
succeeds and does nothing effective.
2. Passing the config as a write option instead trips
`HoodieWriterUtils#validateTableConfig`'s
config-diff rejection, since the option conflicts with the persisted
table config.
So for any table config, the only escapes are `SaveMode.Overwrite`
(recreates the table) or
hand-editing `hoodie.properties`. The error message in #19648 now says
"recreate the table"
because that is the only honest advice.
**What you expected:**
Either `SET TBLPROPERTIES` on a `hoodie.`-prefixed key updates
`hoodie.properties` (with
validation of which configs are safe to change post-creation, since some are
layout-affecting
and genuinely immutable), or the command rejects `hoodie.`-prefixed keys
outright instead of
silently absorbing them into the catalog.
**Steps to reproduce:**
1. Create any Hudi table via Spark SQL.
2. `ALTER TABLE t SET
TBLPROPERTIES('hoodie.datasource.write.slash.separated.date.partitioning'='true')`.
3. Inspect `.hoodie/hoodie.properties` -- the key is absent; inserts do not
slash-separate.
**Suggested fix:**
Split table configs into mutable and immutable sets. For mutable ones, have
`applyPropertySet`/`applyPropertyUnset` write through to `hoodie.properties`
(via
`HoodieTableConfig.update`); for immutable ones, throw with a message naming
the recreate
path. Silently updating only the catalog is the worst of both worlds.
## Environment
**Hudi version:** master (post-#19648)
**Spark version:** 3.5 / 4.x (engine-agnostic: the gap is in
hudi-spark-common)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
