SEPURI-SAI-KRISHNA opened a new pull request, #19703:
URL: https://github.com/apache/hudi/pull/19703
### Describe the issue this Pull Request addresses
Closes #19702.
`HoodieSqlCommonUtils#makePartitionPath` derives the on-disk partition
directory for the partition
DDL commands from the table config, but it only reads two of the three
configs that decide the
layout — hive-style partitioning and URL encoding — and never
`HoodieTableConfig#getSlashSeparatedDatePartitioning`. On a table written
with
`hoodie.datasource.write.slash.separated.date.partitioning=true` the writer
lays a value out as
`2026/01/05` while the DDL commands compute `2026-01-05`, so the two
disagree for every partition
value containing a dash.
Three commands are affected:
| command | consequence today |
|---|---|
| `ALTER TABLE ... ADD PARTITION` | creates a stray `<base>/2026-02-06/`
with a partition metafile in it, and the `hasPartitionMetadata` existence check
looks at the wrong path, so `IF NOT EXISTS` never sees the partition the writer
created |
| `ALTER TABLE ... DROP PARTITION` | targets a directory that does not
exist: reports success and drops nothing |
| `TRUNCATE TABLE ... PARTITION` | same, truncates nothing |
The silent no-ops are the dangerous half: the user is told the partition was
dropped and the data
is still queryable.
### Summary and Changelog
`makePartitionPath` now reads `getSlashSeparatedDatePartitioning` alongside
the two configs it
already honoured, and applies the same `-` -> `/` substitution the write
path performs, so DDL
names the very directory the writer created.
* `HoodieSqlCommonUtils#makePartitionPath` takes a
`slashSeparatedDatePartitioning` flag and
substitutes on the encoded value, after encoding, matching the ordering in
`KeyGenUtils#getRecordPartitionPath` and
`PartitionPathFormatterBase#combine`.
* The substitution is confined to a table partitioned by a **single**
column, mirroring the guard
in `KeyGenUtils#getRecordPartitionPath`. Multi-field slash partitioning is
separately broken and
tracked in #19666; this PR does not change its behavior.
* It is also skipped when hive-style partitioning is on. The config
documents the two as mutually
exclusive, and the write paths do not agree on what the combination
produces (#19669), so there
is no single directory for DDL to name.
`HoodieCatalogTable#extraTableConfig` already forces
`hive_style_partitioning=false` whenever the slash config is set through
SQL, so this only guards
tables created by `df.write`.
* A value with a leading dash is left alone. Substituting would produce a
partition path starting
with `/`, and an absolute relative-partition-path is resolved
inconsistently by
`FSUtils#constructAbsolutePath(String, String)` versus the `StoragePath`
overload used by
`AbstractTableFileSystemView`, so the writer and the file-system view
would disagree on where the
partition lives. Such a value is not a date to begin with. This matches
the guard on the write
path in #19648.
Tests, one per affected command, all on a SQL-created slash table:
* `TestAlterTableAddPartition` — asserts `ADD PARTITION` creates
`2026/02/06` and leaves no
`2026-02-06` behind, and that `ADD PARTITION` for a partition the writer
created is now correctly
rejected as already existing.
* `TestAlterTableDropPartition` — asserts the dropped rows are actually gone
after a clean.
* `TestTruncateTable` — asserts the truncated partition's rows are gone.
Each of the three fails on the unfixed code: `ADD PARTITION` with
`expected: <true> but was: <false>` on the slash directory, and
`DROP`/`TRUNCATE` still returning
the row that should have been removed.
### Impact
Partition DDL on tables using
`hoodie.datasource.write.slash.separated.date.partitioning` starts
naming the correct directory. `ADD PARTITION` stops creating stray
directories, and
`DROP`/`TRUNCATE PARTITION` start actually removing data instead of silently
succeeding.
No behavior change for any table that does not enable the config: the new
branch is guarded on
`getSlashSeparatedDatePartitioning`, which defaults to `false`.
A table that has already accumulated stray dashed directories from `ADD
PARTITION` is not cleaned
up by this change; those directories carry a partition metafile but no data.
### Risk Level
low
Confined to one method, guarded on a config that defaults to false, and the
guarded branch mirrors
the substitution the write path already performs. Verified with the three
new tests, plus
`TestAlterTableAddPartition`, `TestAlterTableDropPartition`,
`TestTruncateTable`,
`TestShowPartitions` and `TestSlashSeparatedPartitionValue` — 46 tests, all
passing — and with each
new test re-run against the unfixed code to confirm it fails there.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
