prashantwason opened a new pull request, #20414:
URL: https://github.com/apache/hudi/pull/20414
### Describe the issue this Pull Request addresses
`TIMESTAMP AS OF '<instant>'` with a Hudi-format instant
(`yyyyMMddHHmmssSSS`) fails when the time-travel relation is inside a `WITH`
clause:
```
org.apache.spark.sql.AnalysisException:
[INVALID_TIME_TRAVEL_TIMESTAMP_EXPR.INPUT] The time travel timestamp expression
"20261022165849221" is invalid. Cannot be casted to the "TIMESTAMP" type.
at
org.apache.spark.sql.catalyst.analysis.TimeTravelSpec$.create(TimeTravelSpec.scala:59)
```
The same literal works at top level and inside a derived table. Example:
```sql
WITH old AS (SELECT id FROM t TIMESTAMP AS OF '20261022165849221'),
new AS (SELECT id FROM t TIMESTAMP AS OF '20261022165851004')
SELECT new.id FROM new LEFT JOIN old ON new.id = old.id WHERE old.id IS NULL
```
Cause: `HoodieCommonSqlParser.convertNativeTimeTravel` rewrites Spark's
`RelationTimeTravel` into Hudi's `TimeTravelRelation` using
`transformDownWithSubqueries`. The CTE definitions of a `WITH` clause are held
in `UnresolvedWith.cteRelations`, which are not children of the node, so the
transform never visits them. Spark's own `TimeTravelSpec` then tries to cast
the Hudi instant to a timestamp and fails.
### Summary and Changelog
Time travel with Hudi instants now works inside CTE definitions.
- `HoodieCommonSqlParser.convertNativeTimeTravel`: also match
`UnresolvedWith` and rewrite each CTE definition recursively (which also covers
nested `WITH` clauses).
- New `TestTimeTravelInCte`: time travel with a Hudi instant at top level,
inside one CTE, inside two joined CTEs, and inside a derived table. Without the
fix the two CTE cases fail with the error above; with it all 4 pass (Spark 3.5
profile, JDK 11).
- Existing `TestTimeTravelTable` passes (9/9) with the change.
### Impact
Fixes a query failure for time travel inside CTEs. No public API or config
change.
### Risk Level
low. The new case only touches `UnresolvedWith` nodes, and only rewrites
`RelationTimeTravel` over an `UnresolvedRelation` within their CTE definitions,
the same rewrite already applied elsewhere in the plan.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
