ad1happy2go opened a new pull request, #18975: URL: https://github.com/apache/hudi/pull/18975
### Change Logs After a successful write + meta-sync, `HoodieSparkSqlWriter.metaSync` refreshes the Spark catalog relation cache for the synced table so later reads in the same session see the new data. That refresh re-resolves the table by name, which for a Hudi table eagerly reads its `.hoodie` metadata from storage. Reported in #18139: with AWS Glue sync, the write and sync succeed, but the job then fails with `AccessDenied` while refreshing a same-named table that belongs to the `default` database (pointing at an unrelated, inaccessible bucket). Two problems, reproduced on Spark 3.5 / 4.0: - An unqualified table name resolves against the session's current/`default` database, so a same-named table there is read by mistake. The name is already qualified with the sync database (`hoodie.datasource.hive_sync.database`); this change preserves that. - The refresh was **unguarded**, so any failure (a transient catalog error, or a same-named table backed by storage the writer cannot access) propagated and failed an **already-committed, already-synced** write. Fix: extract the invalidation into `HoodieSparkSqlWriterInternal.refreshSparkCatalogTableCache`, keep the database-qualified name, and wrap each table's refresh in a `NonFatal` try/catch that logs and continues. Cache invalidation is purely a local-session optimization and must never fail a committed write. ### Impact A failure while refreshing the Spark catalog cache after a successful write+sync no longer fails the write. The only behavioral change: if a refresh genuinely fails, a subsequent read **in the same session** may serve stale cached data (logged at WARN); the committed table is unaffected. ### Risk level: low Behavior change limited to the post-sync, best-effort catalog cache invalidation step. ### Documentation Update None required. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Change Logs and Impact were stated clearly - [x] Adequate tests were added if applicable 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
