suryaprasanna opened a new pull request, #17861:
URL: https://github.com/apache/hudi/pull/17861
…ier for join queries
### Describe the issue this Pull Request addresses
This PR improves debugging experience for Spark queries involving multiple
Hudi tables (especially joins) by including the table name in the job stage
names displayed in the Spark UI. Previously, stage names only showed generic
messages like "Listing all partitions with prefix X" or "Listing all files in N
partitions", making it difficult to identify which table was being processed
when multiple tables were involved in a single query.
### Summary and Changelog
Users gain better observability in the Spark UI when debugging queries
involving multiple Hudi tables.
**Changes:**
- Added `tableConfig` as a class field in `FileSystemBackedTableMetadata`
- Updated `setJobStatus` calls to include table name in stage descriptions:
- "Listing all partitions with prefix X" → "Listing all partitions on
{tableName} with prefix X"
- "Listing all files in N partitions" → "Listing all files in N
partitions from {tableName}"
### Impact
**User-Facing Changes:**
Improved Spark UI stage names that now include the table name, making it
easier to debug and monitor queries involving multiple Hudi tables,
particularly join operations.
**Performance Impact:**
None - this is purely a logging/observability change.
### Risk Level
**None** - This change only affects stage name labels in the Spark UI and
does not modify any functional logic.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
