vamshikrishnakyatham opened a new pull request, #13816: URL: https://github.com/apache/hudi/pull/13816
### Change Logs This PR introduces the `ShowMetadataTableHistoryProcedure`, a new Spark SQL procedure that displays timeline information for both data table and metadata table side-by-side, enabling analysis of metadata table synchronization and evolution. Key features implemented: - **Unified Timeline View**: Shows data table and metadata table timelines in a single result - **Side-by-Side Format**: Metadata table columns first, then data table columns - **Time Formatting**: Proper MM-dd HH:mm:ss formatting for requested/inflight/completed times - **Archive Support**: Includes archived timeline via `showArchived` parameter - **Filtering**: SQL-based filtering and time range support (`startTime`/`endTime`) - **Error Handling**: Graceful handling when metadata table doesn't exist ### Impact **New Public API**: - `show_metadata_table_history` procedure with parameters: `table`, `path`, `limit`, `showArchived`, `filter`, `startTime`, `endTime` - 11-column output schema showing metadata table and data table timeline information ### Risk level: **Low** **Verification performed:** - **Comprehensive test coverage**: 3 focused test cases covering various use cases - **Schema validation**: All output fields properly typed and validated - **Error handling**: Graceful handling of invalid filters, missing tables, and timeline access failures - **Timeline consistency**: Proper handling of both active and archived timelines with correct state mapping ### Documentation Update Update Hudi Spark SQL procedures documentation to include `show_metadata_table_history` usage examples and parameter descriptions. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Change Logs and Impact were stated clearly - [x] Adequate tests were added if applicable - [x] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
