vinlee19 opened a new pull request, #56023:
URL: https://github.com/apache/doris/pull/56023
### What problem does this PR solve?
Following PR [#55404](https://github.com/apache/doris/pull/55404) which
addressed incorrect schema results for Paimon tables, this PR further optimizes
the Paimon time travel implementation by removing redundant code and fixing
consistency issues between snapshot ID and schema ID handling.
#### Key Problem Solved
The core challenge in Paimon time travel is efficiently constructing
tables with specified snapshot ID and schema ID. Previously, these were fetched
separately, causing unnecessary I/O overhead and potential consistency issues.
#### Major Improvements
1. Performance Optimization
- Unified schema and snapshot retrieval: Combined separate API calls for
schema ID and snapshot ID into a single operation
- Removed redundant metadata fetches: Eliminated duplicate calls to Paimon
metadata store
- Optimized branch schema handling: Streamlined branch-specific schema
resolution
2. Bug Fixes
- Fixed timezone-related query errors: Resolved incorrect results when
using Paimon time travel syntax with non-UTC timezones
- Fixed branch schema consistency: Ensured snapshot ID and schema ID are
always fetched atomically to prevent mismatched metadata
- Fixed timestamp parsing issues: Added support for additional timestamp
formats in FOR TIME AS OF clause
3. Enhanced Timestamp Format Support for FOR TIME AS OF
Now supports the following timestamp formats:
- YYYY-MM-DD HH:MM:SS.SSS - Full timestamp with milliseconds (e.g.,
2024-01-15 10:30:45.123)
- YYYY-MM-DD HH:MM:SS - Timestamp with seconds precision (e.g., 2024-01-15
10:30:45)
- YYYY-MM-DD - Date only format (defaults to 00:00:00.000) (e.g.,
2024-01-15)
Example usage:
```
-- Using different timestamp formats
SELECT * FROM paimon_table FOR TIME AS OF "2024-01-15 10:30:45.123";
SELECT * FROM paimon_table FOR TIME AS OF "2024-01-15 10:30:45";
SELECT * FROM paimon_table FOR TIME AS OF "2024-01-15";
```
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [ ] Unit Test
- [x] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [x] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
