codope opened a new pull request, #395:
URL: https://github.com/apache/hudi-rs/pull/395
## Description
Implements LSM timeline reading capability for the hudi-rs core to support
both v6 and v8 timeline formats, following Java implementation patterns with
timeline layout abstraction. This is the foundation PR and we will follow up
with future work on LSM timeline history reading, support for multiple layers
of timeline compaction and manifest parsing for compacted timeline files.
### Core Implementation
- **Timeline Layout Architecture**: Added `TimelineLayoutType` enum to
manage different timeline storage formats.
- **Active Timeline**: Implemented `ActiveTimeline` for older table versions
with layout version 1 (existing format).
- **LSM Timeline**: Added `LsmTimeline` foundation for newer table version 8
with layout version 2 (LSM format).
- **Factory Pattern**: Created `TimelineLayoutFactory` for creating
appropriate timeline implementations based on table configuration.
```rust
Timeline {
layout: TimelineLayoutType::Active(ActiveTimeline) |
TimelineLayoutType::Lsm(LsmTimeline)
}
```
- **layout v1**: Uses `ActiveTimeline` (reads from `.hoodie/` directory)
- **layout v2**: Uses `LsmTimeline` (foundation for `.hoodie/timeline/` LSM
format)
### Configuration Integration
- **Version Detection**: Automatic timeline layout selection based on
`hoodie.table.version` and `hoodie.timeline.layout.version`
- **Graceful Fallback**: Handles invalid table configurations by defaulting
to active timeline layout.
- **Backward Compatibility**: Maintains full compatibility with existing v6
table APIs.
<!--- Please link any related issues and PRs as well. -->
Related to #8
## How are the changes test-covered
- **250/250 core tests** passing
- **Integration tests** for table reading, snapshot queries, and file slices
- **Backward compatibility** verified with existing test data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
