fhan688 opened a new pull request, #3528:
URL: https://github.com/apache/fluss/pull/3528
### Purpose
Linked issue: https://github.com/apache/fluss/issues/3284
This PR adds integration tests for Hudi log table union read based on the
Flink union read test base introduced in #3522.
The tests verify that Fluss can read tiered Hudi snapshot data together
with newly appended Fluss log data for datalake-enabled log tables.
### Brief change log
- Add `FlinkUnionReadLogTableITCase` for Hudi log table union read.
- Verify batch full-type reads for both partitioned and non-partitioned
log tables.
- Verify union read after the tiering job is canceled and new Fluss log
records are appended.
- Verify projection pushdown and partition filter pushdown.
- Verify streaming union read with partition discovery.
- Verify savepoint-based failover and restore for a streaming insert job
reading from a Hudi-backed log table.
- Reuse the Hudi Flink union read test base from #3522 and enable
checkpointing for the tiering job.
### Tests
- `mvn -pl fluss-lake/fluss-lake-hudi -Dcheckstyle.skip=true
spotless:apply`
- `mvn -pl fluss-client,fluss-lake/fluss-lake-hudi -am -DskipITs
-Dcheckstyle.skip=true -DfailIfNoTests=false test-compile`
- `mvn -pl fluss-lake/fluss-lake-hudi -am -DskipITs=false
-Dcheckstyle.skip=true -DfailIfNoTests=false
-Dtest=FlinkUnionReadLogTableITCase test`
### API and Format
No API or storage format changes.
### Documentation
No documentation changes. This PR only adds integration test coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
