XuQianJin-Stars opened a new pull request, #2338:
URL: https://github.com/apache/fluss/pull/2338
### Purpose
Linked issue: close #1979
This PR adds comprehensive test coverage to verify that column projection
(pushdown) works correctly for tables with complex data types (Array, Map, Row)
in Flusk Flink connector.
For example, given a table with schema `(a bigint, b string, c map<string,
int>, d row<d1 timestamp, d2 double>, e array<string>, f date)`, queries like
`SELECT a, c, d, f` should support projection pushdown for these complex types.
Note: This PR focuses on **top-level projection** only. Nested projection
pushdown (e.g., `SELECT d.d1, c['key']`) is not in scope and will be addressed
in a separate FIP (see #2311).
### Brief change log
- **[common]** Added 5 test methods in `ProjectedRowTest` to verify that
`ProjectedRow` correctly handles projection for Array, Map, and Row types
- Test mixed complex types projection
- Test Array-only projection
- Test Map-only projection
- Test null complex values
- Test field reordering with complex types
- **[flink]** Added 6 test methods in `FlinkTableSourceITCase` to verify
projection pushdown in Flink connector
- Test projection with mixed complex types (matching issue #1979 example)
- Test Array types projection
- Test Map types projection
- Test Row types projection
- Test null complex types projection
- Test complex types in log tables
- All tests verify both the execution plan (projection pushdown) and data
correctness
### Tests
**Unit Tests (fluss-common):**
- `ProjectedRowTest.testProjectedRowsWithComplexTypes`
- `ProjectedRowTest.testProjectedRowsWithArrayOnly`
- `ProjectedRowTest.testProjectedRowsWithMapOnly`
- `ProjectedRowTest.testProjectedRowsWithNullComplexTypes`
- `ProjectedRowTest.testProjectedRowsWithReordering`
**Integration Tests (fluss-flink-common):**
- `FlinkTableSourceITCase.testProjectPushDownWithComplexTypes`
- `FlinkTableSourceITCase.testProjectPushDownWithArrayTypes`
- `FlinkTableSourceITCase.testProjectPushDownWithMapTypes`
- `FlinkTableSourceITCase.testProjectPushDownWithRowTypes`
- `FlinkTableSourceITCase.testProjectPushDownWithNullComplexTypes`
- `FlinkTableSourceITCase.testProjectPushDownWithComplexTypesInLogTable`
All tests pass successfully, verifying that the existing `ProjectedRow`
implementation already supports complex type projection.
### API and Format
No API or storage format changes. This PR only adds test coverage for
existing functionality.
### Documentation
No new feature introduced. This PR verifies that the existing projection
pushdown mechanism works correctly with complex data types (Array, Map, Row).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
