weiqingy opened a new pull request, #2063:
URL: https://github.com/apache/auron/pull/2063
<!--
- Start the PR title with the related issue ID, e.g. '[AURON #XXXX] Short
summary...'.
-->
# Which issue does this PR close?
Closes #1851
# Rationale for this change
Per AIP-1, the Flink integration data path requires converting Arrow
vectors returned by the native engine (DataFusion/Rust) back into Flink RowData
so downstream Flink operators can process results.
# What changes are included in this PR?
- FlinkArrowReader orchestrator — zero-copy columnar access via
ColumnarRowData + VectorizedColumnBatch
- 16 ArrowXxxColumnVector wrappers for all 17 supported types
- Decimal fromUnscaledLong optimization for precision ≤ 18
- Batch reset support for streaming pipelines
- 21 unit tests in FlinkArrowReaderTest
# Are there any user-facing changes?
No. Internal API for Flink integration.
# How was this patch tested?
21 tests: ./build/mvn test -pl auron-flink-extension/auron-flink-runtime -am
-Pscala-2.12 -Pflink-1.18 -Pspark-3.5 -DskipBuildNative
-Dtest=FlinkArrowReaderTest
Result: 21 pass, 0 failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
