cshuo opened a new issue, #19857: URL: https://github.com/apache/hudi/issues/19857
## Feature Description Add support for reading VECTOR columns through Flink CDC paths. Flink CDC may read records from data table base files or file slices to construct change records. These paths currently do not consistently preserve the Hudi VECTOR schema or apply the format-specific VECTOR decoding used by the regular Flink readers. As a result, CDC queries can fail or return incorrectly typed values when the selected schema contains VECTOR columns. The issue applies to both supported base-file formats: - Parquet stores VECTOR values using its physical binary encoding and must return them through Flink's ARRAY representation. - Lance stores VECTOR values as fixed-size lists and must return them consistently through the same Flink ARRAY representation. ## Expected Behavior - Flink CDC queries can read VECTOR columns from Parquet and Lance base files. - VECTOR values are returned correctly for CDC records derived from base files and file slices. - Nullable VECTOR values, column projection, field ordering, and CDC row kinds are preserved. - Regular ARRAY columns continue to be handled as regular arrays. - The behavior is consistent across the applicable COW and MOR CDC read paths. ## Scope This work uses the existing Flink VECTOR configuration and Hudi VECTOR data model. It does not introduce a new VECTOR encoding or public API. Parent tracking issue: #19053 Related Lance reader issue: #19821 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
