JingsongLi opened a new pull request, #10958: URL: https://github.com/apache/paimon/pull/10958
### Purpose Add `MultimodalTable.merge_columns` for importing labels or embeddings by key, and `add_columns` for schema-only additions or batched feature computation. Both reuse Data Evolution column updates so untouched media files are retained. The local APIs validate source cardinality, output types, lossless key conversion, and table mode before publication. Results include row counts and schema/snapshot IDs; failures expose their phase and known publication outcome. Schema changes and data commits remain separate, and callers must coordinate a single writer. Preserve row counts when projecting newly added columns that have no physical provider. Store vectors as physical Parquet lists while preserving Arrow metadata, covering ordinary, streaming, and dedicated-vector writers so sparse nullable embeddings remain readable. ### Tests - Python 3.10 / PyArrow 22: **289 passed, 68 subtests passed, 9 skipped, 1 deselected** across column enrichment, vector serialization, multimodal tables, Data Evolution, shard/row-ID updates, shared-shredding MAPs, and format writes. - Added **37 API tests** and **4 streaming Parquet tests**, including media file reuse, composite keys, NULLs, deleted rows, batch/file-group alignment, response loss, stale table handles, signed zero, lossy key casts, and nested metadata. - Additional local REST-catalog verification: successful enrichment plus nine row-filter, column-mask, and denied-read cases rejected before mutation. - Flake8, license headers, diff whitespace, Python 3.6 syntax checks, and the executable documentation example passed. The deselected existing HEVC decoding test requires Torch, which is unavailable in this environment. Native execution was not exercised because the optional runtime is not installed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
