JingsongLi opened a new pull request, #486: URL: https://github.com/apache/paimon-rust/pull/486
## Summary Adds Java-compatible Blob View support to Paimon Rust, including BlobViewStruct serialization, SQL/DataFusion integration, read/write validation, and documentation. The read path also avoids unnecessary BLOB IO by filtering before resolution when predicates do not depend on resolved BLOB fields and batches descriptor range reads per blob file. ## Changes - Add a Java-compatible BlobViewStruct spec and CoreOptions support for blob-view-field and blob-view.resolve.enabled. - Support Blob View writes by storing view fields inline and validating canonical BlobViewStruct payloads. - Resolve Blob View fields on reads through catalog-backed upstream lookups, while preserving raw references when resolution is disabled or unavailable. - Register DataFusion blob_view / sys.blob_view scalar functions and wire catalog-backed scans so SQL queries can resolve Blob Views. - Support __BLOB_VIEW_FIELD in CREATE TABLE and ALTER TABLE ADD COLUMN comments. - Optimize BLOB reads by filtering before resolution when safe and merging nearby descriptor ranges per URI. - Document Blob View usage and dynamic options in docs/src/sql.md. ## Testing - cargo check -p paimon -p paimon-datafusion - cargo test -p paimon blob_view -- --nocapture - cargo test -p paimon table::dedicated_format_file_writer::tests::test_merge_blob_read_requests_merges_nearby_ranges -- --nocapture - cargo test -p paimon-datafusion --test blob_tests test_blob_descriptor_filter_before_resolve_skips_filtered_bad_descriptor -- --nocapture - cargo test -p paimon-datafusion --test blob_tests -- --nocapture - cargo test -p paimon-datafusion sql_context -- --nocapture - git diff --check ## Notes - Blob View resolution currently applies to data-evolution reads. When blob-view.resolve.enabled=false, queries preserve serialized BlobViewStruct bytes. - The range-read optimization is shared by Blob View resolved values and ordinary BlobDescriptor-field reads. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
