Stephen0421 opened a new pull request, #9608: URL: https://github.com/apache/paimon/pull/9608
### Purpose Route descriptor-backed BLOB reads through the table FileIO, and resolve known descriptor columns with `from_descriptor_bytes` instead of the v2-only `from_bytes()` heuristic. - `UriReaderFactory.from_file_io` matches Java `fromFileIO`: HTTP(S) stays on `HttpUriReader`; every other URI reuses the table FileIO (REST tokens). Non-HTTP readers are not LRU-cached, so the factory does not pin FileIO. After pickle, a ResolvingFileIO table still works once `uri_reader_factory` has been materialized. - Convert reader and `OffsetRow.get_blob()` parse descriptor-field bytes with `from_descriptor_bytes` (v1, trailing padding, and `VideoFrameDescriptor` via serde). - After materializing payload (`blob-as-descriptor=false`), descriptor field indices are cleared so row-level `get_blob()` does not re-parse payload as a descriptor. Reader wrappers refresh blob-view lookup only and do not overwrite those indices. - Apply LIMIT before inline convert on merge splits, keep RowKind through the row/batch/row adapter, and keep `sequence.field` on blob-view prescan so overlapping-file merge can still compare. - `read_blobs_concurrent` coalesces exact `BlobRef` + `FileUriReader` only; subclasses keep `new_input_stream()`. Out of scope: when `blob-as-descriptor=true`, Stage 1 still rewrites `BlobViewStruct` to serialized descriptor bytes. Keeping the struct for that mode is a follow-up. ### Tests - `BlobTest`: v1/trailing-padding/`VideoFrameDescriptor` via `from_descriptor_bytes`; table FileIO on convert/`get_blob`/deferred resolve; LIMIT before convert; RowKind through adapters; exact-`BlobRef` coalesce; merge blob-view prescan keeps `sequence.field`; `_refresh_blob_view_lookup` does not clobber descriptor indices - `DedicatedFormatWriterTest.test_blob_view_predicate_and_limit_resolves_filtered_row` - `DedicatedFormatWriterTest.test_blob_view_raw_split_predicate_and_limit_resolves_filtered_row` - `UriReaderFactoryTest`: `from_file_io` reuses FileIO for non-HTTP, does not pin FileIO, and pickles with FileIO in the graph - `ResolvingFileIOTest`: pickle FileIO and table after `uri_reader_factory` is materialized - `RESTTokenFileIOTest.test_close_only_closes_instance_uri_reader_factory` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
