Stephen0421 opened a new pull request, #10057: URL: https://github.com/apache/paimon/pull/10057
### Purpose When `blob-as-descriptor=true`, keep `BlobViewStruct` bytes through Stage 1 so `get_blob()` still resolves through the originating table FileIO. Serialize descriptors only at the Arrow/pandas output boundary. Previously Stage 1 replaced view bytes with a plain descriptor. That dropped the upstream-table association, so `get_blob()` used the **target** FileIO to open the URI. Under REST catalog that is the wrong token (403). - Convert reader still prescans and builds `BlobViewLookup`, but does not rewrite view columns when `blob-as-descriptor=true`. - `TableRead` converts view fields to descriptor bytes in `to_arrow` / `to_pandas` / `to_arrow_batch_reader` (serial and parallel). `to_iterator()` is unchanged. - Cache `blob-as-descriptor` and `_blob_view_output_indices` (positions in `read_type`, not the widened `_scan_read_type`) so non-view row paths skip per-row option lookup. Follow-up to #9608. Native read (`read.native.enabled`) is unchanged and still defaults to false; view tables should stay on the Python reader until native grows a matching hook. ### Tests - `test_blob_view_as_descriptor_get_blob_uses_upstream_file_io`: guarded FileIO; `blob-as-descriptor=true` `get_blob().to_data()` must not read source `.blob` with the target FileIO. - `test_blob_view_as_descriptor_projection_with_predicate_extra_field`: projection `['picture', 'id']` plus a predicate on unprojected `grp`; `_blob_view_output_indices == (0,)`, Arrow has only those two columns, and `picture` is descriptor bytes. - `test_blob_view_fields_resolve_upstream_blob`: `to_arrow` still returns `BlobDescriptor` bytes when `blob-as-descriptor=true`. - `test_blob_view_default_read_uses_upstream_file_io`: default (`blob-as-descriptor=false`) still materializes payload with the upstream FileIO. - `test_table_read_serializes_only_configured_blob_view_fields`: only configured view columns are rewritten; empty indices / `blob-as-descriptor=false` skip serialize. - `test_offset_row_get_blob_view_keeps_per_table_uri_reader`: `get_blob()` keeps the originating table UriReader. - `test_offset_row_get_blob_v1_resolved_blob_view_field` / `test_offset_row_get_blob_resolves_null_blob_view` / `test_offset_row_get_blob_view_struct_without_view_field_indices`: view-struct `get_blob()` for v1, null, and missing view-field indices. - `test_to_iterator_adapters_refresh_blob_view_lookup_after_first_read`: merge `to_iterator` adapters copy lookup after the first convert read. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
