discivigour opened a new pull request, #8488:
URL: https://github.com/apache/paimon/pull/8488
### Purpose
Fix PyPaimon default BlobView reads to resolve upstream blob data with
the upstream table FileIO instead of the downstream table FileIO.
Previously, BlobView fields were first converted from `BlobViewStruct`
bytes to `BlobDescriptor` bytes, then handled by the same descriptor
resolution path as local `blob-descriptor-field` columns. That path
uses the current table FileIO, which is wrong for BlobView values
pointing to blobs in the source/upstream table.
This patch:
- records the upstream FileIO while resolving BlobView fields;
- keeps ordinary `blob-descriptor-field` columns resolved with the
current table FileIO;
- resolves BlobView descriptor bytes with the recorded upstream FileIO;
- keeps BlobView default reads on the concurrent blob read path when
`blob_parallelism > 1`;
- changes descriptor-byte resolution to build a file-backed `UriReader`
from the passed FileIO context.
### Tests
- `python -m py_compile pypaimon/read/reader/
blob_descriptor_convert_reader.py pypaimon/utils/blob_view_lookup.py
pypaimon/table/row/blob.py pypaimon/tests/blob_table_test.py`
- `python -m unittest
pypaimon.tests.blob_table_test.DedicatedFormatWriterTest.test_blob_view
_fields_resolve_upstream_blob
pypaimon.tests.blob_table_test.DedicatedFormatWriterTest.test_blob_view
_default_read_uses_upstream_file_io`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
