leaves12138 commented on code in PR #8808:
URL: https://github.com/apache/paimon/pull/8808#discussion_r3729622029
##########
paimon-python/pypaimon/read/table_read.py:
##########
@@ -832,19 +899,45 @@ def __authed_reader(self, split, auth_result,
blob_parallelism=1):
if extra_fields:
effective_read_type = read_fields + extra_fields
- reader = self._create_split_read(
- split, blob_parallelism=blob_parallelism,
- read_type=effective_read_type).create_reader()
+ filter_fn = auth_result.extract_row_filter()
+ effective_limit = self.limit if limit is None else limit
+ auth_fields = (
+ self._auth_filter_field_names(auth_result, effective_read_type)
+ if filter_fn is not None else set()
+ )
+ inline_blob_fields = (
+ self.table.options.blob_descriptor_fields()
+ | self.table.options.blob_view_fields()
+ )
+ embed_filter = (
+ filter_fn is not None
+ and self.table.options.data_evolution_enabled()
+ )
+ # If the auth filter references an inline BLOB, run it after inline
resolution (in
+ # the split read) so it sees resolved payloads while scalar BLOBs
still defer.
+ post_filter_after_inline = embed_filter and bool(auth_fields &
inline_blob_fields)
+ split_read = self._create_split_read(
+ split,
+ blob_parallelism=blob_parallelism,
+ read_type=effective_read_type,
Review Comment:
When `effective_read_type` contains a hidden auth-filter column, the reader
is later wrapped by `ColumnProjectReader`, which drops `file_io` (it copies
only `blob_field_indices`/`vector_field_indices`). Therefore the metadata fix
in `AuthFilterReader` does not cover this common path. Repro on this head: use
a data-evolution table with `blob-as-descriptor=true`, project `["sample_id",
"payload"]`, and apply an auth filter on the unprojected `score` column.
`to_iterator()` returns the expected rows, but `row.get_blob(1).to_data()`
fails with `ValueError: file_io is required to resolve BlobDescriptor bytes`.
`AuthMaskingReader` has the same metadata loss. Please make both wrappers adopt
the full reader metadata and add coverage for hidden auth-filter columns and
masking in descriptor mode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
