Akash3121 commented on code in PR #11136:
URL: https://github.com/apache/paimon/pull/11136#discussion_r4559905518
##########
paimon-python/pypaimon/catalog/catalog_environment.py:
##########
@@ -111,8 +115,13 @@ def dependency_read_context(self) ->
Optional[CatalogContext]:
from pypaimon.catalog.rest.rest_catalog_loader import RESTCatalogLoader
+ # External REST tables intentionally have no snapshot loader, and a
directly
+ # constructed RESTCatalog may omit `metastore=rest` from
CatalogContext.
Review Comment:
`dependency_read_context()` is updated here to support external REST tables
where `catalog_loader` is `None` and `_catalog_context` is populated, but the
downstream callers for `blob-view-field` reads still gate on
`catalog_environment.catalog_loader is not None` instead of
`catalog_environment.catalog_context() is not None`:
1.
`paimon-python/pypaimon/execution/python/read/reader/blob_descriptor_convert_reader.py`
(lines 89–90 and 136–138):
```python
resolve_enabled = CoreOptions.blob_view_resolve_enabled(
table.options) and self._table.catalog_environment.catalog_loader is
not None
```
and:
```python
if self._table.catalog_environment.catalog_loader is None:
return
```
Unlike Java (`DataEvolutionTableRead` / `KeyValueTableRead`, which check
`catalogContext != null`), this will now skip blob-view resolution for external
REST tables because `catalog_loader` is `None`.
2. `paimon-python/pypaimon/execution/python/read/blob_view_lookup.py`
(lines 310–315):
```python
catalog_loader = catalog_environment.catalog_loader
dependency_context = catalog_environment.dependency_read_context()
if dependency_context is catalog_environment.catalog_context():
catalog = catalog_loader.load()
```
If an external REST table (`catalog_loader=None`) is loaded as an
intermediate table in a nested dependency chain where
`header.X-Paimon-Read-Via` is already present in `catalog_context`,
`dependency_read_context()` returns `context` unchanged and
`catalog_loader.load()` will fail with `AttributeError: 'NoneType' object has
no attribute 'load'`.
3. `paimon-python/pypaimon/multimodal/lance_export.py` (lines 160–162) and
`paimon-python/pypaimon/execution/native/table.py` (lines 34–36) similarly
check `catalog_loader is None`.
Should we update those callers to check `catalog_context() is not None`
(and fall back to `CatalogFactory.create_from_context` when `catalog_loader is
None` in `BlobViewLookup._load_table`) so external REST tables can actually use
this `dependency_read_context()` path?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]