rinzool opened a new issue, #2655:
URL: https://github.com/apache/iceberg-python/issues/2655
### Apache Iceberg version
0.10.0 (latest release)
### Please describe the bug 🐞
# Description
Since version 0.10.0 it is not possible to create a Polaris catalog with
`credential` parameter.
For example:
```python
catalog_parameters = {
"type": "rest",
# ...
"scope": "PRINCIPAL_ROLE:ALL",
"header.Polaris-Realm": "my-realm",
"credential": f"{client_id}:{client_secret}"
}
catalog = load_catalog("default, **catalog_parameters)
```
This code fails with error `RESTError: MissingOrInvalidRealm: Missing or
invalid realm`.
The exact same code works with pyiceberg 0.9.1.
# Cause
Polaris needs header `Polaris-Realm`.
We can see in the full stacktrace (see below) that the error occurs when we
create the AuthManager line 266:
https://github.com/apache/iceberg-python/blob/pyiceberg-0.10.0/pyiceberg/catalog/rest/__init__.py#L266.
But headers are injected **after** initializing AuthManager at line 269:
https://github.com/apache/iceberg-python/blob/pyiceberg-0.10.0/pyiceberg/catalog/rest/__init__.py#L269.
<details>
<summary>Full stacktrace</summary>
```
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/__init__.py:128](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/__init__.py#line=127),
in load_rest(name, conf)
125 def load_rest(name: str, conf: Properties) -> Catalog:
126 from pyiceberg.catalog.rest import RestCatalog
--> 128 return RestCatalog(name, **conf)
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py:234](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py#line=233),
in RestCatalog.__init__(self, name, **properties)
232 super().__init__(name, **properties)
233 self.uri = properties[URI]
--> 234 self._fetch_config()
235 self._session = self._create_session()
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py:364](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py#line=363),
in RestCatalog._fetch_config(self)
361 if warehouse_location := self.properties.get(WAREHOUSE_LOCATION):
362 params[WAREHOUSE_LOCATION] = warehouse_location
--> 364 with self._create_session() as session:
365 response = session.get(self.url(Endpoints.get_config,
prefixed=False), params=params)
366 try:
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py:266](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py#line=265),
in RestCatalog._create_session(self)
264 session.auth =
AuthManagerAdapter(AuthManagerFactory.create(auth_impl or auth_type,
auth_type_config))
265 else:
--> 266 session.auth =
AuthManagerAdapter(self._create_legacy_oauth2_auth_manager(session))
268 # Set HTTP headers
269 self._config_headers(session)
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py:295](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/__init__.py#line=294),
in RestCatalog._create_legacy_oauth2_auth_manager(self, session)
285 auth_url = self.auth_url if client_credentials is not None else None
287 auth_config = {
288 "session": session,
289 "auth_url": auth_url,
(...)
292 "optional_oauth_params": self._extract_optional_oauth_params(),
293 }
--> 295 return AuthManagerFactory.create("legacyoauth2", auth_config)
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py:321](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py#line=320),
in AuthManagerFactory.create(cls, class_or_name, config)
318 except Exception as err:
319 raise ValueError(f"Could not load AuthManager class for
'{class_or_name}'") from err
--> 321 return manager_cls(**config)
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py:94](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py#line=93),
in LegacyOAuth2AuthManager.__init__(self, session, auth_url, credential,
initial_token, optional_oauth_params)
92 self._credential = credential
93 self._optional_oauth_params = optional_oauth_params
---> 94 self._refresh_token()
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py:122](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py#line=121),
in LegacyOAuth2AuthManager._refresh_token(self)
120 def _refresh_token(self) -> None:
121 if self._credential is not None:
--> 122 self._token = self._fetch_access_token(self._credential)
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py:116](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/auth.py#line=115),
in LegacyOAuth2AuthManager._fetch_access_token(self, credential)
114 response.raise_for_status()
115 except HTTPError as exc:
--> 116 _handle_non_200_response(exc, {400: OAuthError, 401: OAuthError})
118 return TokenResponse.model_validate_json(response.text).access_token
File
[/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/response.py:111](https://poc-dwh-qfn.forepaas.io/opt/conda/lib/python3.11/site-packages/pyiceberg/catalog/rest/response.py#line=110),
in _handle_non_200_response(exc, error_handler)
108 errs = ", ".join(err["msg"] for err in e.errors())
109 response = f"RESTError {exc.response.status_code}: Received
unexpected JSON Payload: {exc.response.text}, errors: {errs}"
--> 111 raise exception(response) from exc
RESTError: MissingOrInvalidRealm: Missing or invalid realm
```
</details>
# Solution
Move `self._config_headers(session)` instruction in the beginning of
`_create_session` method.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
