kevinjqliu opened a new issue, #3422: URL: https://github.com/apache/iceberg-python/issues/3422
### Apache Iceberg version None ### Please describe the bug 🐞 ### Summary `RestCatalog._create_session()` expects `auth` to be a dict. When catalog config comes from environment variables, values are strings, so `auth` is received as a string and auth initialization fails. This blocks env-var-based configuration for pluggable REST auth (`basic`, `oauth2`, `google`, `entra`, `custom`) unless string JSON is explicitly decoded first. ### Minimal repro ```bash export PYICEBERG_CATALOG__REST__TYPE=rest export PYICEBERG_CATALOG__REST__URI=http://localhost:8181 export PYICEBERG_CATALOG__REST__AUTH='{"type":"oauth2","oauth2":{"client_id":"id","client_secret":"secret","token_url":"https://auth.example/token"}}' ``` ```python from pyiceberg.catalog import load_catalog load_catalog("rest") ``` ### Actual (without this fix) Expected: catalog initializes and uses the configured auth manager. Actual: initialization fails because `auth` is treated as a string and `.get(...)` is called on it. ### Suggested fix In REST catalog session setup, if `auth` is a string, decode it as JSON before reading `auth.type` and type-specific config. Add regression tests for both: - `PYICEBERG_CATALOG__<NAME>__AUTH` (JSON string) initializes auth manager correctly. - `PYICEBERG_CATALOG__<NAME>__AUTH__...` maps correctly into auth manager configuration. ### Alternative fix (follows current env-var standard) Support flattened auth properties from environment variables instead of requiring a JSON blob in `...__AUTH`. Example: ```bash export PYICEBERG_CATALOG__REST__AUTH__TYPE=oauth2 export PYICEBERG_CATALOG__REST__AUTH__OAUTH2__CLIENT_ID=id export PYICEBERG_CATALOG__REST__AUTH__OAUTH2__CLIENT_SECRET=secret export PYICEBERG_CATALOG__REST__AUTH__OAUTH2__TOKEN_URL=https://auth.example/token ``` This aligns with existing flattened env-var configuration behavior and avoids JSON-in-env quoting/escaping issues. ### Verification Observed with current code path (`Config._from_environment_variables`): ```bash export PYICEBERG_CATALOG__REST__AUTH__TYPE=oauth2 export PYICEBERG_CATALOG__REST__AUTH__OAUTH2__CLIENT_ID=id export PYICEBERG_CATALOG__REST__AUTH__OAUTH2__CLIENT_SECRET=secret ``` Parsed result: ```python {'catalog': {'rest': {'auth.type': 'oauth2', 'auth.oauth2.client-id': 'id', 'auth.oauth2.client-secret': 'secret'}}} ``` This confirms flattened `AUTH__...` env vars are currently stored as dotted keys, not as a nested `auth` object consumed by `RestCatalog._create_session()`. ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
