prochac commented on issue #2862: URL: https://github.com/apache/iceberg-python/issues/2862#issuecomment-3892351956
Confirming that `S3V4RestSigner` is still broken in PyIceberg **0.11.0** when used with the fsspec/s3fs/aiobotocore path. **Setup:** - PyIceberg 0.11.0 (`pyiceberg[pyarrow]`), Python 3.14 - Lakekeeper (Rust Iceberg catalog) with `remote-signing-enabled: true`, `sts-enabled: false` - S3-compatible storage (Hetzner Object Storage) **What works:** OAuth2 client_credentials auth, `catalog.load_table()` **What fails:** `table.scan().to_arrow()` → `botocore.exceptions.ClientError: 403 Forbidden` on `HeadObject` The `S3V4RestSigner` event handler registered via botocore hooks never fires in the async aiobotocore path. Lakekeeper returns `py-io-impl: pyiceberg.io.fsspec.FsspecFileIO` and `s3.signer: S3V4RestSigner` as server overrides, so the client cannot switch to `PyArrowFileIO` via catalog config. **Current workaround:** After `catalog.load_table()`, replace `table.io` with a `PyArrowFileIO` instance that has direct S3 credentials — bypassing remote signing entirely. The Lakekeeper tracking issue (lakekeeper/lakekeeper#18) was closed claiming this was fixed in PyIceberg 0.7.0, but the async path is clearly still broken. This supports the case for the refactor described here — the signer needs to work independently of the FileIO implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
