kdn36 opened a new issue, #4325: URL: https://github.com/apache/iceberg-python/issues/4325
### Apache Iceberg version main (development), and 0.12.0 ### Please describe the bug 🐞 A staged create returns no `metadata-location`. `_response_to_staged_table` matches `storage-credentials` against that metadata location, so it finds none and the staged table's FileIO gets no vended credentials. Writing the table's first data files then falls back to the environment's credentials, or fails without them. `create_table` with the same response keeps the credentials. Seen with the BigLake Iceberg REST catalog in vended-credentials mode. It returns the token only in `storage-credentials`, with the table location as prefix. ```python import responses from pyiceberg.catalog.rest import RestCatalog from pyiceberg.partitioning import UNPARTITIONED_PARTITION_SPEC from pyiceberg.schema import Schema from pyiceberg.table.metadata import new_table_metadata from pyiceberg.table.sorting import UNSORTED_SORT_ORDER from pyiceberg.types import LongType, NestedField uri, location = "http://catalog";, "gs://bucket/ns/t" schema = Schema(NestedField(1, "id", LongType(), required=False)) metadata = new_table_metadata(schema, UNPARTITIONED_PARTITION_SPEC, UNSORTED_SORT_ORDER, location, {}) credentials = [{"prefix": f"{location}/", "config": {"gcs.oauth2.token": "vended"}}] def table_response(metadata_location): return { "metadata-location": metadata_location, "metadata": metadata.model_dump(mode="json", by_alias=True), "config": {}, "storage-credentials": credentials, } with responses.RequestsMock() as mock: mock.get(f"{uri}/v1/config", json={"defaults": {}, "overrides": {}}) catalog = RestCatalog("c", uri=uri) mock.post(f"{uri}/v1/namespaces/ns/tables", json=table_response(f"{location}/metadata/00000.metadata.json")) print(catalog.create_table("ns.t", schema).io.properties.get("gcs.oauth2.token")) # vended mock.replace(responses.POST, f"{uri}/v1/namespaces/ns/tables", json=table_response(None)) tx = catalog.create_table_transaction("ns.t", schema) print(tx._table.io.properties.get("gcs.oauth2.token")) # None ``` A possible fix: when `metadata-location` is missing, match against the table location (`metadata.location`), with a trailing `/` so that it matches a prefix like the one above. The scan-planning path already falls back when it has no location. ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
