Dludora opened a new issue, #2877: URL: https://github.com/apache/iceberg-python/issues/2877
### Apache Iceberg version 0.10.0 (latest release) ### Please describe the bug 🐞 ## Description When using PyIceberg with Aliyun OSS (Object Storage Service) via the S3-compatible endpoint, creating a table or appending data fails during the metadata/data upload phase. The underlying PyArrow S3 filesystem attempts to use aws-chunked encoding with x-amz-content-sha256, which is not supported by Aliyun OSS's S3 compatibility layer for certain operations. This leads to the following error: ```Plaintext OSError: When uploading part for key '...' in bucket '...': AWS Error UNKNOWN (HTTP status 400) during UploadPart operation: Unable to parse ExceptionName: InvalidArgument Message: aws-chunked encoding is not supported with the specified x-amz-content-sha256 value. According to [Aliyun OSS Documentation](https://help.aliyun.com/zh/oss/user-guide/0017-00000804), the S3-compatible API has specific constraints regarding signature versions and chunked encoding. ``` ### Steps to Reproduce Initialize a Catalog with Aliyun OSS S3-compatible endpoint. 1. Set s3.force-virtual-addressing: "true" as required by OSS. 2. Attempt to create a table or write data. 3. Reproducible Code Snippet: ```Python import os from pyiceberg.catalog import load_catalog catalog = load_catalog( "oss_catalog", **{ "type": "sql", "uri": "sqlite:///oss_catalog.db", "warehouse": "s3://your-bucket/path/", "s3.endpoint": "https://s3.oss-cn-wulanchabu.aliyuncs.com";, "s3.region": "cn-wulanchabu", "s3.access-key-id": os.environ["OSS_AK"], "s3.secret-access-key": os.environ["OSS_SK"], "s3.force-virtual-addressing": "true", } ) # This triggers the UploadPart error during metadata write catalog.create_table("default.test_table", schema=pa_schema) ``` ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
