jackye1995 opened a new pull request, #6524: URL: https://github.com/apache/opendal/pull/6524
## Summary This PR adds serde aliases to S3Config fields to support multiple configuration key naming conventions, enabling seamless migration from AWS SDK and other S3-compatible libraries. Users can now use either OpenDAL's native field names or AWS-prefixed alternatives. ## Changes Added `#[serde(alias = "...")]` attributes to 12 key S3 configuration fields, following the exact same patterns used by [Apache Arrow's object_store library](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs). ### Example Usage ```json // Original OpenDAL style (still works) { "bucket": "my-bucket", "access_key_id": "AKIAI...", "secret_access_key": "wJalr..." } // AWS SDK style (now supported!) { "aws_bucket": "my-bucket", "aws_access_key_id": "AKIAI...", "aws_secret_access_key": "wJalr..." } ``` ## Added Aliases | OpenDAL Field | Added Aliases | Arrow Reference | |---------------|---------------|-----------------| | `bucket` | `aws_bucket`, `aws_bucket_name`, `bucket_name` | [L1026](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1026) | | `access_key_id` | `aws_access_key_id` | [L1022](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1022) | | `secret_access_key` | `aws_secret_access_key` | [L1023](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1023) | | `session_token` | `aws_session_token`, `aws_token`, `token` | [L1028](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1028) | | `region` | `aws_region` | [L1025](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1025) | | `endpoint` | `aws_endpoint`, `aws_endpoint_url`, `endpoint_url` | [L1027](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1027) | | `enable_virtual_host_style` | `aws_virtual_hosted_style_request`, `virtual_hosted_style_request` | [L1029-L1030](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1029-L1030) | | `checksum_algorithm` | `aws_checksum_algorithm` | [L1036](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1036) | | `enable_request_payer` | `aws_request_payer`, `request_payer` | [L1044](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1044) | | `server_side_encryption` | `aws_server_side_encryption` | [L1047-L1048](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1047-L1048) | | `server_side_encryption_aws_kms_key_id` | `aws_sse_kms_key_id` | [L1049](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1049) | | `server_side_encryption_customer_key` | `aws_sse_customer_key_base64` | [L1053-L1054](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1053-L1054) | ## Missing Configurations (Future Work) These Arrow configurations are not yet available in OpenDAL and could be added in future PRs: | Arrow Configuration | Aliases | Reference | Description | |-------------------|---------|-----------|-------------| | `DefaultRegion` | `aws_default_region`, `default_region` | [L1024](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1024) | Fallback region when region not specified | | `ImdsV1Fallback` | `aws_imdsv1_fallback`, `imdsv1_fallback` | [L1033](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1033) | Allow fallback to IMDSv1 for credentials | | `UnsignedPayload` | `aws_unsigned_payload`, `unsigned_payload` | [L1035](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1035) | Skip payload signing for performance | | `MetadataEndpoint` | `aws_metadata_endpoint`, `metadata_endpoint` | [L1034](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1034) | Custom metadata service endpoint | | `ContainerCredentialsRelativeUri` | `aws_container_credentials_relative_uri` | [L1037](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1037) | ECS container credentials | | `ContainerCredentialsFullUri` | `aws_container_credentials_full_uri` | [L1038](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1038) | EKS pod credentials | | `ContainerAuthorizationTokenFile` | `aws_container_authorization_token_file` | [L1039](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1039) | EKS auth token file | | `CopyIfNotExists` | `aws_copy_if_not_exists`, `copy_if_not_exists` | [L1041](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1041) | Conditional copy operations | | `ConditionalPut` | `aws_conditional_put`, `conditional_put` | [L1042](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1042) | Conditional put operations | | `SkipSignature` | `aws_skip_signature`, `skip_signature` | [L1040](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1040) | Skip request signing | | `DisableTagging` | `aws_disable_tagging`, `disable_tagging` | [L1043](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1043) | Disable object tagging | | `S3Express` | `aws_s3_express`, `s3_express` | [L1032](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1032) | S3 Express One Zone support | | `BucketKeyEnabled` | `aws_sse_bucket_key_enabled` | [L1051-L1052](https://github.com/apache/arrow-rs-object-store/blob/main/src/aws/builder.rs#L1051-L1052) | SSE-KMS bucket key optimization | ## Testing - ✅ Added comprehensive unit tests covering all aliases - ✅ All existing S3 tests pass (16/16) - ✅ Code compiles without warnings - ✅ Clippy passes without issues ## Backward Compatibility This change is fully backward compatible. All existing configurations continue to work exactly as before, with the new aliases providing additional flexibility. Contributes to #6456 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
