wallyhung opened a new issue, #565: URL: https://github.com/apache/paimon-cpp/issues/565
### Search before asking - [x] I searched existing issues and pull requests for path-style addressing, virtual-hosted addressing, and COS, and found no matching report. ### Paimon-cpp version `main` at `46ffa1d1cc2d531118f3372a2cc94dab036a444d`. The same addressing logic is present in `81829741c8e3c9920a57f6edcf8adaddcc4ef5a4`. ### Minimal reproduce step Configure the S3 filesystem with a custom HTTPS endpoint and explicitly disable path-style access: ```text file-system = s3 s3.endpoint = https://cos.ap-beijing.myqcloud.com s3.region = ap-beijing s3.path-style-access = false s3.access-key = <test access key> s3.secret-key = <test secret key> ``` Read an object such as `s3://examplebucket-1250000000/data/file.parquet`, or capture the request URL with the existing mock HTTP client. The current implementation constructs: ```text https://cos.ap-beijing.myqcloud.com/examplebucket-1250000000/data/file.parquet ``` A virtual-hosted request would be: ```text https://examplebucket-1250000000.cos.ap-beijing.myqcloud.com/data/file.parquet ``` This can be demonstrated without a live COS account: the existing `S3ObjectStoreClientTest.TestCustomEndpointAddressing` sets `s3.path-style-access=false` for `https://s3.example.com` but asserts the path-style URL `https://s3.example.com/bucket/file`. ### What doesn't meet your expectations? There is currently no way to select virtual-hosted addressing for a custom HTTPS endpoint, even with a DNS-compatible, non-dotted bucket name and a matching wildcard TLS certificate. In `S3ObjectStoreClient::Execute`, the custom-endpoint branch includes `endpoint_.scheme != "http"` in the condition that selects path-style addressing. Consequently, HTTPS always selects path style, regardless of `s3.path-style-access=false`. - [Addressing logic](https://github.com/apache/paimon-cpp/blob/46ffa1d1cc2d531118f3372a2cc94dab036a444d/src/paimon/fs/s3/s3_file_system.cpp#L808-L818) - [Existing test](https://github.com/apache/paimon-cpp/blob/46ffa1d1cc2d531118f3372a2cc94dab036a444d/src/paimon/fs/s3/s3_file_system_test.cpp#L151-L163) Please provide an explicit way to use virtual-hosted addressing with a custom HTTPS endpoint, while retaining appropriate handling for IP endpoints, dotted buckets/TLS, and explicitly requested path-style access. A regression test should check the request URL and signed Host header. ### Anything else? Tencent COS is a motivating use case: its documentation states that buckets created after January 1, 2024 support only virtual-hosted-style domains. This makes the current HTTPS addressing behavior incompatible with those buckets. Reference: [Tencent COS configuration for S3-compatible applications](https://intl.cloud.tencent.com/document/product/436/34688?lang=en). Validation scope: this report is based on source inspection and the existing mock-test assertions. I have not executed those tests or performed a live COS read; no observed COS HTTP error is being claimed. ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
