wallyhung opened a new issue, #565:
URL: https://github.com/apache/paimon-cpp/issues/565

   ### Search before asking
   
   - [x] I searched existing issues and pull requests for path-style 
addressing, virtual-hosted addressing, and COS, and found no matching report.
   
   ### Paimon-cpp version
   
   `main` at `46ffa1d1cc2d531118f3372a2cc94dab036a444d`.
   The same addressing logic is present in 
`81829741c8e3c9920a57f6edcf8adaddcc4ef5a4`.
   
   ### Minimal reproduce step
   
   Configure the S3 filesystem with a custom HTTPS endpoint and explicitly 
disable path-style access:
   
   ```text
   file-system = s3
   s3.endpoint = https://cos.ap-beijing.myqcloud.com
   s3.region = ap-beijing
   s3.path-style-access = false
   s3.access-key = <test access key>
   s3.secret-key = <test secret key>
   ```
   
   Read an object such as `s3://examplebucket-1250000000/data/file.parquet`, or 
capture the request URL with the existing mock HTTP client.
   
   The current implementation constructs:
   
   ```text
   
https://cos.ap-beijing.myqcloud.com/examplebucket-1250000000/data/file.parquet
   ```
   
   A virtual-hosted request would be:
   
   ```text
   
https://examplebucket-1250000000.cos.ap-beijing.myqcloud.com/data/file.parquet
   ```
   
   This can be demonstrated without a live COS account: the existing 
`S3ObjectStoreClientTest.TestCustomEndpointAddressing` sets 
`s3.path-style-access=false` for `https://s3.example.com` but asserts the 
path-style URL `https://s3.example.com/bucket/file`.
   
   ### What doesn't meet your expectations?
   
   There is currently no way to select virtual-hosted addressing for a custom 
HTTPS endpoint, even with a DNS-compatible, non-dotted bucket name and a 
matching wildcard TLS certificate.
   
   In `S3ObjectStoreClient::Execute`, the custom-endpoint branch includes 
`endpoint_.scheme != "http"` in the condition that selects path-style 
addressing. Consequently, HTTPS always selects path style, regardless of 
`s3.path-style-access=false`.
   
   - [Addressing 
logic](https://github.com/apache/paimon-cpp/blob/46ffa1d1cc2d531118f3372a2cc94dab036a444d/src/paimon/fs/s3/s3_file_system.cpp#L808-L818)
   - [Existing 
test](https://github.com/apache/paimon-cpp/blob/46ffa1d1cc2d531118f3372a2cc94dab036a444d/src/paimon/fs/s3/s3_file_system_test.cpp#L151-L163)
   
   Please provide an explicit way to use virtual-hosted addressing with a 
custom HTTPS endpoint, while retaining appropriate handling for IP endpoints, 
dotted buckets/TLS, and explicitly requested path-style access. A regression 
test should check the request URL and signed Host header.
   
   ### Anything else?
   
   Tencent COS is a motivating use case: its documentation states that buckets 
created after January 1, 2024 support only virtual-hosted-style domains. This 
makes the current HTTPS addressing behavior incompatible with those buckets.
   
   Reference: [Tencent COS configuration for S3-compatible 
applications](https://intl.cloud.tencent.com/document/product/436/34688?lang=en).
   
   Validation scope: this report is based on source inspection and the existing 
mock-test assertions. I have not executed those tests or performed a live COS 
read; no observed COS HTTP error is being claimed.
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to