liloew commented on PR #1971:
URL:
https://github.com/apache/pekko-connectors/pull/1971#issuecomment-5935762788
Thanks for the detailed review. Your understanding is mostly correct. Let me
clarify the problem based on my experiments.
The issue is a mismatch between the path sent on the wire and the path used
when calculating the signature.
For example, for an object key such as `123=.xml`:
* Pekko HTTP sends the path as `/123=.xml`.
* The S3 canonical-request code percent-encodes `=` and signs
`/123%3D.xml`.
The original vendor SDK requests succeed against both Huawei OBS and Tencent
COS, while
the default Pekko Connectors S3 request can fail with
`SignatureDoesNotMatch`. This suggests that these S3-compatible services do
not normalize the received path in exactly the same way as AWS, or that their
canonicalization rules differ for some characters.
I tested the following object keys with Pekko Connectors S3 and the vendors'
official SDKs:
`@`, `?`, `=`, `;`, `.`, `,`, `+`, `&`, `#`, `!`, `[`, and `]`.
The vendor SDK requests succeeded in all tested cases.
With the default Pekko configuration, both Huawei OBS and Tencent COS failed
for object keys containing `@`, `;`, `,`, `&`, and `!`. With
`reserved-characters = ""`, most of these cases succeeded. The remaining
differences were:
* `123=.xml`: Tencent COS still failed.
* `123+.xml`: both Huawei OBS and Tencent COS failed.
* Plain keys such as `123.xml`, `123?.xml`, `123[.xml`, and `123].xml`
succeeded.
Therefore, the failure is not general. It is reproducible for a subset of
object-key characters, and plain object keys do not fail in my tests.
I have not found vendor documentation explicitly describing a list of
“reserved characters” equivalent to the Pekko configuration option. The
configuration value in this PR is therefore based on observed compatibility
behavior, rather than on a vendor-published character table. I will document
that limitation and include the test matrix.
For Huawei OBS, the tested compatible setting is to remove the affected
characters from the encoded set. For Tencent COS, the same setting fixes most
of the tested cases, but `=` and `+` still require additional investigation.
I do not want to claim that one fixed reserved-character value is universally
correct for all OBS or COS operations without broader testing.
The table below shows the test results:
<img width="3197" height="734" alt="result-matrix"
src="https://github.com/user-attachments/assets/0cbf835b-badd-4c11-902a-d3e195ff31d4";
/>
The images below show the scenarios for Pekko Connectors S3 where reserved
characters are left unset versus set to an empty string (the "prohibited" icon
indicates a failure):
<img width="324" height="277" alt="pekko-obs-default"
src="https://github.com/user-attachments/assets/0d016b94-8f45-407a-88a1-dfcc7b14a003";
/>
<img width="319" height="272" alt="pekko-obs-clear-reversed-characters"
src="https://github.com/user-attachments/assets/a731ea95-2434-47d8-b1c9-6f884c4b40e0";
/>
The following images show the scenario using the native
`esdk-obs-java-bundle`:
<img width="357" height="276" alt="esdk-obs"
src="https://github.com/user-attachments/assets/36cff5d4-adbb-43a6-abe7-5a9eed1354bd";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
