zfarrell opened a new pull request, #771: URL: https://github.com/apache/arrow-rs-object-store/pull/771
# Which issue does this PR close? Closes #271 Closes #270 Also related to #620 (versionId) and #318. # Rationale for this change Right now `Signer::signed_url` can only presign a bare URL. There's no way to add extra query parameters or to sign request headers, and S3 needs both. The big one is multipart uploads. To presign an `UploadPart` request you have to include `partNumber` and `uploadId` as query parameters, and they have to be part of the signature. There's no way to do that today, so you can't hand out presigned URLs for the parts of a multipart upload. Signing headers matters for the same reason. If you want S3 to enforce a checksum, or pin the `Content-Type`, or require SSE, those headers have to be baked into the signature when you presign. Otherwise the client can send whatever it wants. # What changes are included in this PR? - Adds `SignedUrlOptions` and a new `Signer::signed_url_opts` method that takes extra query params and signed `(name, value)` header pairs along with the usual method/path/expiry. - `signed_url_opts` has a default impl, and the existing `signed_url` now just calls it with no options. So nothing changes for existing callers and Azure doesn't have to implement anything. - Implemented for S3 (SigV4) and GCS (GOOG4). Azure keeps the default — its SAS model can't sign arbitrary params, and there's an inline comment saying so. - Fixes two signing bugs the new code surfaced: query values were being encoded with `+` instead of `%20`, which doesn't match the canonical query, and the canonical query now sorts by the encoded `(key, value)` pair the way the spec wants. # Are there any user-facing changes? Yes, but only additions. `SignedUrlOptions` and `signed_url_opts` are new. `signed_url` behaves exactly as before. No breaking changes — the trait method is defaulted, and the internal signing methods that got reworked were all `pub(crate)`. This was tested end-to-end against real S3 (and MinIO): a full multipart round-trip with a sub-5MiB final part, checksum enforcement (accepts a matching body, rejects a tampered one, rejects a missing one), `Content-Type` binding, tampering with `partNumber`/`uploadId`/signed headers all getting a 403, percent-encoded keys, expiry, and `If-None-Match` conditional create. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
