emilk opened a new pull request, #781: URL: https://github.com/apache/arrow-rs-object-store/pull/781
Closes #<!-- file an issue first if the project requires one -->. # Which issue does this PR close? The Azure SAS signer fetches a fresh user delegation key on every signing request, which can be heavily throttled by Azure under load. # Rationale for this change `AzureClient::signer()` calls `get_user_delegation_key()` — a `GetUserDelegationKey` network round-trip (`POST /?restype=service&comp=userdelegationkey`) — on **every** `signed_url` / `signed_urls` call. A user delegation key is reusable until its expiry, so re-fetching it per request is wasteful and, under load, gets throttled. We hit this in production: a workload issuing many presigned-URL requests against Azure Blob drove ~100k `GetUserDelegationKey` POSTs in 2h (~840/min), ~35% of which returned HTTP 503 `ServerBusy`. The retry client backed off 10× (~10s) and then surfaced the error to the caller. # What changes are included in this PR? Cache the `UserDelegationKey` (and the `AzureAccessKey` derived from it) on `AzureClient`: - Fetch a longer-lived key (6h, well under Azure’s 7-day cap) once and reuse it to mint many short-lived SAS tokens. - Refresh only when the cached key no longer covers the requested SAS window, keeping a 5-minute margin to rotate before expiry and to absorb clock skew. - Use the validity window Azure actually granted (`SignedStart` / `SignedExpiry`) rather than what was requested. - The signed SAS tokens themselves are unchanged (still caller-specified `expires_in`). Concurrency: the cache is guarded by a `Mutex`, and the lock is never held across the network fetch — under a cold-cache race a key may be fetched more than once, which is rare and self-correcting. Added a unit test (`test_cached_delegation_key_covers`) for the cache-validity window logic. # Are there any user-facing changes? No public API changes. Presigned-URL generation against Azure now performs far fewer `GetUserDelegationKey` requests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
