ankrgyl opened a new issue, #541: URL: https://github.com/apache/arrow-rs-object-store/issues/541
**Describe the bug** If you try to run ~1,000+ concurrent requests, the mutex on the credentials cache becomes a bottleneck **To Reproduce** Run several thousand concurrent requests, and you'll see performance drop (for our tests, p50 GET latency was 800ms on S3). Tracing revealed that almost all the time is spent acquiring the credentials mutex. **Expected behavior** Since the read:write ratio is so high, we should be able to avoid taking a lock in this case. Tests show that if you do, performance returns to the expected (8-20ms) p50 for HEAD/GET requests on S3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
