wjones127 commented on issue #5366:
URL: https://github.com/apache/arrow-rs/issues/5366#issuecomment-1930287838
Here is the reproduction:
1. Create a Multi-part upload
2. Write 10 MB (trigger a request)
3. Sleep for 30 seconds
4. Try writing more data (this will fail with timeout)
I think we would agree that it might be expected behavior if the sleep in
step (3) was synchronous/blocking. However, this will fail even if the sleep is
async, which makes this feel like a real foot gun.
```rust
use object_store::{aws::AmazonS3Builder, path::Path, ObjectStore};
use tokio::io::AsyncWriteExt;
#[tokio::main]
async fn main() {
// Create an S3 store configured from environment variables
let store =
AmazonS3Builder::from_env().with_bucket_name("lance-performance-testing").build().unwrap();
let (_id, mut writer) =
store.put_multipart(&Path::from("test")).await.unwrap();
// Upload 10 MB of data
let data_chunk = vec![0u8; 1024 * 1024];
for _ in 0..10 {
writer.write_all(&data_chunk).await.unwrap();
}
// Sleep for 30 seconds
tokio::time::sleep(tokio::time::Duration::from_secs(30)).await;
// try to upload 10 more (should fail here)
for _ in 0..10 {
writer.write_all(&data_chunk).await.unwrap();
}
// finish.
writer.shutdown().await.unwrap();
}
```
```
called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error:
Generic { store: "S3", source: Reqwest { retries: 0, max_retries: 10, elapsed:
30.00097806s, retry_timeout: 180s, source: reqwest::Error { kind: Request, url:
Url { scheme: "https", cannot_be_a_base: false, username: "", password: None,
host: Some(Domain("s3.us-east-1.amazonaws.com")), port: None, path:
"/lance-performance-testing/test", query:
Some("partNumber=1&uploadId=mOQTaVt2JtGlMwB2IceAR2olH5FyBgk_oJVbZ_zh9XRDU6VbbIRWmx0KKk3MM6vVb2vCA93jP4xIiY9lAo0BHxULl5jNV0issWUnqVyFk.dA.xt3mmaz5U_GK_g1OqYCn4ZMMZ61xIn22FzDWuU.PQ--"),
fragment: None }, source: TimedOut } } } }
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
