jkosh44 commented on PR #7301:
URL: https://github.com/apache/arrow-rs/pull/7301#issuecomment-2740923854
I have a repro! It turns out that I was looking at the wrong method this
whole time. The error happens when the multipart upload is empty.
Here's a minal repro which works on `main`:
```rust
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let store =
GoogleCloudStorageBuilder::new().with_bucket_name("multipart-complete-header-test").with_application_credentials("/path/to/credentials.json").build().unwrap();
let blob_path = "multi-test/hello.txt".into();
let id = store.create_multipart(&blob_path).await.unwrap();
let part_ids = Vec::new();
store
.complete_multipart(&blob_path, &id, part_ids)
.await
.unwrap();
Ok(())
}
```
Which results in the following error:
```
called `Result::unwrap()` on an `Err` value: Generic { store: "GCS", source:
RetryError { method: PUT, uri:
Some(https://storage.googleapis.com/multipart%2Dcomplete%2Dheader%2Dtest/multi%2Dtest%2Fhello%2Egzip),
retries: 0, max_retries: 10, elapsed: 108.320687ms, retry_timeout: 180s,
inner: Status { status: 411, body: Some("<!DOCTYPE html>\n<html lang=en>\n
<meta charset=utf-8>\n <meta name=viewport content=\"initial-scale=1,
minimum-scale=1, width=device-width\">\n <title>Error 411 (Length
Required)!!1</title>\n <style>\n
*{margin:0;padding:0}html,code{font:15px/22px
arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7%
auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* >
body{background:url(//www.google.com/images/errors/robot.png) 100% 5px
no-repeat;padding-right:205px}p{margin:11px 0
22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media
screen and (max-width:772px){body{background:none;margin-top:0;max-width:none
;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png)
no-repeat;margin-left:-5px}@media only screen and
(min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png)
no-repeat 0% 0%/100%
100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png)
0}}@media only screen and
(-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png)
no-repeat;-webkit-background-size:100%
100%}}#logo{display:inline-block;height:54px;width:150px}\n </style>\n <a
href=//www.google.com/><span id=logo aria-label=Google></span></a>\n
<p><b>411.</b> <ins>That’s an error.</ins>\n <p>POST requests require a
<code>Content-length</code> header. <ins>That’s all we know.</ins>\n") } } }
stack backtrace:
0: rust_begin_unwind
at
/rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/std/src/panicking.rs:692:5
1: core::panicking::panic_fmt
at
/rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/core/src/panicking.rs:75:14
2: core::result::unwrap_failed
at
/rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/core/src/result.rs:1704:5
3: core::result::Result<T,E>::unwrap
at
/home/joe.koshakow/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:1109:23
4: object_store::main::{{closure}}
at ./src/main.rs:35:5
5: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/park.rs:284:60
6: tokio::task::coop::with_budget
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/task/coop/mod.rs:167:5
7: tokio::task::coop::budget
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/task/coop/mod.rs:133:5
8: tokio::runtime::park::CachedParkThread::block_on
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/park.rs:284:31
9: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/context/blocking.rs:66:9
10:
tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
11: tokio::runtime::context::runtime::enter_runtime
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/context/runtime.rs:65:16
12: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
13: tokio::runtime::runtime::Runtime::block_on_inner
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/runtime.rs:370:45
14: tokio::runtime::runtime::Runtime::block_on
at
/home/joe.koshakow/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.44.1/src/runtime/runtime.rs:342:13
15: object_store::main
at ./src/main.rs:40:5
16: core::ops::function::FnOnce::call_once
at
/home/joe.koshakow/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose
backtrace.
Process finished with exit code 101
```
The method in question is here:
https://github.com/apache/arrow-rs/blob/660a3ac22a8ef8601acf4548d65146bc623f653a/object_store/src/gcp/client.rs#L516-L522
What's confusing is that is a `PUT`, but the error message said `POST
requests require a <code>Content-length</code> header` which threw me off.
Adding `.header(&CONTENT_LENGTH, "0")` resolves the issue. Though it's not
clear to me why we have the `PUT` call, we're about to delete the file via
`self.multipart_cleanup(path, multipart_id).await?;`, any ideas?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
