WenyXu opened a new issue, #3725:
URL: https://github.com/apache/incubator-opendal/issues/3725
I have some code like:
```rust
let buf_reader = BufReader::new(reader.compat());
let builder = ParquetRecordBatchStreamBuilder::new(buf_reader)
.await
.context(error::ReadParquetSnafu)?;
let stream = builder
.build()
.context(error::BuildParquetRecordBatchStreamSnafu)?;
```
And I tried to read all records from the stream like:
```rust
while let Some(batch) = stream.try_next(){
...
}
```
Reading all records from the stream took a lot of time, almost 10s. I
checked the metrics of Opendal; only one read request is executed.
I reproduced the operation on my machine, I found each time the `try_next`
is invoked, the program prints :
```
Resuming session
ALPN protocol is Some(b"http/1.1")
Using ciphersuite TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
ECDHE curve is ECParameters { curve_type: NamedCurve, named_group: secp256r1
}
Server DNS name is DnsName("s3.ap-southeast-1.amazonaws.com")
2023-12-07T11:35:40.723300Z DEBUG log: Sending warning alert CloseNotify
calculated scope: 20231207/ap-southeast-1/s3/aws4_request
calculated string to sign: AWS4-HMAC-SHA256
20231207T113540Z
20231207/ap-southeast-1/s3/aws4_request
xxxxx
```
I thought the Opendal S3 backend would read the entire file only once... But
it seems some other operations were executed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
