alamb commented on code in PR #18139:
URL: https://github.com/apache/datafusion/pull/18139#discussion_r2442195352
##########
datafusion-cli/src/object_storage/instrumented.rs:
##########
@@ -119,6 +119,54 @@ impl InstrumentedObjectStore {
!= InstrumentedObjectStoreMode::Disabled as u8
}
+ async fn instrumented_put_opts(
+ &self,
+ location: &Path,
+ payload: PutPayload,
+ opts: PutOptions,
+ ) -> Result<PutResult> {
+ let timestamp = Utc::now();
+ let start = Instant::now();
+ let size = payload.content_length();
+ let ret = self.inner.put_opts(location, payload, opts).await?;
+ let elapsed = start.elapsed();
+
+ self.requests.lock().push(RequestDetails {
+ op: Operation::Put,
+ path: location.clone(),
+ timestamp,
+ duration: Some(elapsed),
+ size: Some(size),
+ range: None,
+ extra_display: None,
+ });
+
+ Ok(ret)
+ }
+
+ async fn instrumented_put_multipart(
+ &self,
+ location: &Path,
+ opts: PutMultipartOptions,
+ ) -> Result<Box<dyn MultipartUpload>> {
+ let timestamp = Utc::now();
+ let start = Instant::now();
+ let ret = self.inner.put_multipart_opts(location, opts).await?;
+ let elapsed = start.elapsed();
+
+ self.requests.lock().push(RequestDetails {
+ op: Operation::Put,
+ path: location.clone(),
+ timestamp,
+ duration: Some(elapsed),
Review Comment:
> Prior to closing the overarching issue for these PRs should document some
of the current caveats for duration metrics?
That sounds good to me -- I suggest putting it in the code (not just the
docs) so it is more discoverable -- maybe a note after the summary output?
```
Object Store Profiling
....
Put
count: 1
duration min: 0.000249s
duration max: 0.000249s
duration avg: 0.000249s
size min: 815 B
size max: 815 B
size avg: 815 B
size sum: 815 B
*** NEW ***
Note: Duration for multipart PUT is time spent initiating a multipart PUT
session with the backing store. It does not include the time to actually upload
data.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
