clintropolis opened a new pull request, #19632: URL: https://github.com/apache/druid/pull/19632
### Description This PR fixes and improves metric tracking for partial segment loading, focusing on the `StorageMonitor`. Prior to this PR only the metadata header was tracked (and only partially, sometimes incorrectly). To assist, a `PartialSegmentDownloadListener` interface has been introduced and passed to the file mapper so that callers can be notified whenever operations like a download happen, which is used to wire it into the `StorageLocation` metrics tracking (since it is no longer `SegmentLocalCacheManager` driving these things when partial loads are involved, instead it is the caller, e.g. query engines that retrieved segments). Additionally, some new metrics: * `storage/virtual/read/count` - number of deep-storage range read operations. * `storage/virtual/read/bytes` - bytes pulled over the wire from range read operations. * `storage/virtual/read/time` - total time spent in range reads (ms) Finally, fixes flakiness in `QueryVirtualStorageTest` caused by the change in #19494 to not emit metrics when values are zeros, which this test was using as a sort of latch to indicate when activity on a StorageLocation had cleared out from a prior test. Instead, this is now done through new `tryWaitForNextEvent` and `awaitMetricQuiescent` methods on `LatchableEmitter` to watch for the monitor to either stabilize or stop emitting. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
