suneet-s opened a new pull request, #14752:
URL: https://github.com/apache/druid/pull/14752
### Description
Add a new metric that is emitted by the compaction task that shows how much
time it spent fetching and analyzing segments. When a datasource is very
fragmented, this could be a large amount of time. This metric will be helpful
to operators as they can monitor this to see if they would benefit from
explicitly specifying the schema in the auto compaction / manual compaction
config if their use case allows it.
Here is a sample of what the metric would look like
```
{"taskIngestionMode":"REPLACE_LEGACY","feed":"metrics","taskType":"compact","metric":"compact/segmentAnalyzer/fetchAndProcessMillis","service":"druid/middleManager","groupId":"coordinator-issued_compact_wikipedia_cgmonfmg_2023-08-04T00:35:38.106Z","host":"localhost:8100","version":"28.0.0-SNAPSHOT","value":78,"dataSource":"wikipedia","taskId":"coordinator-issued_compact_wikipedia_cgmonfmg_2023-08-04T00:35:38.106Z","timestamp":"2023-08-04T00:35:40.605Z"}
```
#### Release note
NEW: `compact/segmentAnalyzer/fetchAndProcessMillis` is now reported by
compaction tasks to indicate how much time was spent fetching and processing
segments to infer the schema
This PR has:
- [x] been self-reviewed.
- [x] added documentation for new or modified features or behaviors.
- [x] a release note entry in the PR description.
- [x] added comments explaining the "why" and the intent of the code
wherever would not be obvious for an unfamiliar reader.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
- [ ] added integration tests.
- [x] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
