clintropolis opened a new pull request #9509: add kinesis lag metric URL: https://github.com/apache/druid/pull/9509 Fixes #6739. ### Description This PR adds an additional set of `SeekableStreamSupervisor` lag metrics, based on time in millis behind the latest offsets of the stream, allowing the Kinesis indexing service extension to have a lag metric. `scheduleReporting ` and `emitLag` were pushed down from `KafkaSupervisor` into `SeekableStreamSupervisor`, and two additional abstract methods, `getPartitionRecordLag` and `getPartitionTimeLag` which collect the two types of lag metrics per partition of the stream. Kafka's existing lag metrics are still collected in the same format for backwards compatibility. ``` ingest/{supervisor type}/lag ingest/{supervisor type}/maxLag ingest/{supervisor type}/avgLag ``` and the new time metrics have a `/time` suffix. ``` ingest/{supervisor type}/lag/time ingest/{supervisor type}/maxLag/time ingest/{supervisor type}/avgLag/time ``` Currently `KafkaSupervisor` only emits the former set of metrics, and `KinesisSupervisor`, after this PR, will emit only the latter. It seems possible for `KafkaSupervisor` to emit both sets of metrics, but I will save that for a follow-up PR. <hr> This PR has: - [ ] been self-reviewed. - [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.) - [ ] added documentation for new or modified features or behaviors. - [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links. - [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/licenses.yaml) - [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [ ] added unit tests or modified existing tests to cover new code paths. - [ ] added integration tests. - [ ] been tested in a test Druid cluster. <hr> ##### Key changed/added classes in this PR * `SeekableStreamSupervisor` * `KafkaSupervisor` * `KinesisSupervisor`
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
