[
https://issues.apache.org/jira/browse/FLINK-32954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Xia updated FLINK-32954:
----------------------------
Description:
Expose the current number of heap timers to metric reporter. Internally, expose
the size of `InternalPriorityQueue` to `MetricGroup`.
This metric can aid in debugging the memory consumption of heap timer.
Currently, we need to dump jvm heap to identify the memory consumption of heap
timer. With this metric, users can quickly get a basic knowledge on the working
condition of heap timers.
The *numOfTimers* metric is only suitable for heap timer. Heap priority queue
(`AbstractHeapPriorityQueue`) has an off-the-shelf member to get size
(`AbstractHeapPriorityQueue#size`). Expose this to metric repose is zero-cost.
The *numOfTimers* metric for RocksDB timer brings performance loss, and is not
supported right now. RocksDB (cached) priority queue does *{*}not{*}* has an
off-the-shelf member to get size. Intuitively, we can add an exclusive counter
for RocksDB priority queue. This counter affects the runtime per-record code
path. Thus, the *numOfTimers* metric for RocksDB timer is currently not
supported. I think there may exist some better/lightweight metrics to let user
learn the work condition of RocksDB timers.
was:
Expose the current number of heap timers to metric reporter. Internally, expose
the size of `InternalPriorityQueue` to `MetricGroup`.
This metric can aid in debugging the memory consumption of heap timer.
Currently, we need to dump jvm heap to identify the memory consumption of heap
timer. With this metric, users can quickly get a basic knowledge on the working
condition of heap timers.
The *numOfTimers* metric is only suitable for heap timer. Heap priority queue
(`AbstractHeapPriorityQueue`) has an off-the-shelf member to get size
(`AbstractHeapPriorityQueue#size`). Expose this to metric repose is zero-cost.
The *numOfTimers* metric for RocksDB timer brings performance loss, and is not
supported right now. RocksDB (cached) priority queue does **not** has an
off-the-shelf member to get size. Intuitively, we can add an exclusive counter
for RocksDB priority queue. This counter affects the runtime per-record code
path. Thus, the *numOfTimers* metric for RocksDB timer is currently not
supported. I think there may exist some better/lightweight metrics to let use
learn the work condition of RocksDB timers.
> Metrics expose number of heap timer
> -----------------------------------
>
> Key: FLINK-32954
> URL: https://issues.apache.org/jira/browse/FLINK-32954
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Metrics, Runtime / State Backends
> Reporter: Rui Xia
> Priority: Major
>
> Expose the current number of heap timers to metric reporter. Internally,
> expose the size of `InternalPriorityQueue` to `MetricGroup`.
>
> This metric can aid in debugging the memory consumption of heap timer.
> Currently, we need to dump jvm heap to identify the memory consumption of
> heap timer. With this metric, users can quickly get a basic knowledge on the
> working condition of heap timers.
>
> The *numOfTimers* metric is only suitable for heap timer. Heap priority queue
> (`AbstractHeapPriorityQueue`) has an off-the-shelf member to get size
> (`AbstractHeapPriorityQueue#size`). Expose this to metric repose is zero-cost.
>
> The *numOfTimers* metric for RocksDB timer brings performance loss, and is
> not supported right now. RocksDB (cached) priority queue does *{*}not{*}* has
> an off-the-shelf member to get size. Intuitively, we can add an exclusive
> counter for RocksDB priority queue. This counter affects the runtime
> per-record code path. Thus, the *numOfTimers* metric for RocksDB timer is
> currently not supported. I think there may exist some better/lightweight
> metrics to let user learn the work condition of RocksDB timers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
