Re: Metrics with labels
Registering the counter is fine, e.g. in `open()`:
lazy val responseCounter: Counter = getRuntimeContext
.getMetricGroup
.addGroup("response_code")
.counter("myResponseCounter")
then, in def asyncInvoke(), I can still only do responseCounter.inc(), but
what I want is responseCounter.withLabelValue(200).inc()
I don't find the docs very useful. Here is a useful example of how I would
expect labels to work:
https://pkg.go.dev/github.com/prometheus/client_golang/prometheus#hdr-A_Basic_Example
On Tue, Oct 17, 2023 at 8:32 PM Chesnay Schepler wrote:
> > I think this is a general issue with the Flink metrics.
>
> Not quite. There are a few instance in Flink were code wasn't updated to
> encode metadata as additional labels, and the RocksDB metrics may be one of
> them.
> Also for RocksDB, you could try setting
> "state.backend.rocksdb.metrics.column-family-as-variable: true" to resolve
> this particular problem.
>
> > If I define a custom metric, it is not supported to use labels
>
> You can do so via MetricGroup#addGroup(String key, String value).
> See
> https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/ops/metrics/#user-variables
>
> On 17/10/2023 14:31, Lars Skjærven wrote:
>
> Hello,
>
> We're experiencing difficulties in using Flink metrics in a generic way
> since various properties are included in the name of the metric itself.
> This makes it difficult to generate sensible (and general) dashboards (with
> aggregations).
>
> One example is the metric for rocksdb estimated live data size (
> state.backend.rocksdb.metrics.estimate-live-data-size). the name appears
> as :
> flink_taskmanager_job_task_operator__state_rocksdb_estimate_live_data_size
> .
>
> If, on the other hand, the state name was included as label, this would
> facilitate aggregation across states, i.e.:
>
> flink_taskmanager_job_task_operator_state_rocksdb_estimate_live_data_size{state_descriptor="my_state_descriptor"}
>
> I think this is a general issue with the Flink metrics. If I define a
> custom metric, it is not supported to use labels (
> https://prometheus.io/docs/practices/naming/#labels) in a dynamic way.
>
> Thanks !
>
> Lars
>
>
>
Re: Metrics with labels
> I think this is a general issue with the Flink metrics.
Not quite. There are a few instance in Flink were code wasn't updated to
encode metadata as additional labels, and the RocksDB metrics may be one
of them.
Also for RocksDB, you could try setting
"state.backend.rocksdb.metrics.column-family-as-variable: true" to
resolve this particular problem.
> If I define a custom metric, it is not supported to use labels
You can do so via MetricGroup#addGroup(String key, String value).
See
https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/ops/metrics/#user-variables
On 17/10/2023 14:31, Lars Skjærven wrote:
Hello,
We're experiencing difficulties in using Flink metrics in a generic
way since various properties are included in the name of the metric
itself. This makes it difficult to generate sensible (and general)
dashboards (with aggregations).
One example is the metric for rocksdb estimated live data size
(state.backend.rocksdb.metrics.estimate-live-data-size). the name
appears as :
flink_taskmanager_job_task_operator__state_rocksdb_estimate_live_data_size
.
If, on the other hand, the state name was included as label, this
would facilitate aggregation across states, i.e.:
flink_taskmanager_job_task_operator_state_rocksdb_estimate_live_data_size{state_descriptor="my_state_descriptor"}
I think this is a general issue with the Flink metrics. If I define a
custom metric, it is not supported to use labels
(https://prometheus.io/docs/practices/naming/#labels) in a dynamic way.
Thanks !
Lars
Metrics with labels
Hello,
We're experiencing difficulties in using Flink metrics in a generic way
since various properties are included in the name of the metric itself.
This makes it difficult to generate sensible (and general) dashboards (with
aggregations).
One example is the metric for rocksdb estimated live data size (
state.backend.rocksdb.metrics.estimate-live-data-size). the name appears as
:
flink_taskmanager_job_task_operator__state_rocksdb_estimate_live_data_size
.
If, on the other hand, the state name was included as label, this would
facilitate aggregation across states, i.e.:
flink_taskmanager_job_task_operator_state_rocksdb_estimate_live_data_size{state_descriptor="my_state_descriptor"}
I think this is a general issue with the Flink metrics. If I define a
custom metric, it is not supported to use labels (
https://prometheus.io/docs/practices/naming/#labels) in a dynamic way.
Thanks !
Lars
