ouyangwulin commented on FLINK-13418:

I use influxdbReporter in product env in NTES .When the flink task growing´╝î the 
tags are increasing fast, and disk io of the influxdb meshine are not enough 
for the increasing index. As the temp method , I change to ssd. So, I think the 
Jira is a good idea.

> Avoid InfluxdbReporter to report unnecessary tags
> -------------------------------------------------
>                 Key: FLINK-13418
>                 URL: https://issues.apache.org/jira/browse/FLINK-13418
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Metrics
>            Reporter: Yun Tang
>            Priority: Major
>             Fix For: 1.10.0
> Currently, when building measurement info within {{InfluxdbReporter}}, it 
> would involve all variables as tags (please see code 
> [here|https://github.com/apache/flink/blob/d57741cef9d4773cc487418baa961254d0d47524/flink-metrics/flink-metrics-influxdb/src/main/java/org/apache/flink/metrics/influxdb/MeasurementInfoProvider.java#L54]).
>  However, user could adjust their own scope format to abort unnecessary 
> scope, while {{InfluxdbReporter}} could report all the scopes as tags to 
> InfluxDB.
> This is due to current {{MetricGroup}} lacks of any method to get necessary 
> scopes but only {{#getScopeComponents()}} or {{#getAllVariables()}}. In other 
> words, InfluxDB need tag-key and tag-value to compose as its tags while we 
> could only get all variables (without any filter acording to scope format) or 
> only scopeComponents (could be treated as tag-value). I think that's why 
> previous implementation have to report all tags.
> From our experience on InfluxDB, as the size of tags contribute to the 
> overall series in InfluxDB, it would never be a good idea to contain too many 
> tags, not to mention the [default value of series per 
> database|https://docs.influxdata.com/influxdb/v1.7/troubleshooting/errors/#error-max-series-per-database-exceeded]
>  is only one million.

This message was sent by Atlassian Jira

Reply via email to