[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-02-05 Thread zentol
Github user zentol commented on the issue: https://github.com/apache/flink/pull/5161 merging. ---

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-31 Thread zentol
Github user zentol commented on the issue: https://github.com/apache/flink/pull/5161 Correction: We still can't display them in the UI since we have no tab for job metrics. ---

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-31 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5161 So this ensures that the parent is actually a `TaskManagerJobMetricGroup`, right? This LGTM for me, now. Could you open a follow-up issue for the operator-name thing? ---

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-31 Thread zentol
Github user zentol commented on the issue: https://github.com/apache/flink/pull/5161 I've rebased the branch and did one more change: ``` this.latencyStats = new LatencyStats(this.metrics.parent().parent().addGroup("latency"), historySize, container.getIndexInSubtaskGroup(), g

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-31 Thread zentol
Github user zentol commented on the issue: https://github.com/apache/flink/pull/5161 yes the display will now work. ---

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-31 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5161 I'd say we merge without operator names/operator id and leave that as a follow-up, issue. To at least make it possible to get latency metrics. WDYT? Btw, with this, can we now show the laten

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-31 Thread zentol
Github user zentol commented on the issue: https://github.com/apache/flink/pull/5161 I'm wary about using the operator name (or _any_ variant that inexplicably exists) as that significantly increases the size of watermarks. I see the use-case, and ideally I would like to have

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-30 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5161 @yew1eb by `operatorName`, do you mean the `name()` or `uid()`, I think both of these could make sense. ---

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-28 Thread yew1eb
Github user yew1eb commented on the issue: https://github.com/apache/flink/pull/5161 Yes, in our production environment, we report and store all metrics to an external Time series database for alarm and visual presentation. When the job is started, we will store the edge struc

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-27 Thread zentol
Github user zentol commented on the issue: https://github.com/apache/flink/pull/5161 @yew1eb Are you suggesting to store the operatorName in the latency marker? ---

[GitHub] flink issue #5161: [FLINK-7608][metric] Refactor latency statistics metric

2018-01-27 Thread yew1eb
Github user yew1eb commented on the issue: https://github.com/apache/flink/pull/5161 @zentol :+1: **1)**, I have no opinion. **2), 3)**, Replace `operatorID` with `operatorName`, will be better? @aljoscha @rmetzger what are your opinions? ---