It will be in the master tomorrow.
On 20.10.2016 18:50, Philipp Bussche wrote:
Thanks Chesnay !
I am not too familiar with the release cycles here but was wondering when
one could expect your fix to be in the master of Flink ? Should I create my
own build for the moment maybe ?
Thanks.
--
Well the issue is the following:
the metric system assumes the following naming scheme for tasks based on
the DataSet API and simple streaming jobs: [CHAIN] operatorName1 [=>
operatorName2 [ ...]]
To retrieve the operator name the above is split by "=>", giving us a
String[] of all operator
Thanks Chesnay,
I am happy to share more around my environment and do additional testing for
this.
Also I would be happy to help fixing if we see there might be an issue in
the code somewhere.
In fact I am still trying to get a Hacktoberfest T-Shirt and I am still pull
requests short ;)
--
This is completely unintended behavior; you should never have to adjust
your topology so the metric system get's the names right.
I'll take a deep look into this tomorrow ;)
Regards,
Chesnay
On 20.10.2016 08:50, Philipp Bussche wrote:
Some further observations: I had a Job which was taking
Some further observations: I had a Job which was taking events of a Kafka
topic and sending it to two sinks whereas for one of them a Map operation
would happen first. When creating one event stream and sending it to the two
sinks the JMX representation was not showing both sinks and the naming of
Thanks Chesnay.
I had a look at how the JMX representation looks like when I look at a Task
Manager which has one of the example Jobs deployed
(https://ci.apache.org/projects/flink/flink-docs-release-1.1/quickstart/run_example_quickstart.html)
and this looks correct.
I assume at this point that
Hello Philipp,
the relevant names are stored in the OperatorMetricGroup/TaskMetricGroup
classes in flink-runtime.
The name for a task is extracted directly from the
TaskDeploymentDescriptor in TaskManagerJobMetricGroup#addTask().
The name for a streaming operator that the metric system uses
Thanks Chesnay, this is on Flink 1.1.3
Please also note that e.g. the first item in the list which has the custom
metric attached to it starts with a leading "(". It might be that the
parsing of the names is not working quite as expected.
I was trying to find out where these names come from but
Hello Philipp,
there is certainly something very wrong here.
What you _should_ see is 6 entries, 1 for each operator; 2-3 more for
the tasks the operators are executed in and the taskmanager stuff.
Usually, operator metrics use the name that you configured, like
"TokenMapStream", whereas
