Hello Philipp, there is certainly something very wrong here.
What you _should_ see is 6 entries, 1 for each operator; 2-3 more for the tasks the operators are executed in and the taskmanager stuff.
Usually, operator metrics use the name that you configured, like "TokenMapStream", whereas tasks use the concatenation of all operator names joined with =>, as in "KafkaSource => TokenMapStream".
I will look into this, I've never seen these issues before. One more thing, which version of Flink are you currently using? Regards, Chesnay On 15.10.2016 00:07, Philipp Bussche wrote:
Hi there, I am struggeling to understand what I am looking at after enabling JMX metric reporting on my taskmanager. The job I am trying this out with has 1 source, 2 map functions (where one is a RichMap) and 3 sinks. This is how I have written my Job: DataStream<Invitation> invitations = streaming .addSource(new FlinkKafkaConsumer09<>( dsp.getPropertyAsString("kafka.invitation.topic"), new InvitationSchema(), kafkaProps)).name("KafkaSource"); invitations.addSink(new PostgresqlInvitationDetailsSink<>(psqlConfig)).name("InvitationDetailSink"); DataStream<Tuple2<String, String>> tokens = invitations .map(new TokenExtractor()).name("TokenMapStream"); tokens.addSink(new PostgresqlInvitationTokenSink<>(psqlConfig)).name("InvitationTokenSink"); DataStream<Tuple4<String, String, String, String>> invitationResponses = invitations .map(new InvitationDetailsExtractor(psqlConfig, tokenToSenderMapping)).name("InvitationDetailsRichMapStream"); invitationResponses.addSink(new Neo4JInvitationSink<>(neo4jConfig)).name("InvitationRelationshipSink"); streaming.execute("InvitationJob"); Somehow I was expecting to have metrics representing the source, the sinks and the operators, however instead of 6 entries in my JMX tree I only have 4. Please see screenshot attached. Also I was somehow expecting the JMX objects to be named like my task / operator names but it has all sorts of prefix/suffix magic around the names. Finally I have one custom metric which obviously is attached to my RichMapFunction (InvitationDetailsExtractor). However the custom metric (invitationDetailsAdded) shows up under an object where one of the keys (which I would expect to be set to the operation name) is a combination of the prefix "Sink" plus the name of the first sink that I am using plus the name of the first map function (which is not the RichMapFunction actually: my custom metric "invitationDetailsAdded" shows up under "(Sink-_InvitationDetailSink-_TokenMapStream" which is very confusing because this metric is actually incremented as part of the InvitationDetailsRichMapStream). Can somebody please explain what I can expect from metrics exposed via JMX (should they really represent my tasks and operations) and why the naming is so strange ? Thanks Philipp <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/n9560/Screen_Shot_2016-10-15_at_00.png> -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Task-and-Operator-Monitoring-via-JMX-naming-tp9560.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.