Hi, Etienne! Thanks for the response. Yes, I ran the test with both the flink and spark runners, and both showed committed and attempted values.
I didn’t actually use MetricsPusher for these tests. I have questions about MetricsPusher, but I’ll put those in another post. -Phil On 2018/11/30 13:57:43, Etienne Chauchot <[email protected]> wrote: > Hi Phil,> > Thanks for using MetricsPusher and Beam in general ! > > - MetricsHttpSink works that way: it filters out committed metrics from the > json output when committed metrics are not> > supported. I checked, Flink runner still does not support committed metrics. > So there should be no committed metrics> > values in the output json.There might be a bug. I'll open a ticket: thx for > pointing out ! You tested on flink and spark> > right? and both output committed metrics values right?> > - there is no default mechanism to fallback committed metrics values on > attempted ones> > - Apache Flink does no make flink Accumulators available in detached mode, so > indeed, metrics are not available in this> > mode.> > CCing dev list.> > Etienne> > Le lundi 26 novembre 2018 à 15:57 -0600, Phil Franklin a écrit :> > > All of the discussion I’ve seen says that Flink and Spark only provided > > attempted metric values, but when I use> > > MetricsHttpSink and look at the JSON it has both attempted and committed > > values (albeit, both the same for my simple> > > testing). Has the metrics processing been updated recently, and I’m just > > missing the change updates? Or are the> > > committed values being defaulted to the attempted values? > > > > > > Also, I’ve seen it mentioned that Flink doesn’t report metrics when in > > detached mode. Is this still the case?> > > > > > > > > Thanks for your help!> >
