Hi again, Etienne! You didn’t mention whether spark is reporting committed values now, but you also didn’t mention opening a ticket concerning the spark output. Am I right in inferring that spark does in fact report committed values?
Thanks! -Phil On 2018/11/30 13:57:43, Etienne Chauchot <[email protected]> wrote: > Hi Phil,> > Thanks for using MetricsPusher and Beam in general ! > > - MetricsHttpSink works that way: it filters out committed metrics from the > json output when committed metrics are not> > supported. I checked, Flink runner still does not support committed metrics. > So there should be no committed metrics> > values in the output json.There might be a bug. I'll open a ticket: thx for > pointing out ! You tested on flink and spark> > right? and both output committed metrics values right?> > - there is no default mechanism to fallback committed metrics values on > attempted ones> > - Apache Flink does no make flink Accumulators available in detached mode, so > indeed, metrics are not available in this> > mode.> > CCing dev list.> > Etienne> > Le lundi 26 novembre 2018 à 15:57 -0600, Phil Franklin a écrit :> > > All of the discussion I’ve seen says that Flink and Spark only provided > > attempted metric values, but when I use> > > MetricsHttpSink and look at the JSON it has both attempted and committed > > values (albeit, both the same for my simple> > > testing). Has the metrics processing been updated recently, and I’m just > > missing the change updates? Or are the> > > committed values being defaulted to the attempted values? > > > > > > Also, I’ve seen it mentioned that Flink doesn’t report metrics when in > > detached mode. Is this still the case?> > > > > > > > > Thanks for your help!> >
