[ https://issues.apache.org/jira/browse/BEAM-9065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17020913#comment-17020913 ]
Etienne Chauchot edited comment on BEAM-9065 at 1/22/20 9:53 AM: ----------------------------------------------------------------- Hey ! Yes, no problem, no hurry, we are still waiting to reproduce the failure in a UTest was (Author: echauchot): Hey ! Yes, no problem, no hurry, we are still waiting to reproduce the failure. > Spark runner accumulates metrics (incorrectly) between runs > ----------------------------------------------------------- > > Key: BEAM-9065 > URL: https://issues.apache.org/jira/browse/BEAM-9065 > Project: Beam > Issue Type: Bug > Components: runner-spark > Reporter: Etienne Chauchot > Assignee: Etienne Chauchot > Priority: Major > Fix For: 2.20.0 > > Time Spent: 40m > Remaining Estimate: 0h > > When pipeline.run() is called, MetricsAccumulator (wrapper of > MetricsContainerStepMap spark accumulator) is initialized. Spark needs this > class to be a singleton for failover. The problem is that when several > pipelines are run inside the same JVM, the initialization of > MetricsAccumulator singleton does not reset the underlying spark accumulator > causing metrics to be accumulated between runs. -- This message was sent by Atlassian Jira (v8.3.4#803005)