[ https://issues.apache.org/jira/browse/BEAM-3310?focusedWorklogId=91405&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91405 ]
ASF GitHub Bot logged work on BEAM-3310: ---------------------------------------- Author: ASF GitHub Bot Created on: 16/Apr/18 17:17 Start Date: 16/Apr/18 17:17 Worklog Time Spent: 10m Work Description: swegner commented on issue #4548: [BEAM-3310] Metrics pusher URL: https://github.com/apache/beam/pull/4548#issuecomment-381681741 > Can we move forward and merge this PR as it is and consider it as the default implementation that java sdks and java runners are free to use (like the philosophy of runner-core-java module)? What do you mean by making it the default? I don't think this replaces existing metrics functionality; for example existing infrastructure includes the ability to support committed values, and querying metrics results via the `PipelineResult` interface. However I'm not opposed to this going in as additional opt-in functionality. > @swegner you gave LGTM, so I think we can merge. The `MetricsPusher` component looks good to me; I haven't reviewed Spark/Flink runner integration. Also note that I am not a Beam Committer, so final LGTM will have to come from someone who is. (@lukecwik is a committer, but is currently on vacation) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 91405) Time Spent: 4h 40m (was: 4.5h) > Push metrics to a backend in an runner agnostic way > --------------------------------------------------- > > Key: BEAM-3310 > URL: https://issues.apache.org/jira/browse/BEAM-3310 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core > Reporter: Etienne Chauchot > Assignee: Etienne Chauchot > Priority: Major > Time Spent: 4h 40m > Remaining Estimate: 0h > > The idea is to avoid relying on the runners to provide access to the metrics > (either at the end of the pipeline or while it runs) because they don't have > all the same capabilities towards metrics (e.g. spark runner configures sinks > like csv, graphite or in memory sinks using the spark engine conf). The > target is to push the metrics in the common runner code so that no matter the > chosen runner, a user can get his metrics out of beam. > Here is the link to the discussion thread on the dev ML: > https://lists.apache.org/thread.html/01a80d62f2df6b84bfa41f05e15fda900178f882877c294fed8be91e@%3Cdev.beam.apache.org%3E > And the design doc: > https://s.apache.org/runner_independent_metrics_extraction -- This message was sent by Atlassian JIRA (v7.6.3#76005)