[
https://issues.apache.org/jira/browse/BEAM-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477023#comment-16477023
]
Etienne Chauchot edited comment on BEAM-3967 at 5/16/18 8:34 AM:
-----------------------------------------------------------------
MetricsPusher is quite a simple piece of software hosted in the runner side
what it does is this:
* there is one instance of it per pipeline.
* reads options provided by the user (what sink to use, what frequency to
push, additional sink related parameters)
* schedules a thread which regularly merges the metrics and pushes them to
the configured metrics sink.
* at each thread tick, it checks the state of the pipeline and if it is in
terminal state, then it terminates the pushing thread.
So, for the communication between SDKs and runners, the only thing needed is
the pipelineOptions all the other things are runner dependent and runner side.
was (Author: echauchot):
MetricsPusher is quite a simple piece of software hosted in the runner side
what it does is this:
* reads options provided by the user (what sink to use, what frequency to
push, additional sink related parameters)
* schedules a thread which regularly merges the metrics of a given pipeline
and pushes them to the configured metrics sink.
* at each thread tick, it checks the status of the pipeline and if it is in
terminal state, then it terminates the pushing thread.
So, for the communication between SDKs and runners, the only thing needed is
the pipelineOptions all the other things are runner dependent and runner side.
> Create a langage agnostic definition of MetricsPusher
> -----------------------------------------------------
>
> Key: BEAM-3967
> URL: https://issues.apache.org/jira/browse/BEAM-3967
> Project: Beam
> Issue Type: Sub-task
> Components: runner-core
> Reporter: Etienne Chauchot
> Assignee: Etienne Chauchot
> Priority: Major
>
> Part of portability effort, we need to define a langage agnostic definition
> of Metrics pusher. Java runners are free to use default java implementation
> of MetricsPusher. But runners or SDKs that are not in java need this.
> Especially Dataflow engine will use it to instantiate the MetricsPusher (see
> https://issues.apache.org/jira/browse/BEAM-3926)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
