[
https://issues.apache.org/jira/browse/BEAM-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Amato updated BEAM-6833:
-----------------------------
Description:
dataflow_metrics.py expects a job object to be passed in, it cannot fully query
the metrics on its own.
Updating this to use the dataflow_client to also query the job itself (non
trivial since the job passed in is a wrapper for the Job proto the
dataflow_client obtains, this wrapper is created by dataflow_runner.) Though we
may not need any of the extra wrapper features, since we only need access to
the graph. But it is unclear if this will work, as the graph DatalowRunner
generates is not the same graph as the one queried out.
The job graph is use to translate step names
was:
dataflow_metrics.py expects a job object to be passed in, it cannot fully query
the metrics on its own.
Updating this to use the dataflow_client to also query the job itself (non
trivial since the job passed in is a wrapper for the Job proto the
dataflow_client obtains, this wrapper is created by dataflow_runner.) Though we
may not need any of the extra wrapper features, since we only need access to
the graph. But it is unclear if this will work, as the graph DatalowRunner
generates is not the same graph as the one queried out.
> dataflow_metrics.py should be able to fully query the metrics
> -------------------------------------------------------------
>
> Key: BEAM-6833
> URL: https://issues.apache.org/jira/browse/BEAM-6833
> Project: Beam
> Issue Type: New Feature
> Components: sdk-py-core
> Reporter: Alex Amato
> Priority: Major
>
> dataflow_metrics.py expects a job object to be passed in, it cannot fully
> query the metrics on its own.
> Updating this to use the dataflow_client to also query the job itself (non
> trivial since the job passed in is a wrapper for the Job proto the
> dataflow_client obtains, this wrapper is created by dataflow_runner.) Though
> we may not need any of the extra wrapper features, since we only need access
> to the graph. But it is unclear if this will work, as the graph DatalowRunner
> generates is not the same graph as the one queried out.
>
> The job graph is use to translate step names
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
