[
https://issues.apache.org/jira/browse/BEAM-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Amato updated BEAM-7025:
-----------------------------
Description:
This is an indication of a user misconfiguring a beam pipeline.
This is because its not possible to get a handle to use the produced
pcollection for that output tag, if .with_outputs is not used. So this should
be disallowed entirely, a run time exception should be thrown.
Note:
The bundle descriptor knows which tags are available for each step. So at
runtime it can be detected. But we need to be careful to not test it on every
element, for performance purposes
i suspect its possible to detect it statically, but may require collecting more
information
But there should be some code path already collects the elements for the bundle
into the different tags when output at that point, at the end of bundle
execution we can check for it which would be cheap
was:
This is an indication of a user misconfiguring a beam pipeline.
This is because its not possible to get a handle to use the produced
pcollection for that output tag, if .with_outputs is not used. So this should
be disallowed entirely, a run time exception should be thrown.
> Python pipelines should not be able to use output tags that are not defined
> in with_outputs.
> --------------------------------------------------------------------------------------------
>
> Key: BEAM-7025
> URL: https://issues.apache.org/jira/browse/BEAM-7025
> Project: Beam
> Issue Type: New Feature
> Components: sdk-py-core
> Reporter: Alex Amato
> Priority: Major
>
> This is an indication of a user misconfiguring a beam pipeline.
> This is because its not possible to get a handle to use the produced
> pcollection for that output tag, if .with_outputs is not used. So this should
> be disallowed entirely, a run time exception should be thrown.
> Note:
> The bundle descriptor knows which tags are available for each step. So at
> runtime it can be detected. But we need to be careful to not test it on every
> element, for performance purposes
>
> i suspect its possible to detect it statically, but may require collecting
> more information
>
> But there should be some code path already collects the elements for the
> bundle into the different tags when output at that point, at the end of
> bundle execution we can check for it which would be cheap
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
