[
https://issues.apache.org/jira/browse/BEAM-9218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027310#comment-17027310
]
Elias Djurfeldt commented on BEAM-9218:
---------------------------------------
I've patched the fix in [https://github.com/apache/beam/pull/10728] and can
confirm that it works.
One thing to note, which might be a separate issue is that when supplying the
`requirements_file` argument to the command that creates the template, the
execution time increases considerably to around 3-4 minutes, instead of the 3
seconds or so when removing it. [~robertwb] Is this a bug or expected behaviour
when working with templates?
Example below takes 3-4 minutes to stage:
{code:java}
python -m run_pipeline \
--project=$PROJECT \
--runner=$RUNNER \
--staging_location=$STAGING_LOCATION \
--temp_location=$TEMP_LOCATION \
--template_location=$TEMPLATE_LOCATION \
--requirements_file=requirements.txt \
--region=REGION \
{code}
Whereas this example only takes 4 seconds or so
{code:java}
python -m run_pipeline \
--project=$PROJECT \
--runner=$RUNNER \
--staging_location=$STAGING_LOCATION \
--temp_location=$TEMP_LOCATION \
--template_location=$TEMPLATE_LOCATION \
--region=REGION \
{code}
> Template staging broken on Beam 2.18.0
> --------------------------------------
>
> Key: BEAM-9218
> URL: https://issues.apache.org/jira/browse/BEAM-9218
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Affects Versions: 2.18.0
> Reporter: Michael Charkin
> Priority: Major
>
> beam 2.18.0 can not stage cloud Dataflow templates
>
> Looks like it is trying to access the RuntimeValueProvider during staging
> causing 'not accessible'
>
> Repo with code to reproduce the issue:
> [https://github.com/firemuzzy/dataflow-templates-bug]
>
> With the help of stack overflow narrowed the issue to the latest beam release
> and not python versions
> [https://stackoverflow.com/questions/59940069/how-do-you-create-a-google-cloud-dataflow-template-with-python-3?noredirect=1#59940069]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
