[
https://issues.apache.org/jira/browse/BEAM-14219?focusedWorklogId=759606&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-759606
]
ASF GitHub Bot logged work on BEAM-14219:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Apr/22 21:44
Start Date: 20/Apr/22 21:44
Worklog Time Spent: 10m
Work Description: y1chi commented on code in PR #17323:
URL: https://github.com/apache/beam/pull/17323#discussion_r854578401
##########
sdks/python/apache_beam/runners/dataflow/dataflow_runner.py:
##########
@@ -393,6 +393,72 @@ def _adjust_pipeline_for_dataflow_v2(self, pipeline):
def _check_for_unsupported_features_on_non_portable_worker(self, pipeline):
pipeline.visit(self.combinefn_visitor())
+ def _enable_sdk_prebuild_if_applicable(self, options):
+ setup_options = options.view_as(SetupOptions)
+ # SDK container image pre-build explicitly enabled or disabled.
+ if (setup_options.prebuild_sdk_container_engine or
+ setup_options.disable_sdk_container_prebuild):
+ return
+ from apache_beam.runners.dataflow.internal import apiclient
+ # prebuild_sdk_container_engine only works with fnapi jobs.
+ if (not apiclient._use_fnapi(options) and
Review Comment:
On a side note, I guess the prebuilding doesn't really involve beam_fn_api,
so I'm wondering if we can backport the boot.go change to non-fnapi python
worker containers, I think the prebuilding has value for jobs on legacy python
batch worker as well, WDYT?
Issue Time Tracking
-------------------
Worklog Id: (was: 759606)
Time Spent: 6h (was: 5h 50m)
> Dataflow runner prebuilds SDK container with cloud build by default
> -------------------------------------------------------------------
>
> Key: BEAM-14219
> URL: https://issues.apache.org/jira/browse/BEAM-14219
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Yichi Zhang
> Assignee: Yichi Zhang
> Priority: P2
> Time Spent: 6h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
