[
https://issues.apache.org/jira/browse/BEAM-6942?focusedWorklogId=226441&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-226441
]
ASF GitHub Bot logged work on BEAM-6942:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Apr/19 00:41
Start Date: 12/Apr/19 00:41
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #8225: [BEAM-6942]
Make modifications to pipeline options to be visible to all views.
URL: https://github.com/apache/beam/pull/8225#discussion_r274727098
##########
File path: sdks/python/apache_beam/options/pipeline_options.py
##########
@@ -796,13 +796,13 @@ def _add_argparse_args(cls, parser):
'"<ENV_VAL>"} }. All fields in the json are optional except '
'command.'))
parser.add_argument(
- '--sdk-worker-parallelism', default=None,
+ '--sdk_worker_parallelism', default=None,
help=('Sets the number of sdk worker processes that will run on each '
'worker node. Default is 1. If 0, it will be automatically set '
'by the runner by looking at different parameters (e.g. number '
'of CPU cores on the worker machine).'))
parser.add_argument(
- '--environment-cache-millis', default=0,
+ '--environment_cache_millis', default=0,
Review comment:
I think `PortableOptions` should contain options that all Portable Runners
are expected to support and `RunnerOptions` may be something that will be
obtained from the runner and represent runner-specific options that SDK cannot
be aware of. Looks like current `RunnerOptions` class is not used, instead we
try to get runner-specific options via JobApi, see
https://github.com/apache/beam/blob/e4cf63b9c69cdd8ce37208bb301595f5f2af12c8/sdks/python/apache_beam/runners/portability/portable_runner.py#L281
and above. More changes may be needed to create PipelineOptions subclasses
dynamically... Oh well. This is beyond the scope of this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 226441)
Time Spent: 8h 20m (was: 8h 10m)
> Pipeline options to experiment propagation is not working in Dataflow runner.
> -----------------------------------------------------------------------------
>
> Key: BEAM-6942
> URL: https://issues.apache.org/jira/browse/BEAM-6942
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Valentyn Tymofieiev
> Priority: Major
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> Relevant code:
> [https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L356-L388]
> 3 experiments/options are affected. We need to fix it in 2.12.0
> cc: [~altay] [~apilloud]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
