[
https://issues.apache.org/jira/browse/BEAM-6942?focusedWorklogId=225833&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-225833
]
ASF GitHub Bot logged work on BEAM-6942:
----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Apr/19 20:36
Start Date: 10/Apr/19 20:36
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #8225: [BEAM-6942]
Make modifications to pipeline options to be visible to all views.
URL: https://github.com/apache/beam/pull/8225#discussion_r274145131
##########
File path: sdks/python/apache_beam/options/pipeline_options.py
##########
@@ -796,13 +796,13 @@ def _add_argparse_args(cls, parser):
'"<ENV_VAL>"} }. All fields in the json are optional except '
'command.'))
parser.add_argument(
- '--sdk-worker-parallelism', default=None,
+ '--sdk_worker_parallelism', default=None,
help=('Sets the number of sdk worker processes that will run on each '
'worker node. Default is 1. If 0, it will be automatically set '
'by the runner by looking at different parameters (e.g. number '
'of CPU cores on the worker machine).'))
parser.add_argument(
- '--environment-cache-millis', default=0,
+ '--environment_cache_millis', default=0,
Review comment:
Further thoughts on this:
1. Why is some subset of Flink runner options defined in PortableOptions. It
seems that we should remove them, since they are not used by PortableRunner?
Any thoughts, @angoenka, @robertwb ?
2. If we need to keep `environment_cache_millis` in `PortableOptions`, is
the default 0 reasonable?
3. We should avoid using dashes in pipeline options. Emitting a warning
seems to be the safest way, failing or converting them to underscores will be
breaking changes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 225833)
Time Spent: 5h 20m (was: 5h 10m)
> Pipeline options to experiment propagation is not working in Dataflow runner.
> -----------------------------------------------------------------------------
>
> Key: BEAM-6942
> URL: https://issues.apache.org/jira/browse/BEAM-6942
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Valentyn Tymofieiev
> Priority: Major
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Relevant code:
> [https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L356-L388]
> 3 experiments/options are affected. We need to fix it in 2.12.0
> cc: [~altay] [~apilloud]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
