[
https://issues.apache.org/jira/browse/BEAM-6942?focusedWorklogId=224780&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-224780
]
ASF GitHub Bot logged work on BEAM-6942:
----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Apr/19 05:56
Start Date: 09/Apr/19 05:56
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #8225: [BEAM-6942]
Make modifications to pipeline options to be visible to all views.
URL: https://github.com/apache/beam/pull/8225#discussion_r273334379
##########
File path: sdks/python/apache_beam/options/pipeline_options.py
##########
@@ -796,13 +796,13 @@ def _add_argparse_args(cls, parser):
'"<ENV_VAL>"} }. All fields in the json are optional except '
'command.'))
parser.add_argument(
- '--sdk-worker-parallelism', default=None,
+ '--sdk_worker_parallelism', default=None,
help=('Sets the number of sdk worker processes that will run on each '
'worker node. Default is 1. If 0, it will be automatically set '
'by the runner by looking at different parameters (e.g. number '
'of CPU cores on the worker machine).'))
parser.add_argument(
- '--environment-cache-millis', default=0,
+ '--environment_cache_millis', default=0,
Review comment:
@goenka, @robertwb, I rootcaused remaining failure on this PR to this typo.
With this PR, but without fixing this typo, PortableRunner tests fail
because the value `environment_cache_millis=10000` set in
https://github.com/apache/beam/blob/37556f2cd3ce32e23e16cb653632eafd7169c8e2/sdks/python/build.gradle#L182
is propagated, and then we end up losing correct path to a temporary location
stored in the environment while running the test. However I don't fully
understand why the test without any changes currently passes. It is possible
that another error is lurking somewhere. Do you have any ideas? Also, where is
`environment_cache_millis` actually used?
cc: @aaltay.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 224780)
Time Spent: 2h 40m (was: 2.5h)
> Pipeline options to experiment propagation is not working in Dataflow runner.
> -----------------------------------------------------------------------------
>
> Key: BEAM-6942
> URL: https://issues.apache.org/jira/browse/BEAM-6942
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Valentyn Tymofieiev
> Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Relevant code:
> [https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L356-L388]
> 3 experiments/options are affected. We need to fix it in 2.12.0
> cc: [~altay] [~apilloud]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
