[
https://issues.apache.org/jira/browse/BEAM-13709?focusedWorklogId=721399&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-721399
]
ASF GitHub Bot logged work on BEAM-13709:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Feb/22 07:13
Start Date: 05/Feb/22 07:13
Worklog Time Spent: 10m
Work Description: tvalentyn edited a comment on pull request #16715:
URL: https://github.com/apache/beam/pull/16715#issuecomment-1030567382
I think the consitency we can pursue here is: when we initialize options
that have store_true / store_false actions, we don't allow values, but we
intepret the fact that the option is specified. In apis that expect K-V pairs
(PipelineOpitions(), from_dictionary(), we need to pass some value, so we only
allow `True` as value and interpret it as: 'option is present', not
necessarily that the corresponding dest needs to be set to `True`. This is what
argparse does, so this would be consistent with the command-line behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 721399)
Time Spent: 3h 40m (was: 3.5h)
> PipelineOptions() and from_dictionary parsing use_public_ips and
> no_use_public_ips differently
> ----------------------------------------------------------------------------------------------
>
> Key: BEAM-13709
> URL: https://issues.apache.org/jira/browse/BEAM-13709
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Minbo Bae
> Assignee: Anand Inguva
> Priority: P2
> Labels: starter, usability
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> {{PipelineOptions}} in Python has two methods to pass a param dict: using in
> constructor {{PipelineOptions(**params)}} or
> {{{}PipelineOptions.from_dictionary(params){}}}.
> But, they work slightly differently:
> *
> [PipelineOptions(**params)|https://github.com/apache/beam/blob/v2.35.0/sdks/python/apache_beam/options/pipeline_options.py#L313-L324]
> discards an option if it is not defined as a dest of {{argparse}} in an
> Option class. For example, {{no_use_public_ips=True}} is ignored and the
> Dataflow job will run with public IPs. To disable public IPs, the option
> dictionary must use {{{}use_public_ips{}}}.
> *
> [PipelineOptions.from_dictionary()|https://github.com/apache/beam/blob/v2.35.0/sdks/python/apache_beam/options/pipeline_options.py#L229]
> skips an option if the option value is {{{}False{}}}. For example,
> {{use_public_ips=False}} is ignored and the Dataflow job will run with public
> IPs. To disable public IPs, the option dictionary must use
> {{no_use_public_ips.}}
> This makes the user very confused, and sometimes the pipeline works in an
> unexpected way.
> We must have the consistent behavior between the two methods, or at least a
> warning about invalid ignored options.
> BEAM-9093 dealt with a similar issue for {{PipelineOptions()}}. Like the
> issue, I guess adding a warning in `PipelineOptions.from_dictionary()` for
> ignored options can help reducing the confusion, if we cannot have two
> methods have exactly the same behavior.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
