David Song created BEAM-8814:
--------------------------------
Summary: --no_auth flag is boolean type and is misleading
Key: BEAM-8814
URL: https://issues.apache.org/jira/browse/BEAM-8814
Project: Beam
Issue Type: Bug
Components: sdk-py-harness
Affects Versions: 2.16.0, 2.15.0, 2.14.0
Environment: Python2, Python3
Reporter: David Song
Fix For: 2.16.0, 2.15.0, 2.14.0
Pipeline options defines a
[no_auth|[https://github.com/apache/beam/blob/master/sdks/python/apache_beam/options/pipeline_options.py#L468]]
flag that is type=bool. This type is known to be ambiguous because it will
expect a value, but anything passed to it will be considered True. For example,
passing in "--no_auth=False" would still evaluate to True. We should instead
use action="store_true" which only detects whether the flag is passed or not.
Furthermore,
[PipelineOptions.from_dictionary|[https://github.com/apache/beam/blob/master/sdks/python/apache_beam/options/pipeline_options.py#L229]]
will assume that boolean flags are passed in without values (e.g. passing
--no_auth, instead of --no_auth=True). This, combined with type=bool failing
without a value, will ensure that it always fails.
sdk_worker_main is the only place that uses from_dictionary (aside from tests),
and it will crash if no_auth flag is passed. Looking at pipeline_options_test,
tests that call
[from_dictionary|[https://github.com/apache/beam/blob/master/sdks/python/apache_beam/options/pipeline_options_test.py#L218]]
will feed in get_all_options, which means it have intended to only be used for
serializing/deserializing flag options.
So from here, to support the no_auth flag:
* we change no_auth so that it is action="store_true", or
* we change sdk_worker_main so that it does not use from_dictionary
Or both.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
