[
https://issues.apache.org/jira/browse/BEAM-5668?focusedWorklogId=151966&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151966
]
ASF GitHub Bot logged work on BEAM-5668:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Oct/18 11:18
Start Date: 06/Oct/18 11:18
Worklog Time Spent: 10m
Work Description: mxm closed pull request #6588: [BEAM-5668] Revert
"[BEAM-5442] Pass unknown SDK pipeline options to Runner (#6557)"
URL: https://github.com/apache/beam/pull/6588
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/python/apache_beam/options/pipeline_options.py
b/sdks/python/apache_beam/options/pipeline_options.py
index bc8c962bcd4..a0059dbb381 100644
--- a/sdks/python/apache_beam/options/pipeline_options.py
+++ b/sdks/python/apache_beam/options/pipeline_options.py
@@ -20,7 +20,6 @@
from __future__ import absolute_import
import argparse
-import logging
from builtins import list
from builtins import object
@@ -214,19 +213,7 @@ def get_all_options(self, drop_default=False):
subset[str(cls)] = cls
for cls in subset.values():
cls._add_argparse_args(parser) # pylint: disable=protected-access
- known_args, unknown_args = parser.parse_known_args(self._flags)
- # Parse args which are not known at this point but might be recognized
- # at a later point in time, i.e. by the actual Runner.
- if unknown_args and unknown_args[0] != '':
- logging.info("Parsing unknown args: %s", unknown_args)
- for arg in unknown_args:
- if arg.startswith('--'):
- parser.add_argument(arg.split('=', 1)[0], nargs='?')
- # repeat parsing with unknown options added
- known_args, unknown_args = parser.parse_known_args(self._flags)
- if unknown_args:
- logging.warn("Discarding unparseable args: %s", unknown_args)
-
+ known_args, _ = parser.parse_known_args(self._flags)
result = vars(known_args)
# Apply the overrides if any
diff --git a/sdks/python/apache_beam/options/pipeline_options_test.py
b/sdks/python/apache_beam/options/pipeline_options_test.py
index 651e7336b39..9c14c25668e 100644
--- a/sdks/python/apache_beam/options/pipeline_options_test.py
+++ b/sdks/python/apache_beam/options/pipeline_options_test.py
@@ -45,24 +45,16 @@ def tearDown(self):
'display_data': [DisplayDataItemMatcher('num_workers', 5)]},
{
'flags': [
- '--profile_cpu', '--profile_location', 'gs://bucket/',
- 'ignored', '-invalid=arg', '--unknown_arg', 'unknown_value',
- '--unknown_flag'
- ],
+ '--profile_cpu', '--profile_location', 'gs://bucket/',
'ignored'],
'expected': {
'profile_cpu': True, 'profile_location': 'gs://bucket/',
'mock_flag': False, 'mock_option': None,
- 'mock_multi_option': None,
- 'unknown_arg': 'unknown_value',
- 'unknown_flag': None},
+ 'mock_multi_option': None},
'display_data': [
DisplayDataItemMatcher('profile_cpu',
True),
DisplayDataItemMatcher('profile_location',
- 'gs://bucket/'),
- DisplayDataItemMatcher('unknown_arg',
- 'unknown_value')
- ]
+ 'gs://bucket/')]
},
{'flags': ['--num_workers', '5', '--mock_flag'],
'expected': {'num_workers': 5,
diff --git a/sdks/python/apache_beam/runners/worker/sdk_worker_main_test.py
b/sdks/python/apache_beam/runners/worker/sdk_worker_main_test.py
index e9b584a339f..6b5972e4ac4 100644
--- a/sdks/python/apache_beam/runners/worker/sdk_worker_main_test.py
+++ b/sdks/python/apache_beam/runners/worker/sdk_worker_main_test.py
@@ -56,8 +56,8 @@ def wrapped_method_for_test():
def test_work_count_default_value(self):
self._check_worker_count('{}', 12)
- def test_parse_pipeline_options(self):
- expected_options = PipelineOptions([])
+ def test_parse_pipeine_options(self):
+ expected_options = PipelineOptions()
expected_options.view_as(
SdkWorkerMainTest.MockOptions).m_m_option = [
'worker_threads=1', 'beam_fn_api'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 151966)
Time Spent: 2h (was: 1h 50m)
> [beam_PostCommit_Python_Verify] [:pythonPostCommit] Many tests failing
> ----------------------------------------------------------------------
>
> Key: BEAM-5668
> URL: https://issues.apache.org/jira/browse/BEAM-5668
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Reporter: Scott Wegner
> Assignee: Maximilian Michels
> Priority: Major
> Labels: currently-failing
> Time Spent: 2h
> Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
> * [Jenkins
> Job|https://builds.apache.org/job/beam_PostCommit_Python_Verify/6181/]
> * [Gradle Build
> Scan|https://scans.gradle.com/s/fcpta7s7wcsnm/console-log?task=:beam-sdks-python:postCommitITTests]
> * [Test source
> code|https://github.com/apache/beam/blob/4c693e7aa1ba71fa83370dd383f4bcebe4690b86/sdks/python/apache_beam/io/gcp/pubsub_integration_test.py#L171]
> * [Dataflow
> job|https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-05_07_00_20-5526009939236014896?project=apache-beam-testing]
> Initial investigation:
> Lots of different looking test failures from the Gradle build scan.
> ----
> _After you've filled out the above details, please [assign the issue to an
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
> Assignee should [treat test failures as
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
> helping to fix the issue or find a more appropriate owner. See [Apache Beam
> Post-Commit
> Policies|https://beam.apache.org/contribute/postcommits-policies]._
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
