You can’t have the same argument in the parse_know_args and XyzOptions
(assuming you get these args in the same way).
From: Sofia’s World <mmistr...@gmail.com>
Sent: Tuesday, December 3, 2024 11:32 PM
To: user@beam.apache.org
Subject: Re: Problem running pipeline in unit tests
Hello
thanks for getting back.I tracked down the problem to one of the pipeline
option classes i have been using for years.
Somehow the argparse library is clashing, not sure if it is my IDE or something
else.....
I have replaced the option class at the bottom of this email with this method
and i will see how it goes.. but i think it's entirely my issue
Kind regards
def parse_known_args(argv):
"""Parses args for the workflow."""
parser = argparse.ArgumentParser()
parser.add_argument('--recipients')
class XyzOptions(PipelineOptions):
@classmethod
def _add_argparse_args(cls, parser):
parser.add_argument('--recipients',
default='mmistr...@gmail.com<mailto:mmistr...@gmail.com>')
parser.add_argument('--key')
parser.add_argument('--sendgridkey')
On Tue, Dec 3, 2024 at 10:05 PM XQ Hu via user
<user@beam.apache.org<mailto:user@beam.apache.org>> wrote:
Can you share the full example to reproduce this?
On Mon, Dec 2, 2024 at 4:30 PM Sofia’s World
<mmistr...@gmail.com<mailto:mmistr...@gmail.com>> wrote:
HI all
i am starting writing beam test in a new environment and suddenly i am
finding this error
while running a very simple test
def test_word_count(self):
with TestPipeline() as p:
input_data = p | 'Start' >> beam.Create(['hello world', 'hello beam'])
counts = (
input_data
| 'One' >>beam.FlatMap(lambda x: x.split(' '))
| 'Two' >> beam.Map(lambda x: (x, 1))
| 'Three' >> beam.CombinePerKey(sum)
)
debugSink = beam.Map(print)
counts | debugSink
the bot says it's an issue with argparse, and perhaps i have some lib clash...
i fixed it long time ago but now i forgot
Could anyone advise how can i address this problem?
Kind regards
Marco
test_word_count
(shareloader.test.test_stock_pick.TestEdgarUtils.test_word_count) ... Missing
pipeline option (runner). Executing pipeline using the default runner:
DirectRunner.
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
'function' object has no attribute '__self__'
ERROR
======================================================================
ERROR: test_word_count
(shareloader.test.test_stock_pick.TestEdgarUtils.test_word_count)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/workspaces/GCP_Experiments/dataflow/shareloader/test/test_stock_pick.py",
line 63, in test_word_count
with TestPipeline() as p:
File
"/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/pipeline.py",
line 620, in __exit__
self.result = self.run()
^^^^^^^^^^
File
"/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/testing/test_pipeline.py",
line 115, in run
result = super().run(
^^^^^^^^^^^^
File
"/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/pipeline.py",
line 570, in run
self._options).run(False)
^^^^^^^^^^
File
"/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/pipeline.py",
line 594, in run
return self.runner.run_pipeline(self, self._options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/runners/direct/direct_runner.py",
line 151, in run_pipeline
all_options = options.get_all_options()
^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/options/pipeline_options.py",
line 351, in get_all_options
cls._add_argparse_args(parser) # pylint: disable=protected-access
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspaces/GCP_Experiments/dataflow/samples/email_pipeline.py", line
46, in _add_argparse_args
parser.add_value_provider_argument('--input_file', type=str,
File
"/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/options/pipeline_options.py",
line 129, in add_value_provider_argument
self.add_argument(*args, **kwargs)
File "/home/codespace/.python/current/lib/python3.12/argparse.py", line 1472,
in add_argument
action = action_class(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^
TypeError: _StoreAction.__init__() got an unexpected keyword argument
'default_value'
