[
https://issues.apache.org/jira/browse/BEAM-12920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17488258#comment-17488258
]
Valentyn Tymofieiev commented on BEAM-12920:
--------------------------------------------
This is likely a duplicate of https://issues.apache.org/jira/browse/BEAM-12914.
Test with additional logging enabled:
{noformat}
Testing started at 11:07 AM ...
/home/valentyn/.pyenv/versions/py39env/bin/python
/home/valentyn/.IdeaIE2019.3/config/plugins/python-ce/helpers/pycharm/_jb_unittest_runner.py
--target ptransform_test.PTransformTypeCheckTestCase.test_pardo_type_inference
Launching unittests with arguments python -m unittest
ptransform_test.PTransformTypeCheckTestCase.test_pardo_type_inference in
/home/valentyn/projects/beam/beam/beam/build/gradleenv/-1734967052/lib/python3.7/site-packages/apache_beam/transforms
/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/__init__.py:79:
UserWarning: This version of Apache Beam has not been sufficiently tested on
Python 3.9. You may encounter bugs or missing features.
warnings.warn(
<function Filter.<locals>.<lambda> at 0x7f003336e5e0> 139638835963360 [<class
'int'>]
2018 0 LOAD_DEREF 0 (fn)
2 LOAD_FAST 0 (x)
4 BUILD_LIST 1
6 LOAD_FAST 1 (args)
8 LIST_EXTEND 1
10 LIST_TO_TUPLE
12 BUILD_MAP 0
14 LOAD_FAST 2 (kwargs)
16 DICT_MERGE 1
18 CALL_FUNCTION_EX 1
20 POP_JUMP_IF_FALSE 28
22 LOAD_FAST 0 (x)
24 BUILD_LIST 1
26 RETURN_VALUE
>> 28 BUILD_LIST 0
30 RETURN_VALUE
0 LOAD_DEREF 0 (fn) Executing simple op LOAD_DEREF
Stack: [Const[<function
PTransformTypeCheckTestCase.test_pardo_type_inference.<locals>.<lambda> at
0x7f003336e550>]] Vars: [<class 'int'>, Union[], Union[]]
{}
2 LOAD_FAST 0 (x) Executing simple op LOAD_FAST
Stack: [Const[<function
PTransformTypeCheckTestCase.test_pardo_type_inference.<locals>.<lambda> at
0x7f003336e550>], <class 'int'>] Vars: [<class 'int'>, Union[], Union[]]
{}
4 BUILD_LIST 1 Executing simple op BUILD_LIST
Stack: [Const[<function
PTransformTypeCheckTestCase.test_pardo_type_inference.<locals>.<lambda> at
0x7f003336e550>], List[int]] Vars: [<class 'int'>, Union[], Union[]]
{}
6 LOAD_FAST 1 (args) Executing simple op LOAD_FAST
Stack: [Const[<function
PTransformTypeCheckTestCase.test_pardo_type_inference.<locals>.<lambda> at
0x7f003336e550>], List[int], Union[]] Vars: [<class 'int'>, Union[], Union[]]
{}
Traceback (most recent call last):
File
"/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/typehints/trivial_inference.py",
line 289, in infer_return_type
return infer_return_type_func(c, input_types, debug, depth)
File
"/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/typehints/trivial_inference.py",
line 507, in infer_return_type_func
raise TypeInferenceError('unable to handle %s' % opname)
apache_beam.typehints.trivial_inference.TypeInferenceError: unable to handle
LIST_EXTEND
8 LIST_EXTEND 1
Any != <class 'int'>
Expected :<class 'int'>
Actual :Any
<Click to see difference>
Traceback (most recent call last):
File
"/home/valentyn/.IdeaIE2019.3/config/plugins/python-ce/helpers/pycharm/teamcity/diff_tools.py",
line 32, in _patched_equals
old(self, first, second, msg)
File "/home/valentyn/.pyenv/versions/3.9.4/lib/python3.9/unittest/case.py",
line 831, in assertEqual
assertion_func(first, second, msg=msg)
File "/home/valentyn/.pyenv/versions/3.9.4/lib/python3.9/unittest/case.py",
line 824, in _baseAssertEqual
raise self.failureException(msg)
AssertionError: <class 'int'> != Any
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/valentyn/.pyenv/versions/3.9.4/lib/python3.9/unittest/case.py",
line 59, in testPartExecutor
yield
File "/home/valentyn/.pyenv/versions/3.9.4/lib/python3.9/unittest/case.py",
line 593, in run
self._callTestMethod(testMethod)
File "/home/valentyn/.pyenv/versions/3.9.4/lib/python3.9/unittest/case.py",
line 550, in _callTestMethod
method()
File
"/home/valentyn/projects/beam/beam/beam/build/gradleenv/-1734967052/lib/python3.7/site-packages/apache_beam/transforms/ptransform_test.py",
line 2484, in test_pardo_type_inference
self.assertEqual(int, beam.Filter(lambda x: False).infer_output_type(int))
{noformat}
> Convert To Bare Types Fails on Generators in Python 3.9
> -------------------------------------------------------
>
> Key: BEAM-12920
> URL: https://issues.apache.org/jira/browse/BEAM-12920
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.32.0
> Environment: Python 3.9.7
> Linux Mint 20.2, Kernel: 5.4.0-84-generic
> Reporter: Jonathan Hourany
> Priority: P2
>
> In Python 3.9 the test
> [test_convert_bare_types|https://github.com/apache/beam/blob/6c1c19d6c711babf5f5f042fa465be20948caff6/sdks/python/apache_beam/typehints/native_type_compatibility.py#L40]
> fails when testing the following Generator test case
> {code:python}
> test_cases += [
> (
> 'bare generator',
> typing.Generator,
> typehints.Generator[typehints.TypeVariable('T_co')]),
> ]
> {code}
> Raising the following {{ValueError}}
> {noformat}
> ./sdks/python/apache_beam/typehints/native_type_compatibility_test.py::NativeTypeCompatibilityTest::test_convert_bare_types
> Failed: [undefined]ValueError: Unsupported Generator with no arguments.
> {noformat}
> ----
> The immediate cause may be stemming from
> [_get_args|https://github.com/apache/beam/blob/6c1c19d6c711babf5f5f042fa465be20948caff6/sdks/python/apache_beam/typehints/native_type_compatibility.py#L40]
> which attempts to access a type's ___args___ attribute. In Python 3.9 bare
> generators no longer instantiate with a default ___args___ attribute. Only
> when instantiated with args do they get one (see example below).
> However, despite it's name the test doesn't create a generator that's
> totally bare, so the deeper cause may be coming from the way
> {{typinghints.Generator}} is constructed.
> h3. Examples
> {code:python}
> # Python 3.7.5
> In [1]: from typing import Generator
> In [2]: Generator.__args__
> Out[2]: (+T_co, -T_contra, +V_co)
> {code}
> ----
> {code:python}
> # Python 3.9.7
> In [1]: from typing import Generator
> In [2]: Generator.__args__
> ---------------------------------------------------------------------------
> AttributeError Traceback (most recent call last)
> <ipython-input-2-3e272b948083> in <module>
> ----> 1 Generator.__args__
> ~/.pyenv/versions/3.9.7/lib/python3.9/typing.py in __getattr__(self, attr)
> 704 if '__origin__' in self.__dict__ and not _is_dunder(attr):
> 705 return getattr(self.__origin__, attr)
> --> 706 raise AttributeError(attr)
> 707
> 708 def __setattr__(self, attr, val):
> AttributeError: __args__
> In [3]: # Initializing with args, however, creates the needed attr
> In [4]: Generator[int, int, int].__args__
> Out[4]: (int, int, int)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
