[
https://issues.apache.org/jira/browse/BEAM-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16839537#comment-16839537
]
Valentyn Tymofieiev edited comment on BEAM-6429 at 5/14/19 3:03 PM:
--------------------------------------------------------------------
I think that it is likely that BEAM-6877 triggered the failure. On Python 3.6,
type inference returns "Any", instead of, say, Tuple. This means that we may
choose a less efficient coder (FastPrimitivesCoder instead of TupleCoder).
However my understanding is that this is an acceptable degradation and such
replacement should still work.
However, I think that somewhere in the code there is a more concerning
degradation where a structured pair KV[Any, Any] becomes Any. This may be a bug
in typehints or in the fn_api_runner. We should try to find this place, then we
may be able to suggest a fix for this issue before a better support for
typehints becomes available.
cc'ing [~robertwb] who is familiar with fn_api_runner and may have some
thoughts where to dig.
cc'ing [~udim] who is working on typehints replacement for awareness.
was (Author: tvalentyn):
I think that it is likely that BEAM-6877 triggered the failure. On Python 3.6,
type inference returns "Any", instead of, say, Tuple. This means that we may
choose a less efficient coder (FastPrimitivesCoder instead of TupleCoder).
However my understanding is that this is an acceptable degradation and such
replacement should still work.
However, I think that somewhere in the code there is a more concerning
degradation where a structured pair KV[Any, Any] becomes Any. This may be a bug
in typehints or in the fn_api_runner. We should try to find this place, then we
may be able to suggest a fix for this issue before a better support for
typehints becomes available.
cc'ing [~robertwb] who is familiar with fn_api_runner and may have some
thoughts where to dig.
cc'ing [~ehudm] who is working on typehints replacement for awareness.
> apache_beam.runners.portability.fn_api_runner_test.FnApiRunnerTest.test_multimap_side_input
> fails in Python 3.6
> ----------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-6429
> URL: https://issues.apache.org/jira/browse/BEAM-6429
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Frederik Bode
> Priority: Minor
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {noformat}
> ERROR: test_multimap_side_input
> (apache_beam.runners.portability.fn_api_runner_test.FnApiRunnerTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner_test.py",
> line 230, in test_multimap_side_input
> equal_to([('a', [1, 3]), ('b', [2])]))
> File "/beam/sdks/python/apache_beam/pipeline.py", line 425, in __exit__
> self.run().wait_until_finish()
> File "/beam/sdks/python/apache_beam/pipeline.py", line 405, in run
> self._options).run(False)
> File "/beam/sdks/python/apache_beam/pipeline.py", line 418, in run
> return self.runner.run_pipeline(self, self._options)
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 265, in run_pipeline
> default_environment=self._default_environment))
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 268, in run_via_runner_api
> return self.run_stages(*self.create_stages(pipeline_proto))
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 355, in run_stages
> safe_coders)
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 449, in run_stage
> elements_by_window = _WindowGroupingBuffer(si, value_coder)
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 191, in __init__
> self._key_coder = coder.wrapped_value_coder.key_coder()
> File "/beam/sdks/python/apache_beam/coders/coders.py", line 177, in key_coder
> raise ValueError('Not a KV coder: %s.' % self)
> ValueError: Not a KV coder: BytesCoder.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
