[
https://issues.apache.org/jira/browse/BEAM-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16839410#comment-16839410
]
Frederik Bode edited comment on BEAM-6429 at 5/14/19 1:25 PM:
--------------------------------------------------------------
I went up the stacktrace, and noticed that the transform containing the
multimap didn't have a `TupleCoder` (like in Python 3.5) as its coder, but a
`FastPrimitivesCoder`. This means there already a difference between Python3.5
and Python3.6 in the construction of the pipeline_proto variable. This led me
to:
{code:java}
pcoll.element_type = typehints.coerce_to_kv_type(
pcoll.element_type, transform_node.full_label){code}
referenced [here |
https://github.com/apache/beam/blob/5c7ee600ba8560102f61622c894633955d208f87/sdks/python/apache_beam/pipeline.py#L638-L639].
```
was (Author: frederik):
Update: `python setup.py nosetests --tests
apache_beam.runners.portability.fn_api_runner_test.py:FnApiRunnerTest.test_multimap_side_input`
fails with a `Not a KV coder: BytesCoder`. This is JIRA issue
[BEAM-6429](https://issues.apache.org/jira/browse/BEAM-6429). I went up the
stacktrace, and noticed that the transform containing the multimap didn't have
a `TupleCoder` (like in Python 3.5) as its coder, but a `FastPrimitivesCoder`.
This is in de `pipeline_proto` variable, which led me to:
```
pcoll.element_type = typehints.coerce_to_kv_type(
pcoll.element_type, transform_node.full_label)
```
(referenced
[here](https://github.com/apache/beam/blob/5c7ee600ba8560102f61622c894633955d208f87/sdks/python/apache_beam/pipeline.py#L638-L639).
), which makes me think this is linked to
> apache_beam.runners.portability.fn_api_runner_test.FnApiRunnerTest.test_multimap_side_input
> fails in Python 3.6
> ----------------------------------------------------------------------------------------------------------------
>
> Key: BEAM-6429
> URL: https://issues.apache.org/jira/browse/BEAM-6429
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Frederik Bode
> Priority: Minor
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {noformat}
> ERROR: test_multimap_side_input
> (apache_beam.runners.portability.fn_api_runner_test.FnApiRunnerTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner_test.py",
> line 230, in test_multimap_side_input
> equal_to([('a', [1, 3]), ('b', [2])]))
> File "/beam/sdks/python/apache_beam/pipeline.py", line 425, in __exit__
> self.run().wait_until_finish()
> File "/beam/sdks/python/apache_beam/pipeline.py", line 405, in run
> self._options).run(False)
> File "/beam/sdks/python/apache_beam/pipeline.py", line 418, in run
> return self.runner.run_pipeline(self, self._options)
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 265, in run_pipeline
> default_environment=self._default_environment))
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 268, in run_via_runner_api
> return self.run_stages(*self.create_stages(pipeline_proto))
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 355, in run_stages
> safe_coders)
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 449, in run_stage
> elements_by_window = _WindowGroupingBuffer(si, value_coder)
> File "/beam/sdks/python/apache_beam/runners/portability/fn_api_runner.py",
> line 191, in __init__
> self._key_coder = coder.wrapped_value_coder.key_coder()
> File "/beam/sdks/python/apache_beam/coders/coders.py", line 177, in key_coder
> raise ValueError('Not a KV coder: %s.' % self)
> ValueError: Not a KV coder: BytesCoder.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
