Chuan Yu Foo created BEAM-3530:
----------------------------------
Summary: DoFn.process should raise exception if something other
than a List is returned
Key: BEAM-3530
URL: https://issues.apache.org/jira/browse/BEAM-3530
Project: Beam
Issue Type: Bug
Components: sdk-py-core
Reporter: Chuan Yu Foo
Assignee: Ahmet Altay
The process method of DoFns can either return values or yield values. In the
case of returning values, it expects a List of elements to be returned. When
returning a single value, it is easy to forget this, and return the value
instead.
Correct way:
{{class SomeDoFn(beam.DoFn)}}
{{ def process(self, elem):}}
{{ return ['a']}}
Incorrect way:
{{class SomeDoFn(beam.DoFn)}}
{{ def process(self, elem):}}
{{ return 'a'}}
A pipeline with the incorrect DoFn will fail will a cryptic error message
without a direct indication that the actual error is due to SomeDoFn returning
an element instead of a List containing that element. This issue is very
time-consuming to track down.
It would be good if the pipeline could raise an exception or otherwise indicate
that the DoFn is incorrectly returning an element instead of a List to make it
easier to identify the error.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
