chrisqiqiu commented on issue #18712:
URL: https://github.com/apache/beam/issues/18712#issuecomment-4369757552
Hi @kennknowles , im new contributor and i'd like to take this issue.
i can confirm that bug still reproduces on master (2.74.0.dev). A DoFn that
mistakenly returns 'abc-processed' instead of ['abc-processed'] silently emits
13 single-character elements ('a', 'b', 'c', ...) — no error, no warning.
To answer your question above — type hints don't catch this in the common
case. I tested four variants:
1. No hints -> silent
2. PEP 484 annotations on def process(self, e: str) -> Iterable[str] ->
silent
3. @beam.typehints.with_output_types(str) -> silent
4. @with_output_types(str) + --runtime_type_check -> caught with error
message ("Returning a str from a ParDo or FlatMap is discouraged") for str,
bytes, and dict. The check just isn't run by default.
Beam already has the check at typehints/typecheck.py, but it's gated behind
two opt-ins. My plan to fix is to promote the isinstance(output, (dict, bytes,
str)) portion to run unconditionally in handle_process_outputs (three
isinstance checks per output and catches the common mistake).
Could you pls assign this to me so I can put up a PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
