robertwb commented on a change in pull request #12352:
URL: https://github.com/apache/beam/pull/12352#discussion_r473208125
##########
File path: sdks/python/apache_beam/examples/snippets/snippets.py
##########
@@ -689,8 +689,9 @@ def examples_wordcount_streaming(argv):
output = (
lines
- | 'DecodeUnicode' >>
- beam.FlatMap(lambda encoded: encoded.decode('utf-8'))
+ | 'DecodeUnicode' >> beam.FlatMap(
+ lambda encoded:
+ (encoded if isinstance(encoded, list) else
encoded.decode('utf-8')))
Review comment:
What I'm really asking was why was the test wrong? Isn't a list the
wrong thing to pass to the next operation, ExtractWords, which uses re.findall?
##########
File path: sdks/python/apache_beam/typehints/typecheck.py
##########
@@ -265,3 +269,74 @@ def visit_transform(self, applied_transform):
transform.get_type_hints(),
applied_transform.full_label),
applied_transform.full_label)
+
+
+class PerformanceTypeCheckVisitor(pipeline.PipelineVisitor):
+ def visit_transform(self, applied_transform):
+ transform = applied_transform.transform
+ full_label = applied_transform.full_label
+
+ # Store output type hints in current transform
+ if hasattr(transform, 'fn'):
+ if not hasattr(transform.fn, '_runtime_output_constraints'):
Review comment:
Use `transform._add_type_constraint_from_consumer` rather than direct
access here as well. Then you can get rid of hasattr(transform, 'fn') and just
write
```
output_type_hints = self.get_output_type_hints(transform)
if output_type_hints:
transform._add_type_constraint_from_consumer(output_type_hints)
```
##########
File path: sdks/python/apache_beam/runners/worker/opcounters.py
##########
@@ -224,8 +230,25 @@ def _observable_callback_inner(value, is_encoded=False):
return _observable_callback_inner
+ def type_check(self, value):
+ # type: (any, bool) -> None
+ for transform_label, type_constraint_tuple in (
+ self.output_type_constraints.items()):
+ parameter_name, constraint = type_constraint_tuple
+ try:
+ _check_instance_type(constraint, value, parameter_name, verbose=True)
+ except TypeCheckError as e:
+ if not transform_label.startswith('ParDo'):
Review comment:
Drop a TODO(BEAM-10710) here as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
