TheNeuralBit commented on code in PR #23069:
URL: https://github.com/apache/beam/pull/23069#discussion_r970103451
##########
sdks/python/apache_beam/runners/interactive/pipeline_fragment.py:
##########
@@ -233,3 +234,9 @@ def visit_transform(self, transform_node):
v = PruneVisitor()
runner_pipeline.visit(v)
return runner_pipeline
+
+
+def should_skip_pruning(transform: AppliedPTransform):
+ return (
+ isinstance(transform.transform, TestStream) or
+ '_DataFrame_' in transform.full_label)
Review Comment:
Also - it might be good to make sure this works for operations on
DeferredSeries objects as well (i.e. do we need to match on "_Series"?). This
could be challenging though. I don't think we've encountered an example of a
series getting pruned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
