lostluck commented on code in PR #32325:
URL: https://github.com/apache/beam/pull/32325#discussion_r1733257870
##########
sdks/python/apache_beam/runners/direct/direct_runner.py:
##########
@@ -516,7 +516,8 @@ def visit_transform(self, applied_ptransform):
if isinstance(applied_ptransform.transform, ExternalTransform):
raise RuntimeError(
"Streaming Python direct runner "
- "does not support cross-language pipelines.")
+ "does not support cross-language pipelines."
+ "Please use other runners such as DataflowRunner.")
Review Comment:
The Beam codebase shouldn't be recommending a non-free runner. It's a
conflict of interest WRT the governance of the project (I think).
I would love for *Python* users to try Prism out before we make the
recommendation though. All I do is run the test suites. I don't try running
proper pipelines at this stage.
WRT Prism, by 2.60.0 it's hopefully passing all the Python Validates Runner
tests, and all the Java Validates Runner Tests that succeed against the Python
Portable Runner. Unfortunately, it's not quite the full set for 2.59.0. For
2.59.0, Prism is presently launching it as "experimental support" in the Java
and Python SDKs.
Unfortunately, because I'm on the release, and I need to finish my talks for
Beam Summit, making Prism documentation on the Beam site and similar isn't
happening until after Beam Summit, in September.
WRT to the specific reason for this message: Prism doesn't care if something
is streaming or not. It has unified execution of bounded and unbounded
pcollections. So xlang is an orthogonal concern.
However, per the above, prism doesn't support *everything* yet, so triggers
or specific windowing strategy configurations, among other things, may prevent
pipeline execution at this time. But TBH it's probably fine for most pipelines.
No "real time" processing time timers either (which also extends to
ProcessContinuation delays for streaming IOs too).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
