kennknowles commented on PR #32165:
URL: https://github.com/apache/beam/pull/32165#issuecomment-2297057635
I think these are the cases:
- choose v2: we don't even do the v1 translation, and `upload_graph` is
turned off / does nothing, portable pipeline staged to GCS
- choose v1: `upload_graph` works and also we will set it automatically if
necessary
- leave up to the service: v1 graph follows the usual v1 path and portable
pipeline staged to GCS
So there might be a corner case for a very large graph that does not choose
v1 or v2. In this case, we will automatically turn on `upload_graph` and stage
both the v1 dataflow graph and the portable pipeline to GCS. I am not certain
if this works. What I said before would suggest it doesn't but I was probably
wrong. Basically if the staging locations or service-side code paths have
collisions then it won't work, otherwise it will.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
