boyuanzz commented on a change in pull request #12419:
URL: https://github.com/apache/beam/pull/12419#discussion_r477462448
##########
File path:
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/HandlesSplits.java
##########
@@ -35,6 +36,12 @@
/** Returns the current progress of the active element as a fraction between
0.0 and 1.0. */
double getProgress();
+ String getPtranformId();
Review comment:
Yes, we can pass these information locally for current transform. But I
also need these information from `process transform` when current transform is
`truncate`. Whether this change is needed depends on the decision on which
transform id and main input id we should use for split from `truncate`. As I
mentioned in previous comment:
I think we should still use transform id and main input id from process
sized elements. There are 3 possible cases:
- We only have splits on window boundary
- We only have element split on the last window
- We have both element splits and window splits.
If we choose transfrom id and input id from truncate for element split, it's
also wrong conceptually. I prefer process sized elements because it will
simplify the implementation on the runner side. For example, the runner doesn't
need to distinguish window splits or element splits. If we treat truncate and
process as a whole, it also make sense to use transform id from process.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
