xintongsong commented on code in PR #19927:
URL: https://github.com/apache/flink/pull/19927#discussion_r895328670
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultPartitionType.java:
##########
@@ -81,7 +81,19 @@ public enum ResultPartitionType {
* in that {@link #PIPELINED_APPROXIMATE} partition can be reconnected
after down stream task
* fails.
*/
- PIPELINED_APPROXIMATE(true, false, ConsumingConstraint.CAN_BE_PIPELINED,
ReleaseBy.UPSTREAM);
+ PIPELINED_APPROXIMATE(true, false, ConsumingConstraint.CAN_BE_PIPELINED,
ReleaseBy.UPSTREAM),
+
+ /**
+ * Hybrid partitions with a bounded (local) buffer pool to support
downstream task to
+ * simultaneous reading and writing shuffle data.
+ *
+ * <p>Hybrid result has the following two characteristics:
+ *
+ * <p>Intermediate data can be consumed any time, whether fully produced
or not.
+ *
+ * <p>Intermediate data can be consumed directly from memory as much as
possible.
+ */
+ HYBRID(true, false, ConsumingConstraint.CAN_BE_PIPELINED,
ReleaseBy.SCHEDULER);
Review Comment:
Yes, the interfaces were intended to expose a characteristic rather than to
be bounded to a specific implementation. The problem was on the caller side.
Instead of relying on the intended characteristic, some callers were relying on
assumptions such as "a result partition type that returns `true` for
`isPipelined` must be `PIPELINED` or `PIPELINED_BOUNDED`".
E.g., in `AdaptiveScheduler#assertPreconditions`, the intention here is to
make sure only `PIPELINED` and `PIPELINED_BOUNDED` are involved. You may take a
look at where `isBlockingOrBlockingPersistentResultPartition` and
`isPipelinedOrPipelinedBoundedResultPartition` for more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
