reswqa opened a new pull request, #21560:
URL: https://github.com/apache/flink/pull/21560
## What is the purpose of the change
*At present, downstream task can be scheduled only if all upstream tasks
finished in adaptive batch scheduler. We can do some improvements to support
schedule downstream tasks when the upstream's some subtask is not completely
finished. After the upstream task's data is totally produced, we need to update
the result partition's information of this subtask to trigger downstream task's
consumption.*
## Brief change log
- *Introduce PartialFinishedInputConsumableDecider.*
- *Serialize and cache shuffle descriptor one by one instead of group.*
- *UnknownInputChannel supports update resultPartitionId when transform to
local input channel.*
- *Introduce ResultPartitionType#isHybridResultPartition.*
- *Adaptive batch scheduler supports schedule downstream tasks when
partial producer finished.*
## Verifying this change
This change added unit tests.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
