This is an automated email from the ASF dual-hosted git repository.
xtsong pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
from e8e9db37e17 [FLINK-29728] TablePlanner prevents Flink from starting is
working directory is a symbolic link
new 840a5198df7 [hotfix] Migrate ResultPartitionFactoryTest to JUnit5 and
AssertJ.
new 7f338e87a77 [FLINK-28889] Full spilling strategy is no longer consider
consuming progress.
new 14392cf4ff2 [FLINK-28889] HsBufferContext supports multiple consumer.
new a2b402d66de [FLINK-28889] Introduce
HsSubpartitionConsumerMemoryDataManager and let HsMemoryDataManager supports
multiple consumer.
new f95c12e9a4a [FLINK-28889] HsFileDataManager supports multiple consumer.
new d65722f29a9 [FLINK-28889] Assign an unique identifier for each
downstream consumer.
new c04985e6ac9 [FLINK-28889] fail fast when multiple consumer is not
allowed but register many times
new d11940c4a78 [FLINK-28889] HsResultPartition support broadcast optimize
The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../ResultPartitionDeploymentDescriptor.java | 5 +
.../network/partition/ResultPartitionFactory.java | 12 ++
.../network/partition/hybrid/HsBufferContext.java | 16 +-
.../network/partition/hybrid/HsConsumerId.java} | 43 ++---
.../io/network/partition/hybrid/HsDataView.java | 4 +-
.../partition/hybrid/HsFileDataManager.java | 8 +-
.../partition/hybrid/HsFullSpillingStrategy.java | 68 +++----
.../partition/hybrid/HsMemoryDataManager.java | 65 ++++---
.../hybrid/HsMemoryDataManagerOperation.java | 15 +-
.../partition/hybrid/HsResultPartition.java | 60 ++++--
.../hybrid/HsSelectiveSpillingStrategy.java | 16 +-
.../partition/hybrid/HsSpillingInfoProvider.java | 34 +++-
...titionView.java => HsSubpartitionConsumer.java} | 20 +-
... HsSubpartitionConsumerInternalOperations.java} | 6 +-
.../HsSubpartitionConsumerMemoryDataManager.java | 185 +++++++++++++++++++
.../partition/hybrid/HsSubpartitionFileReader.java | 3 +-
.../hybrid/HsSubpartitionFileReaderImpl.java | 16 +-
.../hybrid/HsSubpartitionMemoryDataManager.java | 191 +++++++------------
.../network/partition/ResultPartitionBuilder.java | 8 +
.../partition/ResultPartitionFactoryTest.java | 87 +++++----
.../partition/hybrid/HsBufferContextTest.java | 27 ++-
.../network/partition/hybrid/HsConsumerIdTest.java | 26 +--
.../partition/hybrid/HsFileDataManagerTest.java | 34 ++--
.../hybrid/HsFullSpillingStrategyTest.java | 39 +---
.../partition/hybrid/HsMemoryDataManagerTest.java | 20 ++
.../partition/hybrid/HsResultPartitionTest.java | 204 +++++++++++++++++++++
...sSubpartitionConsumerMemoryDataManagerTest.java | 197 ++++++++++++++++++++
.../hybrid/HsSubpartitionFileReaderImplTest.java | 84 ++++++---
.../HsSubpartitionMemoryDataManagerTest.java | 160 ++++++----------
.../partition/hybrid/HsSubpartitionViewTest.java | 60 +++---
.../hybrid/TestingMemoryDataManagerOperation.java | 26 ++-
.../hybrid/TestingSpillingInfoProvider.java | 10 +-
...tingSubpartitionConsumerInternalOperation.java} | 6 +-
.../shuffle/PartitionDescriptorBuilder.java | 9 +-
34 files changed, 1227 insertions(+), 537 deletions(-)
copy
flink-runtime/src/main/java/org/apache/flink/runtime/{rest/handler/async/OperationKey.java
=> io/network/partition/hybrid/HsConsumerId.java} (54%)
rename
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/{HsSubpartitionView.java
=> HsSubpartitionConsumer.java} (94%)
rename
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/{HsSubpartitionViewInternalOperations.java
=> HsSubpartitionConsumerInternalOperations.java} (87%)
create mode 100644
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/HsSubpartitionConsumerMemoryDataManager.java
copy
flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/handlers/JarDeleteHeadersTest.java
=>
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/hybrid/HsConsumerIdTest.java
(56%)
create mode 100644
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/hybrid/HsSubpartitionConsumerMemoryDataManagerTest.java
rename
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/hybrid/{TestingSubpartitionViewInternalOperation.java
=> TestingSubpartitionConsumerInternalOperation.java} (88%)
