This is an automated email from the ASF dual-hosted git repository.
sewen pushed a change to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git.
from bc9f0fc [FLINK-16753][checkpointing] Use CheckpointException to wrap
exceptions thrown from AsyncCheckpointRunnable (#14072)
new c54b12b [backport] Backport some core changes that were part of the
FLIP-27 File Source commit
new b0bfe8b [backport] Relevant test mock changes from
"[FLINK-18907][test] Refactor MockSourceReader"
new 21283ee [backport] Relevant test mock changes from
"[FLINK-18907][task] Add test coverage for watermarks with chained sources"
new c7d2b4f [backport] Relevant test mock changes from
"[FLINK-18907][task] Fix and add test coverage for watermarks gauges with
chained sources"
new bb234ca [backport] Relevant test mock changes from
"[FLINK-18907][test] Add stream status forwarding test for chained sources"
new 9c558bc [refactor][tests] Move some source test utils from
flink-connector-base to flink-core
new 0f7ff28 [FLINK-19457][core] Add a number sequence generating source
for the FLIP-27 source API.
new 99e9a90 [FLINK-19492][core] Consolidate Source Events between Source
API and Split Reader API
new e4e0f23 [FLINK-19448][connector base] Explicitly check for
un-expected condition that would leave an inconsistent state
new 253ffd2 [FLINK-19427][FLINK-19489][tests] Fix test conditions for
'SplitFetcherTest.testNotifiesWhenGoingIdleConcurrent()'
new 1131e05 [FLINK-18044][Connectors/Common] Add the subtask index
information to the SourceReaderContext. (#12647)
new dffc124 [hotfix][runtime] Remove commented-out annotation in
SourceOperator
new 37ccdbe [FLINK-18820] Emit MAX_WATERMARK at the end in SourceOperator
new 1be08c1 [hotfix] Replace finally block with JUnit After method in
SourceOperatorTest.
new 83e3613 [hotfix] Only close the SourceReader and EventTimeLogic in
the SourceOperator if they are not null.
new 4d413df [hotfix] Add a waitUntil() method to the CommonTestUtils.
new 9dce8f0 Backport of "[FLINK-19698][connector/common] Let
SplitEnumerator/SourceReader/SourceCoordinator implement CheckpointListener."
new 2d07471 [FLINK-19698][connector/common] Add a close() method to the
SplitReader.
new 216f469 [FLINK-19384][core] Add common permissive exception
signatures to all methods of Source.
new 2b4d69d [build] Add exclusions for added default methods in
SourceReader and SplitEnumerator interfaces.
new ae23ab6 [refactor][DataStream API] Make DataStreamUtils.collect()
methods more flexible.
The 21 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../SingleThreadMultiplexSourceReaderBase.java | 63 ++++-
.../base/source/reader/SourceReaderBase.java | 11 +-
.../base/source/reader/fetcher/SplitFetcher.java | 10 +-
.../source/reader/splitreader/SplitReader.java | 7 +
.../base/source/reader/SourceReaderBaseTest.java | 16 +-
.../base/source/reader/SourceReaderTestBase.java | 2 +-
.../reader/fetcher/SplitFetcherManagerTest.java | 22 ++
.../source/reader/fetcher/SplitFetcherTest.java | 63 ++---
.../base/source/reader/mocks/MockSourceReader.java | 5 +
.../source/reader/mocks/MockSplitEnumerator.java | 2 +-
.../base/source/reader/mocks/MockSplitReader.java | 3 +
.../source/reader/mocks/TestingSplitReader.java | 20 ++
flink-core/pom.xml | 5 +-
.../apache/flink/api/connector/source/Source.java | 16 +-
.../flink/api/connector/source/SourceReader.java | 19 +-
.../api/connector/source/SourceReaderContext.java | 5 +
.../api/connector/source/SplitEnumerator.java | 17 +-
.../connector}/source/event/NoMoreSplitsEvent.java | 2 +-
.../connector}/source/event/RequestSplitEvent.java | 15 +-
.../connector/source/lib/NumberSequenceSource.java | 253 +++++++++++++++++++++
.../source/lib/util/IteratorSourceEnumerator.java | 90 ++++++++
.../source/lib/util/IteratorSourceReader.java | 148 ++++++++++++
.../source/lib/util/IteratorSourceSplit.java | 34 ++-
.../source/lib/NumberSequenceSourceTest.java | 91 ++++++++
.../api/connector/source/mocks/MockSource.java | 12 +-
.../connector/source/mocks/MockSourceReader.java | 98 ++++++--
.../source/mocks/MockSplitEnumerator.java | 21 +-
.../source}/mocks/TestingReaderContext.java | 11 +-
.../source}/mocks/TestingReaderOutput.java | 2 +-
.../runtime/checkpoint/CheckpointCoordinator.java | 7 +-
.../OperatorCoordinatorCheckpointContext.java | 19 +-
.../coordination/OperatorCoordinator.java | 42 ++--
.../coordination/OperatorCoordinatorHolder.java | 22 +-
.../RecreateOnResetOperatorCoordinator.java | 13 +-
.../source/coordinator/SourceCoordinator.java | 24 +-
.../coordinator/SourceCoordinatorProvider.java | 2 +-
.../checkpoint/CheckpointCoordinatorTest.java | 53 +++++
.../CheckpointCoordinatorTestingUtils.java | 19 +-
.../CoordinatorEventsExactlyOnceITCase.java | 2 +-
.../coordination/MockOperatorCoordinator.java | 2 +-
.../OperatorCoordinatorHolderTest.java | 2 +-
.../RecreateOnResetOperatorCoordinatorTest.java | 3 +-
.../coordination/TestingOperatorCoordinator.java | 2 +-
.../coordinator/SourceCoordinatorProviderTest.java | 2 +-
.../source/coordinator/SourceCoordinatorTest.java | 12 +-
.../coordinator/SourceCoordinatorTestBase.java | 5 +-
.../streaming/api/datastream/DataStreamUtils.java | 136 ++++++++++-
.../streaming/api/operators/SourceOperator.java | 55 ++++-
.../api/operators/SourceOperatorFactory.java | 10 +-
.../collect/CollectSinkOperatorCoordinator.java | 2 +-
.../runtime/tasks/SourceOperatorStreamTask.java | 8 +-
.../api/operators/SourceOperatorTest.java | 156 ++++++++-----
.../source/SourceOperatorEventTimeTest.java | 14 +-
.../operators/source/TestingSourceOperator.java | 13 ++
.../tasks/SourceOperatorStreamTaskTest.java | 57 +++--
.../flink/core/testutils/CommonTestUtils.java | 29 +++
.../source/lib/IteratorSourcesITCase.java | 90 ++++++++
57 files changed, 1590 insertions(+), 274 deletions(-)
rename
{flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base
=>
flink-core/src/main/java/org/apache/flink/api/connector}/source/event/NoMoreSplitsEvent.java
(96%)
rename
{flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base
=>
flink-core/src/main/java/org/apache/flink/api/connector}/source/event/RequestSplitEvent.java
(83%)
create mode 100644
flink-core/src/main/java/org/apache/flink/api/connector/source/lib/NumberSequenceSource.java
create mode 100644
flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceEnumerator.java
create mode 100644
flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceReader.java
copy
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/TestingSourceSplit.java
=>
flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceSplit.java
(55%)
create mode 100644
flink-core/src/test/java/org/apache/flink/api/connector/source/lib/NumberSequenceSourceTest.java
rename
{flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader
=>
flink-core/src/test/java/org/apache/flink/api/connector/source}/mocks/TestingReaderContext.java
(93%)
rename
{flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader
=>
flink-core/src/test/java/org/apache/flink/api/connector/source}/mocks/TestingReaderOutput.java
(96%)
create mode 100644
flink-tests/src/test/java/org/apache/flink/api/connector/source/lib/IteratorSourcesITCase.java
