This is an automated email from the ASF dual-hosted git repository.
sewen pushed a change to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git.
from c24185d [hotfix] Adjust japicmp exclusion to permit adding methods to
SourceReaderContext and SplitEnumeratorContext interfaces
new 9190335 [backport] Backport some core changes that were part of the
FLIP-27 File Source commit
new ecc6a8a [backport] Relevant test mock changes from
"[FLINK-18907][test] Refactor MockSourceReader"
new f79a3a7 [backport] Relevant test mock changes from
"[FLINK-18907][task] Add test coverage for watermarks with chained sources"
new cd3ffa5 [backport] Relevant test mock changes from
"[FLINK-18907][task] Fix and add test coverage for watermarks gauges with
chained sources"
new 242ced9 [backport] Relevant test mock changes from
"[FLINK-18907][test] Add stream status forwarding test for chained sources"
new 7916ad7 [refactor][tests] Move some source test utils from
flink-connector-base to flink-core
new 09ea814 [FLINK-19457][core] Add a number sequence generating source
for the FLIP-27 source API.
new 301046a [FLINK-19492][core] Consolidate Source Events between Source
API and Split Reader API
new cdc0721 [FLINK-19448][connector base] Explicitly check for
un-expected condition that would leave an inconsistent state
new dd2ac7a [FLINK-19427][FLINK-19489][tests] Fix test conditions for
'SplitFetcherTest.testNotifiesWhenGoingIdleConcurrent()'
new 5b2cc0e [FLINK-18044][Connectors/Common] Add the subtask index
information to the SourceReaderContext. (#12647)
new 1f823bd [hotfix][runtime] Remove commented-out annotation in
SourceOperator
new b3ea6da [FLINK-18820] Emit MAX_WATERMARK at the end in SourceOperator
new 8ad5da9 [hotfix] Replace finally block with JUnit After method in
SourceOperatorTest.
new 3463da5 [hotfix] Only close the SourceReader and EventTimeLogic in
the SourceOperator if they are not null.
new 5787a70 [hotfix] Add a waitUntil() method to the CommonTestUtils.
new 5f63e34 Backport of "[FLINK-19698][connector/common] Let
SplitEnumerator/SourceReader/SourceCoordinator implement CheckpointListener."
new 2a77589 [FLINK-19698][connector/common] Add a close() method to the
SplitReader.
new abe4c4b [FLINK-19384][core] Add common permissive exception
signatures to all methods of Source.
new 9cb2419 [build] Add exclusions for added default methods in
SourceReader and SplitEnumerator interfaces.
new 1a8ff89 [refactor][DataStream API] Make DataStreamUtils.collect()
methods more flexible.
The 21 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../SingleThreadMultiplexSourceReaderBase.java | 63 ++++-
.../base/source/reader/SourceReaderBase.java | 11 +-
.../base/source/reader/fetcher/SplitFetcher.java | 10 +-
.../source/reader/splitreader/SplitReader.java | 7 +
.../base/source/reader/SourceReaderBaseTest.java | 16 +-
.../base/source/reader/SourceReaderTestBase.java | 2 +-
.../reader/fetcher/SplitFetcherManagerTest.java | 22 ++
.../source/reader/fetcher/SplitFetcherTest.java | 63 ++---
.../base/source/reader/mocks/MockSourceReader.java | 5 +
.../source/reader/mocks/MockSplitEnumerator.java | 2 +-
.../base/source/reader/mocks/MockSplitReader.java | 3 +
.../source/reader/mocks/TestingSplitReader.java | 20 ++
flink-core/pom.xml | 5 +-
.../apache/flink/api/connector/source/Source.java | 16 +-
.../flink/api/connector/source/SourceReader.java | 19 +-
.../api/connector/source/SourceReaderContext.java | 5 +
.../api/connector/source/SplitEnumerator.java | 17 +-
.../connector}/source/event/NoMoreSplitsEvent.java | 2 +-
.../connector}/source/event/RequestSplitEvent.java | 15 +-
.../connector/source/lib/NumberSequenceSource.java | 253 +++++++++++++++++++++
.../source/lib/util/IteratorSourceEnumerator.java | 90 ++++++++
.../source/lib/util/IteratorSourceReader.java | 148 ++++++++++++
.../source/lib/util/IteratorSourceSplit.java | 34 ++-
.../source/lib/NumberSequenceSourceTest.java | 91 ++++++++
.../api/connector/source/mocks/MockSource.java | 12 +-
.../connector/source/mocks/MockSourceReader.java | 98 ++++++--
.../source/mocks/MockSplitEnumerator.java | 21 +-
.../source}/mocks/TestingReaderContext.java | 11 +-
.../source}/mocks/TestingReaderOutput.java | 2 +-
.../runtime/checkpoint/CheckpointCoordinator.java | 7 +-
.../OperatorCoordinatorCheckpointContext.java | 19 +-
.../coordination/OperatorCoordinator.java | 42 ++--
.../coordination/OperatorCoordinatorHolder.java | 22 +-
.../RecreateOnResetOperatorCoordinator.java | 13 +-
.../source/coordinator/SourceCoordinator.java | 24 +-
.../coordinator/SourceCoordinatorProvider.java | 2 +-
.../checkpoint/CheckpointCoordinatorTest.java | 53 +++++
.../CheckpointCoordinatorTestingUtils.java | 19 +-
.../CoordinatorEventsExactlyOnceITCase.java | 2 +-
.../coordination/MockOperatorCoordinator.java | 2 +-
.../OperatorCoordinatorHolderTest.java | 2 +-
.../RecreateOnResetOperatorCoordinatorTest.java | 3 +-
.../coordination/TestingOperatorCoordinator.java | 2 +-
.../coordinator/SourceCoordinatorProviderTest.java | 2 +-
.../source/coordinator/SourceCoordinatorTest.java | 12 +-
.../coordinator/SourceCoordinatorTestBase.java | 5 +-
.../streaming/api/datastream/DataStreamUtils.java | 136 ++++++++++-
.../streaming/api/operators/SourceOperator.java | 55 ++++-
.../api/operators/SourceOperatorFactory.java | 10 +-
.../collect/CollectSinkOperatorCoordinator.java | 2 +-
.../runtime/tasks/SourceOperatorStreamTask.java | 8 +-
.../api/operators/SourceOperatorTest.java | 156 ++++++++-----
.../source/SourceOperatorEventTimeTest.java | 14 +-
.../operators/source/TestingSourceOperator.java | 13 ++
.../tasks/SourceOperatorStreamTaskTest.java | 57 +++--
.../flink/core/testutils/CommonTestUtils.java | 29 +++
.../source/lib/IteratorSourcesITCase.java | 90 ++++++++
57 files changed, 1590 insertions(+), 274 deletions(-)
rename
{flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base
=>
flink-core/src/main/java/org/apache/flink/api/connector}/source/event/NoMoreSplitsEvent.java
(96%)
rename
{flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base
=>
flink-core/src/main/java/org/apache/flink/api/connector}/source/event/RequestSplitEvent.java
(83%)
create mode 100644
flink-core/src/main/java/org/apache/flink/api/connector/source/lib/NumberSequenceSource.java
create mode 100644
flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceEnumerator.java
create mode 100644
flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceReader.java
copy
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/TestingSourceSplit.java
=>
flink-core/src/main/java/org/apache/flink/api/connector/source/lib/util/IteratorSourceSplit.java
(55%)
create mode 100644
flink-core/src/test/java/org/apache/flink/api/connector/source/lib/NumberSequenceSourceTest.java
rename
{flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader
=>
flink-core/src/test/java/org/apache/flink/api/connector/source}/mocks/TestingReaderContext.java
(93%)
rename
{flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader
=>
flink-core/src/test/java/org/apache/flink/api/connector/source}/mocks/TestingReaderOutput.java
(96%)
create mode 100644
flink-tests/src/test/java/org/apache/flink/api/connector/source/lib/IteratorSourcesITCase.java
