This is an automated email from the ASF dual-hosted git repository.
sewen pushed a change to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git.
from 10ad46b [FLINK-19433] [docs][table] Correct example of FROM_UNIXTIME
function in document
new 1379455 [FLINK-17761][connector/common] Add a constructor taking
capacity as a parameter for FutureCompletingBlockingQueue
new 7c55a64a [refactor][connectors] Backport of the connector-base
exception handling from the Kafka Connector Pull Request
new bc96de6 [hotfix][core] Add to Source Enumerator convenience methods
to assign single split
new 7aa51ea [FLINK-19205][core] Add access to configuration and hostname
in the SourceReaderContext
new 3b57a02 [hotfix][connectors] Add RequestSplitEvent to
'flink-connector-base'
new ae9842b [hotfix][testing] Add a set of parameterizable testing mocks
for the Split Reader API
new 3c31445 [FLINK-19162][connectors] Add 'recycle()' to the
RecordsWithSplitIds to support reuse of heavy objects.
new b8733d7 [FLINK-18680][connectors] Make connector base
RecordsWithSplitIds more lightweight.
new 997fc5f [FLINK-17393][connectors] Wakeup the SplitFetchers more
elegantly.
new 55745d0 [FLINK-19225][connectors] Various small improvements to
SourceReaderBase
new 6a2760c [hotfix][tests] Move constants in SplitFetcherTest relevant
to only one test into test method
new bd8f647 [refactor][core] Eagerly initialize the FetchTask to support
proper unit testing
new e72e485 [FLINK-18128][connectors] Ensure idle split fetchers lead to
availability notifications.
new 0e821ea [FLINK-19223][connectors] Simplify Availability Future Model
in Base Connector
new 406aa9f [FLINK-19245][connectors] Set default capacity for
FutureCompletingBlockingQueue.
new e5a23ca [hotfix][tests] Extend test coverage for
FutureCompletingBlockingQueue.
new 6072f5f [FLINK-17393][connectors] (follow-up) Wakeup the
SplitFetchers more elegantly.
new 0f63cb9 [FLINK-19225][connectors] Various small improvements to
SourceReaderBase (part 2)
new 5b43707 [hotfix][connectors] Improve JavaDocs for
SingleThreadFetcherManager
new f220c24 [FLINK-19250][connectors] Fix error propagation in connector
base (SplitFetcherManager).
new 257a0da [FLINK-19251][connectors] Avoid confusing queue handling in
"SplitReader.handleSplitsChanges()"
new c24185d [hotfix] Adjust japicmp exclusion to permit adding methods to
SourceReaderContext and SplitEnumeratorContext interfaces
The 22 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../base/source/event/RequestSplitEvent.java | 51 ++-
.../base/source/reader/RecordsBySplits.java | 179 ++++++---
.../base/source/reader/RecordsWithSplitIds.java | 29 +-
.../SingleThreadMultiplexSourceReaderBase.java | 25 +-
.../base/source/reader/SourceReaderBase.java | 159 +++++---
.../base/source/reader/SourceReaderOptions.java | 2 +-
.../base/source/reader/SplitsRecordIterator.java | 96 -----
.../base/source/reader/fetcher/AddSplitsTask.java | 20 +-
.../base/source/reader/fetcher/FetchTask.java | 32 +-
.../reader/fetcher/SingleThreadFetcherManager.java | 20 +-
.../base/source/reader/fetcher/SplitFetcher.java | 93 ++---
.../source/reader/fetcher/SplitFetcherManager.java | 13 +-
.../source/reader/fetcher/SplitFetcherTask.java | 6 +-
.../source/reader/splitreader/SplitReader.java | 10 +-
.../source/reader/splitreader/SplitsAddition.java | 5 +
.../FutureCompletingBlockingQueue.java | 443 +++++++++++++++++++--
.../reader/synchronization/FutureNotifier.java | 66 ---
.../base/source/reader/SourceReaderBaseTest.java | 138 +++++--
.../base/source/reader/SourceReaderTestBase.java | 13 +-
.../reader/fetcher/SplitFetcherManagerTest.java | 157 ++++++++
.../source/reader/fetcher/SplitFetcherTest.java | 240 +++++++++--
.../base/source/reader/mocks/MockBaseSource.java | 7 +-
.../base/source/reader/mocks/MockSourceReader.java | 6 +-
.../base/source/reader/mocks/MockSplitReader.java | 60 ++-
...dEmitter.java => PassThroughRecordEmitter.java} | 14 +-
.../source/reader/mocks/TestingReaderContext.java | 78 ++++
.../source/reader/mocks/TestingReaderOutput.java | 71 ++++
.../reader/mocks/TestingRecordsWithSplitIds.java | 30 +-
.../source/reader/mocks/TestingSourceSplit.java | 28 +-
.../source/reader/mocks/TestingSplitReader.java | 69 ++++
.../FutureCompletingBlockingQueueTest.java | 238 +++++++++++
.../reader/synchronization/FutureNotifierTest.java | 131 ------
flink-core/pom.xml | 4 +
.../api/connector/source/SourceReaderContext.java | 12 +
.../connector/source/SplitEnumeratorContext.java | 13 +
.../api/connector/source/SplitsAssignment.java | 8 +
.../connector/source/mocks/MockSourceSplit.java | 3 +
.../streaming/api/operators/SourceOperator.java | 28 +-
.../api/operators/SourceOperatorFactory.java | 13 +-
.../operators/source/TestingSourceOperator.java | 5 +-
40 files changed, 1850 insertions(+), 765 deletions(-)
copy
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/constraints/AbstractConstraint.java
=>
flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/source/event/RequestSplitEvent.java
(54%)
delete mode 100644
flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/source/reader/SplitsRecordIterator.java
delete mode 100644
flink-connectors/flink-connector-base/src/main/java/org/apache/flink/connector/base/source/reader/synchronization/FutureNotifier.java
create mode 100644
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/fetcher/SplitFetcherManagerTest.java
copy
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/{MockRecordEmitter.java
=> PassThroughRecordEmitter.java} (67%)
create mode 100644
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/TestingReaderContext.java
create mode 100644
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/TestingReaderOutput.java
copy
flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/overlays/CompositeContainerOverlay.java
=>
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/TestingRecordsWithSplitIds.java
(57%)
copy
flink-kubernetes/src/main/java/org/apache/flink/kubernetes/kubeclient/Endpoint.java
=>
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/TestingSourceSplit.java
(66%)
create mode 100644
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/mocks/TestingSplitReader.java
create mode 100644
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/synchronization/FutureCompletingBlockingQueueTest.java
delete mode 100644
flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/reader/synchronization/FutureNotifierTest.java
