This is an automated email from the ASF dual-hosted git repository.
martijnvisser pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/flink-connector-pulsar.git
from 40c2705 [hotfix] Enable checkstyle only for source- and testsource
directories, not for generated sources
new d21147e [FLINK-28853][pulsar] Implement pausing Pulsar splits in case
of watermark drift
new 9cce627 [FLINK-28853][connector-base] Add
allow-unaligned-source-splits config option
new b0a1678 [FLINK-28853] Address PR comments / Add Kafka and Pulsar
split pausing tests
new 08e7b19 [hotfix][docs]: Fix minor grammar and spelling mistakes
new dfb5fd8 [FLINK-29580][Connector/Pulsar] Remove
pulsar.consumer.autoUpdatePartitionsIntervalSeconds option. (#21070)
new 618daa2 [FLINK-29433][Connector/Pulsar] Support Auth through the
builder pattern in Pulsar connector. (#21071)
new 62da02a [FLINK-28085][Connector/Pulsar] Close the pending Pulsar
transactions before closing pipeline (#21075)
new c6a20a6 [FLINK-28820][Connector/Pulsar] Improve the writing
performance for PulsarSink (#21074)
new 1c88632 [FLINK-29624][Common][Connector][Filesystem] Upgrade
org.apache.commons:commons-lang3 from 3.3.2 to 3.12.0
new 4f204e8 [FLINK-29709][Connector/Pulsar] Bump the Pulsar to latest
2.10.2 (#21204)
new d6e2e06 [FLINK-28083][Connector/Pulsar] PulsarSource work with
object-reusing DeserializationSchema. (#21205)
new f6ead4c [FLINK-29860][Connectors/Pulsar] InlineElement fields should
be transient (#21262)
new a6ba997 [FLINK-29830][Connector/Pulsar] Create the topic with schema
before consuming messages in PulsarSinkITCase. (#21252)
new d8abb2a [FLINK-26027][Connector/Pulsar] Expose Pulsar producer
metrics and add FLIP-33 sink metrics. (#21249)
new d330e40 [FLINK-30254][Connector/Pulsar] Sync Pulsar updates and set
Flink version to 1.17-SNAPSHOT
new 3199922 [FLINK-30254][Connector/Pulsar] Resolve dependency
convergence errors
The 16 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/push_pr.yml | 6 +-
.github/workflows/weekly.yml | 2 +-
flink-connector-pulsar/pom.xml | 13 +--
.../pulsar/common/metrics/MetricNames.java | 63 ++++++++++++
.../common/metrics/ProducerMetricsInterceptor.java | 67 ++++++++++++
.../pulsar/common/schema/PulsarSchema.java | 15 ++-
.../pulsar/common/schema/PulsarSchemaUtils.java | 9 +-
.../common/utils/PulsarTransactionUtils.java | 2 +
.../connector/pulsar/sink/PulsarSinkBuilder.java | 33 ++++++
.../connector/pulsar/sink/PulsarSinkOptions.java | 16 +++
.../pulsar/sink/config/PulsarSinkConfigUtils.java | 4 -
.../pulsar/sink/config/SinkConfiguration.java | 29 +++---
.../connector/pulsar/sink/writer/PulsarWriter.java | 86 +++++++---------
.../pulsar/sink/writer/router/MessageKeyHash.java | 2 +-
.../sink/writer/router/TopicRoutingMode.java | 2 +-
.../sink/writer/topic/TopicProducerRegister.java | 86 +++++++++++++++-
.../pulsar/source/PulsarSourceBuilder.java | 33 ++++++
.../pulsar/source/PulsarSourceOptions.java | 21 ++--
.../pulsar/source/config/CursorVerification.java | 2 +-
.../source/config/PulsarSourceConfigUtils.java | 5 -
.../pulsar/source/config/SourceConfiguration.java | 16 ++-
.../source/enumerator/PulsarSourceEnumerator.java | 8 ++
.../source/enumerator/assigner/SplitAssigner.java | 3 +
.../enumerator/assigner/SplitAssignerBase.java | 5 +
.../source/enumerator/cursor/StartCursor.java | 4 +-
.../source/enumerator/cursor/StopCursor.java | 6 +-
.../source/reader/PulsarSourceReaderFactory.java | 21 ++--
.../source/reader/emitter/PulsarRecordEmitter.java | 63 ++++++++++--
.../reader/fetcher/PulsarFetcherManagerBase.java | 41 +++++---
.../fetcher/PulsarOrderedFetcherManager.java | 22 ++--
.../fetcher/PulsarUnorderedFetcherManager.java | 18 ++--
.../source/reader/message/PulsarMessage.java | 74 -------------
.../reader/message/PulsarMessageCollector.java | 60 -----------
.../reader/source/PulsarOrderedSourceReader.java | 19 ++--
.../reader/source/PulsarSourceReaderBase.java | 31 ++++--
.../reader/source/PulsarUnorderedSourceReader.java | 41 +++++++-
.../split/PulsarOrderedPartitionSplitReader.java | 13 +--
.../split/PulsarPartitionSplitReaderBase.java | 114 ++++++++++++++++-----
.../split/PulsarUnorderedPartitionSplitReader.java | 27 ++---
.../connector/pulsar/sink/PulsarSinkITCase.java | 6 +-
.../writer/topic/TopicProducerRegisterTest.java | 7 +-
.../source/enumerator/cursor/StopCursorTest.java | 15 ++-
.../source/PulsarOrderedSourceReaderTest.java | 32 ++++++
.../PulsarOrderedPartitionSplitReaderTest.java | 8 +-
.../split/PulsarPartitionSplitReaderTestBase.java | 68 ++++++------
.../pulsar/testutils/function/ControlSource.java | 41 ++++++--
.../testutils/runtime/PulsarRuntimeOperator.java | 4 +
.../testutils/sink/PulsarSinkTestSuiteBase.java | 37 -------
.../src/main/resources/META-INF/NOTICE | 8 +-
pom.xml | 13 ++-
50 files changed, 862 insertions(+), 459 deletions(-)
create mode 100644
flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/common/metrics/MetricNames.java
create mode 100644
flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/common/metrics/ProducerMetricsInterceptor.java
delete mode 100644
flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/source/reader/message/PulsarMessage.java
delete mode 100644
flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/source/reader/message/PulsarMessageCollector.java
delete mode 100644
flink-connector-pulsar/src/test/java/org/apache/flink/connector/pulsar/testutils/sink/PulsarSinkTestSuiteBase.java
