This is an automated email from the ASF dual-hosted git repository.
dependabot[bot] pushed a change to branch
dependabot/maven/org.apache.logging.log4j-log4j-core-2.25.3
in repository https://gitbox.apache.org/repos/asf/flink-connector-kafka.git
discard 8b784fdd Bump org.apache.logging.log4j:log4j-core from 2.25.0 to 2.25.3
add 20ca57d7 [FLINK-38862] Configurable partition discovery interval in
upsert mode
add 5b69b2b2 [FLINK-38876] Support per-cluster offset in Dynamic Kafka
Source (#209)
add dc4efea9 [FLINK-38917][docs] Add documentation about Python
DynamicKafkaSource (#211)
add f2e6b34e [FLINK-38947] Handle errors in onCompletion callback only
once (#214)
add d0b5cca3 [FLINK-38920] create Table source and factory for
DynamicKafkaSource in Flink Table API (#212)
add 5cd6e46f [FLINK-38869] InitializationContext for KafkaSubscriber (#208)
add 160ee7f0 [FLINK-38937] Introduce TestKafkaContainer wrapper for Apache
and Confluent Kafka images
add 2902e3b1 [FLINK-38937] Migrate all Kafka tests to TestKafkaContainer
with KRaft mode
add e75c00e6 [FLINK-38937] Rename KAFKA constant to CP_KAFKA for clarity
add a5505346 [hotfix] Apply spotless formatting
add 7e64c26e [FLINK-25538] Migrate tests from JUnit 4 to JUnit 5
add 1db38de2 [FLINK-25538] Replaces individual
@ExtendWith(TestLoggerExtension.class) annotations with global SPI registration
add 6f04b287 [FLINK-25538] Remove optional public modifiers from test
classes and methods following JUnit 5 best practices.
add 22d09e67 [FLINK-25538] Added ArchUnit rule (BanJunit4Rules) to prevent
future JUnit 4 usage
add cdbd6351 [FLINK-39013] add E2E test for DynamicKafkaTableSource (#218)
add 7ef6c448 [hotfix] Wait for leader election to prevent OUT_OF_ORDER
issues
add da1a90ff [FLINK-39012]Add global kafka enumerator support (#223)
add a9d2f77a [hotfix] Update testcontainers dependency to fix Ubuntu -
Docker compatibility issue
add 12d4d4b8 [FLINK-39037] support query kafka cluster id as metadata in
records of DynamicKafkaTableSource (#222)
add 80dadcb3 [hotfix] Wait for log partitions assignments
add e0969a4b [FLINK-39120] Update Kafka version to 4.2.0
add d6785701 [FLINK-39121][tests] Retry kafka containers startup
add c604ce54 [FLINK-29245] Remove unused RetryRule
add ffb1fc6c [hotfix] Remove unused test infrastructure from KafkaTestBase
add ee987855 [FLINK-38999][lineage] License lineage files
add a42f6a42 [hotfix] Revert [FLINK-38869] InitializationContext for
KafkaSubscriber
add 8eab7c94 [hotfix] Update weekly CI versions
add 741673ef [FLINK-39138] Update `flink-connector-parent` version to 2.0.0
add a7e805a3 [FLINK-39238] Support watermark alignment in dynamic Kafka
reader (#240)
add bd2fd2b2 [FLINK-39338] Fix topic name collisions in
KafkaWriterTestBase for parameterized tests
add c3f80874 [FLINK-39234][tests] Move
createNewTopicAndWaitForPartitionAssignment to a shared utils class
add 7ec6463d [FLINK-39234][tests] Wait for topic intialization in
KafkaWriterTestBase based tests
add 61f81bd0 [FLINK-39514] Release dynamic Kafka split outputs on metadata
shrink (#245)
add 67feb64e Bump org.apache.logging.log4j:log4j-core from 2.25.0 to 2.25.4
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8b784fdd)
\
N -- N -- N
refs/heads/dependabot/maven/org.apache.logging.log4j-log4j-core-2.25.3
(67feb64e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/weekly.yml | 11 +-
.../docs/connectors/datastream/dynamic-kafka.md | 144 ++++-
.../docs/connectors/table/dynamic-kafka.md | 177 ++++++
.../docs/connectors/datastream/dynamic-kafka.md | 137 +++++
.../content/docs/connectors/table/dynamic-kafka.md | 182 ++++++
.../flink-end-to-end-tests-common-kafka/pom.xml | 11 +-
.../tests/util/kafka/KafkaContainerClient.java | 8 +-
.../flink/tests/util/kafka/KafkaSinkE2ECase.java | 18 +-
.../flink/tests/util/kafka/KafkaSourceE2ECase.java | 20 +-
.../util/kafka/SQLClientSchemaRegistryITCase.java | 59 +-
.../flink/tests/util/kafka/SmokeKafkaITCase.java | 12 +-
.../86dfd459-67a9-4b26-9b5c-0b0bbf22681a | 48 --
...c90862 => f902c3ff-48e5-424e-923f-3705c14930a0} | 0
.../archunit-violations/stored.rules | 10 +-
flink-connector-kafka/pom.xml | 11 -
.../kafka/dynamic/metadata/ClusterMetadata.java | 66 +-
.../SingleClusterTopicMetadataService.java | 37 +-
.../kafka/dynamic/source/DynamicKafkaSource.java | 8 +-
.../dynamic/source/DynamicKafkaSourceBuilder.java | 20 +-
.../dynamic/source/DynamicKafkaSourceOptions.java | 53 +-
.../DynamicKafkaSourceEnumStateSerializer.java | 144 ++++-
.../enumerator/DynamicKafkaSourceEnumerator.java | 174 +++++-
.../enumerator/GlobalSplitOwnerAssigner.java | 63 ++
.../source/reader/DynamicKafkaSourceReader.java | 116 +++-
.../KafkaClusterAwareDeserializer.java} | 16 +-
.../kafka/lineage/DefaultKafkaDatasetFacet.java | 19 +
.../lineage/DefaultKafkaDatasetIdentifier.java | 19 +
.../kafka/lineage/DefaultTypeDatasetFacet.java | 19 +
.../connector/kafka/lineage/KafkaDatasetFacet.java | 19 +
.../kafka/lineage/KafkaDatasetFacetProvider.java | 19 +
.../kafka/lineage/KafkaDatasetIdentifier.java | 19 +
.../lineage/KafkaDatasetIdentifierProvider.java | 19 +
.../connector/kafka/lineage/TypeDatasetFacet.java | 19 +
.../kafka/lineage/TypeDatasetFacetProvider.java | 19 +
.../kafka/sink/ExactlyOnceKafkaWriter.java | 6 +
.../kafka/sink/KafkaRecordSerializationSchema.java | 4 +-
.../flink/connector/kafka/sink/KafkaWriter.java | 44 +-
.../kafka/sink/TransactionNamingStrategy.java | 1 +
.../kafka/sink/TwoPhaseCommittingStatefulSink.java | 19 +
.../kafka/sink/internal/BackchannelImpl.java | 2 +
.../kafka/sink/internal/ProducerPoolImpl.java | 4 +
.../TransactionAbortStrategyContextImpl.java | 1 +
.../source/enumerator/KafkaSourceEnumState.java | 1 +
.../enumerator/KafkaSourceEnumStateSerializer.java | 2 +
.../source/enumerator/KafkaSourceEnumerator.java | 64 +-
.../kafka/table/DynamicKafkaConnectorOptions.java | 63 ++
.../table/DynamicKafkaDeserializationSchema.java | 74 ++-
.../kafka/table/DynamicKafkaTableFactory.java | 523 ++++++++++++++++
...micSource.java => DynamicKafkaTableSource.java} | 173 +++---
.../kafka/table/KafkaConnectorOptionsUtil.java | 3 +-
.../connectors/kafka/table/TableDataTypeUtils.java | 19 +
.../table/UpsertKafkaDynamicTableFactory.java | 9 +
.../org.apache.flink.table.factories.Factory | 1 +
.../architecture/TestCodeArchitectureTest.java | 3 +
.../flink/architecture/rules/BanJunit4Rules.java | 46 ++
.../DynamicKafkaSourceClusterMetadataITCase.java | 172 ++++++
.../dynamic/source/DynamicKafkaSourceITTest.java | 399 +++++++++++-
.../DynamicKafkaSourceEnumStateSerializerTest.java | 100 ++-
.../DynamicKafkaSourceEnumeratorTest.java | 677 ++++++++++++++++++++-
.../enumerator/GlobalSplitOwnerAssignerTest.java | 203 ++++++
.../SingleClusterTopicMetadataServiceTest.java | 84 +++
.../reader/DynamicKafkaSourceReaderTest.java | 266 ++++++++
.../DynamicKafkaSourceEnumStateTestUtils.java | 53 ++
.../connector/kafka/lineage/LineageUtilTest.java | 19 +
.../kafka/sink/ExactlyOnceKafkaWriterITCase.java | 3 -
.../sink/FlinkKafkaInternalProducerITCase.java | 9 +-
.../kafka/sink/KafkaCommittableSerializerTest.java | 8 +-
.../KafkaRecordSerializationSchemaBuilderTest.java | 45 +-
.../connector/kafka/sink/KafkaSinkBuilderTest.java | 3 +-
.../connector/kafka/sink/KafkaSinkITCase.java | 37 +-
.../flink/connector/kafka/sink/KafkaSinkTest.java | 19 +
.../kafka/sink/KafkaTransactionLogITCase.java | 23 +-
.../sink/KafkaWriterFaultToleranceITCase.java | 4 +-
.../connector/kafka/sink/KafkaWriterITCase.java | 3 -
.../kafka/sink/KafkaWriterStateSerializerTest.java | 7 +-
.../connector/kafka/sink/KafkaWriterTestBase.java | 24 +-
.../kafka/sink/TransactionIdFactoryTest.java | 7 +-
.../kafka/sink/internal/KafkaCommitterTest.java | 3 -
.../sink/internal/ProducerPoolImplITCase.java | 6 +-
.../sink/internal/TransactionIdFactoryTest.java | 12 +-
.../testutils/KafkaSinkExternalContextFactory.java | 9 +-
.../kafka/source/KafkaSourceBuilderTest.java | 3 -
.../connector/kafka/source/KafkaSourceITCase.java | 17 +-
.../kafka/source/KafkaSourceMigrationITCase.java | 3 +-
.../KafkaSourceEnumStateSerializerTest.java | 8 +-
.../initializer/OffsetsInitializerTest.java | 35 +-
.../enumerator/subscriber/KafkaSubscriberTest.java | 22 +-
.../metrics/KafkaSourceReaderMetricsTest.java | 12 +-
.../KafkaRecordDeserializationSchemaTest.java | 20 +-
.../kafka/testutils/DockerImageVersions.java | 5 +-
.../DynamicKafkaSourceExternalContext.java | 5 +-
.../DynamicKafkaSourceExternalContextFactory.java | 13 +-
.../FlinkKafkaIntegrationCompatibilityTest.java | 250 ++++++++
.../KafkaSourceExternalContextFactory.java | 8 +-
.../flink/connector/kafka/testutils/KafkaUtil.java | 56 +-
.../kafka/testutils/TestKafkaContainer.java | 270 ++++++++
.../TestKafkaContainerValidationTest.java | 47 ++
.../kafka/testutils/TwoKafkaContainers.java | 15 +-
.../kafka/FlinkFixedPartitionerTest.java | 10 +-
.../JSONKeyValueDeserializationSchemaTest.java | 12 +-
.../streaming/connectors/kafka/KafkaTestBase.java | 72 +--
.../connectors/kafka/KafkaTestBaseWithFlink.java | 41 --
.../connectors/kafka/KafkaTestEnvironment.java | 2 +-
.../connectors/kafka/KafkaTestEnvironmentImpl.java | 135 +---
.../metrics/KafkaMetricMutableWrapperTest.java | 8 +-
.../DynamicKafkaDeserializationSchemaTest.java | 139 +++++
.../DynamicKafkaRecordSerializationSchemaTest.java | 26 +-
.../kafka/table/DynamicKafkaTableFactoryTest.java | 104 ++++
.../kafka/table/DynamicKafkaTableITCase.java | 280 +++++++++
.../kafka/table/KafkaConnectorOptionsUtilTest.java | 14 +-
.../kafka/table/KafkaDynamicTableFactoryTest.java | 112 ++--
.../connectors/kafka/table/KafkaTableTestBase.java | 5 +-
.../kafka/table/ReducingUpsertWriterTest.java | 57 +-
.../kafka/table/TableDataTypeUtilsTest.java | 32 +-
.../table/UpsertKafkaDynamicTableFactoryTest.java | 88 ++-
.../org.junit.jupiter.api.extension.Extension | 1 +
flink-python/pom.xml | 6 -
.../src/main/resources/META-INF/NOTICE | 2 +-
pom.xml | 17 +-
tools/maven/checkstyle.xml | 209 ++-----
120 files changed, 6082 insertions(+), 1052 deletions(-)
create mode 100644 docs/content.zh/docs/connectors/table/dynamic-kafka.md
create mode 100644 docs/content/docs/connectors/table/dynamic-kafka.md
copy
flink-connector-kafka/archunit-violations/{27a0a5e4-29c2-4069-b381-952746c90862
=> f902c3ff-48e5-424e-923f-3705c14930a0} (100%)
create mode 100644
flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/dynamic/source/enumerator/GlobalSplitOwnerAssigner.java
copy
flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/dynamic/source/{GetMetadataUpdateEvent.java
=> reader/KafkaClusterAwareDeserializer.java} (70%)
create mode 100644
flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/DynamicKafkaConnectorOptions.java
create mode 100644
flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/DynamicKafkaTableFactory.java
copy
flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/{KafkaDynamicSource.java
=> DynamicKafkaTableSource.java} (82%)
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/architecture/rules/BanJunit4Rules.java
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/connector/kafka/dynamic/source/DynamicKafkaSourceClusterMetadataITCase.java
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/connector/kafka/dynamic/source/enumerator/GlobalSplitOwnerAssignerTest.java
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/connector/kafka/dynamic/source/testutils/DynamicKafkaSourceEnumStateTestUtils.java
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/connector/kafka/testutils/FlinkKafkaIntegrationCompatibilityTest.java
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/connector/kafka/testutils/TestKafkaContainer.java
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/connector/kafka/testutils/TestKafkaContainerValidationTest.java
delete mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/KafkaTestBaseWithFlink.java
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/table/DynamicKafkaDeserializationSchemaTest.java
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/table/DynamicKafkaTableFactoryTest.java
create mode 100644
flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/table/DynamicKafkaTableITCase.java
create mode 100644
flink-connector-kafka/src/test/resources/META-INF/services/org.junit.jupiter.api.extension.Extension
