[
https://issues.apache.org/jira/browse/FLINK-32455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739785#comment-17739785
]
Tzu-Li (Gordon) Tai edited comment on FLINK-32455 at 7/4/23 4:13 AM:
---------------------------------------------------------------------
[~gaoyunhaii] sorry for the slow reply - just finished other stuff and circling
back to this.
> refactor the existing migration tests to the new framework.
The main challenge here is that external connector repos generally need to
build against the latest 2 major version, i.e. for now that would be 1.17.x and
the upcoming 1.18.x version.
Therefore, we can't simply refactor the migration tests in the Kafka connector
to use the new {{MigrationTest}} abstraction, otherwise the code won't be able
to simultaneously build against older versions.
Specifically, the situation is that:
* The Kafka connector code has a test ({{{}KafkaSerializerUpgradeTest{}}})
that depends on {{{}TypeSerializerUpgradeTestBase{}}}, which with FLINK-27518
now implements the new {{{}MigrationTest{}}}.
* After implementing {{{}MigrationTest{}}}, {{TypeSerializerUpgradeTestBase}}
has a breaking change across versions 1.17.x and 1.18.x.
* Therefore, it is not possible to update the test code to simultaneously
build against both versions.
To move forward, I think we need to:
# We might need to revert the changes to middle-layer test util abstractions
like {{TypeSerializerUpgradeTestBase}} that externalized connector code might
be depending on.
# At the same time, formally introduce a replacement for them in the
{{fink-migration-test-utils}} package, and mark the original
{{TypeSerializerUpgradeTestBase}} as deprecated.
# Gradually migrate the test code in the Kafka connector to use
{{flink-migration-test-utils}} after a few connector releases.
What do you think [~gaoyunhaii]?
was (Author: tzulitai):
[~gaoyunhaii] sorry for the slow reply - just finished other stuff and circling
back to this.
> refactor the existing migration tests to the new framework.
The main challenge here is that external connector repos generally need to
build against the latest 2 major version, i.e. for now that would be 1.17.x and
the upcoming 1.18.x version.
Therefore, we can't simply refactor the migration tests in the Kafka connector
to use the new {{MigrationTest}} abstraction, otherwise the code won't be able
to simultaneously build against older versions.
Specifically, the situation is that:
* The Kafka connector code has a test ({{{}KafkaSerializerUpgradeTest{}}})
that depends on {{{}TypeSerializerUpgradeTestBase{}}}, which with FLINK-27518
now implements the new {{{}MigrationTest{}}}.
* After implementing {{{}MigrationTest{}}}, {{TypeSerializerUpgradeTestBase}}
has a breaking change across versions 1.17.x and 1.18.x.
* Therefore, it is not possible to update the test code to simultaneously
build against both versions.
To move forward, I think we need to:
# We might need to revert the changes to middle-layer test util abstractions
like {{TypeSerializerUpgradeTestBase}} that externalized connector code might
be depending on.
# At the same time, formally introduce a replacement for them in the
{{fink-migration-test-utils}} package, and mark the original
{{TypeSerializerUpgradeTestBase}} as deprecated.
# Gradually migrate the test code in the Kafka connector to use
{{flink-migration-test-utils}} after a few connector releases.
What do you think?
> Breaking change in TypeSerializerUpgradeTestBase prevents
> flink-connector-kafka from building against 1.18-SNAPSHOT
> -------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-32455
> URL: https://issues.apache.org/jira/browse/FLINK-32455
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Kafka, Test Infrastructure
> Affects Versions: 1.18.0
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Blocker
> Fix For: 1.18.0
>
>
> FLINK-27518 introduced a breaking signature change to the abstract class
> {{TypeSerializerUpgradeTestBase}}, specifically the abstract
> {{createTestSpecifications}} method signature was changed. This breaks
> downstream test code in externalized connector repos, e.g.
> flink-connector-kafka's {{KafkaSerializerUpgradeTest}}
> Moreover, {{fink-migration-test-utils}} needs to be transitively pulled in by
> downstream test code that depends on flink-core test-jar.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
