[
https://issues.apache.org/jira/browse/FLINK-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017064#comment-17017064
]
Aljoscha Krettek commented on FLINK-15474:
------------------------------------------
I agree that test runtime is a major concern. I have a different POC now that
does things slightly differently: now we store the serializer snapshots in the
release branch of a given Flink version, say {{release-1.10}}. The test that
checks for backwards compatibility with 1.10 downloads the snapshots from that
branch from github. This has the advantage that we only have serializer
snapshots in release branches, and only ever for the version of that release.
There would be no snapshots in the master branch.
The PoC is here:
https://github.com/aljoscha/flink/tree/release-branch-upgrade-test, I added
some snapshots to https://github.com/aljoscha/flink/tree/release-1.10 to test
the approach.
The downside of this approach is also very long test time, the Pojo test takes
about 30 seconds for verification against on previous Flink version. We could
annotate theses tests with a special category and only run them nightly.
([~sewen] gave me this idea)
> In TypeSerializerUpgradeTestBase, create serializer snapshots "on demand"
> -------------------------------------------------------------------------
>
> Key: FLINK-15474
> URL: https://issues.apache.org/jira/browse/FLINK-15474
> Project: Flink
> Issue Type: Improvement
> Components: API / Type Serialization System, Tests
> Affects Versions: 1.9.0, 1.10.0
> Reporter: Aljoscha Krettek
> Assignee: Aljoscha Krettek
> Priority: Major
> Fix For: 1.10.0
>
>
> Currently, we store binary snapshots in the repository for all the different
> serializer upgrade test configurations (see linked POC for which snapshots
> are there for just the POJO serializer). This is hard to maintain, because
> someone has to go back and generate snapshtos from previous Flink versions
> and add them to the repo when updating the tests for a new Flink version.
> It's also problematic from a repository perspective because we keep piling up
> binary snapshots.
> Instead, we can create a snapshot "on demand" from a previous Flink version
> by using a classloader that has the previous Flink jar.
> I created a POC which demonstrated the approach:
> [https://github.com/aljoscha/flink/tree/jit-serializer-test-base]. The
> advantage is that we don't need binary snapshots in the repo anymore,
> updating the tests to a newer Flink version should be as easy as adding a new
> migration version and Flink download url. The downside is that the test now
> downloads Flink releases (in the PoC this is done using the caching infra
> introduced for e2e tests), which is costly and also re-generates the
> snapshots for every test, which is also costly. The test time (minus
> downloading) goes up from about 300 ms to roughly 6 seconds. That's not
> something I would call a unit test. We could call these "integration tests"
> (or even e2e tests) and only run them nightly. Side note, we don't have test
> coverage for serializer upgrades from 1.8 and 1.9 currently, so even doing it
> nightly would be an improvement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
