[
https://issues.apache.org/jira/browse/FLINK-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated FLINK-15474:
--------------------------
Fix Version/s: (was: 1.10.0)
1.11.0
> In TypeSerializerUpgradeTestBase, create serializer snapshots "on demand"
> -------------------------------------------------------------------------
>
> Key: FLINK-15474
> URL: https://issues.apache.org/jira/browse/FLINK-15474
> Project: Flink
> Issue Type: Improvement
> Components: API / Type Serialization System, Tests
> Affects Versions: 1.9.0, 1.10.0
> Reporter: Aljoscha Krettek
> Assignee: Aljoscha Krettek
> Priority: Major
> Fix For: 1.11.0
>
>
> Currently, we store binary snapshots in the repository for all the different
> serializer upgrade test configurations (see linked POC for which snapshots
> are there for just the POJO serializer). This is hard to maintain, because
> someone has to go back and generate snapshtos from previous Flink versions
> and add them to the repo when updating the tests for a new Flink version.
> It's also problematic from a repository perspective because we keep piling up
> binary snapshots.
> Instead, we can create a snapshot "on demand" from a previous Flink version
> by using a classloader that has the previous Flink jar.
> I created a POC which demonstrated the approach:
> [https://github.com/aljoscha/flink/tree/jit-serializer-test-base]. The
> advantage is that we don't need binary snapshots in the repo anymore,
> updating the tests to a newer Flink version should be as easy as adding a new
> migration version and Flink download url. The downside is that the test now
> downloads Flink releases (in the PoC this is done using the caching infra
> introduced for e2e tests), which is costly and also re-generates the
> snapshots for every test, which is also costly. The test time (minus
> downloading) goes up from about 300 ms to roughly 6 seconds. That's not
> something I would call a unit test. We could call these "integration tests"
> (or even e2e tests) and only run them nightly. Side note, we don't have test
> coverage for serializer upgrades from 1.8 and 1.9 currently, so even doing it
> nightly would be an improvement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
