Re: outdated version of com.esotericsoftware.kryo

2021-01-15 Thread Seth Wiesman
To expand on Till's answer, Flink uses Kryo whenever the type system does not recognize a given type. If your job works with Kryo fallback disabled, there should be no usage of Kryo serialization in your snapshots[1]. In theory, it should be possible to upgrade Kryo using Flink's schema migration

Re: outdated version of com.esotericsoftware.kryo

2021-01-15 Thread Till Rohrmann
Yes, I think that Flink uses whatever key-/value-serializer is defined for serializing the keys/values for a savepoint. I haven't looked in detail into the problem but if Kryo offers some migration options, then this could make the problem a bit easier. Cheers, Till On Fri, Jan 15, 2021 at

Re: outdated version of com.esotericsoftware.kryo

2021-01-15 Thread Niels Basjes
Till, So, we save the savepoints using the Kryo native format? But I see that Kryo 2.24 (as mentioned here) is from may 2014 ... that's ... really old ! https://github.com/EsotericSoftware/kryo/releases/tag/kryo-2.24.0 I see that Kryo has some suggestions on the migration options:

Re: outdated version of com.esotericsoftware.kryo

2021-01-15 Thread Till Rohrmann
Hi Fred, one reason why this hasn't been updated is that Flink needs to ensure backwards compatibility. Hence, if the newer Kryo library serializes objects differently, then Flink would need to migrate old savepoints having been stored via the old Kryo serializer to the new format. This is

outdated version of com.esotericsoftware.kryo

2021-01-15 Thread Teunissen, F.G.J. (Fred)
Hi, While I was resolving a dependency version conflict in our project that uses Flink v1.11.2 I noticed that Flink is using the library ‘com.esotericsoftware.kryo:kryo:2.24.0’. Is there a reason why this library isn’t updated