Nikolay Izhikov created IGNITE-29309:
----------------------------------------
Summary: CdcEvent with BinaryObject written by 2.17 can't be read
by 2.18: BinaryObjectExImpl has no serialVersionUID
Key: IGNITE-29309
URL: https://issues.apache.org/jira/browse/IGNITE-29309
Project: Ignite
Issue Type: Bug
Reporter: Nikolay Izhikov
Attachments: CdcEventSerializationCompatibilityTest.patch
{{CdcEvent}} whose key or value is a {{BinaryObject}} and that was serialized
by Ignite 2.17 can't be deserialized by Ignite 2.18 (and master), and vice
versa.
This breaks Kafka based CDC replication (ignite-extensions, cdc-ext) when
{{ignite-cdc.sh}} and {{kafka-to-ignite.sh}} run on different Ignite versions:
{{IgniteToKafkaCdcStreamer}} writes events with plain Java serialization
({{IgniteUtils.toBytes(evt)}}) and {{KafkaToIgniteCdcStreamerApplier}} reads
them with {{ObjectInputStream#readObject()}}. Events already stored in a topic
by 2.17 can't be applied after upgrade of the applier to 2.18.
{noformat}
java.io.InvalidClassException:
org.apache.ignite.internal.binary.BinaryObjectExImpl; local class incompatible:
stream classdesc serialVersionUID = 327652719061348914, local class
serialVersionUID = 7243833274571614066
{noformat}
h3. Root cause
{{BinaryObjectExImpl}} (abstract superclass of {{BinaryObjectImpl}} and
{{BinaryObjectOffheapImpl}}) is {{Serializable}} but has no explicit
{{serialVersionUID}}, so JVM computes it from the class shape (name, modifiers,
interfaces, fields, non-private methods and constructors).
Java serialization writes and checks a class descriptor for every
{{Serializable}} class in the hierarchy, so explicit {{serialVersionUID = 0}}
in {{BinaryObjectImpl}} doesn't help, even though the class is
{{Externalizable}}.
The class shape was changed in 2.18, at least by IGNITE-25186 "Make
BinaryObjectExImpl package private":
||Version||Computed serialVersionUID of BinaryObjectExImpl||
|2.17.0|327652719061348914|
|2.18.0, master|7243833274571614066|
Other classes of the event graph ({{CdcEventImpl}}, {{BinaryObjectImpl}},
{{GridCacheVersion}}, {{GridCacheVersionEx}}) have explicit
{{serialVersionUID}} and are compatible.
h3. Reproducer
{{CdcEventSerializationCompatibilityTest}} (see attached patch, based on the
compatibility framework). Events are serialized to files by a node of one
version and then read and compared by a node of another version, the same way
the Kafka streamer and applier do.
||Direction||Result||
|2.17.0 -> master|FAIL for events with BinaryObject key/value|
|master -> 2.17.0|FAIL for events with BinaryObject key/value|
|2.18.0 -> master|OK|
|master -> 2.18.0|OK|
Events with primitive key and value (with and without conflict version) are
read successfully in all cases.
h3. Notes
* Setting explicit {{serialVersionUID}} equal to the 2.17 value restores
compatibility with 2.17 and older, but breaks compatibility with already
released 2.18.0. To support both, reading side must tolerate both values (e.g.
custom {{ObjectInputStream}} in the applier).
* Why build didn't catch it: {{ClassesGenerator}} (generates
{{classnames.properties}}) fails the build if {{Serializable}} class has no
{{serialVersionUID}}, but it skips abstract classes, anonymous classes, classes
with names ending with {{Future}}/{{FutureAdapter}}, and it runs for
ignite-core and ignite-zookeeper modules only. Abstract classes must be
checked, also.
* Another abstract class with the same problem that is persisted with Java
serialization: {{AbstractEvictionPolicyFactory}} (part of
{{CacheConfiguration}}, stored cache data). Its computed {{serialVersionUID}}
is the same in 2.17.0, 2.18.0 and master for now, explicit value must be set
before it changes.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)