Nikolay Izhikov created IGNITE-29185:
----------------------------------------
Summary: cpMapSnapshot.bin written by Ignite 2.17/2.18 can't be
read by current version (generated IDTO serializer changed UUID format)
Key: IGNITE-29185
URL: https://issues.apache.org/jira/browse/IGNITE-29185
Project: Ignite
Issue Type: Bug
Affects Versions: 2.19
Reporter: Nikolay Izhikov
Attachments: IGNITE-29185-reproducer.patch
h3. Problem
{{cpMapSnapshot.bin}} (earliest checkpoint map snapshot, see
{{CheckpointMarkersStorage#EARLIEST_CP_SNAPSHOT_FILE}}) written by Ignite 2.17
/ 2.18 can't be deserialized by current master (future 2.19).
The file is an {{EarliestCheckpointMapSnapshot}} marshalled with
{{JdkMarshaller}}. In 2.17 and 2.18 the payload is written by hand:
{code:java}
U.writeMap(out, data); // keys (UUID) and values via
out.writeObject(...)
U.writeCollection(out, checkpointIds); // UUID elements via out.writeObject(...)
{code}
Since IGNITE-27709 the class is serialized by generated code, and since
IGNITE-27775 the generator emits typed code for collections. The generated
{{EarliestCheckpointMapSnapshotSerializer}} now writes/reads the outer map keys
and the {{checkpointIds}} elements with {{U.writeUuid / U.readUuid}} (null flag
+ two longs) instead of {{writeObject / readObject}} (Java-serialized
{{java.util.UUID}}). The two formats are not interchangeable, so the file
produced by a previous version can't be read after upgrade.
The header change made by IGNITE-24260 (protocol version byte removed) is *not*
a problem: the reader masks the version byte.
h3. Impact
{{CheckpointMarkersStorage#initialize}} catches the failure, logs
{noformat}
Failed to unmarshal earliest checkpoint map snapshot
{noformat}
deletes the file and rebuilds the earliest checkpoint map from checkpoint
markers. No data is lost, but the optimization is silently lost on the first
start after an upgrade and the node logs an error. PDS files must stay backward
compatible, so the old format has to be readable.
h3. Reproducer
{{CheckpointMapSnapshotCompatibilityTest}} in {{ignite-compatibility}}
(attached): starts a 2.17.0 / 2.18.0 node with persistence, waits for
{{cpMapSnapshot.bin}}, restarts on the same PDS with the current version and
asserts the error above isn't logged.
Test output (same for 2.17.0 and 2.18.0):
{noformat}
class org.apache.ignite.IgniteCheckedException: Failed to deserialize object
with given class loader: null
at
org.apache.ignite.marshaller.jdk.JdkMarshallerImpl.unmarshal0(JdkMarshallerImpl.java:128)
at
org.apache.ignite.marshaller.jdk.JdkMarshallerImpl.unmarshal0(JdkMarshallerImpl.java:135)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:74)
at
org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointMarkersStorage.initialize(CheckpointMarkersStorage.java:209)
at
org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointManager.initializeStorage(CheckpointManager.java:325)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:854)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:3075)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1122)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1713)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1635)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1077)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:587)
at
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1307)
at
org.apache.ignite.compatibility.testframework.junits.IgniteCompatibilityAbstractTest.startGrid(IgniteCompatibilityAbstractTest.java:344)
at
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1246)
at
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1222)
at
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1053)
at
org.apache.ignite.compatibility.persistence.CheckpointMapSnapshotCompatibilityTest.testSnapshotFileReadableByCurrentVersion(CheckpointMapSnapshotCompatibilityTest.java:121)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2464)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.io.EOFException
at
java.base/java.io.ObjectInputStream$BlockDataInputStream.readBoolean(ObjectInputStream.java:3374)
at
java.base/java.io.ObjectInputStream.readBoolean(ObjectInputStream.java:1062)
at
org.apache.ignite.internal.dto.IgniteDataTransferObjectInput.readBoolean(IgniteDataTransferObjectInput.java:99)
at
org.apache.ignite.internal.util.CommonUtils.readUuid(CommonUtils.java:682)
at
org.apache.ignite.internal.processors.cache.persistence.checkpoint.EarliestCheckpointMapSnapshotSerializer.readExternal(EarliestCheckpointMapSnapshotSerializer.java:84)
at
org.apache.ignite.internal.processors.cache.persistence.checkpoint.EarliestCheckpointMapSnapshotSerializer.readExternal(EarliestCheckpointMapSnapshotSerializer.java:38)
at
org.apache.ignite.internal.dto.IgniteDataTransferObject.readIgniteDataTransferObject(IgniteDataTransferObject.java:84)
at
org.apache.ignite.internal.dto.IgniteDataTransferObject.readExternal(IgniteDataTransferObject.java:67)
at
java.base/java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2306)
at
java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2255)
at
java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1733)
at
java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:509)
at
java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:467)
at
org.apache.ignite.marshaller.jdk.JdkMarshallerImpl.unmarshal0(JdkMarshallerImpl.java:120)
... 27 more
{noformat}
h3. Possible fix
Make the generated code (or a hand-written {{readExternalData}} in
{{EarliestCheckpointMapSnapshot}}) able to read the legacy layout, e.g. keep
{{writeObject/readObject}} for this class or detect the old format. Add the
compatibility test to the suite.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
