[
https://issues.apache.org/jira/browse/IGNITE-29185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ignite TC Bot updated IGNITE-29185:
-----------------------------------
Ignite Flags: Release Notes Required (was: Docs Required,Release Notes
Required)
Docs Required was reviewed against the merged fix in
https://github.com/apache/ignite/pull/13721. This restores compatibility of an
internal checkpoint snapshot serialization format for existing persisted data;
it introduces no user-facing configuration, procedure, or API requiring a
documentation change. Release Notes Required is retained.
> cpMapSnapshot.bin written by Ignite 2.17/2.18 can't be read by current
> version (generated IDTO serializer changed UUID format)
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: IGNITE-29185
> URL: https://issues.apache.org/jira/browse/IGNITE-29185
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.19
> Reporter: Nikolay Izhikov
> Assignee: Nikolay Izhikov
> Priority: Major
> Labels: ise
> Fix For: 2.19
>
> Attachments: IGNITE-29185-reproducer.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> h3. Problem
> {{cpMapSnapshot.bin}} (earliest checkpoint map snapshot, see
> {{CheckpointMarkersStorage#EARLIEST_CP_SNAPSHOT_FILE}}) written by Ignite
> 2.17 / 2.18 can't be deserialized by current master (future 2.19).
> The file is an {{EarliestCheckpointMapSnapshot}} marshalled with
> {{JdkMarshaller}}. In 2.17 and 2.18 the payload is written by hand:
> {code:java}
> U.writeMap(out, data); // keys (UUID) and values via
> out.writeObject(...)
> U.writeCollection(out, checkpointIds); // UUID elements via
> out.writeObject(...)
> {code}
> Since IGNITE-27709 the class is serialized by generated code, and since
> IGNITE-27775 the generator emits typed code for collections. The generated
> {{EarliestCheckpointMapSnapshotSerializer}} now writes/reads the outer map
> keys and the {{checkpointIds}} elements with {{U.writeUuid / U.readUuid}}
> (null flag + two longs) instead of {{writeObject / readObject}}
> (Java-serialized {{java.util.UUID}}). The two formats are not
> interchangeable, so the file produced by a previous version can't be read
> after upgrade.
> The header change made by IGNITE-24260 (protocol version byte removed) is
> *not* a problem: the reader masks the version byte.
> h3. Impact
> {{CheckpointMarkersStorage#initialize}} catches the failure, logs
> {noformat}
> Failed to unmarshal earliest checkpoint map snapshot
> {noformat}
> deletes the file and rebuilds the earliest checkpoint map from checkpoint
> markers. No data is lost, but the optimization is silently lost on the first
> start after an upgrade and the node logs an error. PDS files must stay
> backward compatible, so the old format has to be readable.
> h3. Reproducer
> {{CheckpointMapSnapshotCompatibilityTest}} in {{ignite-compatibility}}
> (attached): starts a 2.17.0 / 2.18.0 node with persistence, waits for
> {{cpMapSnapshot.bin}}, restarts on the same PDS with the current version and
> asserts the error above isn't logged.
> Test output (same for 2.17.0 and 2.18.0):
> {noformat}
> class org.apache.ignite.IgniteCheckedException: Failed to deserialize object
> with given class loader: null
> at
> org.apache.ignite.marshaller.jdk.JdkMarshallerImpl.unmarshal0(JdkMarshallerImpl.java:128)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshallerImpl.unmarshal0(JdkMarshallerImpl.java:135)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:74)
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointMarkersStorage.initialize(CheckpointMarkersStorage.java:209)
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointManager.initializeStorage(CheckpointManager.java:325)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:854)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:3075)
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1122)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1713)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1635)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1077)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:587)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1307)
> at
> org.apache.ignite.compatibility.testframework.junits.IgniteCompatibilityAbstractTest.startGrid(IgniteCompatibilityAbstractTest.java:344)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1246)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1222)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1053)
> at
> org.apache.ignite.compatibility.persistence.CheckpointMapSnapshotCompatibilityTest.testSnapshotFileReadableByCurrentVersion(CheckpointMapSnapshotCompatibilityTest.java:121)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:569)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2464)
> at java.base/java.lang.Thread.run(Thread.java:840)
> Caused by: java.io.EOFException
> at
> java.base/java.io.ObjectInputStream$BlockDataInputStream.readBoolean(ObjectInputStream.java:3374)
> at
> java.base/java.io.ObjectInputStream.readBoolean(ObjectInputStream.java:1062)
> at
> org.apache.ignite.internal.dto.IgniteDataTransferObjectInput.readBoolean(IgniteDataTransferObjectInput.java:99)
> at
> org.apache.ignite.internal.util.CommonUtils.readUuid(CommonUtils.java:682)
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.EarliestCheckpointMapSnapshotSerializer.readExternal(EarliestCheckpointMapSnapshotSerializer.java:84)
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.EarliestCheckpointMapSnapshotSerializer.readExternal(EarliestCheckpointMapSnapshotSerializer.java:38)
> at
> org.apache.ignite.internal.dto.IgniteDataTransferObject.readIgniteDataTransferObject(IgniteDataTransferObject.java:84)
> at
> org.apache.ignite.internal.dto.IgniteDataTransferObject.readExternal(IgniteDataTransferObject.java:67)
> at
> java.base/java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2306)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2255)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1733)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:509)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:467)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshallerImpl.unmarshal0(JdkMarshallerImpl.java:120)
> ... 27 more
> {noformat}
> h3. Possible fix
> Make the generated code (or a hand-written {{readExternalData}} in
> {{EarliestCheckpointMapSnapshot}}) able to read the legacy layout, e.g. keep
> {{writeObject/readObject}} for this class or detect the old format. Add the
> compatibility test to the suite.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
