julianhowarth opened a new issue, #3828: URL: https://github.com/apache/fory/issues/3828
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/fory/issues) and found no similar issues. ### Version Fory 1.3.0 Java 21, 25 (but probably is 17+) ### Component(s) Java ### Minimal reproduce step This is quite a specific edge case that we uncovered by accident which shows that a Fory instance (using Java language encoding) can fail to deserialize validly serialized data when reused. Two Fory instances built identically (same registrations, in the same order, same config, using `COMPATIBLE` mode) can disagree about the wire encoding of a class that carries a `readResolve` method, depending on the order each instance first encountered the type at runtime. Data that one instance wrote is then unreadable by the other, failing with: ``` java.lang.IllegalArgumentException: Incompatible compatible field schema for field ... ``` The following test demonstrates the issue - run with `java -cp <my-lib-path>/fory-core-1.3.0.jar ForyClassificationAsymmetryRepro.java` (or `java --add-opens=java.base/java.lang.invoke=ALL-UNNAMED <my-lib-path>/fory-core-1.3.0.jar ForyClassificationAsymmetryRepro.java` if Java 25) ```java public class ForyClassificationAsymmetryRepro { /** * A minimal stand-in for the value classes org.immutables and similar libraries generate, * which carry a readResolve. The mere presece of the method is enough to cause the failure. */ public static final class Code implements Serializable { private final String value; public Code(String value) { this.value = value; } public static Code of(String value) { return new Code(value); } @Override public boolean equals(Object o) { return o instanceof Code c && c.value.equals(value); } @Override public int hashCode() { return value.hashCode(); } private Object readResolve() { return this; } @Override public String toString() { return "Code[" + value + "]"; } } public record GroupPayload(int id, Set<Code> codes) implements Serializable {} public record GeoPayload(List<Code> codes) implements Serializable {} /** Every instance is built identically - registrations, ids and order all match. */ private static Fory newFory() { var fory = Fory.builder() .withLanguage(Language.JAVA) .withRefTracking(true) .requireClassRegistration(true) .withCompatibleMode(CompatibleMode.COMPATIBLE) .build(); fory.register(Code.class, 100); fory.register(GroupPayload.class, 101); fory.register(GeoPayload.class, 102); return fory; } public static void main(String[] args) { var payload1 = new GroupPayload(7, Set.of(Code.of("FR"))); var payload2 = new GeoPayload(List.of(Code.of("GB"))); byte[] bytes1 = newFory().serialize(payload1); byte[] bytes2 = newFory().serialize(payload2); // A fresh reader parses either payload without issue System.out.println("fresh reader, payload1: " + newFory().deserialize(bytes1)); System.out.println("fresh reader, payload2: " + newFory().deserialize(bytes2)); // A reader that has already met Code inside payload1 can no longer read payload2: // "Incompatible compatible field schema for field GeoPayload.codes: nested field schema // mismatch; peer typeId=28 (COMPATIBLE_STRUCT), local typeId=31 (EXT)" var reader = newFory(); System.out.println("stateful reader, payload1: " + reader.deserialize(bytes1)); System.out.println("stateful reader, payload2: " + reader.deserialize(bytes2)); } } ``` ### What did you expect to see? output should be: ``` fresh reader, payload1: GroupPayload[id=7, codes=[Code[FR]]] fresh reader, payload2: GeoPayload[codes=[Code[GB]]] stateful reader, payload1: GroupPayload[id=7, codes=[Code[FR]]] stateful reader, payload2: GeoPayload[codes=[Code[GB]]] ``` This output is seen if the `readResolve` method is removed ### What did you see instead? ``` fresh reader, payload1: GroupPayload[id=7, codes=[Code[FR]]] fresh reader, payload2: GeoPayload[codes=[Code[GB]]] stateful reader, payload1: GroupPayload[id=7, codes=[Code[FR]]] Exception in thread "main" org.apache.fory.exception.DeserializationException: Deserialize failed, read objects are: [null] at org.apache.fory.util.ExceptionUtils.handleReadFailed(ExceptionUtils.java:69) at org.apache.fory.Fory.deserialize(Fory.java:520) at org.apache.fory.Fory.deserialize(Fory.java:413) at ForyClassificationAsymmetryRepro.main(ForyClassificationAsymmetryRepro.java:122) Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Incompatible compatible field schema for field ForyClassificationAsymmetryRepro$GeoPayload.codes: nested field schema mismatch; peer=CollectionFieldType{elementType=RegisteredFieldType{nullable=true, trackingRef=true, typeId=28, userTypeId=-1}, nullable=true, trackingRef=true}, local=CollectionFieldType{elementType=RegisteredFieldType{nullable=true, trackingRef=true, typeId=31, userTypeId=-1}, nullable=true, trackingRef=true} at org.apache.fory.builder.CodecUtils.loadSerializer(CodecUtils.java:210) at org.apache.fory.builder.CodecUtils.loadOrGenCompatibleCodecClass(CodecUtils.java:61) at org.apache.fory.builder.CodecUtils.lambda$loadOrGenCompatibleCodecClass$1(CodecUtils.java:74) at org.apache.fory.builder.JITContext.asyncVisitFory(JITContext.java:159) at org.apache.fory.builder.CodecUtils.loadOrGenCompatibleCodecClass(CodecUtils.java:74) at org.apache.fory.resolver.TypeResolver.lambda$getMetaSharedTypeInfo$1(TypeResolver.java:1206) at org.apache.fory.builder.JITContext.registerSerializerJITCallback(JITContext.java:121) at org.apache.fory.resolver.TypeResolver.getMetaSharedTypeInfo(TypeResolver.java:1204) at org.apache.fory.resolver.TypeResolver.createMetaSharedTypeInfo(TypeResolver.java:1079) at org.apache.fory.resolver.TypeResolver.buildCheckedMetaSharedTypeInfo(TypeResolver.java:1094) at org.apache.fory.resolver.TypeResolver.readSharedTypeDefInfo(TypeResolver.java:903) at org.apache.fory.resolver.TypeResolver.readSharedClassTypeInfo(TypeResolver.java:882) at org.apache.fory.resolver.TypeResolver.readTypeInfo(TypeResolver.java:633) at org.apache.fory.context.ReadContext.readRef(ReadContext.java:532) at org.apache.fory.Fory.deserialize(Fory.java:515) at org.apache.fory.Fory.deserialize(Fory.java:413) at ForyClassificationAsymmetryRepro.main(ForyClassificationAsymmetryRepro.java:122) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) at java.base/java.lang.reflect.Method.invoke(Method.java:565) at jdk.compiler/com.sun.tools.javac.launcher.SourceLauncher.execute(SourceLauncher.java:258) at jdk.compiler/com.sun.tools.javac.launcher.SourceLauncher.run(SourceLauncher.java:138) at jdk.compiler/com.sun.tools.javac.launcher.SourceLauncher.main(SourceLauncher.java:76) Caused by: java.lang.IllegalArgumentException: Incompatible compatible field schema for field ForyClassificationAsymmetryRepro$GeoPayload.codes: nested field schema mismatch; peer=CollectionFieldType{elementType=RegisteredFieldType{nullable=true, trackingRef=true, typeId=28, userTypeId=-1}, nullable=true, trackingRef=true}, local=CollectionFieldType{elementType=RegisteredFieldType{nullable=true, trackingRef=true, typeId=31, userTypeId=-1}, nullable=true, trackingRef=true} at org.apache.fory.meta.FieldInfo.incompatibleField(FieldInfo.java:422) at org.apache.fory.meta.FieldInfo.toDescriptor(FieldInfo.java:156) at org.apache.fory.meta.TypeDef.buildDescriptors(TypeDef.java:464) at org.apache.fory.meta.TypeDef.buildDescriptors(TypeDef.java:365) at org.apache.fory.meta.TypeDef.lambda$getDescriptors$0(TypeDef.java:360) at org.apache.fory.resolver.SharedRegistry.lambda$getOrCreateTypeDefDescriptors$0(SharedRegistry.java:409) at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1724) at org.apache.fory.resolver.SharedRegistry.getOrCreateTypeDefDescriptors(SharedRegistry.java:408) at org.apache.fory.meta.TypeDef.getDescriptors(TypeDef.java:359) at org.apache.fory.resolver.TypeResolver.lambda$createDescriptorGrouper$0(TypeResolver.java:1679) at org.apache.fory.resolver.SharedRegistry.lambda$getOrCreateTypeDefDescriptorGrouper$0(SharedRegistry.java:443) at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1724) at org.apache.fory.resolver.SharedRegistry.getOrCreateTypeDefDescriptorGrouper(SharedRegistry.java:443) at org.apache.fory.resolver.TypeResolver.createDescriptorGrouper(TypeResolver.java:1672) at org.apache.fory.resolver.TypeResolver.createDescriptorGrouper(TypeResolver.java:1663) at org.apache.fory.builder.CompatibleCodecBuilder.lambda$new$0(CompatibleCodecBuilder.java:110) at org.apache.fory.builder.BaseObjectCodecBuilder.lambda$typeResolver$0(BaseObjectCodecBuilder.java:279) at org.apache.fory.builder.JITContext.asyncVisitFory(JITContext.java:159) at org.apache.fory.builder.BaseObjectCodecBuilder.typeResolver(BaseObjectCodecBuilder.java:279) at org.apache.fory.builder.BaseObjectCodecBuilder.typeResolver(BaseObjectCodecBuilder.java:275) at org.apache.fory.builder.CompatibleCodecBuilder.<init>(CompatibleCodecBuilder.java:110) at org.apache.fory.builder.CodecUtils.lambda$loadOrGenCompatibleCodecClass$0(CodecUtils.java:67) at org.apache.fory.builder.CodecUtils.loadSerializer(CodecUtils.java:204) ... 21 more ``` ### Anything Else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
