Tatenda-k commented on code in PR #3861:
URL: https://github.com/apache/fory/pull/3861#discussion_r3616254797
##########
java/fory-core/src/main/java/org/apache/fory/context/WriteContext.java:
##########
@@ -442,6 +444,58 @@ public BufferCallback getBufferCallback() {
return bufferCallback;
}
+ /**
+ * If {@code obj} carries a populated {@link ForyExtraFields} sink whose
remote {@code TypeDef}
+ * differs from the local type, emits that remote schema header and replays
every field (matched
+ * fields from the bean, unmatched fields from the sink) so a downstream
peer with the full schema
+ * can recover them, excluded from writeRef(obj,typeinfo) because it is only
called by methods
+ * which pass collection objects, and from writeNonRef(obj,typeinfo) which
is gated on
+ * useDeclaredTypeInfo and !crossLanguage ie. the path for mononmorphic
elements
+ */
+ public boolean tryWriteExtraFieldsSchema(TypeResolver resolver, TypeInfo
typeInfo, Object obj) {
+ FieldAccessor sinkAccessor = typeInfo.getExtraFieldsSinkAccessor();
+ if (sinkAccessor == null) {
+ return false;
+ }
+ ForyExtraFields extraField = (ForyExtraFields) sinkAccessor.getObject(obj);
+ if (extraField == null || extraField.getTypeDef() == null) {
+ return false;
+ }
+ long sinkTypeDefId = extraField.getTypeDef().getId();
+ // The object was deserialized under a different (fuller) remote schema.
Emit that remote schema
+ // header (from the writer-class TypeInfo) and replay all fields through
the reader-class
+ // serializer that captured them, whose field order follows the remote
schema.
+ TypeInfo headerTypeInfo = resolver.getTypeInfoByTypeDefId(sinkTypeDefId);
+ if (headerTypeInfo == null) {
+ throw new IllegalStateException(
+ "Cannot re-serialize "
+ + obj.getClass().getName()
+ + " with captured extra fields: this Fory instance does not have
the object's original "
+ + "remote schema (TypeDef id "
+ + sinkTypeDefId
+ + ") cached, so it cannot emit the schema header needed to
preserve those fields. "
+ + "Replay needs a Fory that has deserialized a payload carrying
that schema and still "
+ + "has it cached, ie the instance that produced this object.
Each instance "
+ + "retains only a bounded number of distinct remote schemas, so
under very high schema "
+ + "churn the schema may not be retained and the captured fields
cannot be replayed.");
+ }
+ Serializer<?> replaySerializer =
+ resolver.getExtraFieldsWriteSerializer(obj.getClass(), sinkTypeDefId);
+ if (replaySerializer == null) {
+ throw new IllegalStateException(
+ "Cannot replay captured extra fields on "
+ + obj.getClass()
+ + ": remote TypeDef "
+ + sinkTypeDefId
+ + " has not been read into this class on this Fory instance.");
+ }
+ resolver.writeTypeInfo(this, headerTypeInfo);
Review Comment:
I've updated `tryWriteExtraFieldsSchema(...)` to call
`UnknownStructSerializer.writeUnknownStructHeader(...)` when the header is an
`UnknownStruct`, and added `unknownStructHeaderDoesNotCorruptForwardedStream`
to test the above case.
##########
java/fory-core/src/main/java/org/apache/fory/context/WriteContext.java:
##########
@@ -442,6 +444,58 @@ public BufferCallback getBufferCallback() {
return bufferCallback;
}
+ /**
+ * If {@code obj} carries a populated {@link ForyExtraFields} sink whose
remote {@code TypeDef}
+ * differs from the local type, emits that remote schema header and replays
every field (matched
+ * fields from the bean, unmatched fields from the sink) so a downstream
peer with the full schema
+ * can recover them, excluded from writeRef(obj,typeinfo) because it is only
called by methods
+ * which pass collection objects, and from writeNonRef(obj,typeinfo) which
is gated on
+ * useDeclaredTypeInfo and !crossLanguage ie. the path for mononmorphic
elements
+ */
+ public boolean tryWriteExtraFieldsSchema(TypeResolver resolver, TypeInfo
typeInfo, Object obj) {
+ FieldAccessor sinkAccessor = typeInfo.getExtraFieldsSinkAccessor();
+ if (sinkAccessor == null) {
+ return false;
+ }
+ ForyExtraFields extraField = (ForyExtraFields) sinkAccessor.getObject(obj);
+ if (extraField == null || extraField.getTypeDef() == null) {
+ return false;
+ }
+ long sinkTypeDefId = extraField.getTypeDef().getId();
+ // The object was deserialized under a different (fuller) remote schema.
Emit that remote schema
+ // header (from the writer-class TypeInfo) and replay all fields through
the reader-class
+ // serializer that captured them, whose field order follows the remote
schema.
+ TypeInfo headerTypeInfo = resolver.getTypeInfoByTypeDefId(sinkTypeDefId);
+ if (headerTypeInfo == null) {
+ throw new IllegalStateException(
+ "Cannot re-serialize "
+ + obj.getClass().getName()
+ + " with captured extra fields: this Fory instance does not have
the object's original "
+ + "remote schema (TypeDef id "
+ + sinkTypeDefId
+ + ") cached, so it cannot emit the schema header needed to
preserve those fields. "
+ + "Replay needs a Fory that has deserialized a payload carrying
that schema and still "
+ + "has it cached, ie the instance that produced this object.
Each instance "
+ + "retains only a bounded number of distinct remote schemas, so
under very high schema "
+ + "churn the schema may not be retained and the captured fields
cannot be replayed.");
+ }
+ Serializer<?> replaySerializer =
+ resolver.getExtraFieldsWriteSerializer(obj.getClass(), sinkTypeDefId);
+ if (replaySerializer == null) {
+ throw new IllegalStateException(
+ "Cannot replay captured extra fields on "
+ + obj.getClass()
+ + ": remote TypeDef "
+ + sinkTypeDefId
+ + " has not been read into this class on this Fory instance.");
+ }
+ resolver.writeTypeInfo(this, headerTypeInfo);
Review Comment:
I've updated the `writeSharedClassMeta` to deduplicate by `typeDef`
identity. And added `twoRemoteVersionsOfSameClassForwardedInOneGraph` test,
which encompasses the above scenario.
##########
java/fory-core/src/main/java/org/apache/fory/serializer/CompatibleSerializer.java:
##########
@@ -225,6 +238,71 @@ public void write(WriteContext writeContext, T value) {
serializer.write(writeContext, value);
}
+ /**
+ * Writes all fields in remote-sorted order under the remote TypeDef schema:
matched fields are
+ * read from local getters, unmatched fields are read from the {@link
ForyExtraFields} map using
+ * fieldInfo.
+ */
+ private void writeFieldsIncludingExtraFields(
+ WriteContext writeContext, T value, ForyExtraFields extraField) {
+ RefWriter refWriter = writeContext.getRefWriter();
+ Generics generics = writeContext.getGenerics();
+ for (SerializationFieldInfo fieldInfo : allFields) {
+ writeFieldByCodecCategory(writeContext, value, refWriter, generics,
fieldInfo, extraField);
+ }
+ }
+
+ private void writeFieldByCodecCategory(
+ WriteContext writeContext,
+ T value,
+ RefWriter refWriter,
+ Generics generics,
+ SerializationFieldInfo fieldInfo,
+ ForyExtraFields extraField) {
+
+ MemoryBuffer buffer = writeContext.getBuffer();
+
+ boolean useExtraField = false;
+ Object fieldValue = null;
+
+ Object temp = extraField.getOrDefault(fieldInfo.descriptor.getName(),
NOT_FOUND);
+ if (temp != NOT_FOUND) {
+ useExtraField = true;
+ fieldValue = temp; // might be null, which is valid
+ }
+
+ switch (fieldInfo.codecCategory) {
+ case BUILD_IN:
+ if (useExtraField) {
+ AbstractObjectSerializer.writeBuildInFieldValue(
+ writeContext, typeResolver, refWriter, fieldInfo, buffer,
fieldValue);
+ } else {
+ AbstractObjectSerializer.writeBuildInField(
Review Comment:
I've updated `compatibleRead(...)` to stash the original value into the sink
so `writeFieldByCodecCategory(...)` finds an entry for the converter field. For
codegen, I added `FieldConverters.readTargetAndCapture(...)` used whenever the
class declares a sink to capture the raw value into the sink so the sink lookup
in `getFieldValue(...)` no longer returns empty. Added
`roundTripConvertedFieldWithExtraField` test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
