[
https://issues.apache.org/jira/browse/IGNITE-28939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-28939:
--------------------------------------
Labels: IEP-132 compatibility wire-format (was: IEP-132 compatibility)
> Turn blob fields of internal types into normal message fields
> -------------------------------------------------------------
>
> Key: IGNITE-28939
> URL: https://issues.apache.org/jira/browse/IGNITE-28939
> Project: Ignite
> Issue Type: Sub-task
> Components: messaging
> Reporter: Anton Vinogradov
> Assignee: Anton Vinogradov
> Priority: Major
> Labels: IEP-132, compatibility, wire-format
> Fix For: 2.19
>
>
> *Wire format change. Do this before 2.19 is released - after that it would
> break rolling upgrade.*
> h3. Goal
> Remove the question "which marshaller" for fields that hold internal classes.
> h3. Why
> Some fields are sent as blobs only because an internal class was never made a
> {{Message}}. Binary gives no schema for them anyway:
> {{BinaryUtils#isCustomJavaSerialization}} checks the whole class hierarchy,
> so {{Externalizable}} classes fall back to {{OptimizedMarshaller}} inside
> binary. One of these fields is also a mismatch: {{BinaryMetadata}} uses
> binary in {{BinaryMetadataVersionInfo}} and jdk in
> {{MetadataUpdateProposedMessage}}.
> h3. How
> Convert to normal message fields:
> * {{BinaryMetadataVersionInfo#metadata}} and
> {{MetadataUpdateProposedMessage#metadata}} - {{BinaryMetadata}} has simple
> fields: type id, type name, field map, schemas, flags;
> * {{ChangeGlobalStateMessage#baselineTopology}} - {{BaselineTopology}};
> * {{GridJobSiblingsResponse#siblings}} - the implementation is the internal
> {{GridJobSiblingImpl}};
> * {{StoredCacheData#qryEntities}} and {{DynamicCacheChangeRequest#schema}} -
> {{QueryEntityMessage}} already exists.
> Fields that really hold user classes stay blobs: node attributes, service
> affinity key, query entity default values, entry processors, invoke
> arguments, query filters, task results, message topic, cache configuration,
> plugin data and {{Throwable}}.
> h3. On-disk compatibility
> Three of these types also live on disk, so the conversion must not change how
> they are stored:
> * {{BinaryMetadata}} - written by {{BinaryMetadataFileStore#writeMetadata}};
> the format comes from its
> {{Externalizable}} methods;
> * {{BaselineTopology}} - kept in the metastore under
> {{METASTORE_CURR_BLT_KEY}}; {{MetaStorage}} marshals values with jdk;
> * {{QueryEntity}} inside {{StoredCacheData}} - {{cache_data.dat}} files and
> snapshots, jdk again.
> Rules that keep the stored format byte-identical:
> # keep the explicit {{serialVersionUID}}. All of these classes declare one,
> so adding the {{Message}} interface does
> not change the computed id. Without an explicit id, adding an interface
> changes it and old files stop loading.
> # make new wire companion fields {{transient}}, the way {{ccfgBytes}} and
> {{qryEntitiesBytes}} are done in
> {{StoredCacheData}}. Non-transient fields are what the disk format is made of.
> # do not add or remove non-transient fields, and do not touch
> {{writeExternal}}/{{readExternal}}. For
> {{BinaryMetadata}} the {{Externalizable}} methods *are* the on-disk format:
> adding {{Message}} on top is fine,
> dropping {{Externalizable}} is not.
> Precedent: {{StoredCacheData}}, {{CacheConfigurationEnrichment}} and
> {{SnapshotMetadata}} are already both
> {{Serializable}} and {{Message}}, and that coexists with the disk today.
> Verification for this ticket: read a {{cache_data.dat}} and a binary metadata
> file written by the previous build,
> and restore a snapshot taken by it.
> h3. Expected result
> Blob fields: 24 now, about 19 after. The {{BinaryMetadata}} mismatch is gone.
> These fields no longer depend on a marshaller, and usually become smaller on
> the wire.
> h3. How to verify
> Binary metadata tests, node join tests, cluster state change tests, dynamic
> cache start with query entities, job siblings tests. Compare message size
> before and after.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
