[
https://issues.apache.org/jira/browse/IGNITE-28942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-28942:
--------------------------------------
Description:
*Research task, finished. Result: keep {{JdkMarshaller}}. Kept for the record
so the question does not come back.*
h3. Question
Can {{JdkMarshaller}} be dropped and binary used everywhere?
h3. What was checked
*Binary can be self-contained.* When a type is not registered, binary writes
{{UNREGISTERED_TYPE_ID}} plus the class name ({{BinaryWriterExImpl#preWrite}}),
and metadata can stay local ({{registerClassNameLocally}}, {{addMetaLocally}}).
Thin client, JDBC and the standalone WAL reader already run binary with no
cluster at all. So "binary needs discovery" is not true of the format.
*But only with a full footer.* An object written by one context and read by a
completely empty one:
* {{compactFooter = false}}: read succeeds, cycles included, 84 bytes against
121 for jdk;
* {{compactFooter = true}} (the default): read fails with "Cannot find metadata
for object with compact footer", because the field layout comes from metadata
that discovery distributes.
{{compactFooter}} is a node-wide setting: {{BinaryWriterExImpl#postWrite}}
reads {{ctx.isCompactFooter()}}. A per-call full-footer mode would have to be
added.
*The unregistered path is slow, and the reason is known.* 160 us against 24 for
a {{CacheConfiguration}}. {{BinaryContext#registerUserClassDescriptor}} stores
the descriptor in {{descByCls}} only when registration succeeded; otherwise it
returns the descriptor without caching it, so it is rebuilt by reflection on
every call.
*Pre-registration is not closed under nesting.* Marshalling a plain
{{CacheConfiguration}} needs 9 types;
{{javax.cache.configuration.FactoryBuilder$SingletonFactory}} and
{{javax.cache.expiry.EternalExpiryPolicy}} are missing from
{{META-INF/classnames.properties}}. {{failIfUnregistered = true}} does not help
- it fires for internal types too.
*On-disk formats would become a migration.* Cache configurations, metastore,
snapshots and dumps are written with jdk and read by offline tools
({{DumpReader}} builds {{Marshallers.jdk()}} directly). Note that wire
companions in such classes are {{transient}}, so today the two formats do not
interfere - dropping jdk would end that.
*And it would not even remove Java serialization.*
{{BinaryUtils#isCustomJavaSerialization}} walks the whole hierarchy, so every
{{Externalizable}} class and every class with {{writeObject}} goes to
{{OptimizedMarshaller}} anyway. Three implementations would become two, not one.
h3. Answer
Keep {{JdkMarshaller}}. The move costs a new format mode, a descriptor-cache
change, and a migration of data already written on user disks, and it buys one
implementation less plus a few dozen bytes per object.
This does not affect the rest of IGNITE-28935: it removes the *choice* of a
marshaller, not a marshaller. If the picture changes later, only the call sites
have to be revisited.
was:
*Research task. No production change expected from this ticket.*
h3. Goal
Answer, with numbers, whether {{JdkMarshaller}} can be dropped and binary used
everywhere.
h3. Why
"Binary needs discovery" is not true of the format. When a type is not
registered, binary writes {{UNREGISTERED_TYPE_ID}} plus the class name
({{BinaryWriterExImpl#preWrite}}), and metadata can be kept local
({{registerClassNameLocally}}, {{addMetaLocally}}). Thin client, JDBC and the
standalone WAL reader already run binary without any cluster.
First measurements:
* with a full footer the object is self-contained. An object written by one
context is read by a completely empty context, cycles included: 84 bytes
against 121 for jdk on the same POJO.
* with the default compact footer the same object cannot be read: "Cannot find
metadata for object with compact footer". The field layout still comes from
metadata, which discovery distributes.
* the unregistered path is slow today: 160 us against 24 for a
{{CacheConfiguration}}, because the descriptor of an unregistered type is
rebuilt by reflection on every call.
h3. How
Answer these questions:
# a per-call full-footer mode - {{compactFooter}} is a node-wide setting now;
# caching descriptors of unregistered types, and what the numbers become after
that;
# class-name filtering on the unregistered path:
{{JdkMarshallerObjectInputStream}} and {{OptimizedObjectInputStream}} pass the
filter into {{CommonUtils#forName}}, while {{BinaryUtils#doReadClass}} passes
{{null}}. Note that {{forName}} only consults the filter on a class-cache miss,
so any check must run in a fresh JVM;
# on-disk formats: cache configurations, metastore, snapshots and dumps are
written with jdk and read by offline tools, so this is a data migration, not a
code change;
# what stays anyway: binary delegates every {{Externalizable}} type to
{{OptimizedMarshaller}}, so the count goes from three implementations to two,
not to one.
h3. Expected result
A decision backed by numbers: either a plan with the four items above, or a
documented "not worth it" so that the question stops coming back.
> Evaluate replacing JdkMarshaller with binary in a self-contained mode
> ---------------------------------------------------------------------
>
> Key: IGNITE-28942
> URL: https://issues.apache.org/jira/browse/IGNITE-28942
> Project: Ignite
> Issue Type: Sub-task
> Components: messaging
> Reporter: Anton Vinogradov
> Assignee: Anton Vinogradov
> Priority: Major
> Fix For: 2.19
>
>
> *Research task, finished. Result: keep {{JdkMarshaller}}. Kept for the record
> so the question does not come back.*
> h3. Question
> Can {{JdkMarshaller}} be dropped and binary used everywhere?
> h3. What was checked
> *Binary can be self-contained.* When a type is not registered, binary writes
> {{UNREGISTERED_TYPE_ID}} plus the class name
> ({{BinaryWriterExImpl#preWrite}}), and metadata can stay local
> ({{registerClassNameLocally}}, {{addMetaLocally}}). Thin client, JDBC and the
> standalone WAL reader already run binary with no cluster at all. So "binary
> needs discovery" is not true of the format.
> *But only with a full footer.* An object written by one context and read by a
> completely empty one:
> * {{compactFooter = false}}: read succeeds, cycles included, 84 bytes against
> 121 for jdk;
> * {{compactFooter = true}} (the default): read fails with "Cannot find
> metadata for object with compact footer", because the field layout comes from
> metadata that discovery distributes.
> {{compactFooter}} is a node-wide setting: {{BinaryWriterExImpl#postWrite}}
> reads {{ctx.isCompactFooter()}}. A per-call full-footer mode would have to be
> added.
> *The unregistered path is slow, and the reason is known.* 160 us against 24
> for a {{CacheConfiguration}}. {{BinaryContext#registerUserClassDescriptor}}
> stores the descriptor in {{descByCls}} only when registration succeeded;
> otherwise it returns the descriptor without caching it, so it is rebuilt by
> reflection on every call.
> *Pre-registration is not closed under nesting.* Marshalling a plain
> {{CacheConfiguration}} needs 9 types;
> {{javax.cache.configuration.FactoryBuilder$SingletonFactory}} and
> {{javax.cache.expiry.EternalExpiryPolicy}} are missing from
> {{META-INF/classnames.properties}}. {{failIfUnregistered = true}} does not
> help - it fires for internal types too.
> *On-disk formats would become a migration.* Cache configurations, metastore,
> snapshots and dumps are written with jdk and read by offline tools
> ({{DumpReader}} builds {{Marshallers.jdk()}} directly). Note that wire
> companions in such classes are {{transient}}, so today the two formats do not
> interfere - dropping jdk would end that.
> *And it would not even remove Java serialization.*
> {{BinaryUtils#isCustomJavaSerialization}} walks the whole hierarchy, so every
> {{Externalizable}} class and every class with {{writeObject}} goes to
> {{OptimizedMarshaller}} anyway. Three implementations would become two, not
> one.
> h3. Answer
> Keep {{JdkMarshaller}}. The move costs a new format mode, a descriptor-cache
> change, and a migration of data already written on user disks, and it buys
> one implementation less plus a few dozen bytes per object.
> This does not affect the rest of IGNITE-28935: it removes the *choice* of a
> marshaller, not a marshaller. If the picture changes later, only the call
> sites have to be revisited.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
