[
https://issues.apache.org/jira/browse/IGNITE-21230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815191#comment-17815191
]
Pavel Pereslegin commented on IGNITE-21230:
-------------------------------------------
h4. Things about current implementation.
The idea of using IgniteDataTransferObject from AI2 didn't go well.
Mainly due to the need to make all objects externalizable, in addition, our
entries have nested objects. For this to work properly, an intermediate layer
of "flat" externalizable objects is needed.
As a result of various code shuffles and internal discussions, at the moment it
was decided to leave the simplest option possible. Each object is serialized by
its own external serializer. The current implementation of object serializers
is not aware of the data format version.
The version of the current format (1) is written at the very beginning (see the
description in the {{UpdateLogMarshallerImpl}} class).
h4. Ideas for the future data format:
It is necessary to maintain a registry of all types of serializable objects
(nested and external) and their serializers with a specific version.
When writing, you must specify the current version of the object format
(serializer version).
When reading, you need to get the required serializer (object type + version)
from the registry.
In terms of implementation, it is supposed to add some mix of
IgniteDataInput/IgniteDataOutput and ObjectInput/ObjectOutput which should be
able to serialize/deserialize object types that are registered in the
serializer registry.
> Improve serialization of catalog commands
> -----------------------------------------
>
> Key: IGNITE-21230
> URL: https://issues.apache.org/jira/browse/IGNITE-21230
> Project: Ignite
> Issue Type: Improvement
> Reporter: Konstantin Orlov
> Assignee: Pavel Pereslegin
> Priority: Major
> Labels: ignite-3
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Currently, catalog uses java's built-in serialisation in order to persists
> descriptors and update entries. Although it is most simple way to address
> conversion of POJO to byte array, the built-in serialisation is not efficient
> in terms of occupied space. Given that catalog may contain thousands and
> thousands of objects, it may result in increased load on a metastorage.
> To address this issue, let's introduce custom serialisation format for
> catalog's descriptors and update entries.
> NB: we should keep in mind that descriptors may evolve over time, thus we
> should consider support of versioning for descriptors
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
