[
https://issues.apache.org/jira/browse/IGNITE-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-17871:
-----------------------------------
Fix Version/s: 3.0.0-beta2
> Use network serialization for RAFT commands
> -------------------------------------------
>
> Key: IGNITE-17871
> URL: https://issues.apache.org/jira/browse/IGNITE-17871
> Project: Ignite
> Issue Type: Improvement
> Reporter: Ivan Bessonov
> Assignee: Ivan Bessonov
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Problem
> Currently, there are two places where {{Command}} instances are being
> serialized:
> * ActionRequest - here the command property is marked is Marshallable,
> meaning that it will be serialized using a User Object Serialization approach
> * Listener - here command is explicitly serialized using JDKMarshaller, for
> further handling by RAFT. This is the data that will be written to the Log
> and deserialized on followers / learners
> What are the problems?
> For ActionRequest message, command is expected to be its largest part. And
> although writing serialized UOS byte array into a netty socket is faster,
> then optimized marshalling, it feels like overall throughput will be smaller.
> And the reason is that there's an extra step of converting command into a
> byte array, that happens in caller thread.
> For serialization in listeners, using JDKMarshaller is both slow and
> inefficient in terms of space. Obvious example - network serialization of
> SnapshotMeta object, for instance, can be condensed to 8 bytes (assuming we
> optimize "writeShort" and change its message type). JDKMarshaller produces
> 232 bytes. Of course, here most of fields are nulls and real payload will be
> bigger, but JDKMarshaller will always lead to more data simply because it has
> to store schema meta-information.
> h3. Solution
> Making Command an implementation of NetworkMessage will solve both of these
> problems. ActionRequest will not have its "prepareMarshal" phase, listeners
> will have fast and space-efficient serialization algorithm.
> Of course, there must be drawbacks. I'll try to explain what I see at the
> moment.
> * Currently, there's no explicit support for List properties, only
> Collection. It is easy to fix
> * CMG commands use classes like ClusterNode and IgniteProductVersion. We
> should introduce message alternatives
> * I saw some enums being used, they are not natively supported at the
> moment. There are two options:
> ** implement native support. I consider this a dangerous path
> ** store explicit ordinal where it's necessary
> * ByteBuffer support would be really nice to have natively. Should be fast
> to implement also
> One important note: there should be no Marshallable properties in commands,
> because we can't persist them. Information about classes' ids is stored in
> sessions and can change between sessions. The way to achieve it is to pass a
> "null" UOS context into serializator.
> Now about serializator: we can have thread-local buffers to write data to.
> When write is complete, data is copied as a byte[]. Reading will be done
> directly from the byte[].
> Possible optimization for ByteBuffers - we can implement them as slices of
> the byte[] payload instead of copying sub-arrays. Will save some time and
> memory.
> h3. Plan
> Given the volume of changes, I suggest splitting the issue into several
> parts. There are multiple sets of commands in Ignite:
> * Table commands (5 commands at the moment of writing this text)
> * CMG commands (6 commands)
> * Metastorage (19)
> This list goes in order of complexity. Table commands are very simple. CMG
> commands require additional messages for ClusterNodes and such.
> Metastorage commands have complicated structures for conditional updates, and
> there are many of them.
> When all commands are messages, we can safely inherit Command from
> NetworkMessage and remove Marshallable from the ActionRequest's field. In
> total, this looks like 4 separate issues, 4th one being the current one. List
> & ByteBuffer support is already completed in IGNITE-17874.
> Of course, implementation of the OptimizedMarshaller should be done here as
> well. I'd also provide some ideas to whoever's going to implement the
> improvement, there's at least one possible optimization in mind that's hard
> to put in words.
> There are also tests, they can be migrated in current issue as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
