Ivan Bessonov created IGNITE-17871:
--------------------------------------
Summary: Use network serialization for RAFT commands
Key: IGNITE-17871
URL: https://issues.apache.org/jira/browse/IGNITE-17871
Project: Ignite
Issue Type: Improvement
Reporter: Ivan Bessonov
h3. Problem
Currently, there are two places where {{Command}} instances are being
serialized:
* ActionRequest - here the command property is marked is Marshallable, meaning
that it will be serialized using a User Object Serialization approach
* Listener - here command is explicitly serialized using JDKMarshaller, for
further handling by RAFT. This is the data that will be written to the Log and
deserialized on followers / learners
What are the problems?
For ActionRequest message, command is expected to be its largest part. And
although writing serialized UOS byte array into a netty socket is faster, then
optimized marshalling, it feels like overall throughput will be smaller. And
the reason is that there's an extra step of converting command into a byte
array, that happens in caller thread.
For serialization in listeners, using JDKMarshaller is both slow and
inefficient in terms of space. Obvious example - network serialization of
SnapshotMeta object, for instance, can be condensed to 8 bytes (assuming we
optimize "writeShort" and change its message type). JDKMarshaller produces 232
bytes. Of course, here most of fields are nulls and real payload will be
bigger, but JDKMarshaller will always lead to more data simply because it has
to store schema meta-information.
h3. Solution
Making Command an implementation of NetworkMessage will solve both of these
problems. ActionRequest will not have its "prepareMarshal" phase, listeners
will have fast and space-efficient serialization algorithm.
Of course, there must be drawbacks. I'll try to explain what I see at the
moment.
* Currently, there's no explicit support for List properties, only Collection.
It is easy to fix
* CMG commands use classes like ClusterNode and IgniteProductVersion. We
should introduce message alternatives
* I saw some enums being used, they are not natively supported at the moment.
There are two options:
** implement native support. I consider this a dangerous path
** store explicit ordinal where it's necessary
* ByteBuffer support would be really nice to have natively. Should be fast to
implement also
One important note: there should be no Marshallable properties in commands,
because we can't persist them. Information about classes' ids is stored in
sessions and can change between sessions. The way to achieve it is to pass a
"null" UOS context into serializator.
Now about serializator: we can have thread-local buffers to write data to. When
write is complete, data is copied as a byte[]. Reading will be done directly
from the byte[].
Possible optimization for ByteBuffers - we can implement them as slices of the
byte[] payload instead of copying sub-arrays. Will save some time and memory.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
