Re: Improve "direct" serialization

Consider you have a message of 100 bytes. If it is split in between 2 buffers, you can write it as follows: [Chunk 1] 65 - 65 bytes to follow false - incomplete flag data - 65 bytes [Chunk 2] 35 - 35 bytes to follow true - complete flag, this is the end data - 35 bytes 15 окт. 2016 г. 1:06

Re: Improve "direct" serialization

Vova, You still can't write the first buffer to channel before the whole message is serialized. Are you going to have multiple write buffers? I'm confused :) -Val On Fri, Oct 14, 2016 at 1:49 PM, Vladimir Ozerov wrote: > Val, > > No need to copy on write. You simply

Re: Improve "direct" serialization

Vova, I meant the copy on write. To know the length you need to fully marshal message first. This means that you will always copy the array before writing to channel. Unless I'm missing something, this eliminates the purpose of direct serialization. -Val On Thu, Oct 13, 2016 at 11:09 PM,

Re: Improve "direct" serialization

I already tried smth like that when tried to make marshalling parallel. This is how I tried to reapproach it. 1. allocate set of direct buffers per each session (connection). that was 1 large buffer, but sliced with smaller ones over large offheap array. 2. each thread acquires chunk by chunk

Re: Improve "direct" serialization

Valya, Yes, in this design we will copy data into separate buffer on read. But what is important - it will happen only for message which is split between buffers. On Fri, Oct 14, 2016 at 2:33 AM, Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Vladimir, > > We don't write length

Re: Improve "direct" serialization

Writes can be optimized even further: 1) Write to *ByteBuffer *as long as there is a place in it. 2) When it is full - invoke a callback which will submit it to the socket, reset position to 0, and continue marshaling. This way we can probably get rid of write "state" at all. On Thu, Oct 13,

Improve "direct" serialization

Folks, I went through our so-called "direct serialization" and appears to be not very efficient to me. We never write message length. As a result we have to constantly track what was written and what was not, and whether we have a room for the next write. The same goes for reader. As a result