The ticket is created:
https://issues.apache.org/jira/browse/IGNITE-8054
2018-03-27 10:04 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>:
> Sorry, Dmitiry, I meant Ignite, not GridGain (memories of pre-apache
> coding). I am assuming that Alex Menshikov was referring to GridLongList
> class in Ignite.
>
> D.
>
> On Mon, Mar 26, 2018 at 11:52 PM, Dmitry Pavlov <dpavlov....@gmail.com>
> wrote:
>
> > Hi Dmitriy, did you mean GridList?
> >
> > I don't understand what does it mean GridGain compress.
> >
> > вт, 27 мар. 2018 г. в 3:06, Dmitriy Setrakyan <dsetrak...@apache.org>:
> >
> > > Thanks, Alex.
> > >
> > > GridGain automatically compresses all the internal types. Somehow it
> > looks
> > > like the GridLongList may have been mixed. Can you please file a ticket
> > for
> > > 2.5 release?
> > >
> > > D.
> > >
> > > On Mon, Mar 26, 2018 at 4:55 AM, Александр Меньшиков <
> > sharple...@gmail.com
> > > >
> > > wrote:
> > >
> > > > I investigated network loading and found that a big part of internal
> > data
> > > > inside messages is `GridLongList`.
> > > > It is a part of `GridDhtTxFinishRequest`,
> > > > `GridDhtAtomicDeferredUpdateResponse`, `GridDhtAtomicUpdateRequest`,
> > > > `GridNearAtomicFullUpdateRequest` and `NearCacheUpdates`.
> > > >
> > > > So I think it has the sense to optimize `GridLongList` serialization.
> > > >
> > > >
> > > > Here we serialize all elements and don't take into account `idx`
> value:
> > > >
> > > > ```
> > > >
> > > > @Override public boolean writeTo(ByteBuffer buf, MessageWriter
> writer)
> > {
> > > >
> > > > writer.setBuffer(buf);
> > > >
> > > >
> > > >
> > > > if (!writer.isHeaderWritten()) {
> > > >
> > > > if (!writer.writeHeader(directType(), fieldsCount()))
> > > >
> > > > return false;
> > > >
> > > >
> > > >
> > > > writer.onHeaderWritten();
> > > >
> > > > }
> > > >
> > > >
> > > >
> > > > switch (writer.state()) {
> > > >
> > > > case 0:
> > > >
> > > > if (!writer.writeLongArray("arr", arr))
> > > >
> > > > return false;
> > > >
> > > >
> > > >
> > > > writer.incrementState();
> > > >
> > > >
> > > >
> > > > case 1:
> > > >
> > > > if (!writer.writeInt("idx", idx))
> > > >
> > > > return false;
> > > >
> > > >
> > > >
> > > > writer.incrementState();
> > > >
> > > >
> > > >
> > > > }
> > > >
> > > >
> > > >
> > > > return true;
> > > >
> > > > }
> > > >
> > > > ```
> > > >
> > > >
> > > >
> > > > Which is not happening in another serialization method in the same
> > class:
> > > >
> > > >
> > > >
> > > > ```
> > > >
> > > > public static void writeTo(DataOutput out, @Nullable GridLongList
> list)
> > > > throws IOException {
> > > >
> > > > out.writeInt(list != null ? list.idx : -1);
> > > >
> > > >
> > > >
> > > > if (list != null) {
> > > >
> > > > for (int i = 0; i < list.idx; i++)
> > > >
> > > > out.writeLong(list.arr[i]);
> > > >
> > > > }
> > > >
> > > > }
> > > >
> > > > ```
> > > >
> > > >
> > > > So, we can simply reduce messages size by sending only a valuable
> part
> > of
> > > > the array.
> > > > If you don't mind I will create an issue in Jira for this.
> > > >
> > > >
> > > > By the way, `long` is a huge type. As I see in most cases
> > `GridLongList`
> > > > uses for counters.
> > > > And I have checked the possibility of compress `long` into smaller
> > types
> > > as
> > > > `int`, `short` or `byte` in test
> > > > `GridCacheInterceptorAtomicRebalanceTest` (took it by random).
> > > > And found out that all `long` in`GridLongList` can be cast to `int`
> and
> > > 70%
> > > > of them to shorts.
> > > > Such conversion is quite fast about 1.1 (ns) per element (I have
> > checked
> > > it
> > > > by JMH test).
> > > >
> > > >
> > > >
> > > > Of course, there are a lot of ways to compress data,
> > > > but I know proprietary GridGain plug-in has different `MessageWriter`
> > > > implementation.
> > > > So maybe it is unnecessary and some compression already exists in
> this
> > > > proprietary plug-in.
> > > > Does someone know something about it?
> > > >
> > >
> >
>
