Hi Dmitriy, did you mean GridList?
I don't understand what does it mean GridGain compress.
вт, 27 мар. 2018 г. в 3:06, Dmitriy Setrakyan <dsetrak...@apache.org>:
> Thanks, Alex.
>
> GridGain automatically compresses all the internal types. Somehow it looks
> like the GridLongList may have been mixed. Can you please file a ticket for
> 2.5 release?
>
> D.
>
> On Mon, Mar 26, 2018 at 4:55 AM, Александр Меньшиков <sharple...@gmail.com
> >
> wrote:
>
> > I investigated network loading and found that a big part of internal data
> > inside messages is `GridLongList`.
> > It is a part of `GridDhtTxFinishRequest`,
> > `GridDhtAtomicDeferredUpdateResponse`, `GridDhtAtomicUpdateRequest`,
> > `GridNearAtomicFullUpdateRequest` and `NearCacheUpdates`.
> >
> > So I think it has the sense to optimize `GridLongList` serialization.
> >
> >
> > Here we serialize all elements and don't take into account `idx` value:
> >
> > ```
> >
> > @Override public boolean writeTo(ByteBuffer buf, MessageWriter writer) {
> >
> > writer.setBuffer(buf);
> >
> >
> >
> > if (!writer.isHeaderWritten()) {
> >
> > if (!writer.writeHeader(directType(), fieldsCount()))
> >
> > return false;
> >
> >
> >
> > writer.onHeaderWritten();
> >
> > }
> >
> >
> >
> > switch (writer.state()) {
> >
> > case 0:
> >
> > if (!writer.writeLongArray("arr", arr))
> >
> > return false;
> >
> >
> >
> > writer.incrementState();
> >
> >
> >
> > case 1:
> >
> > if (!writer.writeInt("idx", idx))
> >
> > return false;
> >
> >
> >
> > writer.incrementState();
> >
> >
> >
> > }
> >
> >
> >
> > return true;
> >
> > }
> >
> > ```
> >
> >
> >
> > Which is not happening in another serialization method in the same class:
> >
> >
> >
> > ```
> >
> > public static void writeTo(DataOutput out, @Nullable GridLongList list)
> > throws IOException {
> >
> > out.writeInt(list != null ? list.idx : -1);
> >
> >
> >
> > if (list != null) {
> >
> > for (int i = 0; i < list.idx; i++)
> >
> > out.writeLong(list.arr[i]);
> >
> > }
> >
> > }
> >
> > ```
> >
> >
> > So, we can simply reduce messages size by sending only a valuable part of
> > the array.
> > If you don't mind I will create an issue in Jira for this.
> >
> >
> > By the way, `long` is a huge type. As I see in most cases `GridLongList`
> > uses for counters.
> > And I have checked the possibility of compress `long` into smaller types
> as
> > `int`, `short` or `byte` in test
> > `GridCacheInterceptorAtomicRebalanceTest` (took it by random).
> > And found out that all `long` in`GridLongList` can be cast to `int` and
> 70%
> > of them to shorts.
> > Such conversion is quite fast about 1.1 (ns) per element (I have checked
> it
> > by JMH test).
> >
> >
> >
> > Of course, there are a lot of ways to compress data,
> > but I know proprietary GridGain plug-in has different `MessageWriter`
> > implementation.
> > So maybe it is unnecessary and some compression already exists in this
> > proprietary plug-in.
> > Does someone know something about it?
> >
>
