[jira] [Comment Edited] (IGNITE-5097) BinaryMarshaller should write ints in "varint" encoding where it makes sense
[
https://issues.apache.org/jira/browse/IGNITE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089542#comment-16089542
]
Vyacheslav Daradur edited comment on IGNITE-5097 at 7/17/17 9:30 AM:
-
[~vozerov], why do you think that?
As I understand the writing in varint - will be the main approach in future
releases.
May be this task is part of design of a global feature?
BTW: Are there clear ways how to manage via {{BinaryConfiguration}} the static
methods in the {{BinaryUtils}}?
was (Author: daradurvs):
[~vozerov], why do you think that?
As I understand the writing in varint - will be the main approach in future
releases.
May be this task is part of design of a global feature?
BTW: Are there clear ways how to manage via BinaryConfiguration the static
methods in the BinaryUtils?
> BinaryMarshaller should write ints in "varint" encoding where it makes sense
>
>
> Key: IGNITE-5097
> URL: https://issues.apache.org/jira/browse/IGNITE-5097
> Project: Ignite
> Issue Type: Task
> Components: general
>Affects Versions: 2.0
>Reporter: Vladimir Ozerov
>Assignee: Vyacheslav Daradur
> Labels: important, performance
> Fix For: 2.2
>
>
> There are a lot of places in the code where we write integers for some
> special purposes. Quite often their value will be vary small, so that
> applying "varint" format could save a lot of space at the cost of very low
> additional CPU overhead.
> Specifically:
> 1) Array/collection/map lengths
> 2) BigDecimal's (usually will save ~6 bytes)
> 3) Strings
> 4) Enum ordinals
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (IGNITE-5097) BinaryMarshaller should write ints in "varint" encoding where it makes sense
[
https://issues.apache.org/jira/browse/IGNITE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036716#comment-16036716
]
Vyacheslav Daradur edited comment on IGNITE-5097 at 6/5/17 9:31 AM:
[~ptupitsyn],
bq. Not sure though about int<->uint casts.
It is equivalent of {{<<<}} unsigned left bit-shift operator in Java
This is necessary in the case of negative numbers, otherwise it may provide
infinite loop. Covered by added unit test.
bq. 1) Does this work for negative values?
Yes it works for negative values, but we shouldn't use it for, because negative
values always take 5 bytes.
bq. 2) Since we use this thing purely for positive values, should we use uint
in method signature?
Not sure. I'd prefer not to use {{uint}} for compliance with the Java
implementation.
was (Author: daradurvs):
[~ptupitsyn],
bq. Not sure though about int<->uint casts.
It is equivalent of {{>>>}} unsigned right bit-shift operator in Java
This is necessary in the case of negative numbers, otherwise it may provide
infinite loop. Covered by added unit test.
bq. 1) Does this work for negative values?
Yes it works for negative values, but we shouldn't use it for, because negative
values always take 5 bytes.
bq. 2) Since we use this thing purely for positive values, should we use uint
in method signature?
Not sure. I'd prefer not to use {{uint}} for compliance with the Java
implementation.
> BinaryMarshaller should write ints in "varint" encoding where it makes sense
>
>
> Key: IGNITE-5097
> URL: https://issues.apache.org/jira/browse/IGNITE-5097
> Project: Ignite
> Issue Type: Task
> Components: general
>Affects Versions: 2.0
>Reporter: Vladimir Ozerov
>Assignee: Vyacheslav Daradur
> Labels: important, performance
> Fix For: 2.1
>
>
> There are a lot of places in the code where we write integers for some
> special purposes. Quite often their value will be vary small, so that
> applying "varint" format could save a lot of space at the cost of very low
> additional CPU overhead.
> Specifically:
> 1) Array/collection/map lengths
> 2) BigDecimal's (usually will save ~6 bytes)
> 3) Strings
> 4) Enum ordinals
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-5097) BinaryMarshaller should write ints in "varint" encoding where it makes sense
[
https://issues.apache.org/jira/browse/IGNITE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16036716#comment-16036716
]
Vyacheslav Daradur edited comment on IGNITE-5097 at 6/5/17 9:29 AM:
[~ptupitsyn],
bq. Not sure though about int<->uint casts.
It is equivalent of {{>>>}} unsigned right bit-shift operator in Java
This is necessary in the case of negative numbers, otherwise it may provide
infinite loop. Covered by added unit test.
bq. 1) Does this work for negative values?
Yes it works for negative values, but we shouldn't use it for, because negative
values always take 5 bytes.
bq. 2) Since we use this thing purely for positive values, should we use uint
in method signature?
Not sure. I'd prefer not to use {{uint}} for compliance with the Java
implementation.
was (Author: daradurvs):
bq. Not sure though about int<->uint casts.
It is equivalent of {{>>>}} unsigned right bit-shift operator in Java
This is necessary in the case of negative numbers, otherwise it may provide
infinite loop. Covered by added unit test.
bq. 1) Does this work for negative values?
Yes it works for negative values, but we shouldn't use it for, because negative
values always take 5 bytes.
bq. 2) Since we use this thing purely for positive values, should we use uint
in method signature?
Not sure. I'd prefer not to use {{uint}} for compliance with the Java
implementation.
> BinaryMarshaller should write ints in "varint" encoding where it makes sense
>
>
> Key: IGNITE-5097
> URL: https://issues.apache.org/jira/browse/IGNITE-5097
> Project: Ignite
> Issue Type: Task
> Components: general
>Affects Versions: 2.0
>Reporter: Vladimir Ozerov
>Assignee: Vyacheslav Daradur
> Labels: important, performance
> Fix For: 2.1
>
>
> There are a lot of places in the code where we write integers for some
> special purposes. Quite often their value will be vary small, so that
> applying "varint" format could save a lot of space at the cost of very low
> additional CPU overhead.
> Specifically:
> 1) Array/collection/map lengths
> 2) BigDecimal's (usually will save ~6 bytes)
> 3) Strings
> 4) Enum ordinals
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-5097) BinaryMarshaller should write ints in "varint" encoding where it makes sense
[
https://issues.apache.org/jira/browse/IGNITE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012325#comment-16012325
]
Igor Sapego edited comment on IGNITE-5097 at 5/16/17 1:07 PM:
--
Of course, we can still write length in varint, always using 4 bytes, often
leaving first 3 bytes to be {{0x80}}. I'm just not sure, if this makes sense.
If you consider that collection mostly containing objects, it means that most
collections would take pretty much memory (single object header is 24 bytes
long). So, does 3 at most bytes really make decent traffic saving in case of
collections?
was (Author: isapego):
Of course, we can still write length in varint, always using 4 bytes, often
leaving first 3 bytes to be {{0x80}}. I'm just not sure, if this makes sense.
If you consider that collection mostly containing objects, it means that most
collections would take pretty much memory (single object header is 24 bytes
long). So, does 3 at most bytes really make decent traffic economy in case of
collections?
> BinaryMarshaller should write ints in "varint" encoding where it makes sense
>
>
> Key: IGNITE-5097
> URL: https://issues.apache.org/jira/browse/IGNITE-5097
> Project: Ignite
> Issue Type: Task
> Components: general
>Affects Versions: 2.0
>Reporter: Vladimir Ozerov
>Assignee: Vyacheslav Daradur
> Labels: important, performance
> Fix For: 2.1
>
>
> There are a lot of places in the code where we write integers for some
> special purposes. Quite often their value will be vary small, so that
> applying "varint" format could save a lot of space at the cost of very low
> additional CPU overhead.
> Specifically:
> 1) Array/collection/map lengths
> 2) BigDecimal's (usually will save ~6 bytes)
> 3) Strings
> 4) Enum ordinals
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (IGNITE-5097) BinaryMarshaller should write ints in "varint" encoding where it makes sense
[ https://issues.apache.org/jira/browse/IGNITE-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16010700#comment-16010700 ] Pavel Tupitsyn edited comment on IGNITE-5097 at 5/15/17 3:27 PM: - [~isapego] how is it solved in Java? Do we always know collection size before hand? .NET will have the same problem in *some* cases (lazy collections). I guess we shoud just use varlen when possible, and reserve 4 bytes otherwise. was (Author: ptupitsyn): [~isapego] how is it solved in Java? Do we always know collection size before hand? .NET will have the same problem in *some* cases. I guess we shoud just use varlen when possible, and reserve 4 bytes otherwise. > BinaryMarshaller should write ints in "varint" encoding where it makes sense > > > Key: IGNITE-5097 > URL: https://issues.apache.org/jira/browse/IGNITE-5097 > Project: Ignite > Issue Type: Task > Components: general >Affects Versions: 2.0 >Reporter: Vladimir Ozerov >Assignee: Vyacheslav Daradur > Labels: important, performance > Fix For: 2.1 > > > There are a lot of places in the code where we write integers for some > special purposes. Quite often their value will be vary small, so that > applying "varint" format could save a lot of space at the cost of very low > additional CPU overhead. > Specifically: > 1) Array/collection/map lengths > 2) BigDecimal's (usually will save ~6 bytes) > 3) Strings > 4) Enum ordinals -- This message was sent by Atlassian JIRA (v6.3.15#6346)
