Good catch Steve ! I also think about that.
Ticket unassigned — you can proceed with it.
Also you need to ask your jira account for granting rights for further ticket
assignation.
High level review can make i (@zstan) and probably (@ivandasch).
Also you need to explain how to run new (ignite-6499 fix consistent code) with
old base.
see u !
>
>>
>>>Hello igniters,
>>>
>>>while I would like to help on the calcite because H2 optimiser (or the lack
>>>thereof) is really killing us, I think that it would be wiser to start by
>>>contributing on something easier.
>>>
>>>Therefore I will tackle another problem that we have which is the memory
>>>consumption. I stumbled upon this IEP
>>>https://cwiki.apache.org/confluence/display/IGNITE/IEP-2%3A+Binary+object+format+improvements
>>><
>>>https://cwiki.apache.org/confluence/display/IGNITE/IEP-2%3A+Binary+object+format+improvements
>>> >
>>>that is about optimising the binary marshaller.
>>>
>>>The low hanging fruit seemed to be the null compaction so I decided to start
>>>with it. Though I am sure I do see some hidden complexity.
>>>
>>>Here a couple of questions:
>>>- Can I assign myself IGNITE-6499 and attach a patch?
>>>- Who can I contact to help with the review. In the following page
>>>https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>>>< https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute >
>>>there is no one assigned for marshalling.
>>>
>>>On the details:
>>>The compression is disabled by default as it is not compatible with objects
>>>previously marshalled.
>>>
>>>My approach was to go a bit beyond the JIRA. No only do I remove the indexes
>>>to null fields in the footer, I also remove the 0x65 in the objects. I did
>>>not remove them fro the collections and arrays because they are using
>>>absolute positioning.
>>>
>>>I gain between 5% to 20% depending of my test cases. Obviously the smaller
>>>the object and the higher the number of nulls, the higher the compression
>>>rate.
>>>
>>>Based on that I can quite easily add var int compression which is
>>>IGNITE-6418 and should significantly increase the compression rate with a
>>>lot of integers and longs when only using small numbers.
>>>
>>>Next step is to add JMH micro-benchmark to check the impact in terms of
>>>performances.
>>>
>>>
>>>Example on a simple object w/ null compaction
>>>
>>>Length=55 FooterPosition=50
>>>0x67 // ValueType
>>>0x01 // FormatVersion
>>>0x2b 0x00 //Flags userType=true hasSchema=true offset=1 compactFooter=true
>>>0x78 0x66 0xbe 0x44 //TypeId
>>>0xf9 0xcd 0x07 0x57 //Hashcode
>>>0x37 0x00 0x00 0x00 //Length
>>>0x3d 0xa8 0x15 0xe4 //SchemaId
>>>0x32 0x00 0x00 0x00 //Footer position = 50
>>>0x03 0x01 0x00 0x00 0x00 0x03 0x01 0x00 0x00 0x00 0x09 0x03 0x00 0x00 0x00
>>>0x61 0x62 0x63 0x09 0x03 0x00 0x00 0x00 0x61 0x62 0x63
>>>Footer length=5
>>>0x18 0x1d 0x22 0x2a 0x47
>>>
>>>and w/o null compaction
>>>Length=60 FooterPosition=53
>>>0x67 // ValueType
>>>0x01 // FormatVersion
>>>0x2b 0x00 //Flags userType=true hasSchema=true offset=1 compactFooter=true
>>>0x78 0x66 0xbe 0x44 //TypeId
>>>0xa4 0x43 0x0e 0xf5 //Hashcode
>>>0x3c 0x00 0x00 0x00 //Length
>>>0x3d 0xa8 0x15 0xe4 //SchemaId
>>>0x35 0x00 0x00 0x00 //Footer position = 53
>>>0x03 0x01 0x00 0x00 0x00 0x03 0x01 0x00 0x00 0x00 0x09 0x03 0x00 0x00 0x00
>>>0x61 0x62 0x63 0x65 0x65 0x65 0x09 0x03 0x00 0x00 0x00 0x61 0x62 0x63
>>>Footer length=7
>>>0x18 0x1d 0x22 0x2a 0x2b 0x2c 0x2d
>>>
>>>
>>>
>>>
>>>--
>>>Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>
>>
>>
>>
