Hi guys, I've prepared the PR to show my idea. https://github.com/apache/ignite/pull/1951/files
About querying - I've just copied existing tests and have annotated the testing data. https://github.com/apache/ignite/pull/1951/files#diff-c19a9df4058141d059bb577e75244764 It means fields which will be marked by @BinaryCompression will be compressed at marshalling via BinaryMarshaller. This solution has no effect on existing data or project architecture. I'll be glad to see your thougths. 2017-05-15 19:18 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > Dmitriy, > > I have ready prototype. I want to show it. > It is always easier to discuss on example. > > 2017-05-15 19:02 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>: > >> Vyacheslav, >> >> I think it is a bit premature to provide a PR without getting a community >> consensus on the dev list. Please allow some time for the community to >> respond. >> >> D. >> >> On Mon, May 15, 2017 at 6:36 AM, Vyacheslav Daradur <daradu...@gmail.com> >> wrote: >> >> > I created the ticket: https://issues.apache.org/jira/browse/IGNITE-5226 >> > >> > I'll prepare a PR with described solution in couple of days. >> > >> > 2017-05-15 15:05 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: >> > >> > > Hi, Igniters! >> > > >> > > Apache 2.0 is released. >> > > >> > > Let's continue the discussion about a compression design. >> > > >> > > At the moment, I found only one solution which is compatible with >> > querying >> > > and indexing, this is per-objects-field compression. >> > > Per-fields compression means that metadata (a header) of an object >> won't >> > > be compressed, only serialized values of an object fields (in bytes >> array >> > > form) will be compressed. >> > > >> > > This solution have some contentious issues: >> > > - small values, like primitives and short arrays - there isn't sense >> to >> > > compress them; >> > > - there is no possible to use compression with java-predefined types; >> > > >> > > We can provide an annotation, @IgniteCompression - for example, which >> can >> > > be used by users for marking fields to compress. >> > > >> > > Any thoughts? >> > > >> > > Maybe someone already have ready design? >> > > >> > > 2017-04-10 11:06 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: >> > > >> > >> Alexey, >> > >> >> > >> Yes, I've read it. >> > >> >> > >> Ok, let's discuss about public API design. >> > >> >> > >> I think we need to add some a configure entity to CacheConfiguration, >> > >> which will contain the Compressor interface implementation and some >> > usefull >> > >> parameters. >> > >> Or maybe to provide a BinaryMarshaller decorator, which will be >> compress >> > >> data after marshalling. >> > >> >> > >> >> > >> 2017-04-10 10:40 GMT+03:00 Alexey Kuznetsov <akuznet...@apache.org>: >> > >> >> > >>> Vyacheslav, >> > >>> >> > >>> Did you read initial discussion [1] about compression? >> > >>> As far as I remember we agreed to add only some "top-level" API in >> > order >> > >>> to >> > >>> provide a way for >> > >>> Ignite users to inject some sort of custom compression. >> > >>> >> > >>> >> > >>> [1] >> > >>> http://apache-ignite-developers.2346864.n4.nabble.com/Data-c >> > >>> ompression-in-Ignite-2-0-td10099.html >> > >>> >> > >>> On Mon, Apr 10, 2017 at 2:19 PM, daradurvs <daradu...@gmail.com> >> > wrote: >> > >>> >> > >>> > Hi Igniters! >> > >>> > >> > >>> > I am interested in this task. >> > >>> > Provide some kind of pluggable compression SPI support >> > >>> > <https://issues.apache.org/jira/browse/IGNITE-3592> >> > >>> > >> > >>> > I developed a solution on BinaryMarshaller-level, but reviewer has >> > >>> rejected >> > >>> > it. >> > >>> > >> > >>> > Let's continue discussion of task goals and solution design. >> > >>> > As I understood that, the main goal of this task is to store data >> in >> > >>> > compressed form. >> > >>> > This is what I need from Ignite as its user. Compression provides >> > >>> economy >> > >>> > on >> > >>> > servers. >> > >>> > We can store more data on same servers at the cost of increasing >> CPU >> > >>> > utilization. >> > >>> > >> > >>> > I'm researching a possibility of implementation of compression at >> the >> > >>> > cache-level. >> > >>> > >> > >>> > Any thoughts? >> > >>> > >> > >>> > -- >> > >>> > Best regards, >> > >>> > Vyacheslav >> > >>> > >> > >>> > >> > >>> > >> > >>> > >> > >>> > -- >> > >>> > View this message in context: http://apache-ignite- >> > >>> > developers.2346864.n4.nabble.com/Data-compression-in- >> > >>> > Ignite-2-0-tp10099p16317.html >> > >>> > Sent from the Apache Ignite Developers mailing list archive at >> > >>> Nabble.com. >> > >>> > >> > >>> >> > >>> >> > >>> >> > >>> -- >> > >>> Alexey Kuznetsov >> > >>> >> > >> >> > >> >> > >> >> > >> -- >> > >> Best Regards, Vyacheslav >> > >> >> > > >> > > >> > > >> > > -- >> > > Best Regards, Vyacheslav >> > > >> > >> > >> > >> > -- >> > Best Regards, Vyacheslav >> > >> > > > > -- > Best Regards, Vyacheslav > -- Best Regards, Vyacheslav
