Guys, any thoughts? 2017-05-16 13:40 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>:
> Hi guys, > > I've prepared the PR to show my idea. > https://github.com/apache/ignite/pull/1951/files > > About querying - I've just copied existing tests and have annotated the > testing data. > https://github.com/apache/ignite/pull/1951/files#diff- > c19a9df4058141d059bb577e75244764 > > It means fields which will be marked by @BinaryCompression will be > compressed at marshalling via BinaryMarshaller. > > This solution has no effect on existing data or project architecture. > > I'll be glad to see your thougths. > > > 2017-05-15 19:18 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > >> Dmitriy, >> >> I have ready prototype. I want to show it. >> It is always easier to discuss on example. >> >> 2017-05-15 19:02 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>: >> >>> Vyacheslav, >>> >>> I think it is a bit premature to provide a PR without getting a community >>> consensus on the dev list. Please allow some time for the community to >>> respond. >>> >>> D. >>> >>> On Mon, May 15, 2017 at 6:36 AM, Vyacheslav Daradur <daradu...@gmail.com >>> > >>> wrote: >>> >>> > I created the ticket: https://issues.apache.org/jira >>> /browse/IGNITE-5226 >>> > >>> > I'll prepare a PR with described solution in couple of days. >>> > >>> > 2017-05-15 15:05 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: >>> > >>> > > Hi, Igniters! >>> > > >>> > > Apache 2.0 is released. >>> > > >>> > > Let's continue the discussion about a compression design. >>> > > >>> > > At the moment, I found only one solution which is compatible with >>> > querying >>> > > and indexing, this is per-objects-field compression. >>> > > Per-fields compression means that metadata (a header) of an object >>> won't >>> > > be compressed, only serialized values of an object fields (in bytes >>> array >>> > > form) will be compressed. >>> > > >>> > > This solution have some contentious issues: >>> > > - small values, like primitives and short arrays - there isn't sense >>> to >>> > > compress them; >>> > > - there is no possible to use compression with java-predefined types; >>> > > >>> > > We can provide an annotation, @IgniteCompression - for example, >>> which can >>> > > be used by users for marking fields to compress. >>> > > >>> > > Any thoughts? >>> > > >>> > > Maybe someone already have ready design? >>> > > >>> > > 2017-04-10 11:06 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: >>> > > >>> > >> Alexey, >>> > >> >>> > >> Yes, I've read it. >>> > >> >>> > >> Ok, let's discuss about public API design. >>> > >> >>> > >> I think we need to add some a configure entity to >>> CacheConfiguration, >>> > >> which will contain the Compressor interface implementation and some >>> > usefull >>> > >> parameters. >>> > >> Or maybe to provide a BinaryMarshaller decorator, which will be >>> compress >>> > >> data after marshalling. >>> > >> >>> > >> >>> > >> 2017-04-10 10:40 GMT+03:00 Alexey Kuznetsov <akuznet...@apache.org >>> >: >>> > >> >>> > >>> Vyacheslav, >>> > >>> >>> > >>> Did you read initial discussion [1] about compression? >>> > >>> As far as I remember we agreed to add only some "top-level" API in >>> > order >>> > >>> to >>> > >>> provide a way for >>> > >>> Ignite users to inject some sort of custom compression. >>> > >>> >>> > >>> >>> > >>> [1] >>> > >>> http://apache-ignite-developers.2346864.n4.nabble.com/Data-c >>> > >>> ompression-in-Ignite-2-0-td10099.html >>> > >>> >>> > >>> On Mon, Apr 10, 2017 at 2:19 PM, daradurvs <daradu...@gmail.com> >>> > wrote: >>> > >>> >>> > >>> > Hi Igniters! >>> > >>> > >>> > >>> > I am interested in this task. >>> > >>> > Provide some kind of pluggable compression SPI support >>> > >>> > <https://issues.apache.org/jira/browse/IGNITE-3592> >>> > >>> > >>> > >>> > I developed a solution on BinaryMarshaller-level, but reviewer >>> has >>> > >>> rejected >>> > >>> > it. >>> > >>> > >>> > >>> > Let's continue discussion of task goals and solution design. >>> > >>> > As I understood that, the main goal of this task is to store >>> data in >>> > >>> > compressed form. >>> > >>> > This is what I need from Ignite as its user. Compression provides >>> > >>> economy >>> > >>> > on >>> > >>> > servers. >>> > >>> > We can store more data on same servers at the cost of increasing >>> CPU >>> > >>> > utilization. >>> > >>> > >>> > >>> > I'm researching a possibility of implementation of compression >>> at the >>> > >>> > cache-level. >>> > >>> > >>> > >>> > Any thoughts? >>> > >>> > >>> > >>> > -- >>> > >>> > Best regards, >>> > >>> > Vyacheslav >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > -- >>> > >>> > View this message in context: http://apache-ignite- >>> > >>> > developers.2346864.n4.nabble.com/Data-compression-in- >>> > >>> > Ignite-2-0-tp10099p16317.html >>> > >>> > Sent from the Apache Ignite Developers mailing list archive at >>> > >>> Nabble.com. >>> > >>> > >>> > >>> >>> > >>> >>> > >>> >>> > >>> -- >>> > >>> Alexey Kuznetsov >>> > >>> >>> > >> >>> > >> >>> > >> >>> > >> -- >>> > >> Best Regards, Vyacheslav >>> > >> >>> > > >>> > > >>> > > >>> > > -- >>> > > Best Regards, Vyacheslav >>> > > >>> > >>> > >>> > >>> > -- >>> > Best Regards, Vyacheslav >>> > >>> >> >> >> >> -- >> Best Regards, Vyacheslav >> > > > > -- > Best Regards, Vyacheslav > -- Best Regards, Vyacheslav
