Vladimir, I am not sure how to interpret the graphs? What are we looking at?
On Tue, Jun 6, 2017 at 12:33 AM, Vyacheslav Daradur <daradu...@gmail.com> wrote: > Hi, Igniters. > > I've prepared some benchmarking. Results [1]. > > And I've prepared the evaluation in the form of diagrams [2]. > > I hope that helps to interest the community and accelerates a reaction to > this improvment :) > > [1] > https://github.com/daradurvs/ignite-compression/tree/ > master/src/main/resources/result > [2] https://drive.google.com/file/d/0B2CeUAOgrHkoMklyZ25YTEdKcEk/view > > > > 2017-05-24 9:49 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > > > Guys, any thoughts? > > > > 2017-05-16 13:40 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > > > >> Hi guys, > >> > >> I've prepared the PR to show my idea. > >> https://github.com/apache/ignite/pull/1951/files > >> > >> About querying - I've just copied existing tests and have annotated the > >> testing data. > >> https://github.com/apache/ignite/pull/1951/files#diff-c19a9d > >> f4058141d059bb577e75244764 > >> > >> It means fields which will be marked by @BinaryCompression will be > >> compressed at marshalling via BinaryMarshaller. > >> > >> This solution has no effect on existing data or project architecture. > >> > >> I'll be glad to see your thougths. > >> > >> > >> 2017-05-15 19:18 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com>: > >> > >>> Dmitriy, > >>> > >>> I have ready prototype. I want to show it. > >>> It is always easier to discuss on example. > >>> > >>> 2017-05-15 19:02 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>: > >>> > >>>> Vyacheslav, > >>>> > >>>> I think it is a bit premature to provide a PR without getting a > >>>> community > >>>> consensus on the dev list. Please allow some time for the community to > >>>> respond. > >>>> > >>>> D. > >>>> > >>>> On Mon, May 15, 2017 at 6:36 AM, Vyacheslav Daradur < > >>>> daradu...@gmail.com> > >>>> wrote: > >>>> > >>>> > I created the ticket: https://issues.apache.org/jira > >>>> /browse/IGNITE-5226 > >>>> > > >>>> > I'll prepare a PR with described solution in couple of days. > >>>> > > >>>> > 2017-05-15 15:05 GMT+03:00 Vyacheslav Daradur <daradu...@gmail.com > >: > >>>> > > >>>> > > Hi, Igniters! > >>>> > > > >>>> > > Apache 2.0 is released. > >>>> > > > >>>> > > Let's continue the discussion about a compression design. > >>>> > > > >>>> > > At the moment, I found only one solution which is compatible with > >>>> > querying > >>>> > > and indexing, this is per-objects-field compression. > >>>> > > Per-fields compression means that metadata (a header) of an object > >>>> won't > >>>> > > be compressed, only serialized values of an object fields (in > bytes > >>>> array > >>>> > > form) will be compressed. > >>>> > > > >>>> > > This solution have some contentious issues: > >>>> > > - small values, like primitives and short arrays - there isn't > >>>> sense to > >>>> > > compress them; > >>>> > > - there is no possible to use compression with java-predefined > >>>> types; > >>>> > > > >>>> > > We can provide an annotation, @IgniteCompression - for example, > >>>> which can > >>>> > > be used by users for marking fields to compress. > >>>> > > > >>>> > > Any thoughts? > >>>> > > > >>>> > > Maybe someone already have ready design? > >>>> > > > >>>> > > 2017-04-10 11:06 GMT+03:00 Vyacheslav Daradur < > daradu...@gmail.com > >>>> >: > >>>> > > > >>>> > >> Alexey, > >>>> > >> > >>>> > >> Yes, I've read it. > >>>> > >> > >>>> > >> Ok, let's discuss about public API design. > >>>> > >> > >>>> > >> I think we need to add some a configure entity to > >>>> CacheConfiguration, > >>>> > >> which will contain the Compressor interface implementation and > some > >>>> > usefull > >>>> > >> parameters. > >>>> > >> Or maybe to provide a BinaryMarshaller decorator, which will be > >>>> compress > >>>> > >> data after marshalling. > >>>> > >> > >>>> > >> > >>>> > >> 2017-04-10 10:40 GMT+03:00 Alexey Kuznetsov < > akuznet...@apache.org > >>>> >: > >>>> > >> > >>>> > >>> Vyacheslav, > >>>> > >>> > >>>> > >>> Did you read initial discussion [1] about compression? > >>>> > >>> As far as I remember we agreed to add only some "top-level" API > in > >>>> > order > >>>> > >>> to > >>>> > >>> provide a way for > >>>> > >>> Ignite users to inject some sort of custom compression. > >>>> > >>> > >>>> > >>> > >>>> > >>> [1] > >>>> > >>> http://apache-ignite-developers.2346864.n4.nabble.com/Data-c > >>>> > >>> ompression-in-Ignite-2-0-td10099.html > >>>> > >>> > >>>> > >>> On Mon, Apr 10, 2017 at 2:19 PM, daradurvs <daradu...@gmail.com > > > >>>> > wrote: > >>>> > >>> > >>>> > >>> > Hi Igniters! > >>>> > >>> > > >>>> > >>> > I am interested in this task. > >>>> > >>> > Provide some kind of pluggable compression SPI support > >>>> > >>> > <https://issues.apache.org/jira/browse/IGNITE-3592> > >>>> > >>> > > >>>> > >>> > I developed a solution on BinaryMarshaller-level, but reviewer > >>>> has > >>>> > >>> rejected > >>>> > >>> > it. > >>>> > >>> > > >>>> > >>> > Let's continue discussion of task goals and solution design. > >>>> > >>> > As I understood that, the main goal of this task is to store > >>>> data in > >>>> > >>> > compressed form. > >>>> > >>> > This is what I need from Ignite as its user. Compression > >>>> provides > >>>> > >>> economy > >>>> > >>> > on > >>>> > >>> > servers. > >>>> > >>> > We can store more data on same servers at the cost of > >>>> increasing CPU > >>>> > >>> > utilization. > >>>> > >>> > > >>>> > >>> > I'm researching a possibility of implementation of compression > >>>> at the > >>>> > >>> > cache-level. > >>>> > >>> > > >>>> > >>> > Any thoughts? > >>>> > >>> > > >>>> > >>> > -- > >>>> > >>> > Best regards, > >>>> > >>> > Vyacheslav > >>>> > >>> > > >>>> > >>> > > >>>> > >>> > > >>>> > >>> > > >>>> > >>> > -- > >>>> > >>> > View this message in context: http://apache-ignite- > >>>> > >>> > developers.2346864.n4.nabble.com/Data-compression-in- > >>>> > >>> > Ignite-2-0-tp10099p16317.html > >>>> > >>> > Sent from the Apache Ignite Developers mailing list archive at > >>>> > >>> Nabble.com. > >>>> > >>> > > >>>> > >>> > >>>> > >>> > >>>> > >>> > >>>> > >>> -- > >>>> > >>> Alexey Kuznetsov > >>>> > >>> > >>>> > >> > >>>> > >> > >>>> > >> > >>>> > >> -- > >>>> > >> Best Regards, Vyacheslav > >>>> > >> > >>>> > > > >>>> > > > >>>> > > > >>>> > > -- > >>>> > > Best Regards, Vyacheslav > >>>> > > > >>>> > > >>>> > > >>>> > > >>>> > -- > >>>> > Best Regards, Vyacheslav > >>>> > > >>>> > >>> > >>> > >>> > >>> -- > >>> Best Regards, Vyacheslav > >>> > >> > >> > >> > >> -- > >> Best Regards, Vyacheslav > >> > > > > > > > > -- > > Best Regards, Vyacheslav > > > > > > -- > Best Regards, Vyacheslav >
