Yes, that working on serialized data happens in parts right now and it would be great to extend that.
While it would be possible to work on a compact serialized representation, I can't think of a way to work on a snappy/lz4 compressed version. Am 26.11.2014 23:00 schrieb "Viktor Rosenfeld" < viktor.rosenf...@tu-berlin.de>: > Hi Stephan, > > Compressing network buffers as a blob is probably the fastest/easiest way > to > achieve some measure of results. > > But I wonder if it would be possible to implement some operations on the > serialized compressed form. For example, groupings and joins should be easy > to implement. If the field is not accessed in the UDF then it won't have to > be deserialized. > > A complication would be how the compression scheme is passed on to the next > nodes in the computation chain. > > Cheers, > Viktor > > > > -- > View this message in context: > http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Compression-of-network-traffic-tp2568p2637.html > Sent from the Apache Flink (Incubator) Mailing List archive. mailing list > archive at Nabble.com. >
