-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 What reduction in network traffic can we expect from using compression?
- -s On 11/24/2014 10:20 AM, Stephan Ewen wrote: > Will the compression Codec will be inserted in the Netty loops, or > before that? > > In any case, would it make sense to prototype this on the current > code and forward port this to the new network stack later? I assume > the code would mostly be similar, especially all the JNI vs. Java > considerations and tricks. Am 23.11.2014 23:11 schrieb "Ufuk > Celebi" <u...@apache.org>: > >> >> On 23 Nov 2014, at 21:04, Robert Metzger <rmetz...@apache.org> >> wrote: >> >>> Ufuk can probably elaborate on the details, but he told me some >>> time ago that we can add code to compress outgoing network >>> buffers using snappy or other fast compression algorithms. In >>> particular I/O intensive >> applications >>> would benefit from such a change. >> >> I agree and think that compressing shuffle data is generally a >> good idea (given codecs like Snappy et al.) >> >> I've opened a corresponding issue ( >> https://issues.apache.org/jira/browse/FLINK-1275). The network >> layer is going through a set of big changes at the moment. I >> think it makes sense to wait for them to be finished/merged, >> before working on this. >> >> @Viktor: If you want to work on this, feel free to comment on the >> JIRA issue and we can discuss at which points you would have to >> extend the system. >> >> – Ufuk > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJUcvjyAAoJEFWNjCUTnssiCYgQAK5QzsYPW9Q+C8BUJBxfeudg 61zFPW3cfydfD4L6UXghwz/j7addMm5zdY/xJsFgmN4NAXrtacDfr6+KPyB9glZ0 E8E201q8NN6Czek/TtWEG0SSLj+KXqRAwJBHfBCVaCV/lD7wEAjef2sSFpVL5RK9 0RyS5UeCBW/RxTp/fGXWfy2d9G0SSTIy78/u8Y9nxflD6KnZVGZYZA7jrP/h8Wis ry1ZWLPHKpEbaC+MNbYX1mMJaf9lH0liJ/HopCIUzrxfiwskub17NkfVC/0Uwwc3 r1+/pHFYNfO3FLIkXxWYRLgNyGW6LCE1zmM/BGUo1/C9/rtuiRl24io+G+4V/ac6 tZ9jxjwHQkWGFscoep2p7E8pGVezV1QqYv3prWXaZ2+TUKKzlbd6QCL8i2qmymeb ABy88cpN1U4c/p6GDrva1+llyBG1PK07awAfFmK3OA7VK3MLW1pyHuJiLjOZaFP1 r2K9T5/d2YZa7nPFaJs8bLXsV/WMzFJDAOaILcaCDymxF1xnNu7iwAyuJhg/i6ht KaOiM45JnqC++9MQ2D83zyt/+Yc/OT+06hVN7hdRaLFD+OqG7vNv7LZC+cfAWmFE KyJb9kXNVDxhJXAWyOyKN1nzBtDfaugghGBzvX0u3QP/2uGX1Gca2oqCJvNesOM4 dBlCMt88/gkYjWXadjnK =0q6A -----END PGP SIGNATURE-----
