Tim Armstrong has posted comments on this change. Change subject: IMPALA-3766: Applying LZ4 compression on buffers before spilling ......................................................................
Patch Set 1: It would be good to get some numbers on how this compares to the uncompressed version in terms of bytes written to disk and end-to-end performance, I think we should get an idea of the gains before doing the detailed review. * Total uncompressed bytes versus compressed bytes written - you could add an extra profile counter for this since we already have bytes_written_counter_. * End-to-end query time on a release build. * Disk space saved. This is a little tricky since we allocate 8mb file ranges, but don't write the whole 8mb. The files will be sparse so we will save disk space, but you need to do something like this to see it: http://extrabright.com/blog/2010/03/30/how-to-know-if-a-file-on-linux-is-sparse/ -- To view, visit http://gerrit.cloudera.org:8080/3478 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I4d49bd8d6d7643c84cefd1274c18b52907ca1488 Gerrit-PatchSet: 1 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: anujphadke <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: anujphadke <[email protected]> Gerrit-HasComments: No
