Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15454 )
Change subject: IMPALA-3766: optionally compress spilled data ...................................................................... Patch Set 14: (8 comments) Thanks for the feedback, sorry to push out a rebase, I hope it isn't too bad looking at the PS13->14 diff http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/bufferpool/buffer-pool-counters.h File be/src/runtime/bufferpool/buffer-pool-counters.h: http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/bufferpool/buffer-pool-counters.h@55 PS11, Line 55: g for writes to di > nit: Total bytes written to disk. (May be compressed) thanks, that's less ambiguous http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/bufferpool/buffer-pool.cc File be/src/runtime/bufferpool/buffer-pool.cc: http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/bufferpool/buffer-pool.cc@736 PS11, Line 736: [this, page]( > if the write op fails then write_io_ops will not be incremented whereas pre It wasn't an intentional change, but I think it is an edge case that doesn't really matter (if the query failed, you probably don't care about about the exact number of writes). http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/tmp-file-mgr.h File be/src/runtime/tmp-file-mgr.h: http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/tmp-file-mgr.h@161 PS11, Line 161: using 4kb or smaller > nit: HOLE_PUNCH_BLOCK_SIZE_BYTES Done http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/tmp-file-mgr.h@180 PS11, Line 180: /// compression is used. > nit: although its obvious but maybe mention that -1 means compression_level Done http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/tmp-file-mgr.h@291 PS11, Line 291: // called once. Must be called with 'lock_' held. : Status CreateFiles() WARN_UNUSED > update comment thanks for catching http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/tmp-file-mgr.h@482 PS11, Line 482: return be > is this before or after compression? Done http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/tmp-file-mgr.cc File be/src/runtime/tmp-file-mgr.cc: http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/tmp-file-mgr.cc@65 PS11, Line 65: "(Advanced) Limit on the total bytes of compression buffers that will be used for " > nit:maybe mention that this limit is shared across all queries. Good point, updated. http://gerrit.cloudera.org:8080/#/c/15454/11/be/src/runtime/tmp-file-mgr.cc@454 PS11, Line 454: int64_t num_bytes, TmpFile** tmp_file, int64_t* file_offset) { : lock_guard<SpinLock> lock(lock_); : int64_t scratch_range_bytes = : > shouldn't this check be done after the free_ranges_ recycle underneath? if Yes, i'm not sure what I was thinking when I changed this. -- To view, visit http://gerrit.cloudera.org:8080/15454 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I9c08ff9504097f0fee8c32316c5c150136abe659 Gerrit-Change-Number: 15454 Gerrit-PatchSet: 14 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Sahil Takiar <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Thu, 26 Mar 2020 03:35:43 +0000 Gerrit-HasComments: Yes
