Noemi Pap-Takacs has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/18824 )
Change subject: IMPALA-11477: Adding Codegen to sorted-run-merger ...................................................................... IMPALA-11477: Adding Codegen to sorted-run-merger SortedRunMerger is used to merge multiple, already sorted runs. It is used for external merge in the sorter (SortNode), and in KRPC data stream receiver (ExchangeNode). SortedRunMerger builds and maintains a min heap of the sorted input runs. Rewrote SortedRunMerger::Heapify from recursive to iterative and moved to a separate new source file: sorted-run-merger-ir.cc. Added a static Codegen() to SortedRunMerger and call it from the corresponding ExecNodes: SortNode and ExchangeNode. This change lets the merger use the codegened version of TupleRowComparator instead of the interpreted one, which can increase the speed, especially in case of complex comparison expressions. This change also serves as a base for further codegen-related optimizations in the merger. Testing: - run existing E2E sort tests (test-sort.py) - manual testing: run queries that instantiate sort nodes and merging exchange nodes Benchmarking: - did not cause regression on TPCH query set - made merge-intensive queries and IMPALA-4530 (in-memory merge of quicksorted small runs) faster Change-Id: Ic35c7460bdbd54b8ec5872a83680e2f41ceae9fd --- M be/src/codegen/gen_ir_descriptions.py M be/src/codegen/impala-ir.cc M be/src/exec/exchange-node.cc M be/src/exec/exchange-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/CMakeLists.txt M be/src/runtime/data-stream-test.cc M be/src/runtime/krpc-data-stream-recvr.cc M be/src/runtime/krpc-data-stream-recvr.h A be/src/runtime/sorted-run-merger-ir.cc M be/src/runtime/sorted-run-merger.cc M be/src/runtime/sorted-run-merger.h M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 15 files changed, 254 insertions(+), 106 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/24/18824/11 -- To view, visit http://gerrit.cloudera.org:8080/18824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic35c7460bdbd54b8ec5872a83680e2f41ceae9fd Gerrit-Change-Number: 18824 Gerrit-PatchSet: 11 Gerrit-Owner: Noemi Pap-Takacs <npaptak...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Noemi Pap-Takacs <npaptak...@cloudera.com>
