Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/9943 )
Change subject: IMPALA-5706: Parallelise read I/O in sorter ...................................................................... Patch Set 5: Code-Review+1 (3 comments) The patch looks ok to me, just a few comments about comments. http://gerrit.cloudera.org:8080/#/c/9943/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/9943/3//COMMIT_MSG@14 PS3, Line 14: - Remove the hard-coded maximum limit of buffers that can be used : for merging the sorted runs. Instead this number is calculated : based on the available memory through buffer pool. > I gave this a thought after we discussed this offline and I believe that th Thanks for digging into this! http://gerrit.cloudera.org:8080/#/c/9943/3/be/src/runtime/sorter.cc File be/src/runtime/sorter.cc: http://gerrit.cloudera.org:8080/#/c/9943/3/be/src/runtime/sorter.cc@1039 PS3, Line 1039: > The description of is_pinned_ says that "True if all pages in the run are p You don't have to go into details here, but I still think that the current comment is a bit confusing, because the 'next_page' must be pinned in every case, not just in this if block. http://gerrit.cloudera.org:8080/#/c/9943/5/tests/query_test/test_sort.py File tests/query_test/test_sort.py: http://gerrit.cloudera.org:8080/#/c/9943/5/tests/query_test/test_sort.py@137 PS5, Line 137: sort1_profile = self.get_sort_node_profile(1, result.runtime_profile) : assert(sort1_profile.find("TotalMergesPerformed: 7") > 0) : : sort2_profile = self.get_sort_node_profile(3,result.runtime_profile) : assert(sort2_profile.find("TotalMergesPerformed: 2") > 0) Can you add a short explanation about the cause of the difference between the two 'TotalMergesPerformed'? It may be hard to interpret them if someone does not know the full story. -- To view, visit http://gerrit.cloudera.org:8080/9943 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I74857c1694802e81f1cfc765d2b4e8bc644387f9 Gerrit-Change-Number: 9943 Gerrit-PatchSet: 5 Gerrit-Owner: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Comment-Date: Mon, 23 Apr 2018 20:27:26 +0000 Gerrit-HasComments: Yes
