Tim Armstrong has posted comments on this change. Change subject: IMPALA-4866: Hash join node does not apply limits correctly ......................................................................
Patch Set 2: (6 comments) Looking pretty good I think. http://gerrit.cloudera.org:8080/#/c/6778/2/be/src/exec/partitioned-hash-join-node.cc File be/src/exec/partitioned-hash-join-node.cc: Line 582: COUNTER_SET(rows_returned_counter_, num_rows_returned_); Maybe we should set the counter at the bottom of GetNext(), instead of in the multiple places that it is currently set (this relates to my comment about decrementing 'num_rows_returned_' when we truncate the batch). Line 642: out_batch->set_num_rows(out_batch->num_rows() - (num_rows_returned_ - limit_)); There's a bug if the hash join node is in a subplan - SubplanNode may call into HashJoinNode with a partially-filled batch. In that case the number of rows returned is already accounted in 'num_rows_returned_'. Line 643: *eos = true; Shouldn't we decrement 'num_rows_returned_' if we truncated the batch? Otherwise it won't be accurate in the profile. Line 806: num_rows_returned_ += 1; Nit: we usually write this as: ++num_rows_returned_; Line 944: num_rows_returned_ += 1; Nit: we usually write this as: ++num_rows_returned_; http://gerrit.cloudera.org:8080/#/c/6778/2/testdata/workloads/functional-query/queries/QueryTest/single-node-joins-with-limits.test File testdata/workloads/functional-query/queries/QueryTest/single-node-joins-with-limits.test: How long do these take to run? Should they be under exhaustive? -- To view, visit http://gerrit.cloudera.org:8080/6778 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I414124f8bb6f8b2af2df468e1c23418d05a0e29f Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: anujphadke <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Matthew Jacobs <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: anujphadke <[email protected]> Gerrit-HasComments: Yes
