Tim Armstrong has posted comments on this change. Change subject: IMPALA-3586: Implement union passthrough ......................................................................
Patch Set 4: (6 comments) http://gerrit.cloudera.org:8080/#/c/5816/4/be/src/exec/union-node.cc File be/src/exec/union-node.cc: Line 133: row_batch->set_num_rows(limit_ - num_rows_returned_); > Done. I added saving the number of rows. I am not sure that this case is po Do we have coverage of that kind of plan shape though? If someone else comes along and turns on passthrough in subplans. http://gerrit.cloudera.org:8080/#/c/5816/6/be/src/exec/union-node.cc File be/src/exec/union-node.cc: Line 103: I think we need to reset 'child_batch_' here so that it has the correct 'row_desc' - otherwise they can be out of sync after a sequence of Reset() then Open(). Line 114: } My preference is to remove this code if it's not needed for correctness and we have no specific reason to think that it's important for performance. Line 153: RETURN_IF_ERROR( I think this comment over-explains some resource management things that aren't specific to this code - the MarkNeedsDeepCopy() mechanism and what the non-passthrough case does. We could shorten to something like: // Even though the child is at eos, it's not OK to Close() it here. Once we close the child, // the row batches that it produced are invalid. Setting 'needs_deep_copy' lets us safely // close the child in the next GetNext() call. http://gerrit.cloudera.org:8080/#/c/5816/4/fe/src/main/java/org/apache/impala/planner/UnionNode.java File fe/src/main/java/org/apache/impala/planner/UnionNode.java: Line 266: analyzer, children_.get(i).getTupleIds(), resultExprLists_.get(i))); > Not exactly sure what you mean. We are guaranteed that the layout will be i nullableTupleIds_ in child PlanNode is essentially part of the row layout and this code isn't checking that those are equivalent. I think currently all single-tuple rows have no nullable tuples, but I think it's too subtle to implicitly assume that. Maybe add a precondition check that the input tuple isn't nullable? http://gerrit.cloudera.org:8080/#/c/5816/4/testdata/workloads/functional-planner/queries/PlannerTest/union.test File testdata/workloads/functional-planner/queries/PlannerTest/union.test: Line 3103: ==== > Can you add a couple of tests where there is a table scan on one branch of Was this addressed? -- To view, visit http://gerrit.cloudera.org:8080/5816 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ia8f6d5062724ba5b78174c3227a7a796d10d8416 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Taras Bobrovytsky <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Taras Bobrovytsky <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
