Matthew Jacobs has posted comments on this change. Change subject: IMPALA-3742: partitions DMLs for Kudu tables ......................................................................
Patch Set 6: (2 comments) http://gerrit.cloudera.org:8080/#/c/6037/6/be/src/runtime/data-stream-sender.cc File be/src/runtime/data-stream-sender.cc: PS6, Line 446: TupleRow* current_row = batch->GetRow(i); : uint32_t partition; : RETURN_IF_ERROR(partitioner_->Partition(current_row, &partition)); : RETURN_IF_ERROR(channels_[partition % num_channels]->AddRow(current_row)); > Could the partitioner take a batch and produce a pre-sized vector of partit Henry, would you be opposed to #2 above? It would avoid iterating over the batch twice, though at the aesthetic cost of having to share channels_ with the partitioner directly. I'm fine either way. http://gerrit.cloudera.org:8080/#/c/6037/6/testdata/workloads/functional-planner/queries/PlannerTest/kudu.test File testdata/workloads/functional-planner/queries/PlannerTest/kudu.test: Line 53: 01:EXCHANGE [KUDU(10)] > this is very expensive, we need to avoid this for single-row inserts (and a we were talking about trying to figure out the right cost-based logic separately, but I agree the trivial cases like this are clearly quite bad. -- To view, visit http://gerrit.cloudera.org:8080/6037 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic10b3295159354888efcde3df76b0edb24161515 Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Henry Robinson <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Matthew Jacobs <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-HasComments: Yes
