Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/15963 )
Change subject: IMPALA-6692: Trigger sort node run before hitting memory limit. ...................................................................... Patch Set 15: > Patch Set 15: Code-Review+1 > > Thanks for the changes! I can update to +2 if no one else has comments. > > One more thing to think about is potentially finding a good default for > sort_run_bytes_limit. I agree with doing it in another commit, but it could > be good to know where we will track this, e.g. in IMPALA-6692 or a new jira. Thank you, Csaba. Yes, once this get merged, I can file a follow up JIRA to track what is the best default value for sort_run_bytes_limit. I'm also interested to refine how we buffer OutboundRowBatch in krpc-data-stream-sender. Even with this IMPALA-6692 solution, with around 512mb sort_run_bytes_limit, we will still have 2-3s back pressure anytime we do intermediate sort. We can utilize that time in data stream sender to do more buffering and serialization of next RowBatch while waiting for the pipeline to unblock. Since IMPALA-5444 and IMPALA-9692 (part 3) get in first, it looks like this patch will hit merge conflict. I can prepare a rebase and submit a new patchset later today. -- To view, visit http://gerrit.cloudera.org:8080/15963 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a0ba7c4bae4f1d300d4d9d7f594f63ced06a240 Gerrit-Change-Number: 15963 Gerrit-PatchSet: 15 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: David Rorke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Wed, 01 Jul 2020 19:35:09 +0000 Gerrit-HasComments: No
