Riza Suminto has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/15963 )

Change subject: IMPALA-6692: Trigger sort node run before hitting memory limit.
......................................................................


Patch Set 15:

> Patch Set 15: Code-Review+1
>
> Thanks for the changes! I can update to +2 if no one else has comments.
>
> One more thing to think about is potentially finding a good default for 
> sort_run_bytes_limit. I agree with doing it in another commit, but it could 
> be good to know where we will track this, e.g. in IMPALA-6692 or a new jira.

Thank you, Csaba.
Yes, once this get merged, I can file a follow up JIRA to track what is the 
best default value for sort_run_bytes_limit.

I'm also interested to refine how we buffer OutboundRowBatch in 
krpc-data-stream-sender.
Even with this IMPALA-6692 solution, with around 512mb sort_run_bytes_limit, we 
will still have 2-3s back pressure anytime we do intermediate sort.
We can utilize that time in data stream sender to do more buffering and 
serialization of next RowBatch while waiting for the pipeline to unblock.

Since IMPALA-5444 and IMPALA-9692 (part 3) get in first, it looks like this 
patch will hit merge conflict. I can prepare a rebase and submit a new patchset 
later today.


--
To view, visit http://gerrit.cloudera.org:8080/15963
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2a0ba7c4bae4f1d300d4d9d7f594f63ced06a240
Gerrit-Change-Number: 15963
Gerrit-PatchSet: 15
Gerrit-Owner: Riza Suminto <riza.sumi...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: David Rorke <dro...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Comment-Date: Wed, 01 Jul 2020 19:35:09 +0000
Gerrit-HasComments: No

Reply via email to