Abhishek Rawat has posted comments on this change. ( http://gerrit.cloudera.org:8080/21927 )
Change subject: IMPALA-13445: Ignore num partition for unpartitioned writes ...................................................................... Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/21927/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/21927/4//COMMIT_MSG@18 PS4, Line 18: 1. If the insert is unpartitioned, use the byte-based estimate fully. : Shuffling should only happen if num writers is less than num input : fragment instances. : 2. If the insert is partitioned, try to plan at least one writer for : each shuffling executor nodes, but do not exceed number of : partitions. Should byte based estimate always be used regardless of partitioned or unpartitioned inserts for cases such as LIMIT or when we can estimate the number of rows being inserted are fairly small (even if the number of rows scanned is much larger). I guess for small inserts, it's not a performance issue but maybe memory estimation issue. -- To view, visit http://gerrit.cloudera.org:8080/21927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51ab8fc35a5489351a88d372b28642b35449acfc Gerrit-Change-Number: 21927 Gerrit-PatchSet: 4 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: David Rorke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Fri, 18 Oct 2024 17:58:26 +0000 Gerrit-HasComments: Yes
