Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21927
Change subject: IMPALA-13445: Ignore num partition for unpartitioned writes ...................................................................... IMPALA-13445: Ignore num partition for unpartitioned writes When cost-based planning is used, writer parallelism is limited by the number of partitions. In the unpartitioned insert scenario, there will be just single partitions. That leads to a single fs writer only, which causes slow writes. This patch fixes the issue by distinguishing between partitioned insert and unpartitioned insert. If the insert is unpartitioned, use the byte-based estimate fully. This patch also does minor cleanup in DistributedPlanner.java. Testing: - In test_executor_groups.py, move insert tests from test_query_cpu_count_divisor_default into separate test_query_cpu_count_on_insert. Add two new insert test cases there. - Add and pass CardinalityTest.testByteBasedNumWriters(). - Pass test_executor_groups.py. Change-Id: I51ab8fc35a5489351a88d372b28642b35449acfc --- M be/src/service/query-state-record.cc M fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/test/java/org/apache/impala/planner/CardinalityTest.java M tests/custom_cluster/test_executor_groups.py 5 files changed, 159 insertions(+), 67 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/21927/1 -- To view, visit http://gerrit.cloudera.org:8080/21927 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I51ab8fc35a5489351a88d372b28642b35449acfc Gerrit-Change-Number: 21927 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto <[email protected]>
