Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20581
Change subject: IMPALA-12429: Reduce parallelism for TPC-DS q51a and q67a test. ...................................................................... IMPALA-12429: Reduce parallelism for TPC-DS q51a and q67a test. TestTpcdsQueryWithProcessingCost.test_tpcds_q51a and TestTpcdsQuery.test_tpcds_q67a has been intermittently failing with memory oversubscription error. The fact that test minicluster start 3 impalad in single host probably make admission control less effective in preventing these queries from running in parallel with others. This patch keep both test, but reduce max_fragment_instances_per_node from 4 to 2 to lower its memory requirement. Before patch: q51a Max Per-Host Resource Reservation: Memory=3.08GB Threads=129 Per-Host Resource Estimates: Memory=124.24GB Per Host Min Memory Reservation: localhost:27001(2.93 GB) localhost:27002(1.97 GB) localhost:27000(2.82 GB) Per Host Number of Fragment Instances: localhost:27001(115) localhost:27002(79) localhost:27000(119) Admission result: Admitted immediately Cluster Memory Admitted: 33.00 GB Per Node Peak Memory Usage: localhost:27000(2.84 GB) localhost:27002(1.99 GB) localhost:27001(2.95 GB) Per Node Bytes Read: localhost:27000(62.08 MB) localhost:27002(45.71 MB) localhost:27001(47.39 MB) q67a Max Per-Host Resource Reservation: Memory=2.15GB Threads=105 Per-Host Resource Estimates: Memory=4.48GB Per Host Min Memory Reservation: localhost:27001(2.13 GB) localhost:27002(2.13 GB) localhost:27000(2.15 GB) Per Host Number of Fragment Instances: localhost:27001(76) localhost:27002(76) localhost:27000(105) Cluster Memory Admitted: 13.44 GB Per Node Peak Memory Usage: localhost:27000(2.24 GB) localhost:27002(2.21 GB) localhost:27001(2.21 GB) Per Node Bytes Read: localhost:27000(112.79 MB) localhost:27002(109.57 MB) localhost:27001(105.16 MB) After patch: q51a Max Per-Host Resource Reservation: Memory=2.00GB Threads=79 Per-Host Resource Estimates: Memory=118.75GB Per Host Min Memory Reservation: localhost:27001(1.84 GB) localhost:27002(1.28 GB) localhost:27000(1.86 GB) Per Host Number of Fragment Instances: localhost:27001(65) localhost:27002(46) localhost:27000(74) Cluster Memory Admitted: 33.00 GB Per Node Peak Memory Usage: localhost:27000(1.88 GB) localhost:27002(1.31 GB) localhost:27001(1.88 GB) Per Node Bytes Read: localhost:27000(62.08 MB) localhost:27002(45.71 MB) localhost:27001(47.39 MB) q67a Max Per-Host Resource Reservation: Memory=1.31GB Threads=85 Per-Host Resource Estimates: Memory=3.76GB Per Host Min Memory Reservation: localhost:27001(1.29 GB) localhost:27002(1.29 GB) localhost:27000(1.31 GB) Per Host Number of Fragment Instances: localhost:27001(56) localhost:27002(56) localhost:27000(85) Cluster Memory Admitted: 11.28 GB Per Node Peak Memory Usage: localhost:27000(1.35 GB) localhost:27002(1.32 GB) localhost:27001(1.33 GB) Per Node Bytes Read: localhost:27000(112.79 MB) localhost:27002(109.57 MB) localhost:27001(105.16 MB) Testing: - Pass test_tpcds_queries.py in local machine. Change-Id: I6ae5aeb97a8353d5eaa4d85e3f600513f42f7cf4 --- M tests/query_test/test_tpcds_queries.py 1 file changed, 13 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/20581/1 -- To view, visit http://gerrit.cloudera.org:8080/20581 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I6ae5aeb97a8353d5eaa4d85e3f600513f42f7cf4 Gerrit-Change-Number: 20581 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto <[email protected]>
