Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/8822 )
Change subject: IMPALA-6070: Parallelize another bit of data load. ...................................................................... Patch Set 1: https://jenkins.impala.io/view/Utility/job/pre-review-test/87/ passed just fine. The logs from https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/798/consoleFull look fine to me. 00:53:42 Creating many block table OK (Took: 0 min 23 sec) 00:53:42 Started Loading Kudu functional in background; pid 19994. 00:53:42 Loading Kudu functional (logging to /home/ubuntu/Impala/logs/data_loading/load-kudu.log)... 00:53:42 Started Loading Kudu TPCH in background; pid 19995. 00:53:42 Loading Kudu TPCH (logging to /home/ubuntu/Impala/logs/data_loading/load-kudu-tpch.log)... 00:53:42 Started Loading Hive UDFs in background; pid 19996. 00:53:42 Loading Hive UDFs (logging to /home/ubuntu/Impala/logs/data_loading/build-and-copy-hive-udfs.log)... 00:54:24 Loading Hive UDFs OK (Took: 0 min 42 sec) 00:56:56 Loading workload 'functional-query' using exploration strategy 'core' in table formats 'kudu/none/none' OK (Took: 3 min 14 sec) 00:57:38 Loading workload 'tpch' using exploration strategy 'core' in table formats 'kudu/none/none' OK (Took: 3 min 56 sec) 00:57:38 Running custom post-load steps (logging to /home/ubuntu/Impala/logs/data_loading/custom-post-load-steps.log)... Comparing to some arbitrary other run of this job (https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/796/consoleFull; see below) this seems to save 4 minutes. The overall run time change here is still within the run-to-run noise, but I think this will move the baseline down 3-4 minutes. 19:26:09 Loading Kudu functional (logging to /home/ubuntu/Impala/logs/data_loading/load-kudu.log)... 19:29:38 Loading workload 'functional-query' using exploration strategy 'core' in table formats 'kudu/none/none' OK (Took: 3 min 29 sec) 19:29:38 Loading Kudu TPCH (logging to /home/ubuntu/Impala/logs/data_loading/load-kudu-tpch.log)... 19:33:38 Loading workload 'tpch' using exploration strategy 'core' in table formats 'kudu/none/none' OK (Took: 4 min 0 sec) 19:33:38 Loading Hive UDFs (logging to /home/ubuntu/Impala/logs/data_loading/build-and-copy-hive-udfs.log)... 19:34:19 Loading Hive UDFs OK (Took: 0 min 41 sec) 19:34:19 Running custom post-load steps (logging to /home/ubuntu/Impala/logs/data_loading/custom-post-load-steps.log)... -- To view, visit http://gerrit.cloudera.org:8080/8822 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7e93ee5a77ec9271b980b88bef7ad512ecbe0407 Gerrit-Change-Number: 8822 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger <phi...@cloudera.com> Gerrit-Reviewer: Philip Zeyliger <phi...@cloudera.com> Gerrit-Comment-Date: Wed, 13 Dec 2017 22:31:32 +0000 Gerrit-HasComments: No
