[
https://issues.apache.org/jira/browse/HIVE-14353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396893#comment-15396893
]
Nemon Lou edited comment on HIVE-14353 at 7/29/16 1:08 AM:
-----------------------------------------------------------
||queries||CBO_total_time||CBO_time_in_SelectOP||
|q27| 266.494|80.5 |
|q7 | 328.259|98.8 |
|q68| 369.159|105 |
|q46| 392.777|91.75|
I just run a few of them because of time limit. The time spent in selectOP is
calculated by adding up total times spent for selectOP in one executor ,and
then divide number of cores.(4 in my case).
Also,I have run q46 without projection pruning.And total time is 266.226,time
spent in selectOP is 0.125 seconds.
was (Author: nemon):
||queries||CBO_total_time||CBO_time_in_SelectOP||
|q27| 266.494|251 |
|q7 | 328.259|98.8 |
|q68| 369.159|105 |
|q46| 392.777|91.75|
I just run a few of them because of time limit. The time spent in selectOP is
calculated by adding up total times spent for selectOP in one executor ,and
then divide number of cores.(4 in my case).
Also,I have run q46 without projection pruning.And total time is 266.226,time
spent in selectOP is 0.125 seconds.
> Performance degradation after Projection Pruning in CBO
> --------------------------------------------------------
>
> Key: HIVE-14353
> URL: https://issues.apache.org/jira/browse/HIVE-14353
> Project: Hive
> Issue Type: Bug
> Components: CBO, Logical Optimizer
> Affects Versions: 1.2.1
> Reporter: Nemon Lou
> Attachments: q46_cbo_no_projection_prune_explain.txt,
> q46_cbo_projection_prune_explain.rar
>
>
> TPC-DS with factor 1024.
> Hive on Spark.
> With and without projection prunning,time spent are quite different.
> The way to disable projection prunning : disable HiveRelFieldTrimmer in code
> and compile a new jar.
> ||queries||CBO_no_projection_prune||CBO||
> |q27| 160|251 |
> |q7 | 200|312 |
> |q88| 701|1092|
> |q68| 234|345 |
> |q39| 53|78 |
> |q73| 160|228 |
> |q31| 463|659 |
> |q79| 242|343 |
> |q46| 256|363 |
> |q60| 271|382 |
> |q66| 198|278 |
> |q34| 155|217 |
> |q19| 184|256 |
> |q26| 154|214 |
> |q56| 262|364 |
> |q75| 942|1303|
> |q71| 288|388 |
> |q25| 329|442 |
> |q52| 142|190 |
> |q42| 142|189 |
> |q3 | 139|185 |
> |q98| 153|203 |
> |q89| 187|248 |
> |q58| 264|340 |
> |q43| 127|162 |
> |q32| 174|221 |
> |q96| 156|197 |
> |q70| 320|404 |
> |q29| 499|629 |
> |q18| 266|329 |
> |q21| 76 |92 |
> |q90| 139|165 |
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
