[
https://issues.apache.org/jira/browse/HIVE-14353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396905#comment-15396905
]
Pengcheng Xiong commented on HIVE-14353:
----------------------------------------
[~nemon], thanks for your data. I am bit confused for the CBO_time_in_SelectOP
that you mentioned... What i would like to see is how much time did we spend on
compiling the query (I mean the time spent on the application of
HiveRelFieldTrimmer). For example, for q27, the time difference is 251-160=91.
If the time spent on the application of HiveRelFieldTrimmer is around 90s, then
that means we should improve the compilation. Otherwise if the time spent on
the application of HiveRelFieldTrimmer is around 1-5s, then that means the
difference comes from somewhere else rather than compilation.. Thanks.
> Performance degradation after Projection Pruning in CBO
> --------------------------------------------------------
>
> Key: HIVE-14353
> URL: https://issues.apache.org/jira/browse/HIVE-14353
> Project: Hive
> Issue Type: Bug
> Components: CBO, Logical Optimizer
> Affects Versions: 1.2.1
> Reporter: Nemon Lou
>
> TPC-DS with factor 1024.
> Hive on Spark.
> With and without projection prunning,time spent are quite different.
> The way to disable projection prunning : disable HiveRelFieldTrimmer in code
> and compile a new jar.
> ||queries||CBO_no_projection_prune||CBO||
> |q27| 160|251 |
> |q7 | 200|312 |
> |q88| 701|1092|
> |q68| 234|345 |
> |q39| 53|78 |
> |q73| 160|228 |
> |q31| 463|659 |
> |q79| 242|343 |
> |q46| 256|363 |
> |q60| 271|382 |
> |q66| 198|278 |
> |q34| 155|217 |
> |q19| 184|256 |
> |q26| 154|214 |
> |q56| 262|364 |
> |q75| 942|1303|
> |q71| 288|388 |
> |q25| 329|442 |
> |q52| 142|190 |
> |q42| 142|189 |
> |q3 | 139|185 |
> |q98| 153|203 |
> |q89| 187|248 |
> |q58| 264|340 |
> |q43| 127|162 |
> |q32| 174|221 |
> |q96| 156|197 |
> |q70| 320|404 |
> |q29| 499|629 |
> |q18| 266|329 |
> |q21| 76 |92 |
> |q90| 139|165 |
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
