[
https://issues.apache.org/jira/browse/HIVE-27751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramesh Kumar Thangarajan resolved HIVE-27751.
---------------------------------------------
Fix Version/s: 4.1.0
Resolution: Fixed
> Log Query Compilation summary in an accumulated way
> ---------------------------------------------------
>
> Key: HIVE-27751
> URL: https://issues.apache.org/jira/browse/HIVE-27751
> Project: Hive
> Issue Type: Task
> Components: Hive
> Reporter: Ramesh Kumar Thangarajan
> Assignee: Ramesh Kumar Thangarajan
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> Query Compilation summary is very useful for reading and collecting all the
> measures of compile time in a single place. It is also useful in debugging a
> performance issue in the query compilation phase and also to report and
> compare with various runs
> In order to run test this. Please set the config hive.compile.print.summary
> to true in any q file and run the test to see the Query Compilation Summary
> in the logs. One example of the output is below. The order of operations are
> maintained while print the summary too:
> {code:java}
> Query Compilation Summary
> ----------------------------------------------------------------------------------------------
> waitCompile
> 0 ms
> parse
> 4 ms
> getTableConstraints - HS2-cache
> 69 ms
> optimizer - Calcite: Plan generation
> 257 ms
> optimizer - Calcite: Prejoin ordering transformation
> 20 ms
> optimizer - Calcite: Postjoin ordering transformation
> 24 ms
> optimizer
> 705 ms
> optimizer - HiveOpConverterPostProc
> 0 ms
> optimizer - Generator
> 24 ms
> optimizer - PartitionColumnsSeparator
> 1 ms
> optimizer - SyntheticJoinPredicate
> 2 ms
> optimizer - SimplePredicatePushDown
> 8 ms
> optimizer - RedundantDynamicPruningConditionsRemoval
> 0 ms
> optimizer - SortedDynPartitionTimeGranularityOptimizer
> 2 ms
> optimizer - PartitionPruner
> 3 ms
> optimizer - PartitionConditionRemover
> 2 ms
> optimizer - GroupByOptimizer
> 2 ms
> optimizer - ColumnPruner
> 10 ms
> optimizer - CountDistinctRewriteProc
> 1 ms
> optimizer - SamplePruner
> 1 ms
> optimizer - MapJoinProcessor
> 2 ms
> optimizer - BucketingSortingReduceSinkOptimizer
> 2 ms
> optimizer - UnionProcessor
> 2 ms
> optimizer - JoinReorder
> 0 ms
> optimizer - FixedBucketPruningOptimizer
> 2 ms
> optimizer - BucketVersionPopulator
> 2 ms
> optimizer - NonBlockingOpDeDupProc
> 1 ms
> optimizer - IdentityProjectRemover
> 0 ms
> optimizer - LimitPushdownOptimizer
> 2 ms
> optimizer - OrderlessLimitPushDownOptimizer
> 1 ms
> optimizer - StatsOptimizer
> 0 ms
> optimizer - SimpleFetchOptimizer
> 0 ms
> TezCompiler - Run top n key optimization
> 2 ms
> TezCompiler - Setup dynamic partition pruning
> 3 ms
> optimizer - Merge single column semi-join reducers to composite
> 0 ms
> partition-retrieving
> 1 ms
> TezCompiler - Setup stats in the operator plan
> 78 ms
> TezCompiler - Sorted dynamic partition optimization
> 3 ms
> TezCompiler - Reduce Sink de-duplication
> 4 ms
> TezCompiler - Run the optimizations that use stats for optimization
> 5 ms
> TezCompiler - Run reduce sink after join algorithm selection
> 2 ms
> TezCompiler - Run remove dynamic pruning by size
> 1 ms
> TezCompiler - Run cycle analysis for partition pruning
> 0 ms
> TezCompiler - Remove redundant semijoin reduction
> 1 ms
> TezCompiler - Shared scans optimization
> 13 ms
> TezCompiler - markOperatorsWithUnstableRuntimeStats
> 1 ms
> TezCompiler - generateTaskTree
> 31 ms
> TezCompiler - optimizeTaskPlan
> 156 ms
> TezCompiler
> 323 ms
> semanticAnalyze
> 2628 ms
> compile
> 2633 ms
> ----------------------------------------------------------------------------------------------
> {code}
> {{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
