wangyum commented on pull request #31901: URL: https://github.com/apache/spark/pull/31901#issuecomment-803558990
Before this pr:
```
19:23:58.074 WARN org.apache.spark.sql.TPCDSQuerySuite:
=== Metrics of Analyzer/Optimizer Rules ===
Total number of runs: 407441
Total time: 41.217085463 seconds
Rule
Effective Time / Total Time Effective Runs / Total
Runs
org.apache.spark.sql.catalyst.optimizer.Optimizer$OptimizeSubqueries
3459364804 / 7273617141 49 / 2778
org.apache.spark.sql.catalyst.optimizer.ColumnPruning
554550102 / 2852279204 338 / 6402
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggregateFunctions
830781928 / 2459212474 49 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubquery
1917946262 / 2067630600 51 / 2141
org.apache.spark.sql.catalyst.analysis.DecimalPrecision
1456077159 / 1681591081 361 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences
741392690 / 902859224 815 / 2141
org.apache.spark.sql.catalyst.optimizer.PruneFilters
21731791 / 866239104 5 / 5013
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$ImplicitTypeCasts
330274264 / 782021776 78 / 2141
org.apache.spark.sql.catalyst.optimizer.InferFiltersFromConstraints
644004888 / 776597471 284 / 1389
org.apache.spark.sql.catalyst.optimizer.ReorderJoin
647382677 / 729812324 177 / 3624
org.apache.spark.sql.catalyst.analysis.UpdateAttributeNullability
10876695 / 693912132 12 / 3448
org.apache.spark.sql.catalyst.optimizer.BooleanSimplification
14626317 / 658650063 16 / 3624
org.apache.spark.sql.execution.datasources.PruneFileSourcePartitions
567434042 / 610800777 209 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveMissingReferences
11282754 / 603081344 10 / 2141
org.apache.spark.sql.catalyst.optimizer.PushDownPredicates
447678284 / 580794636 1002 / 6635
org.apache.spark.sql.catalyst.optimizer.RemoveRedundantAliases
21103967 / 522693682 29 / 3624
org.apache.spark.sql.catalyst.optimizer.OptimizeCsvJsonExprs
0 / 452119884 0 / 3624
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$FunctionArgumentConversion
236070931 / 447828356 22 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$WidenSetOperationTypes
291743240 / 426849131 24 / 2141
org.apache.spark.sql.execution.datasources.SchemaPruning
0 / 392620647 0 / 1389
org.apache.spark.sql.catalyst.optimizer.OptimizeUpdateFields
0 / 371380231 0 / 4359
org.apache.spark.sql.catalyst.optimizer.NullPropagation
25185090 / 364626484 60 / 3624
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$CaseWhenCoercion
148082198 / 355006700 33 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ExtractGenerator
0 / 345835417 0 / 2141
org.apache.spark.sql.catalyst.optimizer.FoldablePropagation
2839171 / 321384305 6 / 3624
org.apache.spark.sql.catalyst.optimizer.RewriteCorrelatedScalarSubquery
5107341 / 309518387 8 / 3624
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions
136245772 / 299997876 426 / 2141
org.apache.spark.sql.catalyst.optimizer.SimplifyBinaryComparison
255511 / 276737636 1 / 3624
org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog
155606665 / 275166367 24 / 2141
org.apache.spark.sql.catalyst.optimizer.SimplifyConditionals
0 / 257901787 0 / 3624
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$DateTimeOperations
0 / 257026064 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$IntegralDivision
0 / 252694314 0 / 2141
org.apache.spark.sql.catalyst.optimizer.PushFoldableIntoBranches
6912323 / 251903640 19 / 3624
org.apache.spark.sql.catalyst.optimizer.SimplifyExtractValueOps
0 / 243932140 0 / 3624
org.apache.spark.sql.catalyst.optimizer.ConstantFolding
69263491 / 239972524 221 / 3624
org.apache.spark.sql.catalyst.analysis.TypeCoercion$PromoteStrings
8767486 / 239312714 11 / 2141
org.apache.spark.sql.catalyst.optimizer.UnwrapCastInBinaryComparison
4464067 / 232861362 8 / 3624
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$Division
18825655 / 230242583 10 / 2141
org.apache.spark.sql.catalyst.optimizer.ReorderAssociativeOperator
0 / 227408914 0 / 3624
org.apache.spark.sql.catalyst.analysis.ResolveTimeZone
108526256 / 227205992 537 / 2141
org.apache.spark.sql.catalyst.optimizer.SimplifyCaseConversionExpressions
0 / 225276712 0 / 3624
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$InConversion
3227516 / 224466926 4 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveWindowFrame
12058194 / 219537686 37 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercion$BooleanEquality
0 / 215863072 0 / 2141
org.apache.spark.sql.catalyst.optimizer.RemoveDispensableExpressions
0 / 209234062 0 / 3624
org.apache.spark.sql.execution.datasources.FindDataSourceTable
177261972 / 208675610 296 / 2141
org.apache.spark.sql.catalyst.analysis.CTESubstitution
44178100 / 207656236 58 / 735
org.apache.spark.sql.catalyst.analysis.ResolveCreateNamedStruct
0 / 205571016 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$IfCoercion
0 / 202086288 0 / 2141
org.apache.spark.sql.catalyst.optimizer.OptimizeIn
9926506 / 200067451 27 / 3624
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$StringLiteralCoercion
0 / 200004878 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$EltCoercion
0 / 192989682 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveBinaryArithmetic
5938308 / 192018980 25 / 2141
org.apache.spark.sql.catalyst.optimizer.SimplifyCasts
19440256 / 189351500 83 / 3624
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$ConcatCoercion
0 / 185365923 0 / 2141
org.apache.spark.sql.catalyst.optimizer.LikeSimplification
674471 / 177286705 1 / 3624
org.apache.spark.sql.catalyst.optimizer.ReplaceNullWithFalseInPredicate
2842551 / 177044278 17 / 3624
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations
118491086 / 176494105 296 / 2151
org.apache.spark.sql.catalyst.analysis.PullOutNondeterministic
0 / 168290835 0 / 665
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveWindowOrder
9828199 / 167493133 28 / 2141
org.apache.spark.sql.catalyst.analysis.TimeWindowing
0 / 162946057 0 / 2141
org.apache.spark.sql.catalyst.optimizer.OptimizeWindowFunctions
0 / 162401843 0 / 3624
org.apache.spark.sql.catalyst.analysis.Analyzer$AddMetadataColumns
0 / 160907757 0 / 2151
org.apache.spark.sql.catalyst.optimizer.RemoveNoopOperators
20361364 / 159451417 214 / 7791
org.apache.spark.sql.execution.aggregate.ResolveEncodersInScalaAgg
0 / 150540658 0 / 2141
org.apache.spark.sql.catalyst.analysis.ResolveHigherOrderFunctions
0 / 150173456 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$WindowFrameCoercion
0 / 137450079 0 / 2141
org.apache.spark.sql.execution.dynamicpruning.PartitionPruning
122398390 / 131163171 222 / 1389
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$StackCoercion
0 / 128164923 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ExtractWindowExpressions
37023937 / 127981109 38 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$MapZipWithCoercion
0 / 127855021 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveGroupingAnalytics
23205519 / 118920458 12 / 2141
org.apache.spark.sql.catalyst.analysis.ApplyCharTypePadding
64824954 / 112460168 106 / 665
org.apache.spark.sql.catalyst.optimizer.ConstantPropagation
0 / 106118066 0 / 3624
org.apache.spark.sql.catalyst.optimizer.DecimalAggregates
56241264 / 103958587 124 / 1513
org.apache.spark.sql.catalyst.analysis.ResolveLambdaVariables
0 / 98114927 0 / 2141
org.apache.spark.sql.catalyst.optimizer.PushDownLeftSemiAntiJoin
4284759 / 93655171 32 / 3624
org.apache.spark.sql.catalyst.optimizer.RemoveNoopUnion
0 / 90742352 0 / 1389
org.apache.spark.sql.execution.python.ExtractPythonUDFs
0 / 90481091 0 / 1389
org.apache.spark.sql.catalyst.optimizer.CollapseProject
32147700 / 88138920 191 / 5013
org.apache.spark.sql.execution.datasources.v2.V2ScanRelationPushDown
0 / 85994865 0 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRandomSeed
0 / 83825129 0 / 2141
org.apache.spark.sql.catalyst.analysis.CleanupAliases
45394242 / 80992428 514 / 1179
org.apache.spark.sql.catalyst.optimizer.RemoveRedundantAggregates
0 / 76924970 0 / 3624
org.apache.spark.sql.execution.dynamicpruning.CleanupDynamicPruningFilters
0 / 73585707 0 / 1389
org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates
18971095 / 72745695 16 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveTables
0 / 72132943 0 / 2151
org.apache.spark.sql.catalyst.optimizer.CollapseRepartition
0 / 71574775 0 / 3624
org.apache.spark.sql.catalyst.optimizer.EliminateOuterJoin
9824553 / 70741859 15 / 3624
org.apache.spark.sql.catalyst.optimizer.ComputeCurrentTime
0 / 70233534 0 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAliases
9516490 / 66119504 53 / 2141
org.apache.spark.sql.catalyst.optimizer.SimplifyConditionalsInPredicate
1950867 / 65731553 19 / 3624
org.apache.spark.sql.catalyst.optimizer.OptimizeRepartition
0 / 64678672 0 / 3624
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer
0 / 64325053 0 / 2141
org.apache.spark.sql.catalyst.optimizer.CombineUnions
7361644 / 63872643 48 / 5013
org.apache.spark.sql.catalyst.optimizer.GetCurrentDatabaseAndCatalog
0 / 60852185 0 / 1389
org.apache.spark.sql.catalyst.optimizer.LimitPushDownThroughWindow
0 / 59078747 0 / 3624
org.apache.spark.sql.catalyst.optimizer.EliminateSerialization
0 / 58972531 0 / 3624
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveNewInstance
0 / 57356096 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$GlobalAggregates
3158587 / 56607389 82 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveUpCast
0 / 55416152 0 / 2141
org.apache.spark.sql.catalyst.optimizer.TransposeWindow
0 / 53243101 0 / 3624
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveOrdinalInOrderByAndGroupBy
2689837 / 51448574 8 / 2141
org.apache.spark.sql.catalyst.optimizer.ReplaceExpressions
241958 / 49203258 2 / 1389
org.apache.spark.sql.catalyst.optimizer.ReplaceUpdateFieldsExpression
0 / 49170134 0 / 1389
org.apache.spark.sql.catalyst.optimizer.EliminateLimits
428978 / 48305281 11 / 3624
org.apache.spark.sql.execution.datasources.ResolveSQLOnFile
0 / 48206191 0 / 2141
org.apache.spark.sql.catalyst.analysis.ResolveUnion
0 / 47625269 0 / 2141
org.apache.spark.sql.catalyst.optimizer.CollapseWindow
0 / 46484925 0 / 3624
org.apache.spark.sql.execution.datasources.PreprocessTableCreation
7488318 / 46444682 7 / 665
org.apache.spark.sql.catalyst.optimizer.PushProjectionThroughUnion
1969261 / 45163860 4 / 3624
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggAliasInGroupBy
0 / 44659099 0 / 2141
org.apache.spark.sql.catalyst.analysis.ResolveInlineTables
0 / 43737249 0 / 2141
org.apache.spark.sql.catalyst.optimizer.RewritePredicateSubquery
24748250 / 43218989 26 / 1389
org.apache.spark.sql.catalyst.optimizer.ReassignLambdaVariableID
0 / 42470875 0 / 1389
org.apache.spark.sql.catalyst.optimizer.EliminateResolvedHint
0 / 42470534 0 / 1389
org.apache.spark.sql.catalyst.optimizer.PushLeftSemiLeftAntiThroughJoin
1391712 / 41855536 3 / 3624
org.apache.spark.sql.catalyst.analysis.ResolveCatalogs
0 / 41744035 0 / 2151
org.apache.spark.sql.catalyst.optimizer.EliminateMapObjects
0 / 40267282 0 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveNamespace
0 / 40103109 0 / 2151
org.apache.spark.sql.catalyst.optimizer.LimitPushDown
0 / 39885579 0 / 3624
org.apache.spark.sql.catalyst.optimizer.CombineFilters
0 / 39179703 0 / 3624
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubqueryColumnAliases
0 / 37673126 0 / 2141
org.apache.spark.sql.catalyst.optimizer.NormalizeFloatingNumbers
0 / 37289065 0 / 1389
org.apache.spark.sql.catalyst.optimizer.PropagateEmptyRelation
4064628 / 36452001 5 / 2783
org.apache.spark.sql.catalyst.analysis.Analyzer$HandleNullInputsForUDF
0 / 36314386 0 / 665
org.apache.spark.sql.catalyst.optimizer.RewriteNonCorrelatedExists
0 / 35675344 0 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveGenerate
0 / 34874853 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$LookupFunctions
0 / 33665275 0 / 675
org.apache.spark.sql.catalyst.analysis.ResolveTableValuedFunctions
0 / 33609320 0 / 2151
org.apache.spark.sql.catalyst.optimizer.EliminateSorts
557492 / 33057122 2 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolvePivot
0 / 32398981 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveNaturalAndUsingJoin
0 / 32114392 0 / 2141
org.apache.spark.sql.execution.datasources.FallBackFileSourceV2
0 / 31687366 0 / 2141
org.apache.spark.sql.catalyst.optimizer.ReplaceDeduplicateWithAggregate
0 / 31540710 0 / 1431
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveInsertInto
0 / 31303611 0 / 2151
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveOutputRelation
0 / 31269729 0 / 2141
org.apache.spark.sql.catalyst.analysis.ResolvePartitionSpec
0 / 31159186 0 / 2151
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveEncodersInUDF
0 / 30312585 0 / 665
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveUserSpecifiedColumns
0 / 29266793 0 / 2151
org.apache.spark.sql.catalyst.optimizer.PushExtraPredicateThroughJoin
4074877 / 29060408 7 / 1393
org.apache.spark.sql.catalyst.analysis.EliminateSubqueryAliases
22386100 / 27379356 296 / 1389
org.apache.spark.sql.catalyst.optimizer.PushPredicateThroughNonJoin
0 / 19727080 0 / 1389
org.apache.spark.sql.execution.analysis.DetectAmbiguousSelfJoin
0 / 17321108 0 / 665
org.apache.spark.sql.catalyst.optimizer.RewriteDistinctAggregates
0 / 17130396 0 / 1389
org.apache.spark.sql.catalyst.optimizer.ExtractPythonUDFFromJoinCondition
0 / 16212421 0 / 1389
org.apache.spark.sql.execution.python.ExtractPythonUDFFromAggregate
0 / 16097371 0 / 1389
org.apache.spark.sql.catalyst.analysis.SubstituteUnresolvedOrdinals
2291646 / 15083864 8 / 735
org.apache.spark.sql.catalyst.analysis.EliminateView
0 / 14423140 0 / 1389
org.apache.spark.sql.execution.python.ExtractGroupingPythonUDFFromAggregate
0 / 13924484 0 / 1389
org.apache.spark.sql.catalyst.optimizer.ReplaceExceptWithFilter
0 / 13882442 0 / 1431
org.apache.spark.sql.catalyst.optimizer.RemoveRepetitionFromGroupExpressions
1114414 / 13842538 2 / 1391
org.apache.spark.sql.catalyst.optimizer.RewriteExceptAll
0 / 12267008 0 / 1431
org.apache.spark.sql.execution.datasources.v2.V2Writes
0 / 11154273 0 / 1389
org.apache.spark.sql.catalyst.optimizer.InferFiltersFromGenerate
0 / 11076409 0 / 1389
org.apache.spark.sql.catalyst.optimizer.ObjectSerializerPruning
0 / 10973496 0 / 1389
org.apache.spark.sql.catalyst.optimizer.RemoveLiteralFromGroupExpressions
0 / 10820735 0 / 1391
org.apache.spark.sql.catalyst.analysis.Analyzer$WindowsSubstitution
0 / 10494983 0 / 735
org.apache.spark.sql.catalyst.optimizer.CombineTypedFilters
0 / 10316684 0 / 1389
org.apache.spark.sql.catalyst.optimizer.ReplaceIntersectWithSemiJoin
1784460 / 10130166 24 / 1431
org.apache.spark.sql.catalyst.optimizer.ReplaceExceptWithAntiJoin
973501 / 9767367 1 / 1431
org.apache.spark.sql.catalyst.optimizer.ReplaceDistinctWithAggregate
2202293 / 9597890 42 / 1431
org.apache.spark.sql.catalyst.optimizer.OptimizeLimitZero
0 / 9000817 0 / 1389
org.apache.spark.sql.catalyst.optimizer.RewriteIntersectAll
0 / 8834253 0 / 1431
org.apache.spark.sql.catalyst.analysis.UpdateOuterReferences
0 / 8764186 0 / 665
org.apache.spark.sql.execution.datasources.DataSourceAnalysis
2456760 / 8498247 24 / 665
org.apache.spark.sql.catalyst.analysis.EliminateUnions
0 / 8494086 0 / 735
org.apache.spark.sql.catalyst.analysis.ResolveHints$ResolveJoinStrategyHints
0 / 7425893 0 / 675
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAlterTableChanges
0 / 6966019 0 / 665
org.apache.spark.sql.catalyst.analysis.ResolveHints$ResolveCoalesceHints
0 / 6911635 0 / 675
org.apache.spark.sql.catalyst.analysis.ResolveCommandsWithIfExists
0 / 6744681 0 / 665
org.apache.spark.sql.execution.datasources.PreprocessTableInsertion
0 / 6354045 0 / 665
org.apache.spark.sql.catalyst.analysis.ResolveHints$RemoveAllHints
0 / 6145726 0 / 665
org.apache.spark.sql.catalyst.optimizer.EliminateAggregateFilter
0 / 6011771 0 / 3624
org.apache.spark.sql.catalyst.optimizer.CheckCartesianProducts
0 / 5949828 0 / 2778
org.apache.spark.sql.catalyst.optimizer.CombineConcats
0 / 4467882 0 / 3624
org.apache.spark.sql.catalyst.optimizer.CostBasedJoinReorder
0 / 3704450 0 / 1389
org.apache.spark.sql.execution.OptimizeMetadataOnlyQuery
0 / 3488431 0 / 1389
org.apache.spark.sql.catalyst.optimizer.EliminateDistinct
0 / 3068115 0 / 1389
org.apache.spark.sql.catalyst.analysis.ResolveHints$DisableHints
0 / 1531442 0 / 675
```
After this pr:
```
19:20:49.183 WARN org.apache.spark.sql.TPCDSQuerySuite:
=== Metrics of Analyzer/Optimizer Rules ===
Total number of runs: 407395
Total time: 34.801544775 seconds
Rule
Effective Time / Total Time Effective Runs / Total
Runs
org.apache.spark.sql.catalyst.optimizer.Optimizer$OptimizeSubqueries
3187947814 / 6644582977 49 / 2778
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggregateFunctions
791813409 / 2288352046 49 / 2141
org.apache.spark.sql.catalyst.optimizer.ColumnPruning
485946590 / 2276408285 335 / 6401
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubquery
1895942724 / 2006154058 51 / 2141
org.apache.spark.sql.catalyst.analysis.DecimalPrecision
1210679550 / 1391366516 361 / 2141
org.apache.spark.sql.catalyst.optimizer.InferFiltersFromConstraints
721109407 / 835950522 284 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences
572499783 / 725995054 815 / 2141
org.apache.spark.sql.catalyst.optimizer.PruneFilters
19135828 / 696976637 5 / 5012
org.apache.spark.sql.catalyst.analysis.UpdateAttributeNullability
10856939 / 647795278 12 / 3448
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$ImplicitTypeCasts
301313199 / 631767828 78 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveMissingReferences
10171735 / 571812382 10 / 2141
org.apache.spark.sql.catalyst.optimizer.BooleanSimplification
12116466 / 567023853 16 / 3623
org.apache.spark.sql.catalyst.optimizer.RemoveRedundantAliases
14709526 / 470582394 29 / 3623
org.apache.spark.sql.execution.datasources.PruneFileSourcePartitions
428210354 / 470267955 209 / 1389
org.apache.spark.sql.catalyst.optimizer.PushDownPredicates
368708753 / 459852514 999 / 6634
org.apache.spark.sql.catalyst.optimizer.ReorderJoin
335167605 / 398352520 176 / 3623
org.apache.spark.sql.catalyst.optimizer.OptimizeCsvJsonExprs
0 / 376867104 0 / 3623
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$WidenSetOperationTypes
246240254 / 369841628 24 / 2141
org.apache.spark.sql.execution.datasources.SchemaPruning
0 / 355418010 0 / 1389
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$FunctionArgumentConversion
148160582 / 321067114 22 / 2141
org.apache.spark.sql.catalyst.optimizer.FoldablePropagation
3512250 / 298636116 6 / 3623
org.apache.spark.sql.catalyst.analysis.Analyzer$ExtractGenerator
0 / 265088799 0 / 2141
org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog
157355983 / 259126590 24 / 2141
org.apache.spark.sql.catalyst.optimizer.NullPropagation
16283311 / 251786857 44 / 3623
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions
108190512 / 242785071 426 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$CaseWhenCoercion
131938944 / 241967046 33 / 2141
org.apache.spark.sql.catalyst.optimizer.SimplifyBinaryComparison
272907 / 238604794 1 / 3623
org.apache.spark.sql.catalyst.optimizer.SimplifyConditionals
0 / 212754945 0 / 3623
org.apache.spark.sql.catalyst.optimizer.PushFoldableIntoBranches
6538075 / 212124151 20 / 3623
org.apache.spark.sql.catalyst.optimizer.UnwrapCastInBinaryComparison
4048477 / 210167101 8 / 3623
org.apache.spark.sql.catalyst.optimizer.ConstantFolding
62704250 / 207895760 219 / 3623
org.apache.spark.sql.catalyst.analysis.CTESubstitution
42936893 / 204813186 58 / 735
org.apache.spark.sql.catalyst.analysis.TypeCoercion$PromoteStrings
8090984 / 200608865 11 / 2141
org.apache.spark.sql.catalyst.optimizer.SimplifyCaseConversionExpressions
0 / 193802033 0 / 3623
org.apache.spark.sql.catalyst.analysis.ResolveTimeZone
98996067 / 189887683 537 / 2141
org.apache.spark.sql.execution.datasources.FindDataSourceTable
163652549 / 189723965 296 / 2141
org.apache.spark.sql.catalyst.optimizer.RewriteCorrelatedScalarSubquery
5072527 / 187662600 8 / 3623
org.apache.spark.sql.catalyst.optimizer.ReorderAssociativeOperator
0 / 184300830 0 / 3623
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$Division
19447010 / 183294771 10 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$InConversion
3278833 / 180420638 4 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercion$BooleanEquality
0 / 179031146 0 / 2141
org.apache.spark.sql.catalyst.optimizer.SimplifyExtractValueOps
0 / 178917033 0 / 3623
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$StringLiteralCoercion
0 / 170818541 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$DateTimeOperations
0 / 169515865 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$IfCoercion
0 / 169027389 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$IntegralDivision
0 / 167919905 0 / 2141
org.apache.spark.sql.catalyst.optimizer.OptimizeUpdateFields
0 / 166598755 0 / 4358
org.apache.spark.sql.catalyst.optimizer.OptimizeIn
7078167 / 166071136 27 / 3623
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveBinaryArithmetic
5831247 / 163639681 25 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations
108927809 / 159568405 296 / 2151
org.apache.spark.sql.catalyst.optimizer.SimplifyCasts
16133997 / 158711617 83 / 3623
org.apache.spark.sql.catalyst.optimizer.RemoveDispensableExpressions
0 / 157812395 0 / 3623
org.apache.spark.sql.catalyst.optimizer.LikeSimplification
731219 / 153753185 1 / 3623
org.apache.spark.sql.catalyst.optimizer.ReplaceNullWithFalseInPredicate
1989842 / 149298864 17 / 3623
org.apache.spark.sql.catalyst.analysis.ResolveCreateNamedStruct
0 / 140379120 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$ConcatCoercion
0 / 138616802 0 / 2141
org.apache.spark.sql.catalyst.optimizer.OptimizeWindowFunctions
0 / 132845471 0 / 3623
org.apache.spark.sql.catalyst.analysis.TimeWindowing
0 / 131163499 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$EltCoercion
0 / 130479486 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveWindowOrder
6574334 / 121824438 28 / 2141
org.apache.spark.sql.catalyst.analysis.ResolveHigherOrderFunctions
0 / 114267659 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveWindowFrame
5553055 / 110975706 37 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$MapZipWithCoercion
0 / 110007546 0 / 2141
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$WindowFrameCoercion
0 / 109886578 0 / 2141
org.apache.spark.sql.catalyst.optimizer.RemoveNoopOperators
16885284 / 109470038 213 / 7790
org.apache.spark.sql.execution.dynamicpruning.PartitionPruning
101132556 / 109418855 222 / 1389
org.apache.spark.sql.catalyst.analysis.ApplyCharTypePadding
64700613 / 109056775 106 / 665
org.apache.spark.sql.catalyst.analysis.TypeCoercionBase$StackCoercion
0 / 107711558 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ExtractWindowExpressions
26581041 / 107586045 38 / 2141
org.apache.spark.sql.execution.aggregate.ResolveEncodersInScalaAgg
0 / 107451058 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$AddMetadataColumns
0 / 105330817 0 / 2151
org.apache.spark.sql.catalyst.optimizer.ConstantPropagation
0 / 85703067 0 / 3623
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveGroupingAnalytics
21926898 / 84729481 12 / 2141
org.apache.spark.sql.execution.python.ExtractPythonUDFs
0 / 84225080 0 / 1389
org.apache.spark.sql.catalyst.optimizer.RemoveNoopUnion
0 / 84138791 0 / 1389
org.apache.spark.sql.catalyst.optimizer.PushDownLeftSemiAntiJoin
3828841 / 81042619 32 / 3623
org.apache.spark.sql.catalyst.optimizer.CollapseProject
29006524 / 80522092 188 / 5012
org.apache.spark.sql.catalyst.analysis.ResolveLambdaVariables
0 / 79404306 0 / 2141
org.apache.spark.sql.execution.datasources.v2.V2ScanRelationPushDown
0 / 76819282 0 / 1389
org.apache.spark.sql.catalyst.analysis.CleanupAliases
41832073 / 73999977 514 / 1179
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRandomSeed
0 / 72832470 0 / 2141
org.apache.spark.sql.catalyst.optimizer.DecimalAggregates
27441059 / 70954995 124 / 1513
org.apache.spark.sql.execution.dynamicpruning.CleanupDynamicPruningFilters
0 / 66211569 0 / 1389
org.apache.spark.sql.catalyst.optimizer.CollapseRepartition
0 / 62738683 0 / 3623
org.apache.spark.sql.catalyst.optimizer.ReplaceUpdateFieldsExpression
0 / 62148688 0 / 1389
org.apache.spark.sql.catalyst.optimizer.EliminateOuterJoin
8447285 / 60438670 15 / 3623
org.apache.spark.sql.catalyst.analysis.PullOutNondeterministic
0 / 59813330 0 / 665
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveTables
0 / 58117890 0 / 2151
org.apache.spark.sql.catalyst.optimizer.SimplifyConditionalsInPredicate
1508788 / 57342124 20 / 3623
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAliases
10736150 / 57202130 53 / 2141
org.apache.spark.sql.catalyst.optimizer.ComputeCurrentTime
0 / 56992025 0 / 1389
org.apache.spark.sql.catalyst.optimizer.CombineUnions
6366672 / 56483290 48 / 5012
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer
0 / 53273714 0 / 2141
org.apache.spark.sql.catalyst.optimizer.GetCurrentDatabaseAndCatalog
0 / 52510777 0 / 1389
org.apache.spark.sql.catalyst.optimizer.RemoveRedundantAggregates
0 / 48833758 0 / 3623
org.apache.spark.sql.catalyst.analysis.Analyzer$GlobalAggregates
3017199 / 48774564 82 / 2141
org.apache.spark.sql.catalyst.optimizer.EliminateLimits
505068 / 48749551 11 / 3623
org.apache.spark.sql.catalyst.optimizer.EliminateSerialization
0 / 48564056 0 / 3623
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveNewInstance
0 / 48417148 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveUpCast
0 / 48092004 0 / 2141
org.apache.spark.sql.catalyst.optimizer.RewritePredicateSubquery
17909018 / 46921409 26 / 1389
org.apache.spark.sql.catalyst.optimizer.NormalizeFloatingNumbers
0 / 45951107 0 / 1389
org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates
17548952 / 42800713 16 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveOrdinalInOrderByAndGroupBy
1319508 / 42612744 8 / 2141
org.apache.spark.sql.catalyst.optimizer.TransposeWindow
0 / 41810245 0 / 3623
org.apache.spark.sql.catalyst.optimizer.ReplaceExpressions
246644 / 41676496 2 / 1389
org.apache.spark.sql.catalyst.optimizer.CollapseWindow
0 / 40713724 0 / 3623
org.apache.spark.sql.catalyst.optimizer.ReassignLambdaVariableID
0 / 40113987 0 / 1389
org.apache.spark.sql.catalyst.analysis.ResolveCatalogs
0 / 38439435 0 / 2151
org.apache.spark.sql.catalyst.analysis.ResolveUnion
0 / 37929673 0 / 2141
org.apache.spark.sql.catalyst.optimizer.RewriteNonCorrelatedExists
0 / 37915363 0 / 1389
org.apache.spark.sql.catalyst.optimizer.EliminateMapObjects
0 / 37676082 0 / 1389
org.apache.spark.sql.catalyst.optimizer.PushLeftSemiLeftAntiThroughJoin
1441014 / 37218033 3 / 3623
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggAliasInGroupBy
0 / 37119917 0 / 2141
org.apache.spark.sql.catalyst.optimizer.PushProjectionThroughUnion
1989323 / 36987655 4 / 3623
org.apache.spark.sql.catalyst.optimizer.EliminateResolvedHint
0 / 35106688 0 / 1389
org.apache.spark.sql.catalyst.optimizer.LimitPushDownThroughWindow
0 / 35025111 0 / 3623
org.apache.spark.sql.catalyst.optimizer.LimitPushDown
0 / 35003526 0 / 3623
org.apache.spark.sql.catalyst.optimizer.CombineFilters
0 / 35002506 0 / 3623
org.apache.spark.sql.execution.datasources.PreprocessTableCreation
6997955 / 34806015 7 / 665
org.apache.spark.sql.catalyst.optimizer.OptimizeRepartition
0 / 34460212 0 / 3623
org.apache.spark.sql.execution.datasources.FallBackFileSourceV2
0 / 33817781 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveNamespace
0 / 33816969 0 / 2151
org.apache.spark.sql.catalyst.analysis.Analyzer$HandleNullInputsForUDF
0 / 33606387 0 / 665
org.apache.spark.sql.catalyst.optimizer.PropagateEmptyRelation
3507699 / 32850408 5 / 2783
org.apache.spark.sql.catalyst.analysis.Analyzer$LookupFunctions
0 / 30858879 0 / 675
org.apache.spark.sql.execution.datasources.ResolveSQLOnFile
0 / 30577197 0 / 2141
org.apache.spark.sql.catalyst.optimizer.EliminateSorts
705474 / 30334385 2 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolvePivot
0 / 29561352 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveGenerate
0 / 28867184 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveOutputRelation
0 / 28505610 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveInsertInto
0 / 28160498 0 / 2151
org.apache.spark.sql.catalyst.optimizer.ReplaceDeduplicateWithAggregate
0 / 27949918 0 / 1431
org.apache.spark.sql.catalyst.analysis.ResolveTableValuedFunctions
0 / 27795588 0 / 2151
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveEncodersInUDF
0 / 27686323 0 / 665
org.apache.spark.sql.catalyst.analysis.ResolveInlineTables
0 / 27576064 0 / 2141
org.apache.spark.sql.catalyst.analysis.ResolvePartitionSpec
0 / 27425328 0 / 2151
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubqueryColumnAliases
0 / 27046064 0 / 2141
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveNaturalAndUsingJoin
0 / 26922025 0 / 2141
org.apache.spark.sql.catalyst.optimizer.PushExtraPredicateThroughJoin
4260815 / 25801044 7 / 1393
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveUserSpecifiedColumns
0 / 25738491 0 / 2151
org.apache.spark.sql.catalyst.analysis.EliminateSubqueryAliases
16536843 / 21034764 296 / 1389
org.apache.spark.sql.catalyst.optimizer.PushPredicateThroughNonJoin
0 / 18230332 0 / 1389
org.apache.spark.sql.catalyst.optimizer.RewriteDistinctAggregates
0 / 15619359 0 / 1389
org.apache.spark.sql.execution.analysis.DetectAmbiguousSelfJoin
0 / 15424755 0 / 665
org.apache.spark.sql.catalyst.analysis.SubstituteUnresolvedOrdinals
2152410 / 14785799 8 / 735
org.apache.spark.sql.catalyst.optimizer.ExtractPythonUDFFromJoinCondition
0 / 14691822 0 / 1389
org.apache.spark.sql.execution.python.ExtractPythonUDFFromAggregate
0 / 14488308 0 / 1389
org.apache.spark.sql.execution.python.ExtractGroupingPythonUDFFromAggregate
0 / 12947972 0 / 1389
org.apache.spark.sql.catalyst.analysis.EliminateView
0 / 12601664 0 / 1389
org.apache.spark.sql.catalyst.optimizer.RemoveRepetitionFromGroupExpressions
923862 / 12317031 2 / 1391
org.apache.spark.sql.catalyst.optimizer.ReplaceExceptWithFilter
0 / 11704687 0 / 1431
org.apache.spark.sql.catalyst.optimizer.RewriteExceptAll
0 / 11161156 0 / 1431
org.apache.spark.sql.catalyst.optimizer.InferFiltersFromGenerate
0 / 10349077 0 / 1389
org.apache.spark.sql.catalyst.optimizer.ObjectSerializerPruning
0 / 10090026 0 / 1389
org.apache.spark.sql.execution.datasources.v2.V2Writes
0 / 10062388 0 / 1389
org.apache.spark.sql.catalyst.optimizer.CombineTypedFilters
0 / 9887692 0 / 1389
org.apache.spark.sql.catalyst.analysis.Analyzer$WindowsSubstitution
0 / 9816823 0 / 735
org.apache.spark.sql.catalyst.optimizer.RemoveLiteralFromGroupExpressions
0 / 9542760 0 / 1391
org.apache.spark.sql.catalyst.optimizer.ReplaceIntersectWithSemiJoin
1859861 / 9281119 24 / 1431
org.apache.spark.sql.catalyst.optimizer.ReplaceDistinctWithAggregate
1986456 / 8370500 42 / 1431
org.apache.spark.sql.catalyst.optimizer.RewriteIntersectAll
0 / 8125096 0 / 1431
org.apache.spark.sql.catalyst.analysis.EliminateUnions
0 / 8104101 0 / 735
org.apache.spark.sql.catalyst.optimizer.ReplaceExceptWithAntiJoin
507389 / 8092301 1 / 1431
org.apache.spark.sql.catalyst.optimizer.OptimizeLimitZero
0 / 7931249 0 / 1389
org.apache.spark.sql.execution.datasources.DataSourceAnalysis
2462611 / 7913135 24 / 665
org.apache.spark.sql.catalyst.analysis.UpdateOuterReferences
0 / 7500488 0 / 665
org.apache.spark.sql.catalyst.analysis.ResolveHints$ResolveJoinStrategyHints
0 / 7303284 0 / 675
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAlterTableChanges
0 / 6274915 0 / 665
org.apache.spark.sql.catalyst.analysis.ResolveCommandsWithIfExists
0 / 6211839 0 / 665
org.apache.spark.sql.catalyst.analysis.ResolveHints$ResolveCoalesceHints
0 / 6150465 0 / 675
org.apache.spark.sql.execution.datasources.PreprocessTableInsertion
0 / 6003880 0 / 665
org.apache.spark.sql.catalyst.analysis.ResolveHints$RemoveAllHints
0 / 5882558 0 / 665
org.apache.spark.sql.catalyst.optimizer.CheckCartesianProducts
0 / 5336536 0 / 2778
org.apache.spark.sql.catalyst.optimizer.EliminateAggregateFilter
0 / 5205627 0 / 3623
org.apache.spark.sql.catalyst.optimizer.CombineConcats
0 / 3983214 0 / 3623
org.apache.spark.sql.execution.OptimizeMetadataOnlyQuery
0 / 3364063 0 / 1389
org.apache.spark.sql.catalyst.optimizer.CostBasedJoinReorder
0 / 3272873 0 / 1389
org.apache.spark.sql.catalyst.optimizer.EliminateDistinct
0 / 2649763 0 / 1389
org.apache.spark.sql.catalyst.analysis.ResolveHints$DisableHints
0 / 1525243 0 / 675
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
