Eric5553 commented on a change in pull request #26977: [SPARK-30326][SQL] Raise
exception if analyzer exceed max iterations
URL: https://github.com/apache/spark/pull/26977#discussion_r361798780
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala
##########
@@ -45,16 +45,23 @@ abstract class RuleExecutor[TreeType <: TreeNode[_]]
extends Logging {
* An execution strategy for rules that indicates the maximum number of
executions. If the
* execution reaches fix point (i.e. converge) before maxIterations, it will
stop.
*/
- abstract class Strategy { def maxIterations: Int }
+ abstract class Strategy {
+
+ /** The maximum number of executions. */
+ def maxIterations: Int
+
+ /** Whether to throw exception when exceeding the maximum number. */
+ def errorOnExceed: Boolean
Review comment:
> Why did you add this in not `RuleExecutor` but `Strategy`?
@maropu The setting `SQLConf.ANALYZER_MAX_ITERATIONS` only controls a subset
of Analyzer Rule batches (Hints, Substitution, Resolution and Cleanup). For
other Batches with `Once` strategy, the original RuleExecutor logic will not
raise exception or log warning. Which means within Analyzer, it will take
different actions for different Batches/Strategy. So I think the
`errorOnExceed` behavior is more like binding with maxIteration in Strategy. By
doing this, we could have finer granularity to support the feature and provide
better flexibility to control behaviors for different batches in the future.
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
