This is an automated email from the ASF dual-hosted git repository.

zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new fce7d51a3760 [SPARK-60461][MLLIB] Restore Strategy constructor binary 
compatibility
fce7d51a3760 is described below

commit fce7d51a3760953097fe34df8aa9a4f5326c74ff
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Fri Oct 23 08:21:05 2026 +0800

    [SPARK-60461][MLLIB] Restore Strategy constructor binary compatibility
    
    ### What changes were proposed in this pull request?
    
    Restore the 15-argument `Strategy` constructor used in Spark 4.2, 
delegating to the primary constructor with pruning enabled. Keep `bootstrap` in 
its original parameter position, after `minWeightFractionPerNode`, so its 
constructor default getter continues to return `false`. Make `pruneTree` 
`private[spark]`, preserve both `bootstrap` and `pruneTree` in `Strategy.copy`, 
and remove the corresponding MiMa constructor exclusion.
    
    Please backport to `branch-4.x` and `branch-4.3`.
    
    ### Why are the changes needed?
    
    #55763 added `pruneTree` before `bootstrap`, changing the JVM constructor 
from 15 to 16 parameters. Existing applications compiled against Spark 4.2 fail 
with `NoSuchMethodError` on the current 4.3 branch. For example, a client 
compiled against 4.2 with `new Strategy(Algo.Classification, Gini, maxDepth = 
3)` fails at construction.
    
    Adding only the old constructor would leave another compatibility problem: 
the 15th constructor default getter would return the new pruning default 
(`true`) instead of the old bootstrap default (`false`). Retaining the original 
parameter positions preserves both linkage and behavior.
    
    `private[spark] var bootstrap` restricts Scala access to the stored 
property and its accessors. It does not restrict callers from supplying that 
constructor argument or make the constructor package-private. Spark 4.2's 
15-argument constructor was public, so the compatibility constructor must also 
remain public.
    
    For example, code in a package outside Spark can supply `bootstrap` either 
by name, using defaults for the other optional arguments, or explicitly as the 
last of all 15 arguments:
    
    ```scala
    package example
    
    import org.apache.spark.mllib.tree.configuration.{Algo, QuantileStrategy, 
Strategy}
    import org.apache.spark.mllib.tree.impurity.Gini
    
    object StrategyClient {
      val named = new Strategy(Algo.Classification, Gini, maxDepth = 3, 
bootstrap = true)
    
      val explicit = new Strategy(
        Algo.Classification, Gini, 3, 2, 32, QuantileStrategy.Sort,
        Map.empty[Int, Int], 1, 0.0, 256, 1.0, false, 10, 0.0, true)
    }
    ```
    
    Both calls compile against Spark 4.2; the final `true` in the second call 
is the `bootstrap` argument. The compiler fills the omitted defaults in the 
first call and invokes the same public 15-argument constructor.
    
    Reading or writing the property afterward is still rejected in that 
external package. These examples intentionally do not compile:
    
    ```scala
    object InvalidAccess {
      val bootstrap = StrategyClient.named.bootstrap
      StrategyClient.named.bootstrap = false
    }
    ```
    
    The constructor examples compile against both Spark 4.2 and the patched 
`Strategy`, while both property-access examples are rejected by the Scala 
compiler in each version.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes. Applications compiled against Spark 4.2 can construct `Strategy` on 
Spark 4.3 again, preserving the default and explicit bootstrap values. Pruning 
remains enabled by default.
    
    ### How was this patch tested?
    
    Added three regression tests in `DecisionTreeSuite` covering the old JVM 
constructor signature, the bootstrap constructor default getter, and 
independent pruning/bootstrapping configuration, including copying all four 
combinations of the two flags. All three passed using the ScalaTest runner with 
the `Strategy` test-name filter with locally cached Spark dependencies.
    
    Compiled a separate Scala client against the published Spark 4.2.0 MLlib 
artifact. It reproduces `NoSuchMethodError` with the unpatched 4.3 class and 
runs successfully with the patched class, preserving the default bootstrap 
value and both explicit boolean values. The changed class and regression suite 
were compiled directly with Scala 2.13.18. Full SBT and MiMa runs were not 
performed.
    
    Audited the `Strategy` constructor/factory call sites, both training 
copies, ML classifier parameter conversion, and recursive node conversion; no 
further propagation changes were needed. Also ran the two existing pruning 
regression tests and three temporary integration tests covering legacy 
decision-tree/random-forest classification and regression, ML classifiers, and 
gradient boosting with pruning enabled and disabled. All eight focused tests 
passed.
    
    Checked whitespace, ASCII, and changed-line lengths. The patch applies 
cleanly to `branch-4.3`.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: OpenAI Codex (GPT-6)
    
    Closes #59704 from zhengruifeng/ml-strategy-constructor-compatibility.
    
    Authored-by: Ruifeng Zheng <[email protected]>
    Signed-off-by: Ruifeng Zheng <[email protected]>
---
 .../spark/mllib/tree/configuration/Strategy.scala  | 35 +++++++++++++++++---
 .../spark/mllib/tree/DecisionTreeSuite.scala       | 37 ++++++++++++++++++++++
 project/MimaExcludes.scala                         |  2 --
 3 files changed, 67 insertions(+), 7 deletions(-)

diff --git 
a/mllib/src/main/scala/org/apache/spark/mllib/tree/configuration/Strategy.scala 
b/mllib/src/main/scala/org/apache/spark/mllib/tree/configuration/Strategy.scala
index ab6c6f746756..12115a6035bf 100644
--- 
a/mllib/src/main/scala/org/apache/spark/mllib/tree/configuration/Strategy.scala
+++ 
b/mllib/src/main/scala/org/apache/spark/mllib/tree/configuration/Strategy.scala
@@ -65,8 +65,6 @@ import org.apache.spark.mllib.tree.impurity.{Entropy, Gini, 
Impurity, Variance}
  *                           E.g. 10 means that the cache will get 
checkpointed every 10 updates. If
  *                           the checkpoint directory is not set in
  *                           [[org.apache.spark.SparkContext]], this setting 
is ignored.
- * @param pruneTree If this is true, the final training tree will undergo a 
pruning in which
- *                  nodes with the same prediction are merged.
  */
 @Since("1.0.0")
 class Strategy @Since("1.3.0") (
@@ -84,8 +82,8 @@ class Strategy @Since("1.3.0") (
     @Since("1.2.0") @BeanProperty var useNodeIdCache: Boolean = false,
     @Since("1.2.0") @BeanProperty var checkpointInterval: Int = 10,
     @Since("3.0.0") @BeanProperty var minWeightFractionPerNode: Double = 0.0,
-    @Since("4.3.0") @BeanProperty var pruneTree: Boolean = true,
-    @BeanProperty private[spark] var bootstrap: Boolean = false) extends 
Serializable {
+    @BeanProperty private[spark] var bootstrap: Boolean = false,
+    @BeanProperty private[spark] var pruneTree: Boolean = true) extends 
Serializable {
 
   /**
    */
@@ -102,6 +100,31 @@ class Strategy @Since("1.3.0") (
   }
 
   // scalastyle:off argcount
+  /**
+   * Retains the constructor signature used before the pruning parameter was 
introduced.
+   */
+  def this(
+      algo: Algo,
+      impurity: Impurity,
+      maxDepth: Int,
+      numClasses: Int,
+      maxBins: Int,
+      quantileCalculationStrategy: QuantileStrategy,
+      categoricalFeaturesInfo: Map[Int, Int],
+      minInstancesPerNode: Int,
+      minInfoGain: Double,
+      maxMemoryInMB: Int,
+      subsamplingRate: Double,
+      useNodeIdCache: Boolean,
+      checkpointInterval: Int,
+      minWeightFractionPerNode: Double,
+      bootstrap: Boolean) = {
+    this(algo, impurity, maxDepth, numClasses, maxBins, 
quantileCalculationStrategy,
+      categoricalFeaturesInfo, minInstancesPerNode, minInfoGain, maxMemoryInMB,
+      subsamplingRate, useNodeIdCache, checkpointInterval, 
minWeightFractionPerNode,
+      bootstrap, pruneTree = true)
+  }
+
   /**
    * Backwards compatible constructor for 
[[org.apache.spark.mllib.tree.configuration.Strategy]]
    */
@@ -204,7 +227,9 @@ class Strategy @Since("1.3.0") (
     new Strategy(algo, impurity, maxDepth, numClasses, maxBins,
       quantileCalculationStrategy, categoricalFeaturesInfo, 
minInstancesPerNode,
       minInfoGain, maxMemoryInMB, subsamplingRate, useNodeIdCache,
-      checkpointInterval, minWeightFractionPerNode, pruneTree)
+      checkpointInterval, minWeightFractionPerNode,
+      bootstrap = bootstrap,
+      pruneTree = pruneTree)
   }
 }
 
diff --git 
a/mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala 
b/mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala
index 2f5579acea46..488f7b9e200e 100644
--- a/mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala
+++ b/mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala
@@ -25,6 +25,7 @@ import org.apache.spark.mllib.linalg.Vectors
 import org.apache.spark.mllib.regression.LabeledPoint
 import org.apache.spark.mllib.tree.configuration.Algo._
 import org.apache.spark.mllib.tree.configuration.FeatureType._
+import org.apache.spark.mllib.tree.configuration.QuantileStrategy.Sort
 import org.apache.spark.mllib.tree.configuration.Strategy
 import org.apache.spark.mllib.tree.impurity.{Entropy, Gini, Variance}
 import org.apache.spark.mllib.tree.model._
@@ -35,6 +36,42 @@ import org.apache.spark.util.Utils
 
 class DecisionTreeSuite extends SparkFunSuite with MLlibTestSparkContext {
 
+  /////////////////////////////////////////////////////////////////////////////
+  // Tests for Strategy configuration
+  /////////////////////////////////////////////////////////////////////////////
+
+  test("Strategy should retain the constructor signature used by Spark 4.2") {
+    assert(classOf[Strategy].getConstructors.exists(_.getParameterCount == 15))
+    for (bootstrap <- Seq(false, true)) {
+      val strategy = new Strategy(Classification, Gini, 3, 2, 32, Sort, 
Map.empty[Int, Int],
+        1, 0.0, 256, 1.0, false, 10, 0.0, bootstrap)
+
+      assert(strategy.bootstrap == bootstrap)
+      assert(strategy.pruneTree)
+    }
+  }
+
+  test("Strategy should retain the bootstrap default getter used by compiled 
callers") {
+    val bootstrapDefault = 
Strategy.getClass.getMethod("$lessinit$greater$default$15")
+    assert(bootstrapDefault.invoke(Strategy) == java.lang.Boolean.FALSE)
+  }
+
+  test("Strategy should configure pruning independently of bootstrapping") {
+    for {
+      bootstrap <- Seq(false, true)
+      pruneTree <- Seq(false, true)
+    } {
+      val strategy = new Strategy(Classification, Gini, 3,
+        bootstrap = bootstrap, pruneTree = pruneTree)
+      assert(strategy.bootstrap == bootstrap)
+      assert(strategy.pruneTree == pruneTree)
+
+      val copied = strategy.copy
+      assert(copied.bootstrap == bootstrap)
+      assert(copied.pruneTree == pruneTree)
+    }
+  }
+
   /////////////////////////////////////////////////////////////////////////////
   // Tests calling train()
   /////////////////////////////////////////////////////////////////////////////
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala
index 5e5bdf374e7c..f2029f1e97d9 100644
--- a/project/MimaExcludes.scala
+++ b/project/MimaExcludes.scala
@@ -116,8 +116,6 @@ object MimaExcludes {
     
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.TaskContext.addTaskInterruptListener"),
     // [SPARK-56700][SS] Make DataStreamReader.name public
     
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.sql.streaming.DataStreamReader.name"),
-    // [SPARK-34591][ML] Add pruneTree parameter to Strategy
-    
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.mllib.tree.configuration.Strategy.this"),
     // [SPARK-56395][SQL] Add NEAREST BY top-K ranking join
     
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.sql.Dataset.nearestByJoin"),
     // [SPARK-57332][SQL] MySQLDialect no longer overrides the visit methods 
below; the public


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to