Github user manishamde commented on a diff in the pull request:
https://github.com/apache/spark/pull/3374#discussion_r20624623
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/tree/GradientBoostedTreesSuite.scala
---
@@ -23,104 +23,95 @@ import org.apache.spark.mllib.regression.LabeledPoint
import org.apache.spark.mllib.tree.configuration.Algo._
import org.apache.spark.mllib.tree.configuration.{BoostingStrategy,
Strategy}
import org.apache.spark.mllib.tree.impurity.Variance
-import org.apache.spark.mllib.tree.loss.{SquaredError, LogLoss}
+import org.apache.spark.mllib.tree.loss.{AbsoluteError, SquaredError,
LogLoss}
import org.apache.spark.mllib.util.MLlibTestSparkContext
/**
- * Test suite for [[GradientBoosting]].
+ * Test suite for [[GradientBoostedTrees]].
*/
-class GradientBoostingSuite extends FunSuite with MLlibTestSparkContext {
+class GradientBoostedTreesSuite extends FunSuite with
MLlibTestSparkContext {
test("Regression with continuous features: SquaredError") {
- GradientBoostingSuite.testCombinations.foreach {
+ GradientBoostedTreesSuite.testCombinations.foreach {
case (numIterations, learningRate, subsamplingRate) =>
val arr =
EnsembleTestHelper.generateOrderedLabeledPoints(numFeatures = 10, 100)
- val rdd = sc.parallelize(arr)
- val categoricalFeaturesInfo = Map.empty[Int, Int]
+ val rdd = sc.parallelize(arr, 2)
- val remappedInput = rdd.map(x => new LabeledPoint((x.label * 2) -
1, x.features))
val treeStrategy = new Strategy(algo = Regression, impurity =
Variance, maxDepth = 2,
- numClassesForClassification = 2, categoricalFeaturesInfo =
categoricalFeaturesInfo,
- subsamplingRate = subsamplingRate)
-
- val dt = DecisionTree.train(remappedInput, treeStrategy)
-
- val boostingStrategy = new BoostingStrategy(Regression,
numIterations, SquaredError,
- learningRate, 1, treeStrategy)
+ categoricalFeaturesInfo = Map.empty, subsamplingRate =
subsamplingRate)
+ val boostingStrategy =
+ new BoostingStrategy(treeStrategy, SquaredError, numIterations,
learningRate)
- val gbt = GradientBoosting.trainRegressor(rdd, boostingStrategy)
- assert(gbt.weakHypotheses.size === numIterations)
- val gbtTree = gbt.weakHypotheses(0)
+ val gbt = GradientBoostedTrees.train(rdd, boostingStrategy)
+ assert(gbt.trees.size === numIterations)
EnsembleTestHelper.validateRegressor(gbt, arr, 0.03)
+ val remappedInput = rdd.map(x => new LabeledPoint((x.label * 2) -
1, x.features))
+ val dt = DecisionTree.train(remappedInput, treeStrategy)
+
// Make sure trees are the same.
- assert(gbtTree.toString == dt.toString)
+ assert(gbt.trees.head.toString == dt.toString)
}
}
test("Regression with continuous features: Absolute Error") {
- GradientBoostingSuite.testCombinations.foreach {
+ GradientBoostedTreesSuite.testCombinations.foreach {
case (numIterations, learningRate, subsamplingRate) =>
val arr =
EnsembleTestHelper.generateOrderedLabeledPoints(numFeatures = 10, 100)
- val rdd = sc.parallelize(arr)
- val categoricalFeaturesInfo = Map.empty[Int, Int]
+ val rdd = sc.parallelize(arr, 2)
- val remappedInput = rdd.map(x => new LabeledPoint((x.label * 2) -
1, x.features))
val treeStrategy = new Strategy(algo = Regression, impurity =
Variance, maxDepth = 2,
- numClassesForClassification = 2, categoricalFeaturesInfo =
categoricalFeaturesInfo,
- subsamplingRate = subsamplingRate)
-
- val dt = DecisionTree.train(remappedInput, treeStrategy)
+ categoricalFeaturesInfo = Map.empty, subsamplingRate =
subsamplingRate)
+ val boostingStrategy =
+ new BoostingStrategy(treeStrategy, AbsoluteError, numIterations,
learningRate)
- val boostingStrategy = new BoostingStrategy(Regression,
numIterations, SquaredError,
--- End diff --
Here are my findings. I added two more test cases with numIterations = 100.
```
numIterations = 10, learningRate = 1.0, subsamplingRate = 1.0
metric = 0.8400000000000005
numIterations = 100, learningRate = 1.0, subsamplingRate = 1.0
metric = 0.5344090056285183
numIterations = 10, learningRate = 0.1, subsamplingRate = 1.0
metric = 0.08399999999999984
numIterations = 10, learningRate = 1.0, subsamplingRate = 0.75
metric = 0.8102205882352937
numIterations = 100, learningRate = 1.0, subsamplingRate = 0.75
metric = 0.565608647936787
numIterations = 10, learningRate = 0.1, subsamplingRate = 0.75
metric = 0.11179411764705861
```
A learning rate of 1 doesn't work very well especially with low number of
iterations. Our default learning rate is 0.1 which should be fine.
Suggestion: We remove the learningRate = 1 option from the absolute error
test. I can do more testing to check what settings work well for our GBT model
and include it as a part of the documentation. I will also compare with
scikit-learn to see how much additional loss do we get from an ideal
implementation during the documentation phase.
cc: @jkbradley
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
