Github user yinxusen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11053#discussion_r53387890
--- Diff: docs/ml-guide.md ---
@@ -864,91 +304,12 @@ The `ParamMap` which produces the best evaluation
metric is selected as the best
<div class="codetabs">
<div data-lang="scala" markdown="1">
-{% highlight scala %}
-import org.apache.spark.ml.evaluation.RegressionEvaluator
-import org.apache.spark.ml.regression.LinearRegression
-import org.apache.spark.ml.tuning.{ParamGridBuilder, TrainValidationSplit}
-
-// Prepare training and test data.
-val data =
sqlContext.read.format("libsvm").load("data/mllib/sample_linear_regression_data.txt")
-val Array(training, test) = data.randomSplit(Array(0.9, 0.1), seed = 12345)
-
-val lr = new LinearRegression()
-
-// We use a ParamGridBuilder to construct a grid of parameters to search
over.
-// TrainValidationSplit will try all combinations of values and determine
best model using
-// the evaluator.
-val paramGrid = new ParamGridBuilder()
- .addGrid(lr.regParam, Array(0.1, 0.01))
- .addGrid(lr.fitIntercept)
- .addGrid(lr.elasticNetParam, Array(0.0, 0.5, 1.0))
- .build()
-
-// In this case the estimator is simply the linear regression.
-// A TrainValidationSplit requires an Estimator, a set of Estimator
ParamMaps, and an Evaluator.
-val trainValidationSplit = new TrainValidationSplit()
- .setEstimator(lr)
- .setEvaluator(new RegressionEvaluator)
- .setEstimatorParamMaps(paramGrid)
- // 80% of the data will be used for training and the remaining 20% for
validation.
- .setTrainRatio(0.8)
-
-// Run train validation split, and choose the best set of parameters.
-val model = trainValidationSplit.fit(training)
-
-// Make predictions on test data. model is the model with combination of
parameters
-// that performed best.
-model.transform(test)
- .select("features", "label", "prediction")
- .show()
-
+{% include_example
scala/org/apache/spark/examples/ml/ModelSelectionViaTrainValidationSplitExample.scala
%}
{% endhighlight %}
--- End diff --
remove the line
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
