Github user jkbradley commented on a diff in the pull request:
    --- Diff: docs/ ---
    @@ -60,152 +60,37 @@ MLlib is under active development.
     The APIs marked `Experimental`/`DeveloperApi` may change in future 
     and the migration guide below will explain all changes between releases.
    -## From 1.6 to 2.0
    +## From 2.0 to 2.1
     ### Breaking changes
    -There were several breaking changes in Spark 2.0, which are outlined below.
    -**Linear algebra classes for DataFrame-based APIs**
    -Spark's linear algebra dependencies were moved to a new project, 
    -(see [SPARK-13944]( 
    -As part of this change, the linear algebra classes were copied to a new 
package, ``. 
    -The DataFrame-based APIs in `` now depend on the `` 
    -leading to a few breaking changes, predominantly in various model classes 
    -(see [SPARK-14810]( for 
a full list).
    -**Note:** the RDD-based APIs in `spark.mllib` continue to depend on the 
previous package `spark.mllib.linalg`.
    -_Converting vectors and matrices_
    -While most pipeline components support backward compatibility for loading, 
    -some existing `DataFrames` and pipelines in Spark versions prior to 2.0, 
that contain vector or matrix 
    -columns, may need to be migrated to the new `` vector and matrix 
    -Utilities for converting `DataFrame` columns from `spark.mllib.linalg` to 
`` types
    -(and vice versa) can be found in `spark.mllib.util.MLUtils`.
    -There are also utility methods available for converting single instances 
    -vectors and matrices. Use the `asML` method on a `mllib.linalg.Vector` / 
    -for converting to `ml.linalg` types, and 
    -`mllib.linalg.Vectors.fromML` / `mllib.linalg.Matrices.fromML` 
    -for converting to `mllib.linalg` types.
    -<div class="codetabs">
    -<div data-lang="scala"  markdown="1">
    -{% highlight scala %}
    -import org.apache.spark.mllib.util.MLUtils
    -// convert DataFrame columns
    -val convertedVecDF = MLUtils.convertVectorColumnsToML(vecDF)
    -val convertedMatrixDF = MLUtils.convertMatrixColumnsToML(matrixDF)
    -// convert a single vector or matrix
    -val mlVec: = mllibVec.asML
    -val mlMat: = mllibMat.asML
    -{% endhighlight %}
    -Refer to the [`MLUtils` Scala 
docs](api/scala/index.html#org.apache.spark.mllib.util.MLUtils$) for further 
    -<div data-lang="java" markdown="1">
    -{% highlight java %}
    -import org.apache.spark.mllib.util.MLUtils;
    -import org.apache.spark.sql.Dataset;
    -// convert DataFrame columns
    -Dataset<Row> convertedVecDF = MLUtils.convertVectorColumnsToML(vecDF);
    -Dataset<Row> convertedMatrixDF = 
    -// convert a single vector or matrix mlVec = mllibVec.asML(); mlMat = mllibMat.asML();
    -{% endhighlight %}
    -Refer to the [`MLUtils` Java 
docs](api/java/org/apache/spark/mllib/util/MLUtils.html) for further detail.
    -<div data-lang="python"  markdown="1">
    -{% highlight python %}
    -from pyspark.mllib.util import MLUtils
    -# convert DataFrame columns
    -convertedVecDF = MLUtils.convertVectorColumnsToML(vecDF)
    -convertedMatrixDF = MLUtils.convertMatrixColumnsToML(matrixDF)
    -# convert a single vector or matrix
    -mlVec = mllibVec.asML()
    -mlMat = mllibMat.asML()
    -{% endhighlight %}
    -Refer to the [`MLUtils` Python 
docs](api/python/pyspark.mllib.html#pyspark.mllib.util.MLUtils) for further 
     **Deprecated methods removed**
    -Several deprecated methods were removed in the `spark.mllib` and 
`` packages:
    -* `setScoreCol` in `ml.evaluation.BinaryClassificationEvaluator`
    -* `weights` in `LinearRegression` and `LogisticRegression` in ``
    -* `setMaxNumIterations` in `mllib.optimization.LBFGS` (marked as 
    -* `treeReduce` and `treeAggregate` in `mllib.rdd.RDDFunctions` (these 
functions are available on `RDD`s directly, and were marked as `DeveloperApi`)
    -* `defaultStategy` in `mllib.tree.configuration.Strategy`
    -* `build` in `mllib.tree.Node`
    -* libsvm loaders for multiclass and load/save labeledData methods in 
    -A full list of breaking changes can be found at 
    +* `setLabelCol` in `feature.ChiSqSelectorModel`
    +* `numTrees` in `classification.RandomForestClassificationModel` (This now 
refers to the Param called `numTrees`)
    +* `numTrees` in `regression.RandomForestRegressionModel` (This now refers 
to the Param called `numTrees`)
    +* `model` in `regression.LinearRegressionSummary`
    +* `validateParams` in `PipelineStage`
     ### Deprecations and changes of behavior
    -Deprecations in the `spark.mllib` and `` packages include:
    -* [SPARK-14984](
    - In ``, the `model` field has 
been deprecated.
    -* [SPARK-13784](
    - In `` and 
    - the `numTrees` parameter has been deprecated in favor of `getNumTrees` 
    -* [SPARK-13761](
    - In ``, the `validateParams` method has been 
    - We move all functionality in overridden methods to the corresponding 
    -* [SPARK-14829](
    - In `spark.mllib` package, `LinearRegressionWithSGD`, `LassoWithSGD`, 
`RidgeRegressionWithSGD` and `LogisticRegressionWithSGD` have been deprecated.
    - We encourage users to use `` and 
    -* [SPARK-14900](
    - In `spark.mllib.evaluation.MulticlassMetrics`, the parameters 
`precision`, `recall` and `fMeasure` have been deprecated in favor of 
    -* [SPARK-15644](
    - In `` and ``, the `context` 
method has been deprecated in favor of `session`.
    -* In ``, the `setLabelCol` method has 
been deprecated since it was not used by `ChiSqSelectorModel`.
    +* [SPARK-18592](
    +  Deprecate all setter methods for `DecisionTreeClassificationModel`, 
`GBTClassificationModel`, `RandomForestClassificationModel`, 
`DecisionTreeRegressionModel`, `GBTRegressionModel` and 
     **Changes of behavior**
    -Changes of behavior in the `spark.mllib` and `` packages include:
    -* [SPARK-7780](
    - `spark.mllib.classification.LogisticRegressionWithLBFGS` directly calls 
`` for binary classification now.
    - This will introduce the following behavior changes for 
    -    * The intercept will not be regularized when training binary 
classification model with L1/L2 Updater.
    -    * If users set without regularization, training with or without 
feature scaling will return the same solution by the same convergence rate.
    -* [SPARK-13429](
    - In order to provide better and consistent result with 
    - the default value of 
`spark.mllib.classification.LogisticRegressionWithLBFGS`: `convergenceTol` has 
been changed from 1E-4 to 1E-6.
    -* [SPARK-12363](
    - Fix a bug of `PowerIterationClustering` which will likely change its 
    -* [SPARK-13048](
    - `LDA` using the `EM` optimizer will keep the last checkpoint by default, 
if checkpointing is being used.
    -* [SPARK-12153](
    - `Word2Vec` now respects sentence boundaries. Previously, it did not 
handle them correctly.
    -* [SPARK-10574](
    - `HashingTF` uses `MurmurHash3` as default hash algorithm in both 
`` and `spark.mllib`.
    -* [SPARK-14768](
    - The `expectedType` argument for PySpark `Param` was removed.
    -* [SPARK-14931](
    - Some default `Param` values, which were mismatched between pipelines in 
Scala and Python, have been changed.
    -* [SPARK-13600](
    - `QuantileDiscretizer` now uses 
`spark.sql.DataFrameStatFunctions.approxQuantile` to find splits (previously 
used custom sampling logic).
    - The output buckets will differ for same input data and params.
    +* [SPARK-17870](
    + Fix a bug of `ChiSqSelector` which will likely change its result.
    +* [SPARK-3261](
    + `KMeans` returns potentially fewer than k cluster centers in cases where 
k distinct centroids aren't available or aren't selected.
    +* [SPARK-17389](
    + `KMeans` reduces the default number of steps from 5 to 2 for the 
k-means|| initialization mode.
    +* [SPARK-18481](
    --- End diff --
    Ah, you're right.  I was thinking the "final" change was breaking for 
Scala, but it's not since the model constructor is private.  I'm OK with moving 
those back to changes of behavior, or just removing these items completely.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to