[
https://issues.apache.org/jira/browse/SPARK-14810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280592#comment-15280592
]
Nick Pentreath commented on SPARK-14810:
----------------------------------------
[~josephkb] [~mengxr] [~srowen] I've made a pass through this. I think I've
audited all the excludes added to {{MimaExcludes}} (but will take another pass
to double check). The majority of excludes added relate to (a) private classes
/ methods; (b) @Experimental / DeveloperAPI (c) adding methods to sealed
traits; and (d) the change {{DataFrame}} -> {{Dataset}}.
(d) is a binary incompatible change but affects Java for all of Spark (as we
know). So I've not worried about that.
I will check SPARK-13920 again as it added a lot of excludes (most of them
appear to be for {{DataFrame}} -> {{Dataset}} or private, and all @Experimental
/ DeveloperAPI, but still good to know if anything did change).
So far the the 2 issues are removing deprecated methods:
* SPARK-14089 - 1.1-1.5
** {{BinaryClassificationEvaluator.setScoreCol}}
** {{LBFGS.setMaxNumIterations}} - DeveloperAPI
** {{RDDFunctions.treeReduce}} and {{treeAggregate}} - DeveloperAPI
** {mllib.tree.Strategy.defaultStategy}} - appears to be a spelling error in
the method.
** {{mllib.tree.Node.build}}
** {{MLUtils}} libsvm loaders for multiclass and load/save labeledData methods
* SPARK-14952 - 1.6
** {{ml.LinearRegression.weights}} - @Experimental
** {{ml.LogisticRegression.weights}} - @Experimental
So these are incompatible changes, but I assume are ok. I'm just wondering how
we prefer to document these changes? Migration guide, or somewhere else?
> ML, Graph 2.0 QA: API: Binary incompatible changes
> --------------------------------------------------
>
> Key: SPARK-14810
> URL: https://issues.apache.org/jira/browse/SPARK-14810
> Project: Spark
> Issue Type: Sub-task
> Components: Documentation, GraphX, ML, MLlib
> Reporter: Joseph K. Bradley
> Assignee: Nick Pentreath
>
> Generate a list of binary incompatible changes using MiMa and create new
> JIRAs for issues found. Filter out false positives as needed.
> If you want to take this task, look at the analogous task from the previous
> release QA, and ping the Assignee for advice.
> List of changes since {{1.6.0}} audited - these are "false positives" due to
> being private, @Experimental, DeveloperAPI, etc:
> * SPARK-13686 - Add a constructor parameter `regParam` to
> (Streaming)LinearRegressionWithSGD
> * SPARK-13664 - Replace HadoopFsRelation with FileFormat
> * SPARK-11622 - Make LibSVMRelation extends HadoopFsRelation and Add
> LibSVMOutputWriter
> * SPARK-13920 - MIMA checks should apply to @Experimental and @DeveloperAPI
> APIs
> * SPARK-11011 - UserDefinedType serialization should be strongly typed
> * SPARK-13817 - Re-enable MiMA and removes object DataFrame
> * SPARK-13927 - add row/column iterator to local matrices - (add methods to
> sealed trait)
> * SPARK-13948 - MiMa Check should catch if the visibility change to `private`
> - (DataFrame -> Dataset)
> * SPARK-11262 - Unit test for gradient, loss layers, memory management -
> (private class)
> * SPARK-13430 - moved featureCol from LinearRegressionModelSummary to
> LinearRegressionSummary - (private class)
> * SPARK-13048 - keepLastCheckpoint option for LDA EM optimizer - (private
> class)
> * SPARK-14734 - Add conversions between mllib and ml Vector, Matrix types -
> (private methods added)
> * SPARK-14861 - Replace internal usages of SQLContext with SparkSession -
> (private class)
> Binary incompatible changes:
> * SPARK-14089 - Remove methods that has been deprecated since 1.1, 1.2, 1.3,
> 1.4, and 1.5
> * SPARK-14952 - Remove methods deprecated in 1.6
> * DataFrame -> Dataset<Row> changes for Java (this of course applies for all
> of Spark SQL)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
