[
https://issues.apache.org/jira/browse/SPARK-30928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043874#comment-17043874
]
Huaxin Gao edited comment on SPARK-30928 at 2/25/20 6:40 PM:
-------------------------------------------------------------
I audited all the ML, MLLIb, GraphX-related MiMa exclusions added for 3.0:
A few of them are not necessary. Will open a PR to remove those.
The following are false positive:
https://issues.apache.org/jira/browse/SPARK-16872 (private constructor)
https://issues.apache.org/jira/browse/SPARK-25838 (private or protected member
variables)
https://issues.apache.org/jira/browse/SPARK-11215 (protected methods)
https://issues.apache.org/jira/browse/SPARK-26616 (private constructor or
private object)
https://issues.apache.org/jira/browse/SPARK-25765 (private constructor)
The following are caused by inheritance structure change. The external APIs are
still the same for users, so we don't need to document these in migration guide.
https://issues.apache.org/jira/browse/SPARK-29645 (the param is moved from
individual algorithm to shared Params.
https://issues.apache.org/jira/browse/SPARK-28968 (the param is moved from
individual algorithm to shared Params.
https://issues.apache.org/jira/browse/SPARK-30377 (AFT extends Estimator -> AFT
extends Regressor)
Need to check migration guide for the following:
Remove deprecated APIs:
https://issues.apache.org/jira/browse/SPARK-28980
https://issues.apache.org/jira/browse/SPARK-27410
https://issues.apache.org/jira/browse/SPARK-26127
https://issues.apache.org/jira/browse/SPARK-26090
https://issues.apache.org/jira/browse/SPARK-25382
Binary incompatible changes
https://issues.apache.org/jira/browse/SPARK-28780
https://issues.apache.org/jira/browse/SPARK-26133
https://issues.apache.org/jira/browse/SPARK-30144
https://issues.apache.org/jira/browse/SPARK-30630
was (Author: huaxingao):
I went through the v30excludes list to check if everything on the list is
required for exclusion. I found the following can be removed:
//
ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
//
ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
//
ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
//
ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
//
ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
//
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.getInputCol"),
//
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.getOutputCol"),
//
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.inputCol"),
//
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.setInputCol"),
//
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.setOutputCol"),
//
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.outputCol"),
//
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.image.ImageSchema.readImages"),
I will submit a PR to remove these. Is this sufficient for this ticket? Are
there any other things I need to do? cc [~anders8]
> ML, GraphX 3.0 QA: API: Binary incompatible changes
> ---------------------------------------------------
>
> Key: SPARK-30928
> URL: https://issues.apache.org/jira/browse/SPARK-30928
> Project: Spark
> Issue Type: Sub-task
> Components: Documentation, GraphX, ML, MLlib
> Affects Versions: 3.0.0
> Reporter: zhengruifeng
> Priority: Blocker
>
> Generate a list of binary incompatible changes using MiMa and create new
> JIRAs for issues found. Filter out false positives as needed.
> If you want to take this task, look at the analogous task from the previous
> release QA, and ping the Assignee for advice.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
