gatorsmile commented on issue #25558: [SPARK-28855][CORE][ML][SQL][STREAMING] Remove outdated usages of Experimental, Evolving annotations URL: https://github.com/apache/spark/pull/25558#issuecomment-534195698 I went over the PR and below is the list to summarize the changes. Could you confirm whether it is complete? - Removal of Experimental: - org.apache.spark.metrics.source.CodegenMetrics - org.apache.spark.metrics.source.HiveCatalogMetrics - org.apache.spark.partial - org.apache.spark.rdd.combineByKeyWithClassTag - org.apache.spark.rdd.combineByKeyWithClassTag - org.apache.spark.rdd.combineByKeyWithClassTag - org.apache.spark.sql.from_avro [deprecated] - org.apache.spark.sql.to_avro [deprecated] - org.apache.spark.ml.classification.LinearSVC - org.apache.spark.ml.classification.LinearSVCModel - org.apache.spark.ml.classification.LogisticRegressionSummary - org.apache.spark.ml.classification.LogisticRegressionTrainingSummary - org.apache.spark.ml.classification.BinaryLogisticRegressionSummary - org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummary - org.apache.spark.ml.clustering.BisectingKMeansSummary - org.apache.spark.ml.clustering.ClusteringSummary - org.apache.spark.ml.clustering.GaussianMixtureSummary - org.apache.hadoop.fs.Path.KMeansSummary - org.apache.spark.ml.clustering.PowerIterationClustering - org.apache.spark.ml.evaluation.BinaryClassificationEvaluator - org.apache.spark.ml.evaluation.MulticlassClassificationEvaluator - org.apache.spark.ml.evaluation.RegressionEvaluator - org.apache.spark.ml.feature.BucketedRandomProjectionLSHParams - org.apache.spark.ml.feature.BucketedRandomProjectionLSH - org.apache.spark.ml.feature.BucketedRandomProjectionLSHModel - org.apache.spark.ml.feature.FeatureHasher - org.apache.spark.ml.feature.Imputer - org.apache.spark.ml.feature.ImputerModel - org.apache.spark.ml.feature.MinHashLSHModel - org.apache.spark.ml.feature.MinHashLSH - org.apache.spark.ml.feature.RFormula - org.apache.spark.ml.feature.RFormulaModel - org.apache.spark.ml.feature.VectorSizeHint - org.apache.spark.ml.fpm.FPGrowth - org.apache.spark.ml.fpm.FPGrowthModel - org.apache.spark.ml.fpm.PrefixSpan - org.apache.spark.ml.fpm.PrefixSpan.findFrequentSequentialPatterns - org.apache.spark.ml.image.ImageSchema - org.apache.spark.ml.regression.AFTSurvivalRegression - org.apache.spark.ml.regression.AFTSurvivalRegressionModel - org.apache.spark.ml.regression.GeneralizedLinearRegression - org.apache.spark.ml.regression.GeneralizedLinearRegressionModel - org.apache.spark.ml.regression.GeneralizedLinearRegressionSummary - org.apache.spark.ml.regression.GeneralizedLinearRegressionTrainingSummary - org.apache.spark.ml.regression.LinearRegressionTrainingSummary - org.apache.spark.ml.regression.LinearRegressionSummary - org.apache.spark.ml.stat.ChiSquareTest - org.apache.spark.ml.stat.Correlation - org.apache.spark.ml.stat.Correlation.corr - org.apache.spark.ml.stat.KolmogorovSmirnovTest - org.apache.spark.ml.stat.SummaryBuilder - org.apache.spark.ml.stat.Summarizer - org.apache.spark.ml.util.PMMLUtils.loadFromString - pyspark.context.binaryFiles - pyspark.context.binaryRecords - pyspark.ml.fpm.FPGrowth - pyspark.ml.fpm.FPGrowthModel - pyspark.ml.fpm.PrefixSpan - pyspark.ml.feature.BucketedRandomProjectionLSH - pyspark.ml.feature.BucketedRandomProjectionLSHModel - pyspark.ml.feature.ChiSqSelector - pyspark.ml.feature.ChiSqSelectorModel - pyspark.ml.feature.FeatureHasher - pyspark.ml.feature.Imputer - pyspark.ml.feature.ImputerModel - pyspark.ml.feature.MinHashLSH - pyspark.ml.feature.MinHashLSHModel - pyspark.ml.feature.QuantileDiscretizer - pyspark.ml.feature.RFormula - pyspark.ml.feature.RFormulaModel - pyspark.ml.feature.VectorSizeHint - pyspark.ml.classification.LinearSVC - pyspark.ml.classification.LinearSVCModel - pyspark.ml.classification.LogisticRegressionSummary - pyspark.ml.classification.LogisticRegressionTrainingSummary - pyspark.ml.classification.BinaryLogisticRegressionSummary - pyspark.ml.classification.OneVsRest - pyspark.ml.classification.OneVsRestModel - pyspark.ml.clustering.GaussianMixtureSummary - pyspark.ml.clustering.KMeansSummary - pyspark.ml.clustering.BisectingKMeansSummary - pyspark.ml.clustering.PowerIterationClustering - pyspark.ml.stat.ChiSquareTest - pyspark.ml.stat.KolmogorovSmirnovTest - pyspark.ml.stat.Summarizer - pyspark.ml.stat.SummaryBuilder - pyspark.ml.tuning.TrainValidationSplit - pyspark.ml.tuning.TrainValidationSplitModel - pyspark.ml.evaluation.BinaryClassificationEvaluator - pyspark.ml.evaluation.RegressionEvaluator - pyspark.ml.evaluation.MulticlassClassificationEvaluator - pyspark.ml.evaluation.ClusteringEvaluator - The fitMultiple method in all the classes - pyspark.rdd.countApprox - pyspark.rdd.sumApprox - pyspark.rdd.meanApprox - pyspark.rdd.countApproxDistinct - pyspark.sql.functions.pandas_udf - pyspark.sql.GroupedData - pyspark.sql.Window - pyspark.sql.WindowSpec - pyspark.taskcontext - org.apache.spark.sql.Encoder - org.apache.spark.sql.expressions.javalang.typed - org.apache.spark.sql.Dataset.as - org.apache.spark.sql.Dataset.checkpoint - org.apache.spark.sql.Dataset.localCheckpoint - org.apache.spark.sql.Dataset.withWatermark - org.apache.spark.sql.Dataset.joinWith - org.apache.spark.sql.Dataset.select - org.apache.spark.sql.Dataset.reduce - org.apache.spark.sql.Dataset.groupByKey - org.apache.spark.sql.Dataset.filter - org.apache.spark.sql.Dataset.map - org.apache.spark.sql.Dataset.mapPartitions - org.apache.spark.sql.Dataset.flatMap - org.apache.spark.sql.KeyValueGroupedDataset.mapGroupsWithState - org.apache.spark.sql.KeyValueGroupedDataset.flatMapGroupsWithState - org.apache.spark.sql.SQLContext.listenerManager - org.apache.spark.sql.SQLContext.implicits - org.apache.spark.sql.SQLContext.createDataFrame - org.apache.spark.sql.SQLContext.createDataset - org.apache.spark.sql.SQLContext.range - org.apache.spark.sql.SparkSession.listenerManager - org.apache.spark.sql.SparkSession.streams - org.apache.spark.sql.SparkSession.emptyDataset - org.apache.spark.sql.SparkSession.createDataFrame - org.apache.spark.sql.SparkSession.createDataset - org.apache.spark.sql.SparkSession.range - org.apache.spark.sql.SparkSession.implicits - org.apache.spark.sql.catalog.createTable - org.apache.spark.sql.expressions.Aggregator - org.apache.spark.sql.internal.BaseSessionStateBuilder - org.apache.spark.sql.internal.createTable - org.apache.spark.sql.internal.SessionStateBuilder - org.apache.spark.sql.jdbc.JdbcDialects - org.apache.spark.sql.jdbc.JdbcType - org.apache.spark.sql.sources.StreamSourceProvider - org.apache.spark.sql.sources.StreamSinkProvider - org.apache.spark.sql.sources.CatalystScan - org.apache.spark.sql.util.QueryExecutionListener - org.apache.spark.sql.util.ExecutionListenerManager - org.apache.spark.sql.hive.HiveSessionStateBuilder - org.apache.spark.streaming.StreamingContext.getActive - org.apache.spark.streaming.StreamingContext.getActiveOrCreate - org.apache.spark.streaming.api.java.JavaMapWithStateDStream - org.apache.spark.streaming.api.java.mapWithState - org.apache.spark.streaming.dstream.MapWithStateDStream - org.apache.spark.streaming.dstream.PairDStreamFunctions.mapWithState - Removal of Evolving: - org.apache.spark.streaming.kinesis.KinesisInputDStream - org.apache.spark.streaming.kinesis.KinesisInputDStream.Builder - org.apache.spark.streaming.kinesis.SparkAWSCredentials - org.apache.spark.streaming.kinesis.SparkAWSCredentials.Builder - org.apache.spark.sql.types.SQLUserDefinedType - org.apache.spark.sql.vectorized.ArrowColumnVector - org.apache.spark.sql.Encoder - org.apache.spark.sql.types.ObjectType - org.apache.spark.sql.Dataset.as - org.apache.spark.sql.Dataset.isStreaming - org.apache.spark.sql.Dataset.checkpoint - org.apache.spark.sql.Dataset.localCheckpoint - org.apache.spark.sql.Dataset.withWatermark - org.apache.spark.sql.Dataset.joinWith - org.apache.spark.sql.Dataset.select - org.apache.spark.sql.Dataset.reduce - org.apache.spark.sql.Dataset.groupByKey - org.apache.spark.sql.Dataset.filter - org.apache.spark.sql.Dataset.map - org.apache.spark.sql.Dataset.mapPartitions - org.apache.spark.sql.Dataset.flatMap - org.apache.spark.sql.Dataset.writeStream - org.apache.spark.sql.ForeachWriter - org.apache.spark.sql.KeyValueGroupedDataset - org.apache.spark.sql.KeyValueGroupedDataset.mapGroupsWithState - org.apache.spark.sql.KeyValueGroupedDataset.flatMapGroupsWithState - org.apache.spark.sql.SQLContext.listenerManager - org.apache.spark.sql.SQLContext.implicits - org.apache.spark.sql.SQLContext.createDataFrame - org.apache.spark.sql.SQLContext.createDataset - org.apache.spark.sql.SQLContext.readStream - org.apache.spark.sql.SQLContext.range - org.apache.spark.sql.SQLImplicits - org.apache.spark.sql.SparkSession.listenerManager - org.apache.spark.sql.SparkSession.emptyDataset - org.apache.spark.sql.SparkSession.createDataFrame - org.apache.spark.sql.SparkSession.createDataset - org.apache.spark.sql.SparkSession.range - org.apache.spark.sql.SparkSession.readStream - org.apache.spark.sql.SparkSession.implicits - org.apache.spark.sql.util.QueryExecutionListener - org.apache.spark.sql.util.ExecutionListenerManager - org.apache.spark.sql.expressions.Aggregator - org.apache.spark.sql.catalog.createTable
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
