[jira] [Assigned] (SPARK-25989) OneVsRestModel handle empty outputCols incorrectly
[ https://issues.apache.org/jira/browse/SPARK-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25989: - Assignee: zhengruifeng > OneVsRestModel handle empty outputCols incorrectly > -- > > Key: SPARK-25989 > URL: https://issues.apache.org/jira/browse/SPARK-25989 > Project: Spark > Issue Type: Bug > Components: ML >Affects Versions: 3.0.0 >Reporter: zhengruifeng >Assignee: zhengruifeng >Priority: Minor > Fix For: 3.0.0 > > > {\{ml.classification.ClassificationModel}} will ignore empty output columns. > However, \{{OneVsRestModel}} still try to append new column even if its name > is an empty string. > {code:java} > scala> ovrModel.setPredictionCol("").transform(test).show > +-+++---+ > |label| features| rawPrediction| | > +-+++---+ > | 0.0|(4,[0,1,2,3],[-0|[-0.0965652626152...|2.0| > | 0.0|(4,[0,1,2,3],[-0|[0.07880609384635...|2.0| > | 0.0|(4,[0,1,2,3],[-1|[0.01891571586984...|2.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.72409973016524...|0.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.48045978946729...|2.0| > | 0.0|(4,[0,1,2,3],[0.3...|[1.05496616040758...|0.0| > | 0.0|(4,[0,1,2,3],[0.3...|[0.79508659065535...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.47437469552081...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.23302929670223...|0.0| > | 0.0|(4,[0,1,2,3],[0.8...|[1.79816156359706...|0.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.1564309664080...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.2217906250571...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.9171126308553...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.8316993051998...|1.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.6486206847760...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9252139721697...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9025379528484...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.8518243169707...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.0990190524225...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9973479746889...|2.0| > +-+++---+ > only showing top 20 rows > scala> > ovrModel.setPredictionCol("").setRawPredictionCol("raw").transform(test).show > +-+++---+ > |label| features| raw| | > +-+++---+ > | 0.0|(4,[0,1,2,3],[-0|[-0.0965652626152...|2.0| > | 0.0|(4,[0,1,2,3],[-0|[0.07880609384635...|2.0| > | 0.0|(4,[0,1,2,3],[-1|[0.01891571586984...|2.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.72409973016524...|0.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.48045978946729...|2.0| > | 0.0|(4,[0,1,2,3],[0.3...|[1.05496616040758...|0.0| > | 0.0|(4,[0,1,2,3],[0.3...|[0.79508659065535...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.47437469552081...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.23302929670223...|0.0| > | 0.0|(4,[0,1,2,3],[0.8...|[1.79816156359706...|0.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.1564309664080...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.2217906250571...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.9171126308553...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.8316993051998...|1.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.6486206847760...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9252139721697...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9025379528484...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.8518243169707...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.0990190524225...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9973479746889...|2.0| > +-+++---+ > only showing top 20 rows > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-25989) OneVsRestModel handle empty outputCols incorrectly
[ https://issues.apache.org/jira/browse/SPARK-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25989: Assignee: (was: Apache Spark) > OneVsRestModel handle empty outputCols incorrectly > -- > > Key: SPARK-25989 > URL: https://issues.apache.org/jira/browse/SPARK-25989 > Project: Spark > Issue Type: Bug > Components: ML >Affects Versions: 3.0.0 >Reporter: zhengruifeng >Priority: Minor > > {\{ml.classification.ClassificationModel}} will ignore empty output columns. > However, \{{OneVsRestModel}} still try to append new column even if its name > is an empty string. > {code:java} > scala> ovrModel.setPredictionCol("").transform(test).show > +-+++---+ > |label| features| rawPrediction| | > +-+++---+ > | 0.0|(4,[0,1,2,3],[-0|[-0.0965652626152...|2.0| > | 0.0|(4,[0,1,2,3],[-0|[0.07880609384635...|2.0| > | 0.0|(4,[0,1,2,3],[-1|[0.01891571586984...|2.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.72409973016524...|0.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.48045978946729...|2.0| > | 0.0|(4,[0,1,2,3],[0.3...|[1.05496616040758...|0.0| > | 0.0|(4,[0,1,2,3],[0.3...|[0.79508659065535...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.47437469552081...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.23302929670223...|0.0| > | 0.0|(4,[0,1,2,3],[0.8...|[1.79816156359706...|0.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.1564309664080...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.2217906250571...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.9171126308553...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.8316993051998...|1.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.6486206847760...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9252139721697...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9025379528484...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.8518243169707...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.0990190524225...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9973479746889...|2.0| > +-+++---+ > only showing top 20 rows > scala> > ovrModel.setPredictionCol("").setRawPredictionCol("raw").transform(test).show > +-+++---+ > |label| features| raw| | > +-+++---+ > | 0.0|(4,[0,1,2,3],[-0|[-0.0965652626152...|2.0| > | 0.0|(4,[0,1,2,3],[-0|[0.07880609384635...|2.0| > | 0.0|(4,[0,1,2,3],[-1|[0.01891571586984...|2.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.72409973016524...|0.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.48045978946729...|2.0| > | 0.0|(4,[0,1,2,3],[0.3...|[1.05496616040758...|0.0| > | 0.0|(4,[0,1,2,3],[0.3...|[0.79508659065535...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.47437469552081...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.23302929670223...|0.0| > | 0.0|(4,[0,1,2,3],[0.8...|[1.79816156359706...|0.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.1564309664080...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.2217906250571...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.9171126308553...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.8316993051998...|1.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.6486206847760...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9252139721697...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9025379528484...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.8518243169707...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.0990190524225...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9973479746889...|2.0| > +-+++---+ > only showing top 20 rows > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-25989) OneVsRestModel handle empty outputCols incorrectly
[ https://issues.apache.org/jira/browse/SPARK-25989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25989: Assignee: Apache Spark > OneVsRestModel handle empty outputCols incorrectly > -- > > Key: SPARK-25989 > URL: https://issues.apache.org/jira/browse/SPARK-25989 > Project: Spark > Issue Type: Bug > Components: ML >Affects Versions: 3.0.0 >Reporter: zhengruifeng >Assignee: Apache Spark >Priority: Minor > > {\{ml.classification.ClassificationModel}} will ignore empty output columns. > However, \{{OneVsRestModel}} still try to append new column even if its name > is an empty string. > {code:java} > scala> ovrModel.setPredictionCol("").transform(test).show > +-+++---+ > |label| features| rawPrediction| | > +-+++---+ > | 0.0|(4,[0,1,2,3],[-0|[-0.0965652626152...|2.0| > | 0.0|(4,[0,1,2,3],[-0|[0.07880609384635...|2.0| > | 0.0|(4,[0,1,2,3],[-1|[0.01891571586984...|2.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.72409973016524...|0.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.48045978946729...|2.0| > | 0.0|(4,[0,1,2,3],[0.3...|[1.05496616040758...|0.0| > | 0.0|(4,[0,1,2,3],[0.3...|[0.79508659065535...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.47437469552081...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.23302929670223...|0.0| > | 0.0|(4,[0,1,2,3],[0.8...|[1.79816156359706...|0.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.1564309664080...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.2217906250571...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.9171126308553...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.8316993051998...|1.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.6486206847760...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9252139721697...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9025379528484...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.8518243169707...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.0990190524225...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9973479746889...|2.0| > +-+++---+ > only showing top 20 rows > scala> > ovrModel.setPredictionCol("").setRawPredictionCol("raw").transform(test).show > +-+++---+ > |label| features| raw| | > +-+++---+ > | 0.0|(4,[0,1,2,3],[-0|[-0.0965652626152...|2.0| > | 0.0|(4,[0,1,2,3],[-0|[0.07880609384635...|2.0| > | 0.0|(4,[0,1,2,3],[-1|[0.01891571586984...|2.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.72409973016524...|0.0| > | 0.0|(4,[0,1,2,3],[0.1...|[0.48045978946729...|2.0| > | 0.0|(4,[0,1,2,3],[0.3...|[1.05496616040758...|0.0| > | 0.0|(4,[0,1,2,3],[0.3...|[0.79508659065535...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.47437469552081...|0.0| > | 0.0|(4,[0,1,2,3],[0.6...|[1.23302929670223...|0.0| > | 0.0|(4,[0,1,2,3],[0.8...|[1.79816156359706...|0.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.1564309664080...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-3.2217906250571...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.9171126308553...|1.0| > | 1.0|(4,[0,1,2,3],[-0|[-2.8316993051998...|1.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.6486206847760...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9252139721697...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9025379528484...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.8518243169707...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-1.0990190524225...|2.0| > | 2.0|(4,[0,1,2,3],[-0|[-0.9973479746889...|2.0| > +-+++---+ > only showing top 20 rows > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org