[
https://issues.apache.org/jira/browse/SPARK-7425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14541099#comment-14541099
]
Glenn Weidner commented on SPARK-7425:
--------------------------------------
Thank you for the tip!
In PredictorParams.validateAndTransformSchema, I've replaced
SchemaUtils.checkColumnType with isInstanceOf[NumericType] as shown below
(where NumericType imported from org.apache.spark.sql.types.NumericType).
if (fitting) {
// TODO: Allow other numeric types
//SchemaUtils.checkColumnType(schema, $(labelCol), DoubleType)
val actualDataType = schema($(labelCol)).dataType
require(actualDataType.isInstanceOf[NumericType],
s"Column $labelCol must be of type NumericType but was actually
$actualDataType.")
}
A quick debug run of existing LinearRegressionSuite stepped into modified
PredictorParams.validateAndTransformSchema and verified test still passed with
DoubleType. Next step is to try test with other numeric types.
> spark.ml Predictor should support other numeric types for label
> ---------------------------------------------------------------
>
> Key: SPARK-7425
> URL: https://issues.apache.org/jira/browse/SPARK-7425
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Joseph K. Bradley
> Priority: Minor
> Labels: starter
>
> Currently, the Predictor abstraction expects the input labelCol type to be
> DoubleType, but we should support other numeric types. This will involve
> updating the PredictorParams.validateAndTransformSchema method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
