Github user thunterdb commented on a diff in the pull request:
https://github.com/apache/spark/pull/12663#discussion_r61259664
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala ---
@@ -126,16 +127,16 @@ final class GBTClassifier @Since("1.4.0") (
override protected def train(dataset: Dataset[_]):
GBTClassificationModel = {
val categoricalFeatures: Map[Int, Int] =
MetadataUtils.getCategoricalFeatures(dataset.schema($(featuresCol)))
- val numClasses: Int =
MetadataUtils.getNumClasses(dataset.schema($(labelCol))) match {
- case Some(n: Int) => n
- case None => throw new IllegalArgumentException("GBTClassifier was
given input" +
- s" with invalid label column ${$(labelCol)}, without the number of
classes" +
- " specified. See StringIndexer.")
- // TODO: Automatically index labels: SPARK-7126
- }
- require(numClasses == 2,
- s"GBTClassifier only supports binary classification but was given
numClasses = $numClasses")
- val oldDataset: RDD[LabeledPoint] = extractLabeledPoints(dataset)
+ // We copy and modify this from Classifier.extractLabeledPoints since
GBT only supports
+ // 2 classes now. This lets us provide a more precise error message.
+ val oldDataset: RDD[LabeledPoint] =
+ dataset.select(col($(labelCol)).cast(DoubleType),
col($(featuresCol))).rdd.map {
+ case Row(label: Double, features: Vector) =>
+ require(label % 1 == 0 && label >= 0 && label < 2,
s"GBTClassifier was given" +
--- End diff --
I think you can simplify this check to `label == 0.0 || label == 1.0`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
