[
https://issues.apache.org/jira/browse/SPARK-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14509538#comment-14509538
]
Glenn Weidner edited comment on SPARK-6290 at 4/23/15 6:45 PM:
---------------------------------------------------------------
Thank you Joseph for the quick reply and my apologies for not including test
scenario details. The "java.lang.IllegalArgumentException: requirement
failed:" was normal for this case since scala.Predef.require throws the
exception when the requirement expression is false. Note another test scenario
was created from org.apache.spark.ml.param.ParamsSuite which resulted in an
exception being thrown in getParam method:
test("SPARK-6290") {
val field = new StructField("missing", StringType, false)
val schema = new StructType(Array(field))
// Force require expression in Params.checkInputColumn to fail since actual
is StringType
solver.checkInputColumn2(schema, "missing", BooleanType)
}
org.apache.spark.ml.param.TestParams.missing()
java.lang.NoSuchMethodException: org.apache.spark.ml.param.TestParams.missing()
at java.lang.Class.getMethod(Class.java:1670)
at org.apache.spark.ml.param.Params$class.getParam(params.scala:163)
at org.apache.spark.ml.param.TestParams.getParam(TestParams.scala:23)
at
org.apache.spark.ml.param.Params$$anonfun$checkInputColumn$1.apply(params.scala:209)
at
org.apache.spark.ml.param.Params$$anonfun$checkInputColumn$1.apply(params.scala:207)
at scala.Predef$.require(Predef.scala:233)
at
org.apache.spark.ml.param.Params$class.checkInputColumn(params.scala:206)
at
org.apache.spark.ml.param.TestParams.checkInputColumn(TestParams.scala:23)
at
org.apache.spark.ml.param.TestParams.checkInputColumn2(TestParams.scala:32)
at
org.apache.spark.ml.param.ParamsSuite$$anonfun$5.apply$mcV$sp(ParamsSuite.scala:119)
at
org.apache.spark.ml.param.ParamsSuite$$anonfun$5.apply(ParamsSuite.scala:110)
at
org.apache.spark.ml.param.ParamsSuite$$anonfun$5.apply(ParamsSuite.scala:110)
However, using one of the existing parameters (e.g., "inputCol") defined in
TestParams.scala allowed getParam to complete successfully.
was (Author: gweidner):
Thank you Joseph for the quick reply and my apologies for not including test
scenario details. The "java.lang.IllegalArgumentException: requirement
failed:" was normal for this case since scala.Predef.require throws the
exception when the requirement expression is false. Note another test scenario
was created from org.apache.spark.ml.param.ParamsSuite which resulted in an
exception being thrown in getParam method:
test("SPARK-6290") {
val field = new StructField("missing", StringType, false)
val schema = new StructType(Array(field))
// Force require expression in Params.checkInputColumn to fail since actual
is StringType
solver.checkInputColumn2(schema, "missing", BooleanType)
}
org.apache.spark.ml.param.TestParams.missing()
java.lang.NoSuchMethodException: org.apache.spark.ml.param.TestParams.missing()
at java.lang.Class.getMethod(Class.java:1670)
at org.apache.spark.ml.param.Params$class.getParam(params.scala:163)
at org.apache.spark.ml.param.TestParams.getParam(TestParams.scala:23)
at
org.apache.spark.ml.param.Params$$anonfun$checkInputColumn$1.apply(params.scala:209)
at
org.apache.spark.ml.param.Params$$anonfun$checkInputColumn$1.apply(params.scala:207)
at scala.Predef$.require(Predef.scala:233)
at
org.apache.spark.ml.param.Params$class.checkInputColumn(params.scala:206)
at
org.apache.spark.ml.param.TestParams.checkInputColumn(TestParams.scala:23)
at
org.apache.spark.ml.param.TestParams.checkInputColumn2(TestParams.scala:32)
at
org.apache.spark.ml.param.ParamsSuite$$anonfun$5.apply$mcV$sp(ParamsSuite.scala:119)
at
org.apache.spark.ml.param.ParamsSuite$$anonfun$5.apply(ParamsSuite.scala:110)
at
org.apache.spark.ml.param.ParamsSuite$$anonfun$5.apply(ParamsSuite.scala:110)
However, using one of the existing parameters (e.g., "inputCol") defined in
TestParams.scala allowed getParam to complete successfully.
> spark.ml.param.Params.checkInputColumn bug upon error
> -----------------------------------------------------
>
> Key: SPARK-6290
> URL: https://issues.apache.org/jira/browse/SPARK-6290
> Project: Spark
> Issue Type: Bug
> Components: ML
> Affects Versions: 1.3.0
> Reporter: Joseph K. Bradley
> Priority: Minor
>
> In checkInputColumn, if data types do not match, it tries to print an error
> message with this in it:
> {code}
> Column param description: ${getParam(colName)}"
> {code}
> However, getParam cannot be called on the string colName; it needs the
> parameter name, which this method is not given. This causes a weird error
> which users may find hard to understand.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
