Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/15428#discussion_r83760784
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala ---
@@ -270,10 +270,10 @@ private[ml] trait HasFitIntercept extends Params {
private[ml] trait HasHandleInvalid extends Params {
/**
- * Param for how to handle invalid entries. Options are skip (which will
filter out rows with bad values), or error (which will throw an error). More
options may be added later.
+ * Param for how to handle invalid entries. Options are skip (which will
filter out rows with bad values), or error (which will throw an error), or keep
(which will keep the bad values in certain way). More options may be added
later.
* @group param
*/
- final val handleInvalid: Param[String] = new Param[String](this,
"handleInvalid", "how to handle invalid entries. Options are skip (which will
filter out rows with bad values), or error (which will throw an error). More
options may be added later", ParamValidators.inArray(Array("skip", "error")))
+ final val handleInvalid: Param[String] = new Param[String](this,
"handleInvalid", "how to handle invalid entries. Options are skip (which will
filter out rows with bad values), or error (which will throw an error), or keep
(which will keep the bad values in certain way). More options may be added
later", ParamValidators.inArray(Array("skip", "keep", "error")))
--- End diff --
Since this is used by StringIndexer, which will not support "keep," we
should not modify HasHandleInvalid. I recommend copying out the shared param
and just putting it in Bucketizer and QuantileDiscretizer, rather than trying
to reuse HasHandleInvalid. That will let you specialize the documentation too
so that you can be more specific.
HasHandleInvalid may not be a good shared Param yet, but perhaps in the
future..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
