[
https://issues.apache.org/jira/browse/SPARK-26172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16716831#comment-16716831
]
ASF GitHub Bot commented on SPARK-26172:
----------------------------------------
SparkQA commented on issue #23144: [SPARK-26172][ML][WIP] Unify String Params'
case-insensitivity in ML
URL: https://github.com/apache/spark/pull/23144#issuecomment-446160698
**[Test build #99967 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99967/testReport)**
for PR 23144 at commit
[`11bd906`](https://github.com/apache/spark/commit/11bd9065a850cea6ec1678fc8883746a393e3e02).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Unify String Params' case-insensitivity in ML
> ---------------------------------------------
>
> Key: SPARK-26172
> URL: https://issues.apache.org/jira/browse/SPARK-26172
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Affects Versions: 3.0.0
> Reporter: zhengruifeng
> Priority: Major
>
> For now, there are three ways to deal with case-insensitivity in ML:
> 1, support case-insensitivity, e.g. {{LogisticRegression}};
> 2, support case-insensitivity, but with getter returning the lower case value
> (not the value passed to setter), e.g. {{ALS}},{{DecisionTreeClassifier}};
> 3, do not support case-insensitivity, e.g. {{NaiveBayes}}
>
> This situation result in confusion in usage.
> I think we should choose the *first* way to support case-insensitivity of all
> non-columnName string params, including:
> * LogisticRegression: family
> * MultilayerPerceptronClassifier: {{solver}}
> * NaiveBayes: modelType
> * DecisionTreeClassifier: impurity
> * RandomForestClassifier: featureSubsetStrategy, impurity
> * GBTClassifier: featureSubsetStrategy, impurity, {{lossType}}
> * {{}}
> * LinearRegression: solver, loss
> * GeneralizedLinearRegression: family, link, solver
> * DecisionTreeRegressor: impurity
> * RandomForestRegressor: featureSubsetStrategy, impurity
> * GBTRegressor: featureSubsetStrategy, impurity, {{lossType}}
> * {{}}
> * {\{KMeans: }}initMode
> * LDA: optimizer
> * PowerIterationClustering\{{: }}initMode
> *
> * ALS: coldStartStrategy, intermediateStorageLevel, finalStorageLevel
> *
> * Bucketizer: handleInvalid
> * ChiSqSelector: selectorType
> * Imputer: strategy
> * QuantileDiscretizer: handleInvalid
> * RFormula: handleInvalid, stringIndexerOrderType
> * StringIndexer: handleInvalid, stringOrderType
> * VectorAssembler: handleInvalid
> * VectorIndexer: handleInvalid
> * VectorSizeHint: handleInvalid
> * OneHotEncoderEstimator: handleInvalid (*this will be let alone until the
> breaking change*)
> *
> * BinaryClassificationEvaluator: metricName
> * MulticlassClassificationEvaluator: metricName
> * RegressionEvaluator: metricName
> * ClusteringEvaluator: metricName, distanceMeasure
>
>
>
> To to this:
> * methods {{lowerCaseInArray}} and {{upperCaseInArray}} are created in
> {{ParamValidators}} to check case-insensitivity;
> * methods {{{{$$(param: Param[String])}}}} and {{%%(param: Param[String])}}
> are created in trait {{Params}} to lower/upper the param value conveniently,
> and this can minimize the modifications in existing codes, since in many
> cases we only need to change {{$(param)}} to {{$$\{param}}};
> * in *SharedParamsCodeGen*, *handleInvalid* and *{{distanceMeasure}}* are
> updated to use lowerCaseInArray
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
