GitHub user yanboliang reopened a pull request:
https://github.com/apache/spark/pull/12414
[SPARK-14657][SPARKR][ML] RFormula w/o intercept should output reference
category when encoding string terms
## What changes were proposed in this pull request?
Please see [SPARK-14657](https://issues.apache.org/jira/browse/SPARK-14657)
for detail of this bug.
I searched online and test some other cases, found when we fit R glm
model(or other models powered by R formula) w/o intercept on a dataset
including string/category features, one of the categories in the first category
feature is being used as reference category, we will not drop any category for
that feature.
I think we should keep consistent semantics between Spark RFormula and R
formula.
## How was this patch tested?
Add standard unit tests.
cc @mengxr
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yanboliang/spark spark-14657
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12414.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12414
----
commit db68beab8076d9fedcb4be56e750ffa83031f23d
Author: Yanbo Liang <[email protected]>
Date: 2016-04-15T09:35:52Z
formula w/o intercept, we should output reference category when encoding
string terms
commit 379d05473404122d18292320aa8e33c9b1a37e8b
Author: Yanbo Liang <[email protected]>
Date: 2017-05-22T15:33:34Z
Update test suites.
commit 097da707036fc695de1b797d7dd861f4f3ea79f7
Author: Yanbo Liang <[email protected]>
Date: 2017-06-27T03:12:50Z
Rename to keepReferenceCategory and add test cases.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
