GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/139
[HIVEMALL-182][SPARK][WIP] Add an optimizer rule to filter out columns with
low variances
## What changes were proposed in this pull request?
This pr added a new optimizer rule `VarianceThreshold` in Spark.
TODO
- Add docs in gitbook
- Add more tests
- Brush up `VarianceThreshold` code
## What type of PR is it?
Feature
## What is the Jira issue?
https://issues.apache.org/jira/browse/HIVEMALL-182
## How was this patch tested?
Added tests in `FeatureSelectionRuleSuite`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/maropu/incubator-hivemall HIVEMALL-182
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hivemall/pull/139.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #139
----
commit dc5af08c6a18fb9f4cebf7d7f619cbd053165e1f
Author: Takeshi Yamamuro <yamamuro@...>
Date: 2018-03-29T22:26:40Z
Add an optimizer rule to filter out columns with low variances
----
---
