GitHub user hhbyyh opened a pull request:
https://github.com/apache/spark/pull/10939
[SPARK-13028] [ML] Add MaxAbsScaler to ML.feature as a transformer
jira: https://issues.apache.org/jira/browse/SPARK-13028
MaxAbsScaler works in a very similar way as MinMaxScaler, but scales in a
way that the training data lies within the range [-1, 1] by dividing through
the largest maximum value in each feature. The motivation to use this scaling
include robustness to very small standard deviations of features and preserving
zero entries in sparse data.
Unlike StandardScaler and MinMaxScaler, MaxAbsScaler does not shift/center
the data, and thus does not destroy any sparsity.
Something similar from sklearn:
http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MaxAbsScaler.html#sklearn.preprocessing.MaxAbsScaler
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hhbyyh/spark maxabs
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10939.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10939
----
commit a9215b551af870a36bd9ccfb297e7879c421688f
Author: Yuhao Yang <[email protected]>
Date: 2016-01-26T16:35:30Z
max abs scaler
commit 8ab07476868b1a792db24a1f8dd4ab8b9802843d
Author: Yuhao Yang <[email protected]>
Date: 2016-01-27T01:18:36Z
Merge remote-tracking branch 'upstream/master' into maxabs
commit 91ef8f350554ef01b09dee42ffcb2934fbb5f867
Author: Yuhao Yang <[email protected]>
Date: 2016-01-27T02:53:43Z
ut added
commit cb10bb6f2fa7a5aed84a83db43a9ec2229a016f0
Author: Yuhao Yang <[email protected]>
Date: 2016-01-27T03:10:04Z
remove minmax
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
