[
https://issues.apache.org/jira/browse/SPARK-7514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14537527#comment-14537527
]
yuhao yang commented on SPARK-7514:
-----------------------------------
Hi Joseph, that a good idea. I did a simple google:
weka: Class Normalize, takes scaling factor and translation ( same concepts as
scale and newBase).
http://weka.sourceforge.net/doc.dev/weka/filters/unsupervised/attribute/Normalize.html
sklearn.preprocessing.MinMaxScaler, takes min and scale, yet in array format,
http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html
some implements basic MinMax and takes no extra parameters:
http://docs.pervasive.com/products/DataRush/DF63/javadoc/com/pervasive/datarush/analytics/functions/StatsFunctions.html
http://help.sap.com/saphelp_hanaplatform/helpdata/en/e3/f29fafd4ac43339a1a39407884e545/content.htm?frameset=/en/e6/5c78507a424be58e52877496e2b516/frameset.htm¤t_toc=/en/32/731a7719f14e488b1f4ab0afae995b/plain.htm&node_id=52
> Add MinMaxNormalizer to feature transformation
> ----------------------------------------------
>
> Key: SPARK-7514
> URL: https://issues.apache.org/jira/browse/SPARK-7514
> Project: Spark
> Issue Type: New Feature
> Components: MLlib
> Reporter: yuhao yang
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Add a popular scaling method to feature component, which is commonly known as
> min-max normalization or Rescaling.
> Core function is,
> Normalized( x ) = (x - min) / (max - min) * scale + newBase
> where newBase and scale are parameters of the VectorTransformer. newBase is
> the new minimum number for the feature, and scale controls the range after
> transformation. This is a little complicated than the basic MinMax
> normalization, yet it provides flexibility so that users can control the
> range more specifically. like [0.1, 0.9] in some NN application.
> for case that max == min, 0.5 is used as the raw value.
> reference:
> http://en.wikipedia.org/wiki/Feature_scaling
> http://stn.spotfire.com/spotfire_client_help/index.htm#norm/norm_scale_between_0_and_1.htm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
