Github user atalwalkar commented on a diff in the pull request:
https://github.com/apache/spark/pull/2068#discussion_r16514371
--- Diff: docs/mllib-feature-extraction.md ---
@@ -70,4 +70,110 @@ for((synonym, cosineSimilarity) <- synonyms) {
</div>
</div>
-## TFIDF
\ No newline at end of file
+## TFIDF
+
+## StandardScaler
+
+Standardizes features by scaling to unit variance and/or removing the mean
using column summary
+statistics on the samples in the training set. For example, RBF kernel of
Support Vector Machines
+or the L1 and L2 regularized linear models typically assume that all
features have unit variance
+and/or zero mean.
+
+Standardization can not only improve the convergence rate during the
optimization process, but also
+avoid the problem that when training linear models with regularization
against a feature having
+a variance that is orders of magnitude larger than others, it might
dominate the objective function
+and make the estimator unable to learn from other features correctly as
expected.
--- End diff --
Suggested edit: "Standardization can improve the convergence rate during
the optimization process, and also prevents against features with very large
variances exerting an overly large influence during model training."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
