Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/702#discussion_r12460330
--- Diff: docs/mllib-optimization.md ---
@@ -128,10 +128,24 @@ is sampled, i.e. `$|S|=$ miniBatchFraction $\cdot n =
1$`, then the algorithm is
standard SGD. In that case, the step direction depends from the uniformly
random sampling of the
point.
+### Limited-memory BFGS
+[Limited-memory BFGS
(L-BFGS)](http://en.wikipedia.org/wiki/Limited-memory_BFGS) is an optimization
+algorithm in the family of quasi-Newton methods to solve the optimization
problems of the form
+`$\min_{\wv \in\R^d} \; f(\wv)$`. The L-BFGS approximates the objective
function locally as a quadratic
+without evaluating the second partial derivatives of the objective
function to construct the
+Hessian matrix. The Hessian matrix is approximated by previous gradient
evaluations, so there is no
+vertical scalability issue (the number of training features) when
computing the Hessian matrix
+explicitly in Newton method. As a result, L-BFGS often achieves rapider
convergence compared with
--- End diff --
`Newton's method`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
