On Mon, May 13, 2013 at 9:24 PM, Tom Marthaler <[email protected]> wrote:
> Is there a way to make the feature weights deterministic, no matter the > order of the input training vectors? > Well, not if you use *STOCHASTIC* gradient descent. That is part of the point, actually. But seriously, this is likely to be true of any online logistic regression algorithms. You can build deterministic on-line algorithms for linear regression but even there large sparse systems make gradient descent more palatable. If you make multiple passes through the input, then you could switch to BFGS or use update averaging to accelerate convergence after SGD has gotten you near the solution, but getting near the solution depends a bit on randomized ordering of examples. Hope this helps.
