[
https://issues.apache.org/jira/browse/MAHOUT-342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847506#action_12847506
]
Ted Dunning commented on MAHOUT-342:
------------------------------------
The SGD implementation (not yet committed) should provide a useful starting
point here.
The general consensus that I have seen lately is that the major areas of
scalability motivate the following features:
- support of large sparse feature sets
- support for on-the-fly feature combinations
- sparse and lazy updates
- regularization of various forms (L1, L2 and elastic lasso at least)
These are all handled very well handled by recently described online gradient
descent algorithms.
Some recent work has indicated that for logistic regression at least that small
scale parallelization is quite feasible for these gradient descent algorithms
by sharing updates between different threads scanning the training data, but
that if these sharing events are separated by more than a few hundred training
examples that convergence is not significantly accelerated. This is easy to
deal with in a threaded architecture, but essentially impossible in currently
common map-reduce architectures.
See, for instance,
http://books.nips.cc/papers/files/nips22/NIPS2009_0942.pdf
On the other hand, we can build a good sparse online trainer that works very,
very well.
> [GSOC] Implement Map/Reduce Enabled Neural Networks
> ---------------------------------------------------
>
> Key: MAHOUT-342
> URL: https://issues.apache.org/jira/browse/MAHOUT-342
> Project: Mahout
> Issue Type: New Feature
> Reporter: Grant Ingersoll
>
> Per the Ng. et. al. paper, implement a neural network with back propagation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
