GitHub user hhbyyh opened a pull request:
https://github.com/apache/spark/pull/7609
[SPARK-9273] [MLlib] Add Convolutional Neural network to Spark MLlib
jira: https://issues.apache.org/jira/browse/SPARK-9273
Here's an initial implementation of Convolutional Neural network based on
Spark. The basic design follows the existing optimization framework (train a
batch, sum the gradient, update weights) and should have good scalability.
Currently it supports SGD training.
A typical driver program looks like
```
val topology = new CNNTopology
topology.addLayer(CNNLayer.buildInputLayer(new Scale(28, 28)))
topology.addLayer(CNNLayer.buildConvLayer(6, new Scale(5, 5)))
topology.addLayer(CNNLayer.buildSampLayer(new Scale(2, 2)))
topology.addLayer(CNNLayer.buildConvLayer(12, new Scale(5, 5)))
topology.addLayer(CNNLayer.buildSampLayer(new Scale(2, 2)))
topology.addLayer(CNNLayer.buildOutputLayer(10))
val cnn: CNN = new CNN(topology).setMaxIterations(5000).setMiniBatchSize(16)
val lines = sc.textFile("dataset/train.format", 8)
val data: RDD[LabeledPoint] = ...
cnn.train(data)
```
I tried on Mnist and it can get error rate less than 0.03%.
I'd like to collect some opinions and suggestions and give users a
prototype to try CNN on Spark. I checked the new design from #1290 and current
CNN can be integrated with slight effort.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hhbyyh/spark cnn
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7609.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7609
----
commit 4ba273213b5d0c3adcd3784de40e640aff1df045
Author: Yuhao Yang <[email protected]>
Date: 2015-07-23T06:39:07Z
initial cnn
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
