Very cool! ---david
On Friday, November 21, 2014 3:00:42 AM UTC+1, Chiyuan Zhang wrote: > > https://github.com/pluskid/Mocha.jl > Mocha is a Deep Learning framework for Julia <http://julialang.org/>, > inspired by the C++ Deep Learning framework Caffe > <http://caffe.berkeleyvision.org/>. Since this is the first time I post > announcement here, change logs of the last two releases are listed: > > v0.0.2 2014.11.20 > > - Infrastructure > - Ability to import caffe trained model > - Properly release all the allocated resources upon backend shutdown > - Network > - Sigmoid activation function > - Power, Split, Element-wise layers > - Local Response Normalization layer > - Channel Pooling layer > - Dropout Layer > - Documentation > - Complete MNIST demo > - Complete CIFAR-10 demo > - Major part of User's Guide > > <https://github.com/pluskid/Mocha.jl/blob/master/NEWS.md#v001-20141113>v0.0.1 > 2014.11.13 > > - Backend > - Pure Julia CPU > - Julia + C++ Extension CPU > - CUDA + cuDNN GPU > - Infrastructure > - Evaluate on validation set during training > - Automaticly saving and recovering from snapshots > - Network > - Convolution layer, mean and max pooling layer, fully connected > layer, softmax loss layer > - ReLU activation function > - L2 Regularization > - Solver > - SGD with momentum > - Documentation > - Demo code of LeNet on MNIST > - Tutorial document on the MNIST demo (half finished) > > > > Below is a copy of the README file: > ---------------------------------------------------------------- > > Mocha is a Deep Learning framework for Julia <http://julialang.org/>, > inspired by the C++ Deep Learning framework Caffe > <http://caffe.berkeleyvision.org/>. Mocha support multiple backends: > > - Pure Julia CPU Backend: Implemented in pure Julia; Runs out of the > box without any external dependency; Reasonably fast on small models > thanks > to Julia's LLVM-based just-in-time (JIT) compiler and Performance > Annotations > > <http://julia.readthedocs.org/en/latest/manual/performance-tips/#performance-annotations> > that > eliminate unnecessary bound checkings. > - CPU Backend with Native Extension: Some bottleneck computations > (Convolution and Pooling) have C++ implementations. When compiled and > enabled, could be faster than the pure Julia backend. > - CUDA + cuDNN: An interface to NVidia® cuDNN > <https://developer.nvidia.com/cuDNN> GPU accelerated deep learning > library. When run with CUDA GPU devices, could be much faster depending on > the size of the problem (e.g. on MNIST CUDA backend is roughly 20 times > faster than the pure Julia backend). > > <https://github.com/pluskid/Mocha.jl#installation>Installation > > To install the release version, simply run > > Pkg.add("Mocha") > > in Julia console. To install the latest development version, run the > following command instead: > > Pkg.clone("https://github.com/pluskid/Mocha.jl.git";) > > Then you can run the built-in unit tests with > > Pkg.test("Mocha") > > to verify that everything is functioning properly on your machine. > <https://github.com/pluskid/Mocha.jl#hello-world>Hello World > > Please refer to the MNIST tutorial > <http://mochajl.readthedocs.org/en/latest/tutorial/mnist.html> on how > prepare the MNIST dataset for the following example. The complete code for > this example is located at examples/mnist/mnist.jl > <https://github.com/pluskid/Mocha.jl/blob/master/examples/mnist/mnist.jl>. > See below for detailed documentation of other tutorials and user's guide. > > using Mocha > > data = > HDF5DataLayer(name="train-data",source="train-data-list.txt",batch_size=64) > conv = > ConvolutionLayer(name="conv1",n_filter=20,kernel=(5,5),bottoms=[:data],tops=[:conv]) > pool = > PoolingLayer(name="pool1",kernel=(2,2),stride=(2,2),bottoms=[:conv],tops=[:pool]) > conv2 = > ConvolutionLayer(name="conv2",n_filter=50,kernel=(5,5),bottoms=[:pool],tops=[:conv2]) > pool2 = > PoolingLayer(name="pool2",kernel=(2,2),stride=(2,2),bottoms=[:conv2],tops=[:pool2]) > fc1 = > InnerProductLayer(name="ip1",output_dim=500,neuron=Neurons.ReLU(),bottoms=[:pool2], > tops=[:ip1]) > fc2 = InnerProductLayer(name="ip2",output_dim=10,bottoms=[:ip1],tops=[:ip2]) > loss = SoftmaxLossLayer(name="loss",bottoms=[:ip2,:label]) > > sys = System(CuDNNBackend())init(sys) > > common_layers = [conv, pool, conv2, pool2, fc1, fc2] > net = Net("MNIST-train", sys, [data, common_layers..., loss]) > > params = SolverParameters(max_iter=10000, regu_coef=0.0005, momentum=0.9, > lr_policy=LRPolicy.Inv(0.01, 0.0001, 0.75)) > solver = SGD(params) > # report training progress every 100 iterationsadd_coffee_break(solver, > TrainingSummary(), every_n_iter=100) > # save snapshots every 5000 iterationsadd_coffee_break(solver, > Snapshot("snapshots", auto_load=true), > every_n_iter=5000) > # show performance on test data every 1000 iterations > data_test = > HDF5DataLayer(name="test-data",source="test-data-list.txt",batch_size=100) > accuracy = AccuracyLayer(name="test-accuracy",bottoms=[:ip2, :label]) > test_net = Net("MNIST-test", sys, [data_test, common_layers..., > accuracy])add_coffee_break(solver, ValidationPerformance(test_net), > every_n_iter=1000) > solve(solver, net) > destroy(net)destroy(test_net)shutdown(sys) > > <https://github.com/pluskid/Mocha.jl#documentation>Documentation > > The Mocha documentation is hosted on readthedocs.org > <http://mochajl.readthedocs.org/>. >
