[jira] [Comment Edited] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150613#comment-15150613 ] Asim Jalis edited comment on SPARK-9273 at 2/17/16 3:04 PM: Convolutional Neural Networks are significantly different from generic Artificial Neural Networks that this should be reopened. Unless the ANN functions provide way to get CNN functionality I don’t see this as a duplicate. was (Author: asimjalis): Convolutional Neural Networks are significantly different from Artificial Neural Networks that this should be reopened. Unless the ANN functions provide way to get CNN functionality I don’t see this as a duplicate. > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang >Assignee: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003468#comment-15003468 ] yuhao yang edited comment on SPARK-9273 at 11/17/15 5:36 AM: - Reopen the jira since it's an independent feature and under active development. I will hold the code at https://github.com/hhbyyh/spark/tree/cnnMaster and post major updates here. was (Author: yuhaoyan): Reopen the jira since it's an independent feature and under active development. > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang >Assignee: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992447#comment-14992447 ] Alexander Ulanov edited comment on SPARK-9273 at 11/5/15 8:50 PM: -- Hi Yuhao. Sounds good! Thanks for refactoring the code to support ANN internal interface. Also, I was able to run your example. It shows increasing accuracy while training however it is not very fast. Does it work with LBFGS? There is a good explanation how to use matrices multiplication in convolution: http://cs231n.github.io/convolutional-networks/. Basically, one needs to roll all image patches (regions that will be convolved) the into vectors and stack them together in a matrix. The weights of convolutional layer also should be rolled into vectors and stacked. Multiplying two mentioned matrices provides the convolution result that can be unrolled to 3d matrix, however it would not be necessary for this implementation. We can discuss it offline if you wish. Besides the optimization, there are few more things to be done. It includes unit tests for new layers, gradient test, representing pooling layer as functional layer, and performance comparison with the other implementation of CNN. You can take a look at the tests I've added for MLP https://issues.apache.org/jira/browse/SPARK-11262 and MLP benchmark at https://github.com/avulanov/ann-benchmark. A separate branch/repo for these developments might be a good thing to do. I'll be happy to help you with this. was (Author: avulanov): Hi Yuhao. Sounds good! Thanks for refactoring the code to support ANN internal interface. Also, I was able to run your example. It shows increasing accuracy while training however it is not very fast. There is a good explanation how to use matrices multiplication in convolution: http://cs231n.github.io/convolutional-networks/. Basically, one needs to roll all image patches (regions that will be convolved) the into vectors and stack them together in a matrix. The weights of convolutional layer also should be rolled into vectors and stacked. Multiplying two mentioned matrices provides the convolution result that can be unrolled to 3d matrix, however it would not be necessary for this implementation. We can discuss it offline if you wish. Besides the optimization, there are few more things to be done. It includes unit tests for new layers, gradient test, representing pooling layer as functional layer, and performance comparison with the other implementation of CNN. You can take a look at the tests I've added for MLP https://issues.apache.org/jira/browse/SPARK-11262 and MLP benchmark at https://github.com/avulanov/ann-benchmark. A separate branch/repo for these developments might be a good thing to do. I'll be happy to help you with this. > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737948#comment-14737948 ] Alexander Ulanov edited comment on SPARK-9273 at 9/10/15 1:18 AM: -- Hi Yuhao! I have few comments regarding the interface and the optimization of your implementation. There are two options of optimizing convolutions: using matrix-matrix multiplication and using FFTs. The latter seems a bit more complicated since we don't have optimized parallel FFT in Spark. It also has to support batch data processing. Instead, if one uses matrix-matrix multiplication for convolution, then it can take advantage of native BLAS and batch computations can be supported straightforward. Another benefit is that we would not need to change the current Layer's input/ouptput type (matrix) to tensor. We can store the unwrapped inputs/outputs as vectors within the input/output matrix. Does it make sense to you? was (Author: avulanov): Hi Yuhao! I have few comments regarding the interface and the optimization of your implementation. There are two options of optimizing convolutions: using matrix-matrix multiplication and using FFTs. The latter seems a bit more complicated since we don't have optimized parallel FFT in Spark. It also has to support batch data processing. Instead, if one uses matrix-matrix multiplication for convolution, then it can take advantage of native BLAS and batch computations can be supported straightforward. Another benefit is that we would not need to change the current Layer's input/ouptput type (matrix) to tensor. We can store the unwrapped inputs/outputs as vectors within the input/output matrix. Do you think that it is reasonable? > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642513#comment-14642513 ] Alexander Ulanov edited comment on SPARK-9273 at 7/27/15 9:54 AM: -- I have not heard about the PR until it was submitted. It would be useful to look at the code, benchmark it and see if it fits our API. I've added the link to the umbrella issue for deep learning https://issues.apache.org/jira/browse/SPARK-5575 was (Author: avulanov): I have not heard about the PR until it was submitted. It would be useful to look at the code, benchmark it and see if it fits our API. > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
