[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-67083534 [Test build #545 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/545/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-15 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-67083539 The test logs have expired...rerunning --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-15 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-67087128 @witgo @ankurdave Thank you for suggestion! I merged #3677 into the ANN code and then I run 100 iterations on mnist (60K instances). The running time improved only 1%.

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-15 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-67087246 @jkbradley Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-67089450 [Test build #545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/545/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-15 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-67089972 It looks like the error is: ``` [error] /home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/src/test/scala/org/apache/spark/mllib/ann/ANNSuite.scala:21:

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-15 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-67091484 @jkbradley This PR references an old MLlib that had this class `LocalSparkContext`. It was substituted with `MLlibTestSparkContext` in one of the latest releases. This

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-15 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-67092852 Yes, Jenkins will test against the master branch, so I'd recommend merging with master (or rebasing if the merge is messy). --- If your project is set up for it, you

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-13 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-66876255 @ankurdave #3677 should be able to help improve this PR performance --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-13 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-66876521 @avulanov I will submit a new PR about `AdaDelta` and `AdaGrad` in the next week It should be able to use in this PR. --- If your project is set up for it, you can

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-12 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-66824545 @mengxr Could you suggest why the build has failed? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-10 Thread bgreeven
Github user bgreeven commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r21603916 --- Diff: docs/mllib-ann.md --- @@ -0,0 +1,239 @@ +--- +layout: global +title: Artificial Neural Networks - MLlib +displayTitle: a

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-66454787 [Test build #24310 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24310/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-66461883 [Test build #24310 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24310/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-66461896 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-09 Thread ZhangBanger
Github user ZhangBanger commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r21571009 --- Diff: docs/mllib-ann.md --- @@ -0,0 +1,239 @@ +--- +layout: global +title: Artificial Neural Networks - MLlib +displayTitle: a

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-05 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-65880449 @jkbradley @manishamde I did performance comparisons with multinomial regression and posted them here: https://github.com/apache/spark/pull/1379#issuecomment-65879536.

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-04 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-65731474 I've submitted a feature request to Spark Jira regarding the ANN-based classifier: https://issues.apache.org/jira/browse/SPARK-4752. It is implemented already and I

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-12-02 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-65264353 Just in case, I've tested the ANN without the hidden layer and it seems to work as multinomial regression, though with (one-half) squared-error cost function and

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-24 Thread Lewuathe
Github user Lewuathe commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-64176809 @avulanov What type of resource manager did you use for checking performance? I'm trying to look into the scalability how performance will increase depending of the

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-24 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-64283686 @manishamde Thanks for the useful references! It seems that model parallelization for ANN is a challenging problem. I asked this question to few presenters on the

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-24 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-64285217 @jkbradley Thank you for good explanation! As for (1), single threaded ANN implemented in C++ shows similar accuracy. (2) seems like some additional coding needs to be

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-24 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-64285419 @Lewuathe Ganglia is used to monitor the cluster performance and it shows that cores are busy all the way and memory underutilized. No resource manager is used. ---

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-19 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r20589805 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-19 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r20591105 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-19 Thread bgreeven
Github user bgreeven commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r20620263 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-17 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-63371703 @jkbradley (a) I found few papers with timing results, however with it was with SVM. Moreover, the results differ from minutes to hours. Could you suggest a paper with

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-17 Thread manishamde
Github user manishamde commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-63375342 @avulanov Thanks for conducting the experiments. Could you plot graphs for the experiments that you conducted with changing number of features and number of machines.

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-17 Thread manishamde
Github user manishamde commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-63375573 I found this reference recently about Netflix's distributed implementation of neural nets that could be relevant for MLlib.

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-17 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-63401446 @avulanov I'll look around for papers which might allow for comparisons; I'm not sure offhand. For experiments, I agree with @manishamde may getting

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-15 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-63196649 @avulanov Thanks for running tests! It would be great if we could calibrate tests somehow. Some options would be: (a) trying to reproduce tests in a published paper

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-12 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62783988 @debasish83 I agree, such setup will not fit into the memory of a singe node. Are you talking about recommender system? As far as I know, they are usually addressed by

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-12 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62823148 @mengxr I've performed a test on mnist8m with our implementation of ANN as you suggested. I used a 5-node cluster. Each node has Xeon 3.3GHz 4 cores with 16GB RAM. My

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-12 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62824356 @avulanov I meant first sparse auto encoder with 1 layer and followed by a deep net...these huge nets are mostly for automated feature extraction...of course we can

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r20181004 --- Diff: docs/mllib-ann.md --- @@ -0,0 +1,223 @@ +--- +layout: global +title: Artificial Neural Networks - MLlib +displayTitle: a

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-11 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62623318 @bgreeven @avulanov About supporting various optimizer, updater, gradient, and error function options, I vote for keeping them as parameters, rather than having

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-08 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62254979 I agree with what @debasish83 said. We should find a suitable solution to weight matrix distributed storage. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-07 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62187026 @witgo Thank you for your suggestion! Could you elaborate how als algorithm design could be used? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-07 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62190595 For matrix factorization we have user x product sparse matrix...You can think of this sparse matrix as the feature matrix for ANN...Now consider two matrices H1 and

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-07 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62191470 f is neural activation...it can be tanh or sigmoid function (they are non-convex, nonlinear) , LRU units (max is convex)...in this PR

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-07 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62195511 I think I am missing something from your description. In ANN we need to compute K+1 weight matrices: W1(NxH1), W2(H1xH2), ... WK(HKxM), where N is the number of inputs

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-07 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62197587 I am considering 1 hidden layer ANN right now...For multiple layers I have not thought on it since to scale it (if you want the model to be distributed) I don't think

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-07 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62199347 I think before doing this we need to consider the benefits versus the current implementation with loops. I can name one - the readability of code will be better with

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-07 Thread debasish83
Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62247730 Our sparse data very often has ~10M users and 1M features...for 1 hidden layer net with 10K nodes, we need 10M x 10K + 1M x 10K doubles...if the model is not

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-06 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62085010 We should use matrix to calculate the forward propagation ,back propagation see http://deeplearning.stanford.edu/wiki/index.php/Neural_Network_Vectorization --- If

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-06 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62086344 @witgo I have implemented ANN in matrix form for Spark with breeze a while ago: https://github.com/avulanov/spark/tree/neuralnetwork. We tested it with @bgreeven and it

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-06 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62089562 We can not use existing Gradient classes,Let the whole iterative process is completed in the form of matrix calculation.Moreover We can use the als algorithm design, cut

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-05 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19922521 --- Diff: docs/mllib-ann.md --- @@ -0,0 +1,223 @@ +--- +layout: global +title: Artificial Neural Networks - MLlib +displayTitle: a

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-04 Thread bgreeven
Github user bgreeven commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61749598 Let's discuss a bit more about making the optimiser, updater, gradient, and error function customizable. Notice that for the current LBFGS algorithm, the error

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-04 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61757505 I think that these 3 parameters should be somehow bound otherwise one can plug a gradient with vector length that does not correspond to the ANN size. We could provide

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-04 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61757723 With regards to performance tests, I am going to use mnist8m (http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multiclass.html) as suggested by @mengxr. I have

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread bgreeven
Github user bgreeven commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19722711 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread bgreeven
Github user bgreeven commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19723239 --- Diff: docs/mllib-ann.md --- @@ -0,0 +1,223 @@ +--- +layout: global +title: Artificial Neural Networks - MLlib +displayTitle: a

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread bgreeven
Github user bgreeven commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19733377 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread bgreeven
Github user bgreeven commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19733442 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61480481 [Test build #22818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22818/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61481496 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61499579 **[Test build #22818 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22818/consoleFull)** for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61499597 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61584664 [Test build #22851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22851/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61590143 [Test build #22851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22851/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61590149 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19717255 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread manishamde
Github user manishamde commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61433892 @bgreeven I haven't studied the implementation details yet but I had a question about the API. I realize that RDD[(Vector, Vector)] is a more general structure for

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19717451 --- Diff: docs/mllib-ann.md --- @@ -0,0 +1,223 @@ +--- +layout: global +title: Artificial Neural Networks - MLlib +displayTitle: a

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19717692 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19717699 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread manishamde
Github user manishamde commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-61436212 @bgreeven Another general suggestion: consider adding logging to the code. It goes a long way in debugging errors and get statuses on long running job. Check

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19717937 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19717985 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/ArtificialNeuralNetwork.scala --- @@ -0,0 +1,528 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread avulanov
Github user avulanov commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19719009 --- Diff: docs/mllib-ann.md --- @@ -0,0 +1,223 @@ +--- +layout: global +title: Artificial Neural Networks - MLlib +displayTitle: a

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-11-02 Thread manishamde
Github user manishamde commented on a diff in the pull request: https://github.com/apache/spark/pull/1290#discussion_r19720701 --- Diff: docs/mllib-ann.md --- @@ -0,0 +1,223 @@ +--- +layout: global +title: Artificial Neural Networks - MLlib +displayTitle: a

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-10-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-60742124 [Test build #22359 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22359/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-60749017 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-10-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-60749010 [Test build #22359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22359/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-10-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-60844377 [Test build #22385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22385/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-10-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-60853271 [Test build #22385 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22385/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-60853279 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-57076438 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20930/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-57077703 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-57077701 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20930/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56952211 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20857/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56957127 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56957114 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20857/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56962296 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20858/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56971412 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20858/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56971423 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56789789 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20793/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56790021 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56790721 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20795/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56791029 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56791103 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20795/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56791106 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56791275 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20797/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56796798 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20793/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56796805 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56798291 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20797/consoleFull) for PR 1290 at commit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56798296 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-22 Thread bgreeven
Github user bgreeven commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56343909 Changed optimiser to LBFGS. Works much faster, but has the disadvantage (due to the increased convergence speed per iteration) that it also starts to exhibit

[GitHub] spark pull request: [MLLIB] [spark-2352] Implementation of an 1-hi...

2014-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-56343891 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20650/consoleFull) for PR 1290 at commit

  1   2   >