Welcome to try and contribute to our BigDL: https://github.com/intel-analytics/BigDL
It's native on Spark and fast by leveraging Intel MKL. 2017-02-23 4:51 GMT-08:00 Joeri Hermans <joeri.raymond.e.herm...@cern.ch>: > Hi Nikita, > > We are actively working on this: https://github.com/cerndb/dist-keras > This will allow you to run Keras on Spark (with distributed optimization > algorithms) through pyspark. I recommend you to check the examples > https://github.com/cerndb/dist-keras/tree/master/examples. However, you > need to be aware that distributed optimization is a research topic, and has > several approaches and caveats you need to be aware of. I wrote a blog post > on this if you like to have some additional information on this topic > https://db-blog.web.cern.ch/blog/joeri-hermans/2017-01- > distributed-deep-learning-apache-spark-and-keras > > However, if you don't want to use a distributed optimization algorithm, we > also support a "sequential trainer" which allows you to train a model on > Spark dataframes. > > Kind regards, > > Joeri > ________________________________________. > From: Nick Pentreath [nick.pentre...@gmail.com] > Sent: 23 February 2017 13:39 > To: dev@spark.apache.org > Subject: Re: Implementation of RNN/LSTM in Spark > > The short answer is there is none and highly unlikely to be inside of > Spark MLlib any time in the near future. > > The best bets are to look at other DL libraries - for JVM there is > Deeplearning4J and BigDL (there are others but these seem to be the most > comprehensive I have come across) - that run on Spark. Also there are > various flavours of TensorFlow / Caffe on Spark. And of course the libs > such as Torch, Keras, Tensorflow, MXNet, Caffe etc. Some of them have Java > or Scala APIs and some form of Spark integration out there in the community > (in varying states of development). > > Integrations with Spark are a bit patchy currently but include the > "XOnSpark" flavours mentioned above and TensorFrames (again, there may be > others). > > On Thu, 23 Feb 2017 at 14:23 n1kt0 <nikita.balysc...@googlemail.com > <mailto:nikita.balysc...@googlemail.com>> wrote: > Hi, > can anyone tell me what the current status about RNNs in Spark is? > > > > -- > View this message in context: http://apache-spark- > developers-list.1001551.n3.nabble.com/Implementation-of-RNN-LSTM-in-Spark- > tp14866p21060.html > Sent from the Apache Spark Developers List mailing list archive at > Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org<mailto: > dev-unsubscr...@spark.apache.org> > > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
