Missed the dev-list last email. Resent it again. Please ignore the duplicated one.
2014-09-06 11:22 GMT+08:00 顾荣 <gurongwal...@gmail.com>: > Hi All, > > This is RongGu from PasaLab at Nanjing Universtiy,China. Actually, we have > been working on a distributed matrix operations library on Spark this > summer. It is a Summer Code project hosted by CSDN and Intel Lab ( > http://code.csdn.net/os_camp/8/proposals/26). Previously, the codebase of > the project is hosted on CSDN's code platform( > https://code.csdn.net/u014252240/sparkmatrixlib) and we have been writing > weekly reports on the blog(http://blog.csdn.net/u014252240). > > Now, the project comes to end now. I have moved the project to github > these days. *Please see the link here *https://github.com/PasaLab/saury . > We name the project Saury and provide documents to help people know it > better. > > Technically, we implement the matrix manipulation on Spark with block > matrix parallel algorithms to distribute large scale matrix computation > among cluster nodes. Also, we take advantage of the native linear algebra > library（e.g BLAS）on each worker node to accelerate the computing process. > That really makes a difference! See the preliminary performance evaluation > report at > https://github.com/PasaLab/saury/wiki/Performance-comparison-on-matrices-multiply > > Currently, we are working on adding more advanced matrix manipulation > algorithms into Saury, such as matrix factorization and diagonalization > algorithms. In fact, Saury contains an alpha version distributed LU > factorization implementation now. Also, we are trying to use Tachyon to > hold and share the matrix data across the cluster with faster speed. > > Best, > Rong > > -- > ------------------ > Rong Gu > Department of Computer Science and Technology > State Key Laboratory for Novel Software Technology > Nanjing University > Email: gurongwal...@gmail.com > Homepage: http://pasa-bigdata.nju.edu.cn/people/ronggu/ > > > 2014-09-06 1:29 GMT+08:00 Jeremy Freeman <freeman.jer...@gmail.com>: > >> Hey all, >> >> Definitely agreed this would be nice! In our own work we've done >> element-wise addition, subtraction, and scalar multiplication of similarly >> partitioned matrices very efficiently with zipping. We've also done >> matrix-matrix multiplication with zipping, but that only works in certain >> circumstances, and it's otherwise very communication intensive (as Shivaram >> says). Another tricky thing with addition / subtraction is how to handle >> sparse vs. dense arrays. >> >> Would be happy to contribute anything we did, but definitely first worth >> knowing what progress has been made from the AMPLab. >> >> -- Jeremy >> >> --------------------- >> jeremy freeman, phd >> neuroscientist >> @thefreemanlab >> >> On Sep 5, 2014, at 12:23 PM, Patrick Wendell <pwend...@gmail.com> wrote: >> >> > Hey There, >> > >> > I believe this is on the roadmap for the 1.2 next release. But >> > Xiangrui can comment on this. >> > >> > - Patrick >> > >> > On Fri, Sep 5, 2014 at 9:18 AM, Yu Ishikawa >> > <yuu.ishikawa+sp...@gmail.com> wrote: >> >> Hi Evan, >> >> >> >> That's sounds interesting. >> >> >> >> Here is the ticket which I created. >> >> https://issues.apache.org/jira/browse/SPARK-3416 >> >> >> >> thanks, >> >> >> >> >> >> >> >> -- >> >> View this message in context: >> http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Add-multiplying-large-scale-matrices-tp8291p8296.html >> >> Sent from the Apache Spark Developers List mailing list archive at >> Nabble.com. >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> >> For additional commands, e-mail: dev-h...@spark.apache.org >> >> >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> > For additional commands, e-mail: dev-h...@spark.apache.org >> > >> >> > > > -- > ------------------ > Rong Gu > Department of Computer Science and Technology > State Key Laboratory for Novel Software Technology > Nanjing University > Phone: +86 15850682791 > Email: gurongwal...@gmail.com > Homepage: http://pasa-bigdata.nju.edu.cn/people/ronggu/ > -- ------------------ Rong Gu Department of Computer Science and Technology State Key Laboratory for Novel Software Technology Nanjing University Phone: +86 15850682791 Email: gurongwal...@gmail.com Homepage: http://pasa-bigdata.nju.edu.cn/people/ronggu/
