On Jul 26, 2013 11:56 PM, "Nick Pentreath" <[email protected]> wrote: > > Thanks for the update on that PR I will definitely take a look. > > > I wonder if they will run into the exact same Colt issues as mahout did?!
Yes i wondered that too since the day i saw spark als example. Jblas is far better choice but as Sebastian has demonstrated bona fide improvements are hard to achieve due to high jni costs, so i would actually have a specific type of matrix to solve specific probems when needed rather than sweepingly generalize it as a dense vector or matrix support. Aside from that, it seems lapack backend is running up to 5x slower on amd hardware that our company unfortunately chose to invest in... argh!.. > > > This DSL looks great, I'm gonna play around with it as soon as I get a chance. > > > > One question - breeze has quite a similar syntax that is a bit simpler in some ways - basically * for matrix multiply and :* for elementwise. Would something similar work here? As i commented before, it just caters to R syntax, along with bunch of other things. If we beleive that there is a reason to inherit syntax vs devising something new, then there are really few candidates, and i dont think Breeze is going to cut it based on adoption level. In particular, in my company it is hard to convince R users to start using scala or java as it is, so I am just scoring points here by making it look familiar to them. Also i want to reserve the colon to command associativity of operation, as scala means it, which is important for optimizing non commutative operations such as elementwise division or matrix multiplication. E.g. there are significant peroformance differences between saying A %*% diagonal === A.times(diagonal) And A %*%: diagonal === diagonal.timesLeft(A). Obviously the latter is n flops and the former is n squared. I dont think breeze made a wise decision by putting a special functional meaning into : . It is reserved for associativity in scala. > > > Would be quite nice to have same syntax but different backends that are swappable ;) > — > Sent from Mailbox for iPhone > > On Sat, Jul 27, 2013 at 2:42 AM, Dmitriy Lyubimov <[email protected]> > wrote: > > > coincidentally, spark mlib just posted a pull request intended to add > > support for dense and sparse vectors, looks quite similar. > > https://github.com/mesos/spark/pull/736. They seem to choose JBlas backing > > for dense stuff (although at a vector level there's probably not much > > reason to) and as-is Colt for sparse stuff. > > On Fri, Jul 26, 2013 at 5:20 PM, Dmitriy Lyubimov <[email protected]> wrote: > >> > >> > >> > >> On Fri, Jul 26, 2013 at 5:07 AM, Ted Dunning <[email protected] >wrote: > >> > >>> This sounds great in principle. I haven't seen any details yet (haven't > >>> had time to look). > >>> > >>> Is there a strong reason to go with the R syntax for multiplication > >>> instead > >>> of the matlab convention that a*b means a.times(b)? > >>> > >> > >> As discussed, but also because matlab style elementwise operators are > >> impossible to keep at proper precedence level in scala. It kind of has to > >> start with either '*' or '%' to keep proper precedence, '.*' will not work > >> unfortunately. And mix along the lines "some of Matlab, some of perhaps > >> completely something else' does not seem appealing at all. > >> > >>
