> > However I do think it's easier than it seems to write the implicits; > it doesn't involve new classes or anything. Yes it's pretty much just > what you wrote. There is a class "Vector" in Spark. This declaration > can be in an object; you don't implement your own class. (Also you can > use "toBreeze" to get Breeze vectors.)
The implicit conversion with the implicit def happens for the first vector in the sum, but not the second vector (see below). At this point I give up, because I spent way too much time. I am so disappointed. So many times I heard "Spark makes simple things easy and complicated things possible". Well, here is the simplest thing you can imagine in linear algebra, but heck, it is not easy or intuitive. It was easier to run a DeepLearning algo (from another library) than add two vectors. If anybody has a workaround other than implementing your own add/substract/scalarMultiply, PLEASE let me know. Here is the code and error from (freshly started) spark-shell: scala> import breeze.linalg.{DenseVector => BDV, SparseVector => BSV, Vector => BV} import breeze.linalg.{DenseVector=>BDV, SparseVector=>BSV, Vector=>BV} scala> import org.apache.spark.mllib.linalg.Vectors import org.apache.spark.mllib.linalg.Vectors scala> val v1 = Vectors.dense(1.0, 2.0, 3.0) v1: org.apache.spark.mllib.linalg.Vector = [1.0,2.0,3.0] scala> import org.apache.spark.mllib.linalg.{Vector =>SparkVector} import org.apache.spark.mllib.linalg.{Vector=>SparkVector} scala> object MyUtils { | implicit def toBreeze(v:SparkVector) = BV(v.toArray) | } warning: there were 1 feature warning(s); re-run with -feature for details defined module MyUtils scala> import MyUtils._ import MyUtils._ scala> v1:BV[Double] res2: breeze.linalg.Vector[Double] = DenseVector(1.0, 2.0, 3.0) scala> v1 + v1 <console>:30: error: could not find implicit value for parameter op: breeze.linalg.operators.OpAdd.Impl2[breeze.linalg.Vector[Double],org.apache.spark.mllib.linalg.Vector,That] v1 + v1 ^