on an algorithm that has a graph data structure.
>
> it looks like there 2 ways to implement this with spark
>
> option 1: use graphx which already provide Vetices and Edges to build out
> the graph pretty nicely.
>
> option 2: use mllib sparse vector / matrix to build out th
hi. i am working on an algorithm that has a graph data structure.
it looks like there 2 ways to implement this with spark
option 1: use graphx which already provide Vetices and Edges to build out
the graph pretty nicely.
option 2: use mllib sparse vector / matrix to build out the graph
Probably worth noting that the factory methods in mllib create an object of
type org.apache.spark.mllib.linalg.Vector which stores data in a similar format
as Breeze vectors
Chris
On Sep 15, 2014, at 3:24 PM, Xiangrui Meng wrote:
> Or you can use the factory method `Vectors.sparse`:
>
> val
Or you can use the factory method `Vectors.sparse`:
val sv = Vectors.sparse(numProducts, productIds.map(x => (x, 1.0)))
where numProducts should be the largest product id plus one.
Best,
Xiangrui
On Mon, Sep 15, 2014 at 12:46 PM, Chris Gore wrote:
> Hi Sameer,
>
> MLLib uses Breeze’s vector fo
Hi Sameer,
MLLib uses Breeze’s vector format under the hood. You can use that.
http://www.scalanlp.org/api/breeze/index.html#breeze.linalg.SparseVector
For example:
import breeze.linalg.{DenseVector => BDV, SparseVector => BSV, Vector => BV}
val numClasses = classes.distinct.count.toInt
val
Hi All,I have transformed the data into following format: First column is user
id, and then all the other columns are class ids. For a user only class ids
that appear in this row have value 1 and others are 0. I need to crease a
sparse vector from this. Does the API for creating a sparse vector
