Re: IDF model error

2014-11-26 Thread Shivani Rao
Thanks Yanbo, I wonder why does SSV does not complain when i create using new SSV(4, Array(1, 3, 5, 7)? Is there no error check for this even in the breeze sparse vector's constructor? That is very strange Shivani On Tue, Nov 25, 2014 at 7:25 PM, Yanbo Liang yanboha...@gmail.com wrote: Hi

IDF model error

2014-11-25 Thread Shivani Rao
Hello Spark fans, I am trying to use the IDF model available in the spark mllib to create an tf-idf representation of a n RDD[Vectors]. Below i have attached my MWE I get the following error java.lang.IndexOutOfBoundsException: 7 not in [-4,4) at

Re: IDF model error

2014-11-25 Thread Yanbo Liang
Hi Shivani, You misunderstand the parameter of SparseVector. class SparseVector( override val size: Int, val indices: Array[Int], val values: Array[Double]) extends Vector { } The first parameter is the total length of the Vector rather than the length of non-zero elements. So it