Asher, can you cast like that? Does that casting work? That is my confusion: I don't know what a DataFrame Vector turns into in terms of an RDD type.
I'll try this, thanks. On Tue, Nov 15, 2016 at 11:25 AM, Asher Krim <ak...@hubspot.com> wrote: > What language are you using? For Java, you might convert the dataframe to > an rdd using something like this: > > df > .toJavaRDD() > .map(row -> (SparseVector)row.getAs(row.fieldIndex("columnName"))); > > On Tue, Nov 15, 2016 at 1:06 PM, Russell Jurney <russell.jur...@gmail.com> > wrote: > >> I have two dataframes with common feature vectors and I need to get the >> cosine similarity of one against the other. It looks like this is possible >> in the RDD based API, mllib, but not in ml. >> >> So, how do I convert my sparse dataframe vectors into something spark >> mllib can use? I've searched, but haven't found anything. >> >> Thanks! >> -- >> Russell Jurney twitter.com/rjurney russell.jur...@gmail.com relato.io >> > > > > -- > Asher Krim > Senior Software Engineer > -- Russell Jurney twitter.com/rjurney russell.jur...@gmail.com relato.io