How to import mllib.rdd.RDDFunctions into the spark-shell

2014-10-28 Thread Stephen Boesch
I seem to recall there were some specific requirements on how to import the implicits. Here is the issue: scala import org.apache.spark.mllib.rdd.RDDFunctions._ console:10: error: object RDDFunctions in package rdd cannot be accessed in package org.apache.spark.mllib.rdd import

Re: How to import mllib.rdd.RDDFunctions into the spark-shell

2014-10-28 Thread Yanbo Liang
Because that org.apache.spark.mllib.rdd.RDDFunctions._ is mllib private class, it can only be called by function in mllib. 2014-10-28 17:09 GMT+08:00 Stephen Boesch java...@gmail.com: I seem to recall there were some specific requirements on how to import the implicits. Here is the issue:

Re: How to import mllib.rdd.RDDFunctions into the spark-shell

2014-10-28 Thread Stephen Boesch
HI Yanbo, That is not the issue: notice that importing the object is fine: scala import org.apache.spark.mllib.rdd.RDDFunctions import org.apache.spark.mllib.rdd.RDDFunctions scala import org.apache.spark.mllib.rdd.RDDFunctions._ console:11: error: object RDDFunctions in package rdd cannot be

Re: How to import mllib.rdd.RDDFunctions into the spark-shell

2014-10-28 Thread Yanbo Liang
Yes, it can import org.apache.spark.mllib.rdd.RDDFunctions but you can not use any method in this class or even new an object of this class. So I infer that if you import org.apache.spark.mllib.rdd.RDDFunctions._, it may call some method of that object. 2014-10-28 17:29 GMT+08:00 Stephen Boesch

Re: How to import mllib.rdd.RDDFunctions into the spark-shell

2014-10-28 Thread Xiangrui Meng
FYI, there is a PR to make mllib.rdd.RDDFunctions public: https://github.com/apache/spark/pull/2907 -Xiangrui On Tue, Oct 28, 2014 at 5:18 AM, Yanbo Liang yanboha...@gmail.com wrote: Yes, it can import org.apache.spark.mllib.rdd.RDDFunctions but you can not use any method in this class or even