you need to instantiate a SQLContext :
val sc : SparkContext = ...
val sqlContext = new SQLContext(sc)
import sqlContext.implicits._
Le mar. 12 mai 2015 à 12:29, SLiZn Liu <sliznmail...@gmail.com> a écrit :
> I added `libraryDependencies += "org.apache.spark" % "spark-sql_2.11" %
> "1.3.1"` to `build.sbt` but the error remains. Do I need to import modules
> other than `import org.apache.spark.sql.{ Row, SQLContext }`?
>
> On Tue, May 12, 2015 at 5:56 PM Olivier Girardot <ssab...@gmail.com>
> wrote:
>
>> toDF is part of spark SQL so you need Spark SQL dependency + import
>> sqlContext.implicits._ to get the toDF method.
>>
>> Regards,
>>
>> Olivier.
>>
>> Le mar. 12 mai 2015 à 11:36, SLiZn Liu <sliznmail...@gmail.com> a écrit :
>>
>>> Hi User Group,
>>>
>>> I’m trying to reproduce the example on Spark SQL Programming Guide
>>> <https://spark.apache.org/docs/latest/sql-programming-guide.html#inferring-the-schema-using-reflection>,
>>> and got a compile error when packaging with sbt:
>>>
>>> [error] myfile.scala:30: value toDF is not a member of
>>> org.apache.spark.rdd.RDD[Person]
>>> [error] val people =
>>> sc.textFile("examples/src/main/resources/people.txt").map(_.split(",")).map(p
>>> => Person(p(0), p(1).trim.toInt)).toDF()
>>> [error]
>>> ^
>>> [error] one error found
>>> [error] (compile:compileIncremental) Compilation failed
>>> [error] Total time: 3 s, completed May 12, 2015 4:11:53 PM
>>>
>>> I double checked my code includes import sqlContext.implicits._ after
>>> reading this post
>>> <https://mail-archives.apache.org/mod_mbox/spark-user/201503.mbox/%3c1426522113299-22083.p...@n3.nabble.com%3E>
>>> on spark mailing list, even tried to use toDF("col1", "col2") suggested
>>> by Xiangrui Meng in that post and got the same error.
>>>
>>> The Spark version is specified in build.sbt file as follows:
>>>
>>> scalaVersion := "2.11.6"
>>> libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "1.3.1" %
>>> "provided"
>>> libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "1.3.1"
>>>
>>> Anyone have ideas the cause of this error?
>>>
>>> REGARDS,
>>> Todd Leo
>>>
>>>
>>
