Hi Devesh,
you have to start your SPARK Shell using the packages. The command is
mentioned below (you can use pyspark instead of spark-shell), anyways all
the required commands for this is mentioned here
https://github.com/databricks/spark-csv and I prefer using the 2.11 version
instead of 2.10 as
download a right version of this jar
http://mvnrepository.com/artifact/com.databricks/spark-csv_2.10 (or 2.11),
and append it to SPARK_CLASSPATH
2016-02-18 11:05 GMT+01:00 Devesh Raj Singh :
> Hi,
>
> I want to read CSV file in pyspark
>
> I am running pyspark on pycharm
> I am trying to load a c
Hi,
I want to read CSV file in pyspark
I am running pyspark on pycharm
I am trying to load a csv using pyspark
import os
import sys
os.environ['SPARK_HOME']="/Users/devesh/Downloads/spark-1.5.1-bin-hadoop2.6"
sys.path.append("/Users/devesh/Downloads/spark-1.5.1-bin-hadoop2.6/python/")
# Now we