Re: Spark 1.1. doesn't work with hive context

2014-08-27 Thread S Malligarjunan
It is my mistake, some how I have added the io.compression.codec property value 
as the above mentioned class. Resolved the problem now
 
Thanks and Regards,
Sankar S.  



On Wednesday, 27 August 2014, 1:23, S Malligarjunan smalligarju...@yahoo.com 
wrote:
 


Hello all,

I have just checked out branch-1.1 
and executed below command
./bin/spark-shell --driver-memory 1G
val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)

val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) 
hiveContext.hql(CREATE TABLE IF NOT EXISTS src (key INT, value STRING)) 
hiveContext.hql(LOAD DATA LOCAL INPATH 'examples/src/main/resources/kv1.txt' 
INTO TABLE src) // Queries are expressed in HiveQL hiveContext.hql(FROM src 
SELECT key, value).collect().foreach(println)

I am getting the following exception

Caused by: java.lang.IllegalArgumentException: Compression codec 
com.hadoop.compression.lzo.LzoCodec not found.
at 
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:135)
at 
org.apache.hadoop.io.compress.CompressionCodecFactory.init(CompressionCodecFactory.java:175)
at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
... 72 more
Caused by: java.lang.ClassNotFoundException: Class 
com.hadoop.compression.lzo.LzoCodec not found

 
Thanks and Regards,
Sankar S.  

Spark 1.1. doesn't work with hive context

2014-08-26 Thread S Malligarjunan
Hello all,

I have just checked out branch-1.1 
and executed below command
./bin/spark-shell --driver-memory 1G
val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)

val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) 
hiveContext.hql(CREATE TABLE IF NOT EXISTS src (key INT, value STRING)) 
hiveContext.hql(LOAD DATA LOCAL INPATH 'examples/src/main/resources/kv1.txt' 
INTO TABLE src) // Queries are expressed in HiveQL hiveContext.hql(FROM src 
SELECT key, value).collect().foreach(println)

I am getting the following exception

Caused by: java.lang.IllegalArgumentException: Compression codec 
com.hadoop.compression.lzo.LzoCodec not found.
at 
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:135)
at 
org.apache.hadoop.io.compress.CompressionCodecFactory.init(CompressionCodecFactory.java:175)
at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
... 72 more
Caused by: java.lang.ClassNotFoundException: Class 
com.hadoop.compression.lzo.LzoCodec not found

 
Thanks and Regards,
Sankar S.