Re: spark 1.6.0 on ec2 doesn't work

2016-01-18 Thread Peter Zhang
Could you run spark-shell at $SPARK_HOME DIR? You can try to change you command run at $SPARK_HOME or, point to README.md with full path. Peter Zhang --  Google Sent with Airmail On January 19, 2016 at 11:26:14, Oleg Ruchovets (oruchov...@gmail.com) wrote: It looks spark is not working fine

SparkR with Hive integration

2016-01-18 Thread Peter Zhang
/warehouse/src 16/01/19 12:11:51 ERROR DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:file:/user/hive/warehouse/src is not a directory or unable to create one) How  to use HDFS instead of local file system(file)? Which parameter should to set? Thanks a lot. Peter Zhang --  Google Sent with Airmail

Re: SparkR with Hive integration

2016-01-18 Thread Peter Zhang
Thanks,  I will try. Peter --  Google Sent with Airmail On January 19, 2016 at 12:44:46, Jeff Zhang (zjf...@gmail.com) wrote: Please make sure you export environment variable HADOOP_CONF_DIR which contains the core-site.xml On Mon, Jan 18, 2016 at 8:23 PM, Peter Zhang <zhangju...@gmail.

Re: error: not found: value StructType on 1.5.2

2015-12-20 Thread Peter Zhang
Hi Eran, Missing import package. import org.apache.spark.sql.types._ will work. please try. Peter Zhang --  Google Sent with Airmail On December 20, 2015 at 21:43:42, Eran Witkon (eranwit...@gmail.com) wrote: Hi, I am using spark-shell with version 1.5.2. scala