Re: Hive on Spark orc file empty

2015-11-16 Thread 张炜
Hi Deepak and all, write() is a function of DataFrame, please check https://spark.apache.org/docs/1.5.1/api/java/org/apache/spark/sql/DataFrame.html , the last one is write() The problem is write to HDFS is successful but esults.write.format("orc").save("yahoo_stocks_orc") has empty folder. Could

Re: Hive on Spark orc file empty

2015-11-16 Thread Deepak Sharma
Sai, I am bit confused here. How are you using write with results? I am using spark 1.4.1 and when i use write , it complains about write not being member of DataFrame. error:value write is not a member of org.apache.spark.sql.DataFrame Thanks Deepak On Mon, Nov 16, 2015 at 4:10 PM, 张炜 wrote: >

Hive on Spark orc file empty

2015-11-16 Thread 张炜
Dear all, I am following this article to try Hive on Spark http://hortonworks.com/hadoop-tutorial/using-hive-with-orc-from-apache-spark/ My environment: Hive 1.2.1 Spark 1.5.1 in a nutshell, I ran spark-shell, created a hive table hiveContext.sql("create table yahoo_orc_table (date STRING, open_