Firstly I don't have ORC data to verify but this should work: df <- loadDF(sqlContext, "data/path", "orc") Secondly, could you check if sparkR.stop() was called? sparkRHive.init() should be called after sparkR.init() - please check if there is any error message there.
_____________________________
From: Prem Sure <premsure...@gmail.com>
Sent: Tuesday, January 5, 2016 8:12 AM
Subject: Re: sparkR ORC support.
To: Sandeep Khurana <sand...@infoworks.io>
Cc: spark users <user@spark.apache.org>, Deepak Sharma <deepakmc...@gmail.com>
Yes Sandeep, also copy hive-site.xml too to spark conf directory.
On Tue, Jan 5, 2016 at 10:07 AM, Sandeep Khurana
<sand...@infoworks.io> wrote:
Also, do I need to setup hive in spark as per the link
http://stackoverflow.com/questions/26360725/accesing-hive-tables-in-spark ?
We might need to copy hdfs-site.xml file to spark conf
directory ?
On Tue, Jan 5, 2016 at 8:28 PM, Sandeep Khurana
<sand...@infoworks.io> wrote:
Deepak
Tried this. Getting this error now
rror in sql(hivecontext, "FROM CATEGORIES SELECT
category_id", "") : unused argument ("")
On Tue, Jan 5, 2016 at 6:48 PM, Deepak Sharma
<deepakmc...@gmail.com> wrote:
Hi Sandeep
can you try this ?
results <-
sql(hivecontext, "FROM test SELECT id","")
Thanks
Deepak
On Tue, Jan 5, 2016 at 5:49 PM, Sandeep
Khurana <sand...@infoworks.io> wrote:
Thanks Deepak.
I tried this as
well. I created a hivecontext with "hivecontext <<- sparkRHive.init(sc) " .
When I tried to
read hive table from this ,
results <-
sql(hivecontext, "FROM test SELECT id")
I get below
error,
Error in
callJMethod(sqlContext, "sql", sqlQuery) : Invalid jobj 2. If SparkR was
restarted, Spark operations need to be re-executed.
Not sure what is causing this? Any leads or ideas? I am
using rstudio.
On Tue, Jan 5, 2016 at 5:35 PM,
Deepak Sharma <deepakmc...@gmail.com> wrote:
Hi Sandeep
I am not sure if ORC can be read directly in R.
But there can be a workaround .First create hive table on top of ORC files and
then access hive table in R.
Thanks
Deepak
On Tue, Jan 5, 2016 at
4:57 PM, Sandeep Khurana <sand...@infoworks.io>
wrote:
Hello
I need to read an ORC files in hdfs in R using spark. I am not
able to find a package to do that.
Can anyone help with documentation or example for this purpose?
--
Architect
Infoworks.io
http://Infoworks.io
--
Thanks
Deepak
www.bigdatabig.com
www.keosha.net
--
Architect
Infoworks.io
http://Infoworks.io
--
Thanks
Deepak
www.bigdatabig.com
www.keosha.net
--
Architect
Infoworks.io
http://Infoworks.io
--
Architect
Infoworks.io
http://Infoworks.io
