;
> Is there a reason you want to call stop? If you do, you would need to call
> the line hivecontext <- sparkRHive.init(sc) again.
>
>
> _
> From: Sandeep Khurana
> Sent: Tuesday, January 12, 2016 5:20 AM
> Subject: Re: sparkR ORC support.
> To: F
would need to call the
line hivecontext <- sparkRHive.init(sc) again.
_
From: Sandeep Khurana
Sent: Tuesday, January 12, 2016 5:20 AM
Subject: Re: sparkR ORC support.
To: Felix Cheung
Cc: spark users , Prem Sure ,
Deepak Sharma , Yanbo Liang
It worked for so
k-client")
>>>
>>> .libPaths(c(file.path(Sys.getenv("SPARK_HOME"), "R", "lib"),
>>> .libPaths()))
>>> library(SparkR)
>>>
>>> sc <- sparkR.init()
>>> hivecontext <- sparkRHive.init(sc)
>>&g
gt;> stop("invalid jobj ", value$id)
>> 7
>> writeJobj(con, object)
>> 6
>> writeObject(con, a)
>> 5
>> writeArgs(rc, args)
>> 4
>> invokeJava(isStatic = TRUE, className, methodName, ...)
>> 3
>> callJStatic("org.apache.s
kRHive.init(sc)
> df <- loadDF(hivecontext, "/data/ingest/sparktest1/", "orc")
>
>
>
> --
> Date: Tue, 12 Jan 2016 14:28:58 +0530
> Subject: Re: sparkR ORC support.
> From: sand...@infoworks.io
> To: felixcheun...@hotmail.com
> CC
Hive.init(sc)df <- loadDF(hivecontext,
"/data/ingest/sparktest1/", "orc")
Date: Tue, 12 Jan 2016 14:28:58 +0530
Subject: Re: sparkR ORC support.
From: sand...@infoworks.io
To: felixcheun...@hotmail.com
CC: yblia...@gmail.com; user@spark.apache.org; premsure...@gmail.com;
t;
> Firstly I don't have ORC data to verify but this should work:
>
> df <- loadDF(sqlContext, "data/path", "orc")
>
> Secondly, could you check if sparkR.stop() was called? sparkRHive.init()
> should be called after sparkR.init() - please check if there i
a/path", "orc")
>>
>> Secondly, could you check if sparkR.stop() was called? sparkRHive.init()
>> should be called after sparkR.init() - please check if there is any error
>> message there.
>>
>> _
>> From: Prem Sure
data to verify but this should work:
>>
>> df <- loadDF(sqlContext, "data/path", "orc")
>>
>> Secondly, could you check if sparkR.stop() was called? sparkRHive.init()
>> should be called after sparkR.init() - please check if there is any error
ge there.
>
> _________________
> From: Prem Sure
> Sent: Tuesday, January 5, 2016 8:12 AM
> Subject: Re: sparkR ORC support.
> To: Sandeep Khurana
> Cc: spark users , Deepak Sharma <
> deepakmc...@gmail.com>
>
>
>
> Yes Sandeep, also copy hive-s
is any error message
there.
_
From: Prem Sure
Sent: Tuesday, January 5, 2016 8:12 AM
Subject: Re: sparkR ORC support.
To: Sandeep Khurana
Cc: spark users , Deepak Sharma
Yes Sandeep, also copy hive-site.xml too to spark conf directory.
On Tue, Jan 5, 2016
Yes Sandeep, also copy hive-site.xml too to spark conf directory.
On Tue, Jan 5, 2016 at 10:07 AM, Sandeep Khurana
wrote:
> Also, do I need to setup hive in spark as per the link
> http://stackoverflow.com/questions/26360725/accesing-hive-tables-in-spark
> ?
>
> We might need to copy hdfs-site.
Also, do I need to setup hive in spark as per the link
http://stackoverflow.com/questions/26360725/accesing-hive-tables-in-spark ?
We might need to copy hdfs-site.xml file to spark conf directory ?
On Tue, Jan 5, 2016 at 8:28 PM, Sandeep Khurana
wrote:
> Deepak
>
> Tried this. Getting this erro
Deepak
Tried this. Getting this error now
rror in sql(hivecontext, "FROM CATEGORIES SELECT category_id", "") :
unused argument ("")
On Tue, Jan 5, 2016 at 6:48 PM, Deepak Sharma wrote:
> Hi Sandeep
> can you try this ?
>
> results <- sql(hivecontext, "FROM test SELECT id","")
>
> Thanks
> D
Hi Sandeep
can you try this ?
results <- sql(hivecontext, "FROM test SELECT id","")
Thanks
Deepak
On Tue, Jan 5, 2016 at 5:49 PM, Sandeep Khurana
wrote:
> Thanks Deepak.
>
> I tried this as well. I created a hivecontext with "hivecontext <<-
> sparkRHive.init(sc) " .
>
> When I tried to r
Thanks Deepak.
I tried this as well. I created a hivecontext with "hivecontext <<-
sparkRHive.init(sc) " .
When I tried to read hive table from this ,
results <- sql(hivecontext, "FROM test SELECT id")
I get below error,
Error in callJMethod(sqlContext, "sql", sqlQuery) :
Invalid jobj 2.
Hi Sandeep
I am not sure if ORC can be read directly in R.
But there can be a workaround .First create hive table on top of ORC files
and then access hive table in R.
Thanks
Deepak
On Tue, Jan 5, 2016 at 4:57 PM, Sandeep Khurana
wrote:
> Hello
>
> I need to read an ORC files in hdfs in R using
17 matches
Mail list logo