RE: SparkR read.df Option type doesn't match

2015-11-27 Thread Felix Cheung
Yes - please see the code example on the SparkR API doc: http://spark.apache.org/docs/latest/api/R/read.df.html Suggestion or contribution to improve the doc is welcome! > Date: Thu, 26 Nov 2015 15:08:31 -0700 > From: s...@phemi.com > To: dev@spark.apache.org > Subject: Re: SparkR read.df

RE: SparkR read.df Option type doesn't match

2015-11-27 Thread liushiqi9
There is a bug at this page in the examples I have file it in the JIRA It's SPARK-12019. I don't know how to change the page. But I think an example that shows how to write options would be great. Like sc <- sparkR.init(master="yarn-client",appName= "SparkR", sparkHome = "/home/spark",

Re: Subtract implementation using broadcast

2015-11-27 Thread Reynold Xin
We need to first implement subtract and intersect in Spark SQL natively first (i.e. add physical operator for them rather than using RDD.subtract/intersect). Then it should be pretty easy to do that, given it is just about injecting the right exchange operators. > On Nov 27, 2015, at 11:19

Problem in running MLlib SVM

2015-11-27 Thread Tarek Elgamal
Hi, I am trying to run the straightforward example of SVm but I am getting low accuracy (around 50%) when I predict using the same data I used for training. I am probably doing the prediction in a wrong way. My code is below. I would appreciate any help. import java.util.List; import

Subtract implementation using broadcast

2015-11-27 Thread Justin Uang
Hi, I have seen massive gains with the broadcast hint for joins with DataFrames, and I was wondering if we have thought about allowing the broadcast hint for the implementation of subtract and intersect. Right now, when I try it, it says that there is no plan for the broadcast hint. Justin