We need to first implement subtract and intersect in Spark SQL natively first (i.e. add physical operator for them rather than using RDD.subtract/intersect).
Then it should be pretty easy to do that, given it is just about injecting the right exchange operators. > On Nov 27, 2015, at 11:19 PM, Justin Uang <justin.u...@gmail.com> wrote: > > Hi, > > I have seen massive gains with the broadcast hint for joins with DataFrames, > and I was wondering if we have thought about allowing the broadcast hint for > the implementation of subtract and intersect. > > Right now, when I try it, it says that there is no plan for the broadcast > hint. > > Justin --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
