We need to first implement subtract and intersect in Spark SQL natively first (i.e. add physical operator for them rather than using RDD.subtract/intersect).
Then it should be pretty easy to do that, given it is just about injecting the right exchange operators. > On Nov 27, 2015, at 11:19 PM, Justin Uang <[email protected]> wrote: > > Hi, > > I have seen massive gains with the broadcast hint for joins with DataFrames, > and I was wondering if we have thought about allowing the broadcast hint for > the implementation of subtract and intersect. > > Right now, when I try it, it says that there is no plan for the broadcast > hint. > > Justin --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
