Hi everyone, I ran a SQL join statement on Spark 1.6.1 like this: select * from table1 a join table2 b on a.c1 = b.c1 where a.c2 < 1000;and it took quite a long time because It is a SortMergeJoin and the two tables are big.
In fact, the size of filter result(select * from a where a.c2 < 1000) is very small, and I think a better solution is to use a BroadcastJoin with the filter result, but I know the physical plan is static and it won't be changed. So, can we make the physical plan more adaptive? (In this example, I mean using a BroadcastHashJoin instead of SortMergeJoin automatically. )
