Solve it by remove lazy identity. 2.HiveContext.sql("cache table feature as "select * from src where ...) which result size is only 100K
---------- Forwarded message ---------- From: Jone Zhang <joyoungzh...@gmail.com> Date: 2017-05-10 19:10 GMT+08:00 Subject: Why spark.sql.autoBroadcastJoinThreshold not available To: "user @spark/'user @spark'/spark users/user@spark" < u...@spark.apache.org> Now i use spark1.6.0 in java I wish the following sql to be executed in BroadcastJoin way *select * from sample join feature* This is my step 1.set spark.sql.autoBroadcastJoinThreshold=100M 2.HiveContext.sql("cache lazy table feature as "select * from src where ...) which result size is only 100K 3.HiveContext.sql("select * from sample join feature") Why the join is SortMergeJoin? Grateful for any idea! Thanks.