wangyum commented on code in PR #59027:
URL: https://github.com/apache/spark/pull/59027#discussion_r4242047563
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -1169,6 +1169,22 @@ object SQLConf {
.bytesConf(ByteUnit.BYTE)
.createWithDefaultString("10MB")
+ val BROADCAST_RANGE_JOIN_ENABLED =
buildConf("spark.sql.join.broadcastRangeJoin.enabled")
+ .doc("When true, a join whose condition is a range predicate is planned as
a " +
+ "broadcast range join instead of a broadcast nested loop join. Supported
join " +
+ "types are inner, left outer, right outer, left semi, and left anti. The
" +
+ "preserved side is streamed. Full outer stays a broadcast nested loop
join. " +
+ "Point-in-range and interval overlap (a.lo < b.hi AND b.lo < a.hi)
broadcast " +
+ "an interval index. A single inequality broadcasts a sorted point index.
" +
+ "Only types the index compares with the join predicate are recognized. "
+
+ "A non-deterministic key stays a nested loop join. " +
+ "Overlapping intervals make the interval index larger than the build
table. " +
Review Comment:
Fixed. Replaced the index-growth warning with: `"Deeply overlapping
intervals can increase candidate evaluation cost during probing."` to
accurately describe the resource profile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
