Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/7379#issuecomment-122188991
Sorry, I shouldn't use the word `SMJ`.
I mean if we are planning to improve the performance of RangeJoin, probably
we can think of it in a more general way, not just binary
ConjunctivePredicates, but also for arbitrary-nary predicates.
e.g.
`join on a.key < b.key`
`join on a.key < b.key and a.key2 > b.key2 and a.key3>=b.key3` ...
I just have kind of feeling, maybe it will be more helpful/easy if we
specify the same `RangePartitioner` (range key as the sorted key) for table(s),
and then make the Cartesian Product for partitions.
And the binary BroadcastRangeJoin just a specific case, and definitely lots
of code can be shared (like the binary search for the closest tuple).
Sorry if I misunderstood something.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
