Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21109
> Regarding the second point, this is not an ordinary range join, but an
equi-join with a secondary range condition.
"an equi-join with a secondary range condition" is a restriction isn't it?
IIRC the logical rewrite approach can work for range join without equi-join,
which has broader use cases. e.g. turning a cartisian join into hash join/SMJ.
I took a quick look at this patch, it's really bulky, and not safe to be
turned on by default without the spill ability. And more importantly, it can't
work for range join without equi-condition, which I think should be a common
use case.
Can you investigate the logical rewrite approach? I'll attach some related
papers if I find them.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
