dwsmith1983 commented on issue #6442:
URL:
https://github.com/apache/datafusion-comet/issues/6442#issuecomment-6756551734
Another shape that hits this, on `main` (acddec463), Spark 4.1: a broadcast
hash join whose condition uses both sides and holds an expression Comet
declines.
```sql
-- t(i int, j int), u(j int, k int)
SET spark.comet.expression.Length.enabled=false;
SELECT /*+ BROADCAST(u) */ t.i FROM t JOIN u
ON t.j = u.j AND length(concat(cast(t.i AS STRING), cast(u.k AS STRING)))
= 2;
```
With AQE off the join records "Expression support is disabled. Set
spark.comet.expression.Length.enabled=true to enable it." With AQE on, the
final `BroadcastHashJoin` over the `BroadcastQueryStage` has no reason. A
condition on one side alone is pushed into a filter, which keeps its reason, so
it does not show the gap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
