Neuw84 commented on issue #6133: URL: https://github.com/apache/datafusion-comet/issues/6133#issuecomment-5798773579
Update: the result is intermittent, not deterministic. Two further full runs today of the same two configurations that returned 0 rows this morning (Comet 1.0.0 end to end, and `CometNativeScan` under the third-party operators) both returned the correct 12,185 rows for q64 at SF1000, same image, same data, 300 shuffle partitions; a run yesterday of the end-to-end configuration was also correct. So the loss is run-dependent. q64's affected join reads `store_sales` through a dynamic-partition-pruning filter on `ss_sold_date_sk`; in the same environment we also see, on the third-party-operator configuration, q5 fail with `SubqueryAdaptiveBroadcastExec does not support the execute() code path` — a DPP subquery evaluated before AQE rewrote it. A timing-dependent evaluation of the scan's DPP `InSubqueryExec` (before the rewrite → exception; before the broadcast is ready → an empty set → every partition pruned → 0 rows) would explain both; we are capturing the caller stack of t he q5 evaluation now and will add it here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
