andygrove opened a new pull request, #2493: URL: https://github.com/apache/datafusion-ballista/pull/2493
# Which issue does this PR close? No separate issue. Split out of #2416 at review request. # Rationale for this change `BallistaQueryPlanner` answers plans that only scan `information_schema` on the client instead of distributing them. The `LocalRun` visitor that decides this uses `LogicalPlan::visit`, which does not descend into subquery expressions. So in ```sql SELECT table_name FROM information_schema.tables WHERE table_name IN (SELECT name FROM big) ``` the only scan it sees is `information_schema.tables`, and `big` is scanned locally. On the optimized plans `BallistaQueryPlanner` receives, most subqueries have already been decorrelated into joins, which mostly hides this. Callers that check an unoptimized plan see it directly. # What changes are included in this PR? - Use `visit_with_subqueries` so scans inside subqueries are counted. - Expose the check as `pub fn scans_only_local_tables(&LogicalPlan) -> bool`, which the Flight SQL frontend in #2416 also needs. - A test with the query above, which fails without the fix. # Are there any user-facing changes? No behaviour change for plans without subqueries. Adds one public function to `ballista-core`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
