[
https://issues.apache.org/jira/browse/DRILL-8290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17684215#comment-17684215
]
ASF GitHub Bot commented on DRILL-8290:
---------------------------------------
vvysotskyi commented on code in PR #2636:
URL: https://github.com/apache/drill/pull/2636#discussion_r1096586279
##########
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java:
##########
@@ -645,10 +646,18 @@ public Void visitOp(PhysicalOperator op,
Collection<PhysicalOperator> collection
}
return null;
}
-
}
protected Pair<SqlNode, RelDataType> validateNode(SqlNode sqlNode) throws
ValidationException, RelConversionException, ForemanSetupException {
+ if (context.getOptions().getOption(ExecConstants.FILE_LISTING_LIMIT0_OPT))
{
+ // Check for a LIMIT 0 in the root portion of the query before validation
+ // because validation of the query's FROM clauses will already trigger
+ // the recursive listing files to which FILE_LISTING_LIMIT0_OPT is meant
+ // to apply.
+ boolean rootSelectLimit0 = FindLimit0SqlVisitor.containsLimit0(sqlNode);
+ context.getPlannerSettings().setRootSelectLimit0(rootSelectLimit0);
Review Comment:
Perhaps with the query option would be simpler to pass the value rather than
extend the planner setting interface and pass it through. And schema config
already has access to query options.
> Short cut recursive file listings for LIMIT 0 queries
> -----------------------------------------------------
>
> Key: DRILL-8290
> URL: https://issues.apache.org/jira/browse/DRILL-8290
> Project: Apache Drill
> Issue Type: Improvement
> Components: Query Planning & Optimization
> Affects Versions: 1.20.2
> Reporter: James Turton
> Assignee: James Turton
> Priority: Minor
> Fix For: 1.21.0
>
>
> The existing LIMIT 0 query optimisations do not prevent a query run against
> the top of a deep DFS directory tree from recursively listing FileStatuses
> for everything within it using a pool of worker threads. This Issue proposes
> a new optimisation whereby such queries will recurse into the directory tree
> on a single thread that returns as soon as any single FileStatus has been
> obtained.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
