[ 
https://issues.apache.org/jira/browse/DRILL-8283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584701#comment-17584701
 ] 

ASF GitHub Bot commented on DRILL-8283:
---------------------------------------

jnturton commented on code in PR #2632:
URL: https://github.com/apache/drill/pull/2632#discussion_r954644543


##########
exec/java-exec/src/main/resources/drill-module.conf:
##########
@@ -115,7 +115,8 @@ drill.exec: {
       text: {
         buffer.size: 262144,
         batch.size: 4000
-      }
+      },
+      recursive_listing_max_size: 10000

Review Comment:
   A limit of 0 (or less) now means no limit and the default, for this PR, is 
now 0. 





> Add a configurable recursive file listing size limit
> ----------------------------------------------------
>
>                 Key: DRILL-8283
>                 URL: https://issues.apache.org/jira/browse/DRILL-8283
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - Other
>    Affects Versions: 1.20.2
>            Reporter: James Turton
>            Assignee: James Turton
>            Priority: Minor
>             Fix For: 1.20.3
>
>
> Currently a malicious or merely unwitting user can crash their Drill foreman 
> by sending
> {code:java}
> select * from dfs.huge_workspace limit 10
> {code}
> causing the query planner to recurse over every file in huge_workspace and 
> culminating in
> {code:java}
> 2022-08-09 15:13:22,251 [1d0da29f-e50c-fd51-43d9-8a5086d52c4e:foreman] ERROR 
> o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred, 
> exiting. Information message: Unable to handle out of memory condition in 
> Foreman.java.lang.OutOfMemoryError: null {code}
> if there are enough files in huge_workspace. A SHOW FILES command can produce 
> the same effect. This issue proposes a new BOOT option named 
> drill.exec.storage.file.recursive_listing_max_size with a default value of, 
> say 10 000. If a file listing task exceeds this limit then the initiating 
> operation is terminated with a UserException preventing runaway resource 
> usage.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to