jnturton opened a new pull request, #2632: URL: https://github.com/apache/drill/pull/2632
# [DRILL-8283](https://issues.apache.org/jira/browse/DRILL-8283): Add a configurable recursive file listing size limit ## Description Currently a malicious or merely unwitting user can crash their Drill foreman by sending ``` select * from dfs.huge_workspace limit 10 ``` causing the query planner to recurse over every file in huge_workspace and culminating in ``` 2022-08-09 15:13:22,251 [1d0da29f-e50c-fd51-43d9-8a5086d52c4e:foreman] ERROR o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in Foreman.java.lang.OutOfMemoryError: null ``` if there are enough files in huge_workspace. A SHOW FILES command can produce the same effect. This issue proposes a new BOOT option named drill.exec.storage.file.recursive_listing_max_size with a default value of, say 10 000. If a file listing task exceeds this limit then the initiating operation is terminated with a UserException preventing runaway resource usage. ## Documentation New entry on https://drill.apache.org/docs/start-up-options/ ## Testing FileSystemUtilTest#testRecursiveListingMaxSize -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@drill.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
