I have two questions. 1. Can you upgrade to the latest version of Drill? There have been many updates since 1.21.1 and it is quite possible that the issue you are encountering has been fixed. 2. What kind of files are you querying?
Best, — C > On Dec 19, 2025, at 04:14, Anupam Gupta <[email protected]> > wrote: > > Hello, > I’m following up on the details we shared earlier regarding our Apache Drill > 1.21.1 setup running in Docker. > As mentioned, we provided: > > * > The Docker Compose file > * > The Drill configuration file > * > The DFS storage plugin configuration > * > The query being executed: > > SELECT * > FROM dfs.shared.`<filename>` > > Could you please let us know if you’ve had a chance to review these details, > or if there are any additional logs or configuration settings (e.g., memory > settings, full query profiles, or Drillbit logs) that would help with further > analysis? > We’re happy to provide any additional information needed to move this forward. > > ________________________________ > From: Anupam Gupta <[email protected]> > Sent: Wednesday, December 17, 2025 12:03 PM > To: [email protected] <[email protected]> > Subject: Re: Issue with Distributed Query Execution and OutOfMemoryError in > Apache Drill > > > CAUTION: External Sender. Please do not click on links or open attachments > from senders you do not trust. > > > We are using Apache Drill version 1.21.1. > As requested, we are attaching the Docker Compose file and the Drill > configuration file that we are currently using to run Drill in Docker. > Additionally, below is the DFS storage plugin configuration we are using: > > { > "type": "file", > "connection": "file:///shared", > "workspaces": { > "shared": { > "location": "/shared", > "writable": true, > "defaultInputFormat": null, > "allowAccessOutsideWorkspace": true > }, > "root": { > "location": "/", > "writable": false, > "defaultInputFormat": null, > "allowAccessOutsideWorkspace": true > } > }, > "formats": { > "parquet": { > "type": "parquet" > }, > "json": { > "type": "json", > "extensions": [ > "json" > ] > }, > "csv": { > "type": "text", > "extensions": [ > "csv" > ], > "lineDelimiter": "\n", > "fieldDelimiter": ",", > "quote": "\"", > "escape": "\"", > "comment": "#", > "extractHeader": true > } > }, > "authMode": "SHARED_USER", > "enabled": true > } > > > The query we are executing is: > > SELECT * > FROM dfs.shared.`<filename>` > > > Please let us know if you need any additional details, such as memory > settings or complete query logs, to further analyze the issue. > > ________________________________ > From: Charles Givre <[email protected]> > Sent: Wednesday, December 17, 2025 1:19 AM > To: [email protected] <[email protected]> > Subject: Re: Issue with Distributed Query Execution and OutOfMemoryError in > Apache Drill > > CAUTION: External Sender. Please do not click on links or open attachments > from senders you do not trust. > > > Hi Anupam, > The obvious cause is that you are running out of memory. We won’t be able to > debug this unless you share your configuration and the query. Also, what > version of Drill are you using? > Best, > — C > >> On Dec 16, 2025, at 05:34, Anupam Gupta <[email protected]> >> wrote: >> >> Hello Apache Drill Support Team, >> We are currently working with Apache Drill to fetch millions to billions of >> records. To support this workload, we have configured multiple Drill nodes >> and are executing queries in distributed mode. >> However, we are observing that only a single fragment appears to be in use >> at a time, and the query does not execute as expected. During execution, the >> Drill UI becomes unresponsive and breaks intermittently. >> Our setup is running Apache Drill using Docker on a Windows machine. While >> executing the queries, we encounter the following error in the Drillbit logs: >> >> Exception in thread "I/O dispatcher 6" java.lang.OutOfMemoryError: Java heap >> space >> >> 07:07:32.552 [Curator-ConnectionStateManager-0] ERROR >> o.apache.curator.utils.ThreadUtils - Unexpected exception in thread: >> Thread[Curator-ConnectionStateManager-0,5,main] >> >> >> We would appreciate your guidance on how to properly configure Apache Drill >> for large-scale distributed queries and resolve the memory and execution >> issues we are facing. >> Thank you for your support. >> >
signature.asc
Description: Message signed with OpenPGP
