Hello Apache Drill Support Team, We are currently working with Apache Drill to fetch millions to billions of records. To support this workload, we have configured multiple Drill nodes and are executing queries in distributed mode. However, we are observing that only a single fragment appears to be in use at a time, and the query does not execute as expected. During execution, the Drill UI becomes unresponsive and breaks intermittently. Our setup is running Apache Drill using Docker on a Windows machine. While executing the queries, we encounter the following error in the Drillbit logs:
Exception in thread "I/O dispatcher 6" java.lang.OutOfMemoryError: Java heap space 07:07:32.552 [Curator-ConnectionStateManager-0] ERROR o.apache.curator.utils.ThreadUtils - Unexpected exception in thread: Thread[Curator-ConnectionStateManager-0,5,main] We would appreciate your guidance on how to properly configure Apache Drill for large-scale distributed queries and resolve the memory and execution issues we are facing. Thank you for your support.
