Can you also share the stack trace from drillbit.log for the below error RESOURCE ERROR: External Sort encountered an error while spilling to disk
And what is the version of Drill that you are running ? Please share the table definition and the number of rows in that table, which is being queried. Thanks, Khurram On Tue, Mar 12, 2019 at 12:17 PM Boaz Ben-Zvi <[email protected]> wrote: > Hi Giovanni, > > The error given by the External-Sort indicates a problem while > spilling the excess memory into disk. > > When you enlarged the memory (from the default 2GB) to 8GB the LAG query > may succeeded without spilling, hence circumvented the issue. > > Yes, you can keep enlarging the memory and run w/o spilling, but better > check and fix the root issue. > > How is your spilling configured - check the filesystem - > "drill.exec.spill.fs" and the directories - "drill.exec.spill.directories" > > The default is the local filesystem, and into /tmp . It is possible > that very little disk space is available for /tmp . > > Thanks, > > -- Boaz > > On 3/12/19 6:03 AM, Giovanni Conte wrote: > > Hello, > > I am doing a LAG and a SUM query over a PCAP dataset of 300Mb. > > I get this error: > > RESOURCE ERROR: External Sort encountered an error while spilling to disk > > > > Then I changetd > > planner.memory.max_query_memory_per_node ---> 8589934592 > > and with this I am able to perform the LAG but not the SUM. > > > > I have no problem of memory since I am working on a server with 72 cores > > and 256 GB of RAM. > > Which other parameter should I change to avoid the RESOURCE ERROR? > > Can I go over 8589934592? > > Thank you very much, > > > > Giovanni > > >
