Re: RESOURCE ERROR: External Sort encountered an error while spilling to disk

2019-03-12 Thread Khurram Faraaz
Can you also share the stack trace from drillbit.log for the below error RESOURCE ERROR: External Sort encountered an error while spilling to disk And what is the version of Drill that you are running ? Please share the table definition and the number of rows in that table, which is being

Json Complex array issue only on HDF, local storage is fine

2019-03-12 Thread Robert Vantol
This is our Json, as you see the imp node is an array of other json: { "id":"21D66BC4F2BA7E24", "imp":[ { "id":"1", "banner":{ "w":160, "h":600, "pos":1, "topframe":1

Re: RESOURCE ERROR: External Sort encountered an error while spilling to disk

2019-03-12 Thread Boaz Ben-Zvi
 Hi Giovanni,     The error given by the External-Sort indicates a problem while spilling the excess memory into disk. When you enlarged the memory (from the default 2GB) to 8GB the LAG query may succeeded without spilling, hence circumvented the issue. Yes, you can keep enlarging the

Re: Update planner/configuration keys

2019-03-12 Thread Kunal Khatua
That is correct... you'll need to run these on the command prompt for the very first time. Like profiles, I believe, the ALTER SYSTEM commands also have a permanent effect. I'm not sure if there is a way for a user to provide a pre-configured set of parameters to use during startup. You could see

RESOURCE ERROR: External Sort encountered an error while spilling to disk

2019-03-12 Thread Giovanni Conte
Hello, I am doing a LAG and a SUM query over a PCAP dataset of 300Mb. I get this error: RESOURCE ERROR: External Sort encountered an error while spilling to disk Then I changetd planner.memory.max_query_memory_per_node ---> 8589934592 and with this I am able to perform the LAG but not the SUM. I

Re: Update planner/configuration keys

2019-03-12 Thread PRAVEEN DEVERACHETTY
Hi Kunal, But where should i keep those statements? is there any drill startup script that i can run these alter staements? i think these scripts only on the drill command prompt right? Thanks, Praveen On Tue, Mar 12, 2019 at 1:02 PM Kunal Khatua wrote: > Executing an "alter system set

Re: Drill 1.15.0 fails with error while quering Parquet 2.0 file

2019-03-12 Thread Kunal Khatua
Hi Denis You seem to be trying to read a Parquet 2.0 format file with a Parquet 1.10 reader that comes with Drill. Is there a specific reason you are using version 2.0 ? ~ Kunal On 3/11/2019 10:13:39 AM, Denis Dudinski wrote: Hello, I have a parquet 2.0 file which contains serialised avro

Re: Update planner/configuration keys

2019-03-12 Thread Kunal Khatua
Executing an "alter system set param=value" usually persists the value. Not sure if that works for an embedded mode.  Could you try and let us know if that works? On 3/11/2019 11:19:13 AM, PRAVEEN DEVERACHETTY wrote: I am using apache drill on windows platform. My requirement is to udpate the