Re: Query hangs on planning

2016-09-01 Thread Sudheesh Katkam
That setting is for off-heap memory. The earlier case hit heap memory limit. > On Sep 1, 2016, at 11:36 AM, Zelaine Fong wrote: > > One other thing ... have you tried tuning the planner.memory_limit > parameter? Based on the earlier stack trace, you're hitting a memory

Re: Query hangs on planning

2016-09-01 Thread Zelaine Fong
One other thing ... have you tried tuning the planner.memory_limit parameter? Based on the earlier stack trace, you're hitting a memory limit during query planning. So, tuning this parameter should help that. The default is 256 MB. -- Zelaine On Thu, Sep 1, 2016 at 11:21 AM, rahul challapalli

Re: Query hangs on planning

2016-09-01 Thread rahul challapalli
While planning we use heap memory. 2GB of heap should be sufficient for what you mentioned. This looks like a bug to me. Can you raise a jira for the same? And it would be super helpful if you can also attach the data set used. Rahul On Wed, Aug 31, 2016 at 9:14 AM, Oscar Morante

Re: Query hangs on planning

2016-08-31 Thread Oscar Morante
Sure, This is what I remember: * Failure - embedded mode on my laptop - drill memory: 2Gb/4Gb (heap/direct) - cpu: 4cores (+hyperthreading) - `planner.width.max_per_node=6` * Success - AWS Cluster 2x c3.8xlarge - drill memory: 16Gb/32Gb - cpu: limited by kubernetes to

Re: Query hangs on planning

2016-08-31 Thread Khurram Faraaz
Can you please share the number of cores on the setup where the query hung as compared to the number of cores on the setup where the query went through successfully. And details of memory from the two scenarios. Thanks, Khurram On Wed, Aug 31, 2016 at 4:50 PM, Oscar Morante

Re: Query hangs on planning

2016-08-31 Thread Oscar Morante
For the record, I think this was just bad memory configuration after all. I retested on bigger machines and everything seems to be working fine. On Tue, Aug 09, 2016 at 10:46:33PM +0530, Khurram Faraaz wrote: Oscar, can you please report a JIRA with the required steps to reproduce the OOM

Re: Query hangs on planning

2016-08-09 Thread Khurram Faraaz
Oscar, can you please report a JIRA with the required steps to reproduce the OOM error. That way someone from the Drill team will take a look and investigate. For others interested here is the stack trace. 2016-08-09 16:51:14,280 [285642de-ab37-de6e-a54c-378aaa4ce50e:foreman] ERROR

Query hangs on planning

2016-08-09 Thread Oscar Morante
Hi there, I've been stuck with this for a while and I'm not sure if I'm running into a bug or I'm just doing something very wrong. I have this stripped-down version of my query: https://gist.github.com/spacepluk/9ab1e1a0cfec6f0efb298f023f4c805b The data is just a single file with one record