This is a log with the `upload_date` lines and `usage <> 'Test'` enabled: https://gist.github.com/spacepluk/d7ac11c0de6859e4bd003d2022b3c55e
The client times out around here (~1.5hours): https://gist.github.com/spacepluk/d7ac11c0de6859e4bd003d2022b3c55e#file-drillbit-log-L178
And it still keeps running for a while until it dies (~2.5hours): https://gist.github.com/spacepluk/d7ac11c0de6859e4bd003d2022b3c55e#file-drillbit-log-L178
The memory settings for this test were: DRILL_HEAP="4G" DRILL_MAX_DIRECT_MEMORY="8G"This is on a laptop with 16G and I should probably lower it, but it seems a bit excessive for such a small query. And I think I got the same results on a 2 node cluster with 8/16. I'm gonna try again on the cluster to make sure.
Thanks, Oscar On Tue, Aug 09, 2016 at 04:13:17PM +0530, Khurram Faraaz wrote:
You mentioned "*But if I uncomment the where clause then it runs for a couple of hours until it runs out of memory.*" Can you please share the OutOfMemory details from drillbit.log and the value of DRILL_MAX_DIRECT_MEMORY Can you also try to see what happens if you retain just this line where upload_date = '2016-08-01' in your where clause, can you check if the explain succeeds. Thanks, Khurram On Tue, Aug 9, 2016 at 4:00 PM, Oscar Morante <[email protected]> wrote:Hi there, I've been stuck with this for a while and I'm not sure if I'm running into a bug or I'm just doing something very wrong. I have this stripped-down version of my query: https://gist.github.com/spacepluk/9ab1e1a0cfec6f0efb298f023f4c805b The data is just a single file with one record (1.5K). Without changing anything, explain takes ~1sec on my machine. But if I uncomment the where clause then it runs for a couple of hours until it runs out of memory. Also if I uncomment the where clause *and* take out the join, then it takes around 30s to plan. Any ideas? Thanks!
signature.asc
Description: Digital signature
