Re: IllegalStateException: Memory was leaked by query - Drill error on wide table, but OK on a narrow but longer table.

2016-03-26 Thread Edmon Begoli
I don't know if it is related, but our /tmp files were filled up. I will try to search, but our machines got hosed up from the problem (queries were running over the last weekend), so I am not sure if I can recover or if I can easily replicate. Will try. On Saturday, March 26, 2016, Abdel Hakim

Re: IllegalStateException: Memory was leaked by query - Drill error on wide table, but OK on a narrow but longer table.

2016-03-26 Thread Abdel Hakim Deneche
Hey Edmon, Can you search the logs for the errorId displayed with this error ? I've seen a similar issue when external sort fails to spill to disk (no space left on disk), it would leak memory and display this error message instead of the original issue (problem spilling to disk). I will open a

IllegalStateException: Memory was leaked by query - Drill error on wide table, but OK on a narrow but longer table.

2016-03-21 Thread Edmon Begoli
We are converting some raw CMS data from csv to parquet using Drill, and using partitioning as we go. Query 1 runs OK on a narrower file: size: 13G Mar 16 18:20 out_revenuej_lds_100_201412.csv (Month 12 file) 13G Mar 16 16:37 out_claimsj_lds_100_2014_q1.csv (Quarter 1 file) q1 has 198 columns;