Hi Ashish, > So you are saying it could not even design to join two physical files of 1> > GB each and show the result on 8 GB RAM?
Thanks for sharing your file sizes. You said your files are compressed, so the actual data size might be 5-10x the on-disk size, so 5-10 GB. You mentioned you untarred the files. How large was the result? 10 GB of memory might be sufficient for your ~10 GB of uncompressed data. Again, configuring Drill with 10 GB on an 8 GB system will lead to swapping. Drill asks for memory only as needed, so Drill will start fine and run fine for a while. But, eventually swapping will kick in, which is fine as long as you understand that the result will be slower-than-expected performance. > FYI: If I ran the same query on plain CSV file (i.e running after untarring > and uncompressed files ), it was successful. This is very helpful! It shows that Drill can handle the data itself. Makes me wonder if there is some bug with how Drill handles decompression. Perhaps someone can try to reproduce this problem. In the meantime, looks like a workaround is to go ahead and use the uncompressed files. Thanks, - Paul
