[jira] [Assigned] (DRILL-4847) Window function query results in OOM Exception.
[ https://issues.apache.org/jira/browse/DRILL-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritesh Maker reassigned DRILL-4847: Assignee: Pritesh Maker (was: Paul Rogers) > Window function query results in OOM Exception. > --- > > Key: DRILL-4847 > URL: https://issues.apache.org/jira/browse/DRILL-4847 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.8.0 > Environment: 4 node cluster CentOS >Reporter: Khurram Faraaz >Assignee: Pritesh Maker >Priority: Critical > Labels: window_function > Attachments: drillbit.log > > > Window function query results in OOM Exception. > Drill version 1.8.0-SNAPSHOT git commit ID: 38ce31ca > MapRBuildVersion 5.1.0.37549.GA > {noformat} > 0: jdbc:drill:schema=dfs.tmp> SELECT clientname, audiencekey, spendprofileid, > postalcd, provincecd, provincename, postalcode_json, country_json, > province_json, town_json, dma_json, msa_json, ROW_NUMBER() OVER (PARTITION BY > spendprofileid ORDER BY (CASE WHEN postalcd IS NULL THEN 9 ELSE 0 END) ASC, > provincecd ASC) as rn FROM `MD593.parquet` limit 3; > Error: RESOURCE ERROR: One or more nodes ran out of memory while executing > the query. > Failure while allocating buffer. > Fragment 0:0 > [Error Id: 2287fe71-f0cb-469a-a563-11580fceb1c5 on centos-01.qa.lab:31010] > (state=,code=0) > {noformat} > Stack trace from drillbit.log > {noformat} > 2016-08-16 07:25:44,590 [284d4006-9f9d-b893-9352-4f54f9b1d52a:foreman] INFO > o.a.drill.exec.work.foreman.Foreman - Query text for query id > 284d4006-9f9d-b893-9352-4f54f9b1d52a: SELECT clientname, audiencekey, > spendprofileid, postalcd, provincecd, provincename, postalcode_json, > country_json, province_json, town_json, dma_json, msa_json, ROW_NUMBER() OVER > (PARTITION BY spendprofileid ORDER BY (CASE WHEN postalcd IS NULL THEN 9 > ELSE 0 END) ASC, provincecd ASC) as rn FROM `MD593.parquet` limit 3 > ... > 2016-08-16 07:25:46,273 [284d4006-9f9d-b893-9352-4f54f9b1d52a:frag:0:0] INFO > o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to > /tmp/drill/spill/284d4006-9f9d-b893-9352-4f54f9b1d52a_majorfragment0_minorfragment0_operator8/2 > 2016-08-16 07:25:46,283 [284d4006-9f9d-b893-9352-4f54f9b1d52a:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - User Error Occurred > org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: One or more > nodes ran out of memory while executing the query. > Failure while allocating buffer. > [Error Id: 2287fe71-f0cb-469a-a563-11580fceb1c5 ] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543) > ~[drill-common-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:242) > [drill-java-exec-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_101] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_101] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101] > Caused by: org.apache.drill.exec.exception.OutOfMemoryException: Failure > while allocating buffer. > at > org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:187) > ~[vector-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapTransferPair.(RepeatedMapVector.java:331) > ~[vector-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapTransferPair.(RepeatedMapVector.java:307) > ~[vector-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.vector.complex.RepeatedMapVector.getTransferPair(RepeatedMapVector.java:161) > ~[vector-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.record.SimpleVectorWrapper.cloneAndTransfer(SimpleVectorWrapper.java:66) > ~[drill-java-exec-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.record.VectorContainer.cloneAndTransfer(VectorContainer.java:204) > ~[drill-java-exec-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.record.VectorContainer.getTransferClone(VectorContainer.java:157) > ~[drill-java-exec-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.mergeAndSpill(ExternalSortBatch.java:569) > ~[drill-java-exec-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at >
[jira] [Assigned] (DRILL-4847) Window function query results in OOM Exception.
[ https://issues.apache.org/jira/browse/DRILL-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zelaine Fong reassigned DRILL-4847: --- Assignee: Paul Rogers Reassigning to [~Paul.Rogers] based on [~khfaraaz]'s finding that with the new external sort, the testcase still fails with in OOM in external sort. > Window function query results in OOM Exception. > --- > > Key: DRILL-4847 > URL: https://issues.apache.org/jira/browse/DRILL-4847 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.8.0 > Environment: 4 node cluster CentOS >Reporter: Khurram Faraaz >Assignee: Paul Rogers >Priority: Critical > Labels: window_function > Attachments: drillbit.log > > > Window function query results in OOM Exception. > Drill version 1.8.0-SNAPSHOT git commit ID: 38ce31ca > MapRBuildVersion 5.1.0.37549.GA > {noformat} > 0: jdbc:drill:schema=dfs.tmp> SELECT clientname, audiencekey, spendprofileid, > postalcd, provincecd, provincename, postalcode_json, country_json, > province_json, town_json, dma_json, msa_json, ROW_NUMBER() OVER (PARTITION BY > spendprofileid ORDER BY (CASE WHEN postalcd IS NULL THEN 9 ELSE 0 END) ASC, > provincecd ASC) as rn FROM `MD593.parquet` limit 3; > Error: RESOURCE ERROR: One or more nodes ran out of memory while executing > the query. > Failure while allocating buffer. > Fragment 0:0 > [Error Id: 2287fe71-f0cb-469a-a563-11580fceb1c5 on centos-01.qa.lab:31010] > (state=,code=0) > {noformat} > Stack trace from drillbit.log > {noformat} > 2016-08-16 07:25:44,590 [284d4006-9f9d-b893-9352-4f54f9b1d52a:foreman] INFO > o.a.drill.exec.work.foreman.Foreman - Query text for query id > 284d4006-9f9d-b893-9352-4f54f9b1d52a: SELECT clientname, audiencekey, > spendprofileid, postalcd, provincecd, provincename, postalcode_json, > country_json, province_json, town_json, dma_json, msa_json, ROW_NUMBER() OVER > (PARTITION BY spendprofileid ORDER BY (CASE WHEN postalcd IS NULL THEN 9 > ELSE 0 END) ASC, provincecd ASC) as rn FROM `MD593.parquet` limit 3 > ... > 2016-08-16 07:25:46,273 [284d4006-9f9d-b893-9352-4f54f9b1d52a:frag:0:0] INFO > o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to > /tmp/drill/spill/284d4006-9f9d-b893-9352-4f54f9b1d52a_majorfragment0_minorfragment0_operator8/2 > 2016-08-16 07:25:46,283 [284d4006-9f9d-b893-9352-4f54f9b1d52a:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - User Error Occurred > org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: One or more > nodes ran out of memory while executing the query. > Failure while allocating buffer. > [Error Id: 2287fe71-f0cb-469a-a563-11580fceb1c5 ] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543) > ~[drill-common-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:242) > [drill-java-exec-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_101] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_101] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101] > Caused by: org.apache.drill.exec.exception.OutOfMemoryException: Failure > while allocating buffer. > at > org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:187) > ~[vector-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapTransferPair.(RepeatedMapVector.java:331) > ~[vector-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapTransferPair.(RepeatedMapVector.java:307) > ~[vector-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.vector.complex.RepeatedMapVector.getTransferPair(RepeatedMapVector.java:161) > ~[vector-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.record.SimpleVectorWrapper.cloneAndTransfer(SimpleVectorWrapper.java:66) > ~[drill-java-exec-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.record.VectorContainer.cloneAndTransfer(VectorContainer.java:204) > ~[drill-java-exec-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at > org.apache.drill.exec.record.VectorContainer.getTransferClone(VectorContainer.java:157) > ~[drill-java-exec-1.8.0-SNAPSHOT.jar:1.8.0-SNAPSHOT] > at >