[
https://issues.apache.org/jira/browse/DRILL-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643581#comment-14643581
]
Abhishek Girish commented on DRILL-3555:
----------------------------------------
As discussed offline, I'll skip running this query. In summary, this issue
might have regressed. But we never had thorough tests for them, so its also
likely to be an existing issue which was only observed recently.
> Changing defaults for planner.memory.max_query_memory_per_node causes queries
> with window function to fail
> ----------------------------------------------------------------------------------------------------------
>
> Key: DRILL-3555
> URL: https://issues.apache.org/jira/browse/DRILL-3555
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Affects Versions: 1.1.0, 1.2.0
> Environment: 4 Nodes. Direct Memory= 48 GB each
> Reporter: Abhishek Girish
> Assignee: Jinfeng Ni
> Priority: Critical
>
> Changing the default value for planner.memory.max_query_memory_per_node from
> 2 GB to anything higher causes queries with window functions to fail.
> Changed system options
> {code:sql}
> > select * from sys.options where status like '%CHANGE%';
> +-------------------------------------------+----------+---------+----------+-------------+-------------+-----------+------------+
> | name | kind | type | status |
> num_val | string_val | bool_val | float_val |
> +-------------------------------------------+----------+---------+----------+-------------+-------------+-----------+------------+
> | planner.enable_decimal_data_type | BOOLEAN | SYSTEM | CHANGED |
> null | null | true | null |
> | planner.memory.max_query_memory_per_node | LONG | SYSTEM | CHANGED |
> 8589934592 | null | null | null |
> +-------------------------------------------+----------+---------+----------+-------------+-------------+-----------+------------+
> 2 rows selected (0.249 seconds)
> {code}
> Query
> {code:sql}
> > SELECT SUM(ss.ss_net_paid_inc_tax) OVER (PARTITION BY ss.ss_store_sk) FROM
> > store_sales ss LIMIT 20;
> java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR:
> DrillRuntimeException: Adding this batch causes the total size to exceed max
> allowed size. Current runningBytes 1073638500, Incoming batchBytes 127875.
> maxBytes 1073741824
> Fragment 1:0
> [Error Id: 9c2ec9cf-21c6-4d5e-b0d6-7cd59e32c49d on abhi1:31010]
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {code}
> Log:
> {code}
> 2015-07-23 18:16:52,292 [2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:frag:2:2] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:2:2: State change requested RUNNING -->
> FINISHED
> 2015-07-23 18:16:52,292 [2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:frag:2:2] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:2:2: State to report: FINISHED
> 2015-07-23 18:17:05,485 [2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:frag:1:0] ERROR
> o.a.d.e.p.i.s.SortRecordBatchBuilder - Adding this batch causes the total
> size to exceed max allowed size. Current runningBytes 1073638500, Incoming
> batchBytes 127875. maxBytes 1073741824
> 2015-07-23 18:17:05,486 [2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:frag:1:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:1:0: State change requested RUNNING -->
> FAILED
> ...
> 2015-07-23 18:17:05,990 [2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:frag:1:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:1:0: State change requested FAILED -->
> FINISHED
> 2015-07-23 18:17:05,999 [2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:frag:1:0] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: DrillRuntimeException:
> Adding this batch causes the total size to exceed max allowed size. Current
> runningBytes 1073638500, Incoming batchBytes 127875. maxBytes 1073741824
> Fragment 1:0
> [Error Id: 9c2ec9cf-21c6-4d5e-b0d6-7cd59e32c49d on abhi1:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> DrillRuntimeException: Adding this batch causes the total size to exceed max
> allowed size. Current runningBytes 1073638500, Incoming batchBytes 127875.
> maxBytes 1073741824
> Fragment 1:0
> [Error Id: 9c2ec9cf-21c6-4d5e-b0d6-7cd59e32c49d on abhi1:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
> ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_65]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
> Caused by: org.apache.drill.common.exceptions.DrillRuntimeException: Adding
> this batch causes the total size to exceed max allowed size. Current
> runningBytes 1073638500, Incoming batchBytes 127875. maxBytes 1073741824
> at
> org.apache.drill.exec.physical.impl.sort.SortRecordBatchBuilder.add(SortRecordBatchBuilder.java:115)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:380)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:105)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> ...
> 2015-07-23 18:17:06,021 [BitServer-3] WARN o.a.d.e.w.b.ControlMessageHandler
> - Dropping request to cancel fragment.
> 2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:1:0 does not exist.
> 2015-07-23 18:17:06,022 [BitServer-3] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:1:4: State change requested RUNNING -->
> CANCELLATION_REQUESTED
> 2015-07-23 18:17:06,023 [BitServer-3] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 2a4e6e2e-8cfa-ed8f-de56-e6c5517b5da6:1:4: State to report:
> CANCELLATION_REQUESTED
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
