[jira] [Commented] (DRILL-6914) Query with RuntimeFilter and SemiJoin fails with IllegalStateException: Memory was leaked by query
[ https://issues.apache.org/jira/browse/DRILL-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738957#comment-16738957 ] Boaz Ben-Zvi commented on DRILL-6914: - This memory leak can be reproduced on SF1 (on the Mac) by forcing the Hash-Join to *spill*; e.g., by setting this _internal_ option ("spill if the number of batches in memory gets to 1000 "): {code:java} alter system set `exec.hashjoin.max_batches_in_memory` = 1000; {code} (Also removed the irrelevant 'distinct' and the 'cast' parts from the repro query). However could not reproduce by spilling on SF0, or by spilling with a regular (not semi) Hash-Join. Also tried the fix from PR#1600 (DRILL-6947), but it did not cure this memory leak. Maybe [~weijie] has some ideas about the cause for this leak. Looking at the Semi-Join code changes (PR#1522), none seems to me in any conflict with the runtime filter (maybe [~weijie] has a better idea). > Query with RuntimeFilter and SemiJoin fails with IllegalStateException: > Memory was leaked by query > -- > > Key: DRILL-6914 > URL: https://issues.apache.org/jira/browse/DRILL-6914 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.15.0 >Reporter: Abhishek Ravi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.16.0 > > Attachments: 23cc1af3-0e8e-b2c9-a889-a96504988d6c.sys.drill, > 23cc1b7c-5b5c-d123-5e72-6d7d2719df39.sys.drill > > > Following query fails on TPC-H SF 100 dataset when > exec.hashjoin.enable.runtime_filter = true AND planner.enable_semijoin = true. > Note that the query does not fail if any one of them or both are disabled. > {code:sql} > set `exec.hashjoin.enable.runtime_filter` = true; > set `exec.hashjoin.runtime_filter.max.waiting.time` = 1; > set `planner.enable_broadcast_join` = false; > set `planner.enable_semijoin` = true; > select > count(*) as row_count > from > lineitem l1 > where > l1.l_shipdate IN ( > select > distinct(cast(l2.l_shipdate as date)) > from > lineitem l2); > reset `exec.hashjoin.enable.runtime_filter`; > reset `exec.hashjoin.runtime_filter.max.waiting.time`; > reset `planner.enable_broadcast_join`; > reset `planner.enable_semijoin`; > {code} > > {noformat} > Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. > Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > (state=,code=0) > java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Memory was leaked > by query. Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:536) > at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:640) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at > org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:151) > at sqlline.BufferedRows.(BufferedRows.java:37) > at sqlline.SqlLine.print(SqlLine.java:1716) > at sqlline.Commands.execute(Commands.java:949) > at sqlline.Commands.sql(Commands.java:882) > at sqlline.SqlLine.dispatch(SqlLine.java:725) > at sqlline.SqlLine.runCommands(SqlLine.java:1779) > at sqlline.Commands.run(Commands.java:1485) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38) > at sqlline.SqlLine.dispatch(SqlLine.java:722) > at sqlline.SqlLine.initArgs(SqlLine.java:458) > at sqlline.SqlLine.begin(SqlLine.java:514) > at sqlline.SqlLine.start(SqlLine.java:264) > at sqlline.SqlLine.main(SqlLine.java:195) > Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM > ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: > (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123) > at org.apac
[jira] [Commented] (DRILL-6914) Query with RuntimeFilter and SemiJoin fails with IllegalStateException: Memory was leaked by query
[ https://issues.apache.org/jira/browse/DRILL-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16736508#comment-16736508 ] Abhishek Ravi commented on DRILL-6914: -- This attachment refers to the query profile for a scenario where semijoin is *disabled* and runtime filter is *enabled*. The query succeeds.[^23cc1af3-0e8e-b2c9-a889-a96504988d6c.sys.drill] > Query with RuntimeFilter and SemiJoin fails with IllegalStateException: > Memory was leaked by query > -- > > Key: DRILL-6914 > URL: https://issues.apache.org/jira/browse/DRILL-6914 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.15.0 >Reporter: Abhishek Ravi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.16.0 > > Attachments: 23cc1af3-0e8e-b2c9-a889-a96504988d6c.sys.drill, > 23cc1b7c-5b5c-d123-5e72-6d7d2719df39.sys.drill > > > Following query fails on TPC-H SF 100 dataset when > exec.hashjoin.enable.runtime_filter = true AND planner.enable_semijoin = true. > Note that the query does not fail if any one of them or both are disabled. > {code:sql} > set `exec.hashjoin.enable.runtime_filter` = true; > set `exec.hashjoin.runtime_filter.max.waiting.time` = 1; > set `planner.enable_broadcast_join` = false; > set `planner.enable_semijoin` = true; > select > count(*) as row_count > from > lineitem l1 > where > l1.l_shipdate IN ( > select > distinct(cast(l2.l_shipdate as date)) > from > lineitem l2); > reset `exec.hashjoin.enable.runtime_filter`; > reset `exec.hashjoin.runtime_filter.max.waiting.time`; > reset `planner.enable_broadcast_join`; > reset `planner.enable_semijoin`; > {code} > > {noformat} > Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. > Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > (state=,code=0) > java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Memory was leaked > by query. Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:536) > at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:640) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at > org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:151) > at sqlline.BufferedRows.(BufferedRows.java:37) > at sqlline.SqlLine.print(SqlLine.java:1716) > at sqlline.Commands.execute(Commands.java:949) > at sqlline.Commands.sql(Commands.java:882) > at sqlline.SqlLine.dispatch(SqlLine.java:725) > at sqlline.SqlLine.runCommands(SqlLine.java:1779) > at sqlline.Commands.run(Commands.java:1485) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38) > at sqlline.SqlLine.dispatch(SqlLine.java:722) > at sqlline.SqlLine.initArgs(SqlLine.java:458) > at sqlline.SqlLine.begin(SqlLine.java:514) > at sqlline.SqlLine.start(SqlLine.java:264) > at sqlline.SqlLine.main(SqlLine.java:195) > Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM > ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: > (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelR
[jira] [Commented] (DRILL-6914) Query with RuntimeFilter and SemiJoin fails with IllegalStateException: Memory was leaked by query
[ https://issues.apache.org/jira/browse/DRILL-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16736502#comment-16736502 ] Abhishek Ravi commented on DRILL-6914: -- [~ben-zvi] - I have attached the query profile for the failure. > Query with RuntimeFilter and SemiJoin fails with IllegalStateException: > Memory was leaked by query > -- > > Key: DRILL-6914 > URL: https://issues.apache.org/jira/browse/DRILL-6914 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.15.0 >Reporter: Abhishek Ravi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.16.0 > > Attachments: 23cc1b7c-5b5c-d123-5e72-6d7d2719df39.sys.drill > > > Following query fails on TPC-H SF 100 dataset when > exec.hashjoin.enable.runtime_filter = true AND planner.enable_semijoin = true. > Note that the query does not fail if any one of them or both are disabled. > {code:sql} > set `exec.hashjoin.enable.runtime_filter` = true; > set `exec.hashjoin.runtime_filter.max.waiting.time` = 1; > set `planner.enable_broadcast_join` = false; > set `planner.enable_semijoin` = true; > select > count(*) as row_count > from > lineitem l1 > where > l1.l_shipdate IN ( > select > distinct(cast(l2.l_shipdate as date)) > from > lineitem l2); > reset `exec.hashjoin.enable.runtime_filter`; > reset `exec.hashjoin.runtime_filter.max.waiting.time`; > reset `planner.enable_broadcast_join`; > reset `planner.enable_semijoin`; > {code} > > {noformat} > Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. > Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > (state=,code=0) > java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Memory was leaked > by query. Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:536) > at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:640) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at > org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:151) > at sqlline.BufferedRows.(BufferedRows.java:37) > at sqlline.SqlLine.print(SqlLine.java:1716) > at sqlline.Commands.execute(Commands.java:949) > at sqlline.Commands.sql(Commands.java:882) > at sqlline.SqlLine.dispatch(SqlLine.java:725) > at sqlline.SqlLine.runCommands(SqlLine.java:1779) > at sqlline.Commands.run(Commands.java:1485) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38) > at sqlline.SqlLine.dispatch(SqlLine.java:722) > at sqlline.SqlLine.initArgs(SqlLine.java:458) > at sqlline.SqlLine.begin(SqlLine.java:514) > at sqlline.SqlLine.start(SqlLine.java:264) > at sqlline.SqlLine.main(SqlLine.java:195) > Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM > ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: > (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) > at > io.netty.handler
[jira] [Commented] (DRILL-6914) Query with RuntimeFilter and SemiJoin fails with IllegalStateException: Memory was leaked by query
[ https://issues.apache.org/jira/browse/DRILL-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733544#comment-16733544 ] Abhishek Ravi commented on DRILL-6914: -- [~ben-zvi] - When I refer to *either option true*, I mean *specifically setting the other to false*. \{{runtime_filter}} does have some pending issues but this specific issue is seen when both semijoin and runtime_filter features are enabled. Working on something else right now, will re-run and attach the profile for failed query soon. > Query with RuntimeFilter and SemiJoin fails with IllegalStateException: > Memory was leaked by query > -- > > Key: DRILL-6914 > URL: https://issues.apache.org/jira/browse/DRILL-6914 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.15.0 >Reporter: Abhishek Ravi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.16.0 > > > Following query fails on TPC-H SF 100 dataset when > exec.hashjoin.enable.runtime_filter = true AND planner.enable_semijoin = true. > Note that the query does not fail if any one of them or both are disabled. > {code:sql} > set `exec.hashjoin.enable.runtime_filter` = true; > set `exec.hashjoin.runtime_filter.max.waiting.time` = 1; > set `planner.enable_broadcast_join` = false; > set `planner.enable_semijoin` = true; > select > count(*) as row_count > from > lineitem l1 > where > l1.l_shipdate IN ( > select > distinct(cast(l2.l_shipdate as date)) > from > lineitem l2); > reset `exec.hashjoin.enable.runtime_filter`; > reset `exec.hashjoin.runtime_filter.max.waiting.time`; > reset `planner.enable_broadcast_join`; > reset `planner.enable_semijoin`; > {code} > > {noformat} > Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. > Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > (state=,code=0) > java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Memory was leaked > by query. Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:536) > at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:640) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at > org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:151) > at sqlline.BufferedRows.(BufferedRows.java:37) > at sqlline.SqlLine.print(SqlLine.java:1716) > at sqlline.Commands.execute(Commands.java:949) > at sqlline.Commands.sql(Commands.java:882) > at sqlline.SqlLine.dispatch(SqlLine.java:725) > at sqlline.SqlLine.runCommands(SqlLine.java:1779) > at sqlline.Commands.run(Commands.java:1485) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38) > at sqlline.SqlLine.dispatch(SqlLine.java:722) > at sqlline.SqlLine.initArgs(SqlLine.java:458) > at sqlline.SqlLine.begin(SqlLine.java:514) > at sqlline.SqlLine.start(SqlLine.java:264) > at sqlline.SqlLine.main(SqlLine.java:195) > Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM > ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: > (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannel
[jira] [Commented] (DRILL-6914) Query with RuntimeFilter and SemiJoin fails with IllegalStateException: Memory was leaked by query
[ https://issues.apache.org/jira/browse/DRILL-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733484#comment-16733484 ] Boaz Ben-Zvi commented on DRILL-6914: - The `enable_semijoin` option is true by default, so setting it TRUE does not change anything; it would be useful to see if setting it to FALSE makes any difference. The `runtime_filter` option is part of the "Bloom filter" feature, which AFAIK still has some issues, hence is not enabled by default. And [~aravi5] - please attach the profile of the failed query so we could see the physical plan used, etc. > Query with RuntimeFilter and SemiJoin fails with IllegalStateException: > Memory was leaked by query > -- > > Key: DRILL-6914 > URL: https://issues.apache.org/jira/browse/DRILL-6914 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.15.0 >Reporter: Abhishek Ravi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.16.0 > > > Following query fails on TPC-H SF 100 dataset when > exec.hashjoin.enable.runtime_filter = true AND planner.enable_semijoin = true. > Note that the query does not fail if any one of them or both are disabled. > {code:sql} > set `exec.hashjoin.enable.runtime_filter` = true; > set `exec.hashjoin.runtime_filter.max.waiting.time` = 1; > set `planner.enable_broadcast_join` = false; > set `planner.enable_semijoin` = true; > select > count(*) as row_count > from > lineitem l1 > where > l1.l_shipdate IN ( > select > distinct(cast(l2.l_shipdate as date)) > from > lineitem l2); > reset `exec.hashjoin.enable.runtime_filter`; > reset `exec.hashjoin.runtime_filter.max.waiting.time`; > reset `planner.enable_broadcast_join`; > reset `planner.enable_semijoin`; > {code} > > {noformat} > Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. > Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > (state=,code=0) > java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Memory was leaked > by query. Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:536) > at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:640) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at > org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:151) > at sqlline.BufferedRows.(BufferedRows.java:37) > at sqlline.SqlLine.print(SqlLine.java:1716) > at sqlline.Commands.execute(Commands.java:949) > at sqlline.Commands.sql(Commands.java:882) > at sqlline.SqlLine.dispatch(SqlLine.java:725) > at sqlline.SqlLine.runCommands(SqlLine.java:1779) > at sqlline.Commands.run(Commands.java:1485) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38) > at sqlline.SqlLine.dispatch(SqlLine.java:722) > at sqlline.SqlLine.initArgs(SqlLine.java:458) > at sqlline.SqlLine.begin(SqlLine.java:514) > at sqlline.SqlLine.start(SqlLine.java:264) > at sqlline.SqlLine.main(SqlLine.java:195) > Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM > ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: > (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChann
[jira] [Commented] (DRILL-6914) Query with RuntimeFilter and SemiJoin fails with IllegalStateException: Memory was leaked by query
[ https://issues.apache.org/jira/browse/DRILL-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733347#comment-16733347 ] Abhishek Ravi commented on DRILL-6914: -- Yes, as mentioned earlier, the issue is seen with SF 100 and ONLY when both \{{planner.enable_semijoin}} and \{{exec.hashjoin.enable.runtime_filter}} options are set to \{{true}}. The issue is not seen if either one is disabled. > Query with RuntimeFilter and SemiJoin fails with IllegalStateException: > Memory was leaked by query > -- > > Key: DRILL-6914 > URL: https://issues.apache.org/jira/browse/DRILL-6914 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.15.0 >Reporter: Abhishek Ravi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.16.0 > > > Following query fails on TPC-H SF 100 dataset when > exec.hashjoin.enable.runtime_filter = true AND planner.enable_semijoin = true. > Note that the query does not fail if any one of them or both are disabled. > {code:sql} > set `exec.hashjoin.enable.runtime_filter` = true; > set `exec.hashjoin.runtime_filter.max.waiting.time` = 1; > set `planner.enable_broadcast_join` = false; > set `planner.enable_semijoin` = true; > select > count(*) as row_count > from > lineitem l1 > where > l1.l_shipdate IN ( > select > distinct(cast(l2.l_shipdate as date)) > from > lineitem l2); > reset `exec.hashjoin.enable.runtime_filter`; > reset `exec.hashjoin.runtime_filter.max.waiting.time`; > reset `planner.enable_broadcast_join`; > reset `planner.enable_semijoin`; > {code} > > {noformat} > Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. > Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > (state=,code=0) > java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Memory was leaked > by query. Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:536) > at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:640) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at > org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:151) > at sqlline.BufferedRows.(BufferedRows.java:37) > at sqlline.SqlLine.print(SqlLine.java:1716) > at sqlline.Commands.execute(Commands.java:949) > at sqlline.Commands.sql(Commands.java:882) > at sqlline.SqlLine.dispatch(SqlLine.java:725) > at sqlline.SqlLine.runCommands(SqlLine.java:1779) > at sqlline.Commands.run(Commands.java:1485) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38) > at sqlline.SqlLine.dispatch(SqlLine.java:722) > at sqlline.SqlLine.initArgs(SqlLine.java:458) > at sqlline.SqlLine.begin(SqlLine.java:514) > at sqlline.SqlLine.start(SqlLine.java:264) > at sqlline.SqlLine.main(SqlLine.java:195) > Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM > ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: > (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) > at > io.netty.channel.AbstractChannel
[jira] [Commented] (DRILL-6914) Query with RuntimeFilter and SemiJoin fails with IllegalStateException: Memory was leaked by query
[ https://issues.apache.org/jira/browse/DRILL-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732543#comment-16732543 ] Boaz Ben-Zvi commented on DRILL-6914: - The error does not show on SF1. `planner.enable_semijoin` is true by default. [~aravi5] - can you retest with this option set to FALSE ? Can the profile of the failed query be attached to this Jira ? > Query with RuntimeFilter and SemiJoin fails with IllegalStateException: > Memory was leaked by query > -- > > Key: DRILL-6914 > URL: https://issues.apache.org/jira/browse/DRILL-6914 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.15.0 >Reporter: Abhishek Ravi >Assignee: Boaz Ben-Zvi >Priority: Major > Fix For: 1.16.0 > > > Following query fails on TPC-H SF 100 dataset when > exec.hashjoin.enable.runtime_filter = true AND planner.enable_semijoin = true. > Note that the query does not fail if any one of them or both are disabled. > {code:sql} > set `exec.hashjoin.enable.runtime_filter` = true; > set `exec.hashjoin.runtime_filter.max.waiting.time` = 1; > set `planner.enable_broadcast_join` = false; > set `planner.enable_semijoin` = true; > select > count(*) as row_count > from > lineitem l1 > where > l1.l_shipdate IN ( > select > distinct(cast(l2.l_shipdate as date)) > from > lineitem l2); > reset `exec.hashjoin.enable.runtime_filter`; > reset `exec.hashjoin.runtime_filter.max.waiting.time`; > reset `planner.enable_broadcast_join`; > reset `planner.enable_semijoin`; > {code} > > {noformat} > Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. > Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > (state=,code=0) > java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Memory was leaked > by query. Memory leaked: (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:536) > at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:640) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at > org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:151) > at sqlline.BufferedRows.(BufferedRows.java:37) > at sqlline.SqlLine.print(SqlLine.java:1716) > at sqlline.Commands.execute(Commands.java:949) > at sqlline.Commands.sql(Commands.java:882) > at sqlline.SqlLine.dispatch(SqlLine.java:725) > at sqlline.SqlLine.runCommands(SqlLine.java:1779) > at sqlline.Commands.run(Commands.java:1485) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38) > at sqlline.SqlLine.dispatch(SqlLine.java:722) > at sqlline.SqlLine.initArgs(SqlLine.java:458) > at sqlline.SqlLine.begin(SqlLine.java:514) > at sqlline.SqlLine.start(SqlLine.java:264) > at sqlline.SqlLine.main(SqlLine.java:195) > Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM > ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: > (134217728) > Allocator(frag:1:0) 800/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannel