[ https://issues.apache.org/jira/browse/DRILL-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227693#comment-15227693 ]
ASF GitHub Bot commented on DRILL-3743: --------------------------------------- Github user adeneche commented on a diff in the pull request: https://github.com/apache/drill/pull/460#discussion_r58650141 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java --- @@ -354,17 +335,57 @@ public void success(QueryId queryId, ByteBuf buf) { @Override public void interrupted(final InterruptedException ex) { - logger.warn("Interrupted while waiting for query results from Drillbit", ex); - if (!isTerminal.compareAndSet(false, true)) { + logger.warn("Received multiple responses to run query request."); return; } - closeFuture.removeListener(closeListener); - // Throw an interrupted UserException? - resultsListener.submissionFailed(UserException.systemError(ex).build(logger)); + resultsListener.submissionFailed(UserException.systemError(ex) --- End diff -- We shouldn't use a "system error" when we want to display a proper error message. System errors are meant for unexpected errors that don't have a "nice" error message yet. This may require adding a new "client" user exception type, to make it easier to differentiate errors that happen on the client side. This is may be too much of a change for this pull request, so it's fine if you just create a JIRA to address this change later, as I think there are other places in the client code that could also throw a client error instead of a system error. > query hangs on sqlline once Drillbit on foreman node is killed > -------------------------------------------------------------- > > Key: DRILL-3743 > URL: https://issues.apache.org/jira/browse/DRILL-3743 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow > Affects Versions: 1.2.0 > Environment: 4 node cluster CentOS > Reporter: Khurram Faraaz > Assignee: Sudheesh Katkam > Priority: Critical > Fix For: Future > > > sqlline/query hangs once Drillbit (on Foreman node) is killed. (kill -9 <PID>) > query was issued from the Foreman node. The query returns many records, and > it is a long running query. > Steps to reproduce the problem. > set planner.slice_target=1 > 1. clush -g khurram service mapr-warden stop > 2. clush -g khurram service mapr-warden start > 3. ./sqlline -u "jdbc:drill:schema=dfs.tmp" > 0: jdbc:drill:schema=dfs.tmp> select * from `twoKeyJsn.json` limit 2000000; > 4. Immediately from another console do a jps and kill the Drillbit process > (in this case foreman) while the query is being run on sqlline. You will > notice that sqlline just hangs, we do not see any exceptions or errors being > reported on sqlline prompt or in drillbit.log or drillbit.out > I do see this Exception in sqlline.log on the node from where sqlline was > started > {code} > 2015-09-04 18:45:12,069 [Client-1] INFO o.a.d.e.rpc.user.QueryResultHandler > - User Error Occurred > org.apache.drill.common.exceptions.UserException: CONNECTION ERROR: > Connection /10.10.100.201:53425 <--> /10.10.100.201:31010 (user client) > closed unexpectedly. > [Error Id: ec316cfd-c9a5-4905-98e3-da20cb799ba5 ] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524) > ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.user.QueryResultHandler$SubmissionListener$ChannelClosedListener.operationComplete(QueryResultHandler.java:298) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise$LateListeners.run(DefaultPromise.java:845) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise$LateListenerNotifier.run(DefaultPromise.java:873) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:254) > [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na] > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > 2015-09-04 18:45:12,069 [Client-1] INFO > o.a.d.j.i.DrillResultSetImpl$ResultsListener - [#7] Query failed: > org.apache.drill.common.exceptions.UserException: CONNECTION ERROR: > Connection /10.10.100.201:53425 <--> /10.10.100.201:31010 (user client) > closed unexpectedly. > [Error Id: ec316cfd-c9a5-4905-98e3-da20cb799ba5 ] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524) > ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.user.QueryResultHandler$SubmissionListener$ChannelClosedListener.operationComplete(QueryResultHandler.java:298) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise$LateListeners.run(DefaultPromise.java:845) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise$LateListenerNotifier.run(DefaultPromise.java:873) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:254) > [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na] > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > 2015-09-04 18:45:12,071 [Client-1] ERROR o.a.d.e.rpc.user.QueryResultHandler > - SYSTEM ERROR: ChannelClosedException > [Error Id: c53c477f-f1cf-4458-8620-b1e11ba31701 ] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > ChannelClosedException > [Error Id: c53c477f-f1cf-4458-8620-b1e11ba31701 ] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524) > ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.user.QueryResultHandler$SubmissionListener.failed(QueryResultHandler.java:312) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.setException(CoordinationQueue.java:103) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:788) > [netty-transport-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:689) > [netty-transport-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1114) > [netty-transport-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:705) > [netty-transport-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:32) > [netty-transport-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:980) > [netty-transport-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1032) > [netty-transport-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:965) > [netty-transport-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:254) > [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na] > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > Caused by: org.apache.drill.exec.rpc.ChannelClosedException: null > ... 18 common frames omitted > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)