Hi Jason,
Thanks for your help. I have set this parameter to 0 on every drillbit and it
works like a charm now.
Regarding your questions, there was no particular query that triggered this
issue. Every query longer than 30 seconds was impacted.
Please find below the log messages I received :
2016-03-24 14:18:31,368 [290c16d8-4244-d664-4562-5b156f3e6fff:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query id
290c16d8-4244-d664-4562-5b156f3e6fff: select count(columns[1]) from
hdfs.lemo.mails
2016-03-24 14:18:31,417 [290c16d8-4244-d664-4562-5b156f3e6fff:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Time: 2ms total, 2.056748ms avg, 2ms max.
2016-03-24 14:18:31,417 [290c16d8-4244-d664-4562-5b156f3e6fff:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Earliest start: 1.391000 μs, Latest start: 1.391000 μs, Average
start: 1.391000 μs .
2016-03-24 14:18:31,466 [290c16d8-4244-d664-4562-5b156f3e6fff:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 290c16d8-4244-d664-4562-5b156f3e6fff:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2016-03-24 14:18:31,466 [290c16d8-4244-d664-4562-5b156f3e6fff:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 290c16d8-4244-d664-4562-5b156f3e6fff:0:0:
State to report: RUNNING
2016-03-24 14:19:01,570 [UserServer-1] INFO o.a.drill.exec.rpc.user.UserServer
- RPC connection /39.6.64.20:31010 <--> /39.6.64.22:53976 (user client) timed
out. Timeout was set to 30 seconds. Closing connection.
2016-03-24 14:19:01,579 [CONTROL-rpc-event-queue] INFO
o.a.d.e.w.fragment.FragmentExecutor - 290c16d8-4244-d664-4562-5b156f3e6fff:0:0:
State change requested RUNNING --> CANCELLATION_REQUESTED
2016-03-24 14:19:01,580 [CONTROL-rpc-event-queue] INFO
o.a.d.e.w.f.FragmentStatusReporter - 290c16d8-4244-d664-4562-5b156f3e6fff:0:0:
State to report: CANCELLATION_REQUESTED
2016-03-24 14:19:01,591 [290c16d8-4244-d664-4562-5b156f3e6fff:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 290c16d8-4244-d664-4562-5b156f3e6fff:0:0:
State change requested CANCELLATION_REQUESTED --> FINISHED
2016-03-24 14:19:01,591 [290c16d8-4244-d664-4562-5b156f3e6fff:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 290c16d8-4244-d664-4562-5b156f3e6fff:0:0:
State to report: CANCELLED
2016-03-24 14:19:01,626 [UserServer-1] INFO
o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate query
result to initiating client. This would happenif a client is disconnected
before response notice can be sent.
org.apache.drill.exec.rpc.ChannelClosedException: null
at
org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89)
[drill-rpc-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67)
[drill-rpc-1.6.0.jar:1.6.0]
at
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:788)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:689)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1114)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:705)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:32)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:980)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1032)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:965)
[netty-transport-4.0.27.Final.jar:4.0.27.Final]
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:254)
[netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[netty-common-4.0.27.Final.jar:4.0.27.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
2016-03-24 14:19:01,626 [UserServer-1] WARN
o.a.drill.exec.work.foreman.Foreman - Dropping request to move to FAILED state
as query is already at CANCELED state (which is terminal).
-----Message d'origine-----
De : Jason Altekruse [mailto:[email protected]]
Envoyé : jeudi 24 mars 2016 21:27
À : [email protected]
Objet : Re: How to modify connection timeout delay ?
This can be set in drill-override.conf, but this should not be an issue even
for long running queries as we should be sending a heartbeat signal back
throughout a query's execution, even if the query has not yet produced any real
data. Can you share the query you are running and any errors you can find in
the logs?
To adjust the timeout you can set a higher value for the
drill.exec.user.timeout in conf/drill-override.conf, the value is specified in
seconds.
Jason Altekruse
Software Engineer at Dremio
Apache Drill Committer
On Thu, Mar 24, 2016 at 9:20 AM, COUERON Damien (i-BP - MICROPOLE) <
[email protected]> wrote:
> Hi,
>
> I'm trying to use Drill with the sqlline command line on linux but I'm
> not able to maintain a connection to my drillbits alive long enough to
> get the results back.
> The connection is reset after 30 seconds even while a query is running!
>
> I did find the "timeout" variable and set it to -1 but it does not
> change a thing.
>
> The drillbit.log file contains the following lines :
> 2016-03-24 17:15:37,651 [UserServer-1] INFO
> o.a.drill.exec.rpc.user.UserServer - RPC connection /39.6.64.15:31010
> <--> /39.6.64.22:46641 (user client) timed out. Timeout was set to 30
> seconds. Closing connection.
>
> Anyone knows where I can modify this setting ?
>
> Best regards,
> Damien
>
>
> -----Message d'origine-----
> De : Andries Engelbrecht [mailto:[email protected]]
> Envoyé : jeudi 24 mars 2016 16:47
> À : [email protected]
> Objet : Re: JDBC storage plugin fails
>
> I have also seen this fail if the supplied credentials are not valid.
>
> With MySQL it can be a bit tricky as credentials can depend on the
> location of the client, so make sure the MySQL user credentials are
> for % since Drill is a distributed system and any of the nodes may
> connect to MySQL.
>
> --Andries
>
> > On Mar 24, 2016, at 8:39 AM, Christopher Matta <[email protected]> wrote:
> >
> > Scott,
> > Could you paste the JSON you're using to define the MySQL storage
> > plugin here? I've experienced that you get that error when either
> > the JSON is invalid, or Drill can't find the class you defined.
> >
> > I've created a JIRA to improve this functionality:
> > https://issues.apache.org/jira/browse/DRILL-4533
> >
> >
> >
> > On Thursday, March 24, 2016, Wilburn, Scott
> > <[email protected]> wrote:
> >
> >> Hello,
> >> I am trying to connect to a mysql database using the jdbc storage
> >> plugin using Drill 1.5 described here:
> >> https://drill.apache.org/docs/rdbms-storage-plugin/
> >>
> >> When I try to create the plugin through the web UI, I get the
> >> following vague error:
> >> "Please retry: error (unable to create/ update storage)"
> >>
> >> I tried to see if more information exists in the drillbit logs, but
> >> didn't see anything relevant. Does anyone know where I can find a
> >> better description of the problem, or is JDBC just broken in 1.5? I
> >> found the following Jira with the same description for 1.2
> >> (DRILL-3977). Has it been broken since 1.2, or am I doing something
> stupid?
> >>
> >> Any help would be appreciated.
> >>
> >> Thanks,
> >> Scott Wilburn
> >>
> >>
> >
> > --
> > Chris Matta
> > [email protected]
> > 215-701-3146
>
>
>
> __________ L'intégrité de ce message n'étant pas assurée sur
> Internet, la société i-BP ne peut être tenue responsable de son
> contenu. Si vous n'êtes pas destinataire de ce message, merci de le
> détruire et d'avertir l'expéditeur. The integrity of this message
> cannot be guaranteed on the Internet. The i-BP company cannot
> therefore be considered responsible for the contents. If you are not
> the intended recipient of this message, then please delete it and
> notify the sender. __________
>
__________ L'intégrité de ce message n'étant pas assurée sur Internet, la
société i-BP ne peut être tenue responsable de son contenu. Si vous n'êtes pas
destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.
The integrity of this message cannot be guaranteed on the Internet. The i-BP
company cannot therefore be considered responsible for the contents. If you are
not the intended recipient of this message, then please delete it and notify
the sender. __________
