[
https://issues.apache.org/jira/browse/THRIFT-4598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253801#comment-17253801
]
Stamatis Zampetakis commented on THRIFT-4598:
---------------------------------------------
Apologies for commenting on a closed issue but I just encountered a very
similar stacktrace in some threads of Hiveserver2.
They seem to hang for a long time in sockerRead0 while waiting to read the
beginning of the message. I don't have control over the client so cannot know
if it is secure or insecure but given that other requests by the same client
are handled sucessfuly I am trying to understand how do we end up in this
situation where the thread hangs. Any other ideas apart from secure/insecure
client/server combination that might lead to the same behavior?
{noformat}
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
- locked <0x0000000288af5280> (a java.lang.Object)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
- locked <0x0000000288af52c8> (a sun.security.ssl.AppInputStream)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
- locked <0x0000000283a4f9f0> (a java.io.BufferedInputStream)
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at
org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:376)
at
org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:453)
at
org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:435)
at
org.apache.thrift.transport.TSaslServerTransport.read(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:706)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
> TBinaryProtocol.readMessageBegin() hangs forever in Java
> --------------------------------------------------------
>
> Key: THRIFT-4598
> URL: https://issues.apache.org/jira/browse/THRIFT-4598
> Project: Thrift
> Issue Type: Bug
> Components: Java - Library
> Affects Versions: 0.11.0
> Environment: Ubuntu Linux 16.04 x86_64
> Reporter: Mario Emmenlauer
> Assignee: James E. King III
> Priority: Major
>
> I run some internal tests for the robustness of our thrift Java server. One
> of the robustness tests is to try to connect to the secure server socket with
> an insecure client and vice versa. This may seem a slightly diabolic test but
> it can be a simple user error to forget enabling or disabling encryption. So
> I think thrift should handle this gracefully.
> However it seems that this test leads to an infinitely hanging server thread!
> The thread blocks the jvm (Java 1.8.0_161 x86_64) from ending. I can see in
> jstack that the main method ends and there are almost no threads left, except
> some garbage collectors and finalizers plus
> org.apache.thrift.server.TThreadPoolServer.
> Here is the call stack of the thrift server thread:
> {code:java}
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> at java.net.SocketInputStream.read(SocketInputStream.java:171)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> - locked <0x000000067b390fd8> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:425)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:321)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:225)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> I already reduced the server socket timeout to 30sec and the requestTimeout
> to 10sec. Is there anything else I can do to timeout readMessageBegin()?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
