[
https://issues.apache.org/jira/browse/HBASE-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13853950#comment-13853950
]
Hadoop QA commented on HBASE-8558:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619777/HBASE-8558-0.94.txt
against trunk revision .
ATTACHMENT ID: 12619777
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8244//console
This message is automatically generated.
> Add timeout limit for HBaseClient dataOutputStream
> --------------------------------------------------
>
> Key: HBASE-8558
> URL: https://issues.apache.org/jira/browse/HBASE-8558
> Project: HBase
> Issue Type: Bug
> Components: Client
> Affects Versions: 0.94.5, 0.94.14
> Reporter: wanbin
> Assignee: Liang Xie
> Attachments: HBASE-8558-0.94.txt
>
>
> I run jstack at client host. The result is below.
> "hbase-tablepool-60-thread-34" daemon prio=10 tid=0x00007f1e65a48000
> nid=0x5173 runnable [0x00000000579cc000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked <0x0000000758cb0780> (a sun.nio.ch.Util$2)
> - locked <0x0000000758cb0770> (a
> java.util.Collections$UnmodifiableSet)
> - locked <0x0000000758cb0548> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:336)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:158)
> at
> org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:153)
> at
> org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:114)
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> - locked <0x0000000754e978a0> (a java.io.BufferedOutputStream)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:620)
> - locked <0x0000000754e97880> (a java.io.DataOutputStream)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:86)
> at $Proxy13.multi(Unknown Source)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1395)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1393)
> at
> org.apache.hadoop.hbase.client.ServerCallable.withoutRetries(ServerCallable.java:210)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1402)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1390)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> This thread have hung for one hours
> Meanwhile other thread try to close connection
> "IPC Client (1983049639) connection to
> dump002030.cm6.tbsite.net/10.246.2.30:30020 from admin" daemon prio=10
> tid=0x00007f1e70674800 nid=0x3d76 waiting for monitor entry
> [0x000000004bc0f000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> - waiting to lock <0x0000000754e978a0> (a
> java.io.BufferedOutputStream)
> at java.io.DataOutputStream.flush(DataOutputStream.java:106)
> at java.io.FilterOutputStream.close(FilterOutputStream.java:140)
> at org.apache.hadoop.io.IOUtils.cleanup(IOUtils.java:237)
> at org.apache.hadoop.io.IOUtils.closeStream(IOUtils.java:254)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.close(HBaseClient.java:715)
> - locked <0x0000000754e7b818> (a
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:587)
> dump002030.cm6.tbsite.net is dead regionserver.
> I read hbase sourececode, discover connection.out doesn't set timeout
> this.out = new DataOutputStream
> (new BufferedOutputStream(NetUtils.getOutputStream(socket)));
> I see this mean epoll_wait will block indefinitely.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
