[
https://issues.apache.org/jira/browse/HBASE-14495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935083#comment-14935083
]
stack commented on HBASE-14495:
-------------------------------
Ok. No hanging tests. It picked up an hdfs zombie.
All our tests passed:
{code}
[INFO]
[INFO] Apache HBase ...................................... SUCCESS [2.950s]
[INFO] Apache HBase - Checkstyle ......................... SUCCESS [0.481s]
[INFO] Apache HBase - Resource Bundle .................... SUCCESS [0.153s]
[INFO] Apache HBase - Annotations ........................ SUCCESS [0.756s]
[INFO] Apache HBase - Protocol ........................... SUCCESS [11.708s]
[INFO] Apache HBase - Common ............................. SUCCESS [1:25.021s]
[INFO] Apache HBase - Procedure .......................... SUCCESS [1:52.523s]
[INFO] Apache HBase - Client ............................. SUCCESS [1:21.759s]
[INFO] Apache HBase - Hadoop Compatibility ............... SUCCESS [7.229s]
[INFO] Apache HBase - Hadoop Two Compatibility ........... SUCCESS [7.404s]
[INFO] Apache HBase - Prefix Tree ........................ SUCCESS [10.519s]
[INFO] Apache HBase - Server ............................. SUCCESS
[1:50:17.108s]
[INFO] Apache HBase - Testing Util ....................... SUCCESS [1.121s]
[INFO] Apache HBase - Thrift ............................. SUCCESS [3:22.167s]
[INFO] Apache HBase - Shell .............................. SUCCESS [4:10.544s]
[INFO] Apache HBase - Integration Tests .................. SUCCESS [1.297s]
[INFO] Apache HBase - Examples ........................... SUCCESS [10.038s]
[INFO] Apache HBase - Rest ............................... SUCCESS [9:20.396s]
[INFO] Apache HBase - External Block Cache ............... SUCCESS [0.663s]
[INFO] Apache HBase - Assembly ........................... SUCCESS [1.418s]
[INFO] Apache HBase - Shaded ............................. SUCCESS [0.087s]
[INFO] Apache HBase - Shaded - Client .................... SUCCESS [0.316s]
[INFO] Apache HBase - Shaded - Server .................... SUCCESS [0.527s]
[INFO] Apache HBase - Spark .............................. SUCCESS [1:20.705s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2:14:08.673s
[INFO] Finished at: Tue Sep 29 08:23:45 UTC 2015
[INFO] Final Memory: 107M/738M
{code}
Let me commit since this for sure fixes some current zombies.
> TestHRegion#testFlushCacheWhileScanning goes zombie
> ---------------------------------------------------
>
> Key: HBASE-14495
> URL: https://issues.apache.org/jira/browse/HBASE-14495
> Project: HBase
> Issue Type: Sub-task
> Components: test
> Reporter: stack
> Assignee: stack
> Attachments: 14495.txt, 14495.txt, 14495v3.txt, 14495v6.txt,
> 14495v7.txt, 14495v9.txt
>
>
> This test goes zombie on us, most recently, here:
> https://builds.apache.org/job/PreCommit-HBASE-Build/15744//console
> It does not fail on my internal rig runs nor locally on laptop when run in a
> loop.
> Its hung up in close of the region:
> {code}
> "main" prio=10 tid=0x00007fc49800a800 nid=0x6053 in Object.wait()
> [0x00007fc4a02c9000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000007d07c3478> (a java.lang.Object)
> at
> org.apache.hadoop.hbase.regionserver.MultiVersionConcurrencyControl.waitForRead(MultiVersionConcurrencyControl.java:207)
> - locked <0x00000007d07c3478> (a java.lang.Object)
> at
> org.apache.hadoop.hbase.regionserver.MultiVersionConcurrencyControl.completeAndWait(MultiVersionConcurrencyControl.java:143)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalPrepareFlushCache(HRegion.java:2257)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2061)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2026)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2016)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1423)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1344)
> - locked <0x00000007d07c34a8> (a java.lang.Object)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1295)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.closeRegionAndWAL(HBaseTestingUtility.java:352)
> at
> org.apache.hadoop.hbase.regionserver.TestHRegion.testFlushCacheWhileScanning(TestHRegion.java:3756)
> {code}
> It is waiting on mvcc to catch up.
> There is this comment at the point where we are hung:
> // TODO: Lets see if we hang here, if there is a scenario where
> an outstanding reader
> // with a read point is in advance of this write point.
> mvcc.completeAndWait(writeEntry);
> The above came in with HBASE-12751. The comment was added at v29:
> https://issues.apache.org/jira/secure/attachment/12754775/12751.rebased.v29.txt
> Looks like I added it so must have had predilection that this might be
> dodgy... Let me take a look...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
