[
https://issues.apache.org/jira/browse/CURATOR-559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan Zimmerman resolved CURATOR-559.
--------------------------------------
Resolution: Fixed
For Curator 5.0.0 TestThreadLocalRetryLoop now uses a foreground Curator
operation so that the tests are reliable.
> Inconsistent ZK timeouts
> ------------------------
>
> Key: CURATOR-559
> URL: https://issues.apache.org/jira/browse/CURATOR-559
> Project: Apache Curator
> Issue Type: Bug
> Components: Framework
> Affects Versions: 4.2.0, 4.3.0
> Reporter: Grant Digby
> Assignee: Jordan Zimmerman
> Priority: Blocker
> Fix For: 5.0.0, 4.3.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> I've configured a reasonable timeout using BoundedExponentialBackoffRetry,
> and generally it works as I'd expect if ZK is down when I make a call like
> "create.forPath". But if ZK is unavailable when I call acquire on an
> InterProcessReadWriteLock, it takes far longer before it finally times out.
> I call acquire which is wrapped in "RetryLoop.callWithRetry" and it goes onto
> call findProtectedNodeInForeground which is also wrapped in
> "RetryLoop.callWithRetry". If I've configured the
> BoundedExponentialBackoffRetry to retry 20 times, the inner retry tries 20
> times for every one of the 20 outer retry loops, so it retries 400 times.
>
> This class recreates it, if you put break points at the commented sections
> and bring ZK down you can see the different times until it disconnects and
> the stack traces which I've included below.
>
> {code:java}
> public class GoCurator {
> public static void main(String[] args) throws Exception {
> CuratorFramework cf = CuratorFrameworkFactory.newClient(
> "localhost:2181",
> new BoundedExponentialBackoffRetry(200, 10000, 20)
> );
> cf.start();
> String root = "/myRoot";
> if(cf.checkExists().forPath(root) == null) {
> // Stacktrace A showing what happens if ZK is down for this call
> cf.create().forPath(root);
> }
> InterProcessReadWriteLock lcok = new InterProcessReadWriteLock(cf,
> "/grant/myLock");
> // See stacktrace B showing the nested re-try if ZK is down for this call
> lcok.readLock().acquire();
> lcok.readLock().release();
> System.out.println("done");
> } {code}
>
> Stacktrace A (if ZK is down when I'm calling create().forPath). This shows
> the single retry loop so it exist after the correct number of attempts:
>
> {code:java}
> java.lang.Thread.State: WAITING
> at java.lang.Object.wait(Object.java:-1)
> at java.lang.Object.wait(Object.java:502)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1499)
> at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1487)
> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2617)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:242)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:231)
> at
> org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:228)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:219)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:41)
> at com.gebatech.curator.GoCurator.main(GoCurator.java:25) {code}
> Stacktrace B (if ZK is down when I call
> InterProcessReadWriteLock#readLock#acquire). This shows the nested re-try
> loop so it doesn't exit until 20*20 attempts.
>
> {code:java}
> java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Unsafe.java:-1)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
> at
> org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:434)
> at
> org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:56)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.findProtectedNodeInForeground(CreateBuilderImpl.java:1239)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.access$1700(CreateBuilderImpl.java:51)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl$17.call(CreateBuilderImpl.java:1167)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl$17.call(CreateBuilderImpl.java:1156)
> at
> org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:1153)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:607)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:597)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:575)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:51)
> at
> org.apache.curator.framework.recipes.locks.StandardLockInternalsDriver.createsTheLock(StandardLockInternalsDriver.java:54)
> at
> org.apache.curator.framework.recipes.locks.LockInternals.attemptLock(LockInternals.java:225)
> at
> org.apache.curator.framework.recipes.locks.InterProcessMutex.internalLock(InterProcessMutex.java:237)
> at
> org.apache.curator.framework.recipes.locks.InterProcessMutex.acquire(InterProcessMutex.java:89)
> at com.gebatech.curator.GoCurator.main(GoCurator.java:29) {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
