[
https://issues.apache.org/jira/browse/HADOOP-13716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569723#comment-15569723
]
Steve Loughran edited comment on HADOOP-13716 at 10/12/16 8:05 PM:
-------------------------------------------------------------------
example from a test
{code}
int reps = eventually(
() -> ++count == 4,
50,
() -> 10,
(timeout, ex) -> new Exception("timeout after " + count)
);
{code}
This will evaluate the first closure until it succeeds or the total wait time
exceeds 50; the operation for interval calculation is always 10. Returns the
number of iterations it took.
Another example, evaluate a closure until it succeeds. Here it doesn't; the
exception caught on the final unsuccessful iteration is rethrown.
{code}
@Test
public void testEvalLambdaFailures() throws Throwable {
try {
evaluate(() -> {
throw new IOException("oops");
},
TIMEOUT,
retry);
fail("should not have got here");
} catch (IOException expected) {
assertExceptionContains("oops", expected);
}
}
{code}
As well as being Lambda friendly, the patch is just invoking {{Callable<T>}},
so works with Java & and normal interface/implementation classes; that's how I
plan to initially use it for branch-2 code. I just want to be confident we have
something which will move to lambda-expressions with ease.
Also, I've added a FailFastException, similar to that in S3ATestUtils. If a
closure throws that: no attempt to retry. Code that knows what it is doing can
use that to bail out fast on a condition which will never be met.
I'm looking at doing some S3A work, in particular handling inconsistency
problems in some of the FS contract root tests, though I think someone ought to
be able to migrate {{waitFor()}} code to it when they want better exception
propagation, failure handling or alternative backoff strategies
was (Author: ste...@apache.org):
example from a test
{code}
int reps = eventually(
() -> ++count == 4,
50,
() -> 10,
(timeout, ex) -> new Exception("timeout after " + count)
);
{code}
This will evaluate the first closure until it succeeds or the total wait time
exceeds 50; the operation for interval calculation is always 10. Returns the
number of iterations it took.
Another example, evaluate a closure until it succeeds. Here it doesn't; the e
{code}
@Test
public void testEvalLambdaFailures() throws Throwable {
try {
evaluate(() -> {
throw new IOException("oops");
},
TIMEOUT,
retry);
fail("should not have got here");
} catch (IOException expected) {
assertExceptionContains("oops", expected);
}
}
{code}
As well as being Lambda friendly, the patch is just invoking {{Callable<T>}},
so works with Java & and normal interface/implementation classes; that's how I
plan to initially use it for branch-2 code. I just want to be confident we have
something which will move to lambda-expressions with ease.
I'm looking at doing some S3A work, in particular handling inconsistency
problems in some of the FS contract root tests, though I think someone ought to
be able to migrate {{waitFor()}} code to it when they want better exception
propagation, failure handling or alternative backoff strategies
> Add an Eventually class to for tests to await eventual completion of lambda
> expressions/test clauses
> ----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-13716
> URL: https://issues.apache.org/jira/browse/HADOOP-13716
> Project: Hadoop Common
> Issue Type: New Feature
> Components: test
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-13716-001.patch
>
>
> To make our tests robust against timing problems and eventual consistent
> stores, we need to do more spin & wait for state.
> We have some code in {{GenericTestUtils.waitFor}} to await a condition being
> met, but the predicate it calls doesn't throw exceptions, there's no way for
> a probe to throw an exception, and all you get is the eventual "timed out"
> message.
> We can do better, and in closure-ready languages (scala & scalatest, groovy
> and some slider code) we've examples to follow. Some of that work has been
> reimplemented slightly in {{S3ATestUtils.eventually}}
> I propose adding a class in the test tree, {{Eventually}} to be a
> successor/replacement for these.
> # has an eventually/waitfor operation taking a predicate that throws an
> exception
> # has an "evaluate" exception which tries to evaluate an answer until the
> operation stops raising an exception. (again, from scalatest)
> # plugin backoff strategies (from Scalatest; lets you do exponential as well
> as linear)
> # option of adding a special handler to generate the failure exception (e.g.
> run more detailed diagnostics for the exception text, etc).
> # be Java 8 lambda expression friendly
> # be testable and tested itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
