Steve Loughran commented on HADOOP-13716:

example from a test
    int reps = eventually(
        () -> ++count == 4,
        () -> 10,
        (timeout, ex) -> new Exception("timeout after " + count)
This will evaluate the first closure until it succeeds or the total wait time 
exceeds 50; the operation for interval calculation is always 10. Returns the 
number of iterations it took.

Another example, evaluate a closure until it succeeds. Here it doesn't; the e
  public void testEvalLambdaFailures() throws Throwable {
    try {
      evaluate(() -> {
            throw new IOException("oops");
      fail("should not have got here");
    } catch (IOException expected) {
      assertExceptionContains("oops", expected);

As well as being Lambda friendly, the patch is just invoking {{Callable<T>}}, 
so works with Java & and normal interface/implementation classes; that's how I 
plan to initially use it for branch-2 code. I just want to be confident we have 
something which will move to lambda-expressions with ease.

I'm looking at doing some S3A work, in particular handling inconsistency 
problems in some of the FS contract root tests, though I think someone ought to 
be able to migrate {{waitFor()}} code to it when they want better exception 
propagation, failure handling or alternative backoff strategies

> Add an Eventually class to await eventual completion of lambda expressions
> --------------------------------------------------------------------------
>                 Key: HADOOP-13716
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13716
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: test
>    Affects Versions: 2.8.0
>            Reporter: Steve Loughran
> To make our tests robust against timing problems and eventual consistent 
> stores, we need to do more spin & wait for state.
> We have some code in {{GenericTestUtils.waitFor}} to await a condition being 
> met, but the predicate it calls doesn't throw exceptions, there's no way for 
> a probe to throw an exception, and all you get is the eventual "timed out" 
> message. 
> We can do better, and in closure-ready languages (scala & scalatest, groovy 
> and some slider code) we've examples to follow. Some of that work has been 
> reimplemented slightly in {{S3ATestUtils.eventually}}
> I propose adding a class in the test tree, {{Eventually}} to be a 
> successor/replacement for these.
> # has an eventually/waitfor operation taking a predicate that throws an 
> exception
> # has an "evaluate" exception which tries to evaluate an answer until the 
> operation stops raising an exception. (again, from scalatest)
> # plugin backoff strategies (from Scalatest; lets you do exponential as well 
> as linear)
> # option of adding a special handler to generate the failure exception (e.g. 
> run more detailed diagnostics for the exception text, etc).
> # be Java 8 lambda expression friendly
> # be testable and tested itself.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

Reply via email to