Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Gregg Wonderly]

The completely less thread consuming alternative is call backs.   VMS used/uses 
Asynchronous System Traps (ASTs) for everything in the 1980s.  It was a very 
performant and friendly way to allow small software modules to be written to do 
one thing and used as the call back for asynchronous system behaviors.  I wrote 
several symbionts which did serial connection scripting for connecting to 
printers remotely, process management for process reuse on PMDF to speed up 
mail flow through the queue, and other things interactive with the kernel to 
alter process priorities to provide longer term scheduling on a heavily loaded 
750.  It was like heaven to have such a small amount of code (lots of 
functions, some global data).   I am just suggesting that this kind of thing 
was a major part of more than one programming environment.

Having 100’s of threads blocked on various locks adds considerably to the 
overhead of scheduling but also complicates cache use and burdens the developer 
with looping and waiting in ways that increase bugs in code that should not 
exist.  I really feel that the kind of thing that CompletionStage provides to 
the developer is something that should of been more prevalent from the start.  
It inverts programatic logic in some cases, but call backs really are the best 
way to react to asynchronous eventing in software.  

Gregg Wonderly

> On Sep 21, 2016, at 4:25 PM, Benjamin Manes  wrote:
> 
> My limited understanding is that the original API was only CompletableFuture 
> and that CompletionStage introduced as a compromise. It did not appear to be 
> an attempt to strictly follow an interface-implementation separation, e.g. 
> collections. As you said #toCompletableFuture() may throw an UOE, which means 
> some use-cases can't rely on CompletionState which limits its usefulness. In 
> my case that would be an AsyncLoadingCache with a synchronous LoadingCache 
> view. I think having to code that the resulting implementation would be worse 
> if it called toCompletableFuture, caught the exception, and then adapted as 
> you said. 
> 
> When the new future class was introduced it was stated,
> 
> "In other words, we (j.u.c) are not now in a position to dictate a common 
> interface for all SettableFuture, FutureValue, Promise, ListenableFuture, etc 
> like APIs. And as we've seen, different audiences want/need different subsets 
> of this API exposed as interfaces for their usages, and are in any case 
> unlikely to want change all their existing interfaces. However, what we can 
> do is provide a common underlying implementation that is as fast, scalable, 
> space-conserving, carefully-specified, and reliable as possible. It should 
> then be easy and attractive for others creating or reworking higher-level 
> APIs to relay all functionality to the CompletableFuture implementation."  - 
> Doug Lea, '12
> 
> I've gradually come to terms using CF as part of an API and haven't 
> experienced a downside yet.
> 
> On Wed, Sep 21, 2016 at 1:43 PM, Martin Buchholz  > wrote:
> (Sorry to re-open this discussion)
> 
> The separation of a read-only CompletionStage from CompletableFuture is 
> great.  I'm a fan of the scala style Promise/Future split as described in 
> http://docs.scala-lang.org/overviews/core/futures.html 
> , but: we need to 
> re-add (safe, read-only) blocking methods like join.  Java is not Node.js, 
> where there are no threads but there is a universal event loop.  Java 
> programmers are used to Future, where the *only* way to use a future's value 
> is to block waiting for it.  The existing CompletionStage methods are a 
> better scaling alternative to blocking all the time, but blocking is almost 
> always eventually necessary in Java.  For example, junit test methods that 
> start any asynchronous computation need to block until the computation is 
> done, before returning.
> 
> As Viktor has pointed out, users can always implement blocking themselves by 
> writing
> 
> static  CompletableFuture toCompletableFuture(CompletionStage 
> stage) {
> CompletableFuture f = new CompletableFuture<>();
> stage.handle((T t, Throwable ex) -> {
>  if (ex != null) f.completeExceptionally(ex);
>  else f.complete(t);
>  return null;
>  });
> return f;
> }
> 
> static  T join(CompletionStage stage) {
> return toCompletableFuture(stage).join();
> }
> 
> but unlike Viktor, I think it's unreasonable to not provide this for users 
> (especially when we can do so more efficiently).  What is happening instead 
> is API providers not using CompletionStage as return values in public APIs 
> because of the lack of convenient blocking, and instead returning 
> CompletableFuture, which is a tragic software engineering failure.
> 
> 

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[k...@kodewerk.com]

> On Sep 28, 2016, at 5:05 PM, Gregg Wonderly  wrote:
> 
> The completely less thread consuming alternative is call backs.  

This is a technique that we commonly used in a system that a group of use built 
in the 90s. The system had many variants on a number of different data flows 
through the system. It could take days for a number of the services to complete 
so synchronous calls simply were not a practical solution. async turned out to 
be so simple that it was used all over.

Regards,
Kirk

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Benjamin Manes]

>
> I don't see any support in junit or testng for multi-threaded tests.


TestNG has basic support on @Test for running a test method concurrently.
This assumes synchronous code that does not perform complex coordination,
e.g. simple writes into a ConcurrentMap. Specifically the annotation
provides "threadPoolSize", "invocationCount", and "invocationTimeOut". It
does support multi-threaded test execution at the class or method level, or
isolation by JVM forking, for faster build times.

For test methods I most often use Awaitility
 with a JCiP-style test harness
. Lots of other options out
there depending on your needs.

I do think Doug's original design statement, quoted earlier, is probably
the best answer. It seems impossible for j.u.c. to provide the perfect API
for everyone for any given scenario. Instead CF is a building block that
you can limit or enhance by exposing a custom class. That may mean a
proliferation of subsets or over use of returning CF, but that could happen
regardless. Since it is easy to compose with lambdas, I consider the
approach taken pragmatic and the best of unsatisfying options.

Other than a few minor enhancements coming in JDK9, I'm pretty happy with
CF as is.

On Mon, Sep 26, 2016 at 4:18 PM, Martin Buchholz 
wrote:

>
>
> On Mon, Sep 26, 2016 at 7:55 AM, Viktor Klang 
> wrote:
>
>>
>>>
>> Test methods,
>>>
>>
>> Yeah, I thought so as well, but it turns out that when you have tons of
>> async tests, not being able to start new tests until either that timeout or
>> result makes for a very slow test suite, so that's why most serious test
>> frameworks are growing support for dealing with async code. Then all you
>> want to be able to limit is work-in-progress (sloppily called parallelism)
>>
>
> I don't see any support in junit or testng for multi-threaded tests.
>  jtreg uses the strategy of having a set of reusable JVM processes, each of
> which is running only one test at a time, which provides "pretty good"
> isolation, and seems to work well.
>
>
>> main methods
>>>
>>
>> That's a common scenario, but one that can be solved by having
>> non-daemonic pooled worker threads.
>>
>
> Do you have docs for such a thread pool?
>
>
>> and unix process reapers are all examples where it's reasonable to block
>>> forever.
>>>
>>
>> What about waitpid() + WNOHANG?
>>
>
> Are you suggesting periodic polling is better than blocking?
>
>
>> S: "I think it's unreasonable to not provide this for users
 (especially when we can do so more efficiently)." <- If efficiency is
 desired then blocking is definitely not the right solution.

>>>
>>> What about efficiently providing isComplete?
>>>
>>
>> In my experience isComplete is virtually useless without being able to
>> extract the value, in which case you may as well introduce a non-blocking
>> `Optional poll()`
>>
>
> Do you support adding the Polling methods from
> http://www.scala-lang.org/api/2.12.0-RC1/scala/concurrent/Future.html
> to CompletionStage, i.e. isDone and getNow?
>
>
>> The result may already be available without actually blocking.  It may
>>> even be known to be available immediately.  One would like to get the value
>>> without additional allocation.
>>>
>>
>> I've seen that use-case :), and it tends to either be a situation where
>> the value is available by pure luck (or…scheduling artifacts) or when one
>> is keeping CompletionStages where strict values could be kept instead
>> (think rebinding a value on completion).
>>
>> Reading what your'e writing, may I dare propose that what you're after is
>> something along the lines of a: PollableCompletionStage which sits in
>> between CompletionStage and CompletableFuture?
>>
>
> I've been waffling!  Right now, I'm leaning towards having Java APIs
> return fully mutable CompletableFutures as Benjamin and Pavel suggested
> upthread.  Why?  Because a completion stage can be thought of as all of:
> - a consumer of an upstream value
> - a subscription to the event that makes the upstream value available, and
> - the producer of a possible value for downstream consumers.
> Cancellation could be interpreted as a request to unsubscribe from
> upstream producer or a notification to downstream consumers that no value
> will be forthcoming, or both.
>
> Here's a problem with jdk9 minimalCompletionStage: even if you are happy
> with the minimality of the source stage (perhaps it's shared) you might
> want downstream stages to be mutable, but the methods such as thenApply
> also create minimal completion stages.  If you want to convert to a mutable
> future, you can try calling toCompletableFuture, but there's no guarantee
> that will work, even if it doesn't throw UOE.  You can hand-construct a
> CompletableFuture which is then completed in a Runnable via thenRun, but
> then this is opaque to the CompletableFuture implementation, and
> 

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Dávid Karnok]

If not a straight blocking method, a fluent conversion method to type R
could be useful IMO:

R to(Function, R> converter) {
return converter.apply(this);
}

This way, who needs a blocking get can have a function prepared with a
blocking operation:

 Function blockingGet() {
   cs -> {
   Object[] result = { null, null };
   CountDownLatch cdl = new CountDownLatch();
   cs.whenComplete((v, e) -> {
   if (e != null) {
  result[1] = e;
   } else {
  result[0] = v;
   }
   cdl.countDown();
   });

   try {
   cdl.await();
   } catch (InterruptedException ex) {
   throw new RuntimeException(ex);
   }

   if (result[1] != null) {
  throw new RuntimeException(result[1]);
   }
   return result[0];
   }
}

T t = cs.to(blockingGet());


while others can convert it to a more enabled dataflow system:

cs.to(Flux::fromFuture)
.filter(v -> v < 50)
.defaultIfEmpty(-10)
.subscribe(System.out::println, Throwable::printStackTrace);

2016-09-27 10:39 GMT+02:00 Viktor Klang :

> Seems legit
>
> --
> Cheers,
> √
>
> On Sep 26, 2016 23:29, "Attila Szegedi"  wrote:
>
>> Not at all, you could just have a call to cancel() block until the future
>> completes.
>>
>> *ducks*
>>
>> Attila.
>>
>> > On 25 Sep 2016, at 16:34, Viktor Klang  wrote:
>> >
>> > If that truely is the case then the only way of implementing a readonly
>> > Future is by throwing an exception from cancel...
>> >
>> > --
>> > Cheers,
>> > √
>> >
>> > On Sep 25, 2016 4:20 PM, "Joe Bowbeer"  wrote:
>> >
>> >> This statement regarding what happens after cancel is called is
>> correct:
>> >>
>> >> "*After this method returns, subsequent calls to **isDone**() will
>> always
>> >> return true*. Subsequent calls to isCancelled() will always return true
>> >> if this method returned true."
>> >>
>> >> After cancel returns, the future is completed, hence isDone. If cancel
>> >> returns true, i.e. it was cancelled, then  isCancelled returns true.
>> But,
>> >> for example if the future is already completed when cancel is called,
>> then
>> >> cancel will return false and isCancelled will return false.
>> >>
>> >> On Sep 25, 2016 6:49 AM, "David Holmes" 
>> wrote:
>> >>
>> >>> I think that was meant to read “After this method returns _*true*_,
>> >>> subsequent calls …”
>> >>>
>> >>>
>> >>>
>> >>> David
>>
>
> ___
> Concurrency-interest mailing list
> concurrency-inter...@cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>


-- 
Best regards,
David Karnok

Re: We need to add blocking methods to CompletionStage!

[Viktor Klang]

On Sep 27, 2016 01:18, "Martin Buchholz"  wrote:
>
>
>
> On Mon, Sep 26, 2016 at 7:55 AM, Viktor Klang 
wrote:
>>>
>>>
>>>
>>> Test methods,
>>
>>
>> Yeah, I thought so as well, but it turns out that when you have tons of
async tests, not being able to start new tests until either that timeout or
result makes for a very slow test suite, so that's why most serious test
frameworks are growing support for dealing with async code. Then all you
want to be able to limit is work-in-progress (sloppily called parallelism)
>
>
> I don't see any support in junit or testng for multi-threaded tests.
 jtreg uses the strategy of having a set of reusable JVM processes, each of
which is running only one test at a time, which provides "pretty good"
isolation, and seems to work well.

It's less about multithreading per-se and more about async/non-blocking.

Here's one example of a fwk which supports it:
http://www.scalatest.org/user_guide/async_testing

>
>>>
>>> main methods
>>
>>
>> That's a common scenario, but one that can be solved by having
non-daemonic pooled worker threads.
>
>
> Do you have docs for such a thread pool?

You mean one which has a ThreadFactory which enables setDaemonic(false)?

>
>>>
>>> and unix process reapers are all examples where it's reasonable to
block forever.
>>
>>
>> What about waitpid() + WNOHANG?
>
>
> Are you suggesting periodic polling is better than blocking?

The botion of "better" requires some context to measure against:

>From a liveness, responsiveness and scalability PoV, I'd say, barring
selector support (which is arguably also polling), it's the next best
thing. Since it allows you to monitor N number of external processes with a
low constant factor.

>

 S: "I think it's unreasonable to not provide this for users
(especially when we can do so more efficiently)." <- If efficiency is
desired then blocking is definitely not the right solution.
>>>
>>>
>>> What about efficiently providing isComplete?
>>
>>
>> In my experience isComplete is virtually useless without being able to
extract the value, in which case you may as well introduce a non-blocking
`Optional poll()`
>
>
> Do you support adding the Polling methods from
> http://www.scala-lang.org/api/2.12.0-RC1/scala/concurrent/Future.html
> to CompletionStage, i.e. isDone and getNow?

After thinking about it, I'd say 'No'. Since then people would ask to add
more polling options: isFailed etc.
Having been a part of creating s.c.Future I'd say adding the polling
methods (barring possibly 'value: Option[Try[T]]') has been proven to add
little in terms of value while adding a much larger API surface area.
Since Java lacks value-level try-catch (scala.util.Try) and Java lacks
disjoint union types, the signature of 'getNow' would have to throw, which
is generally a bad thing.

So I think a PollableCompletionStage would be a better option, where the
broader surface area of these kinds of ops can be contained.

Adding a CompletionStage constructor to CompletableFuture would make
conversions simple.

Reiterating Dougs sentiment: there is no API which will appease everyone,
and if everything always should be added, not only does every API become a
kitchen sink, but also all implementations thereof will be needlessly
complex, slow and hard to maintain.

>
>>>
>>> The result may already be available without actually blocking.  It may
even be known to be available immediately.  One would like to get the value
without additional allocation.
>>
>>
>> I've seen that use-case :), and it tends to either be a situation where
the value is available by pure luck (or…scheduling artifacts) or when one
is keeping CompletionStages where strict values could be kept instead
(think rebinding a value on completion).
>>
>> Reading what your'e writing, may I dare propose that what you're after
is something along the lines of a: PollableCompletionStage which sits in
between CompletionStage and CompletableFuture?
>
>
> I've been waffling!  Right now, I'm leaning towards having Java APIs
return fully mutable CompletableFutures as Benjamin and Pavel suggested
upthread.  Why?  Because a completion stage can be thought of as all of:
> - a consumer of an upstream value

Not this, CompletionStage doesn't specify how its value comes to be.

> - a subscription to the event that makes the upstream value available,
and

Never this, since CompletionStage is not cancellable.

> - the producer of a possible value for downstream consumers.

This is exactly what it is.

> Cancellation could be interpreted as a request to unsubscribe from
upstream producer or a notification to downstream consumers that no value
will be forthcoming, or both.

CompletionStage is not cancellable so this concern does not exist.

>
> Here's a problem with jdk9 minimalCompletionStage: even if you are happy
with the minimality of the source stage (perhaps it's shared) you might
want downstream stages to be mutable, but the methods such as 

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Viktor Klang]

Seems legit

-- 
Cheers,
√

On Sep 26, 2016 23:29, "Attila Szegedi"  wrote:

> Not at all, you could just have a call to cancel() block until the future
> completes.
>
> *ducks*
>
> Attila.
>
> > On 25 Sep 2016, at 16:34, Viktor Klang  wrote:
> >
> > If that truely is the case then the only way of implementing a readonly
> > Future is by throwing an exception from cancel...
> >
> > --
> > Cheers,
> > √
> >
> > On Sep 25, 2016 4:20 PM, "Joe Bowbeer"  wrote:
> >
> >> This statement regarding what happens after cancel is called is correct:
> >>
> >> "*After this method returns, subsequent calls to **isDone**() will
> always
> >> return true*. Subsequent calls to isCancelled() will always return true
> >> if this method returned true."
> >>
> >> After cancel returns, the future is completed, hence isDone. If cancel
> >> returns true, i.e. it was cancelled, then  isCancelled returns true.
> But,
> >> for example if the future is already completed when cancel is called,
> then
> >> cancel will return false and isCancelled will return false.
> >>
> >> On Sep 25, 2016 6:49 AM, "David Holmes" 
> wrote:
> >>
> >>> I think that was meant to read “After this method returns _*true*_,
> >>> subsequent calls …”
> >>>
> >>>
> >>>
> >>> David
>

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Martin Buchholz]

I don't remember what happened in 2005, but records say that I wrote:
"""Why not ..
- add to the spec of Future.cancel() a guarantee that subsequent calls
  to Future.isDone() always return true?
"""
which led to the spec:
"""After this method returns, subsequent calls to isDone()

will
always return true."""

which does sort-of seem to contradict
"""This attempt will fail if ... or could not be cancelled for some other
reason."""
although if you squint, you can interpret that as a requirement to throw
rather than return normally if ... could not be cancelled for some other
reason.  There's no @throws clause, but runtime exception specs are often
incomplete!

The Future javadoc is in need of some love, but perhaps there's little
progress because the troubles run deep.

On Sun, Sep 25, 2016 at 11:00 PM, Joe Bowbeer  wrote:

> Cancellation: David, I can see your point. Future.cancel(true) was
> discussed 8/27/05 and the extra text was added to make it clearer that the
> state of Future after cancel is called is separate from the state of any
> associated thread or task.
>
> However, I think the added text corresponded too closely to the only
> implementation of Future.cancel that existed at the time. Elsewhere the
> spec tries to permit Future.cancel to fail for some reason other than
> because the Future had already completed:
>
>   "This attempt will fail if the task [...] could not be cancelled for
> some other reason."
>
>   "returns false if the task could not be cancelled, typically because it
> has already completed normally"
>
> Without the added text, my interpretation would be that if cancel returns
> false and then isDone returns false, then cancel failed for some other
> reason.
>
> On Sun, Sep 25, 2016 at 5:39 PM, Martin Buchholz 
> wrote:
>
>>
>>
>> On Sun, Sep 25, 2016 at 2:22 PM, David Holmes 
>> wrote:
>>
>>>
>>> Yet we somehow added the clarification with no regard as to whether
>>> cancel returned true or not. That seems wrong.
>>>
>>
>> Yikes!  I had always assumed that cancel was not permitted to leave the
>> Future incomplete, perhaps influenced by the wording,
>>
>> """After this method returns, subsequent calls to {@link #isDone} will
>> always return {@code true}."""
>>
>> It's much more in the spirit of Java to throw an exception if the future
>> cannot be completed.  It's never come up, I think.
>>
>
>

Re: We need to add blocking methods to CompletionStage!

[Martin Buchholz]

On Mon, Sep 26, 2016 at 7:55 AM, Viktor Klang 
wrote:

>
>>
> Test methods,
>>
>
> Yeah, I thought so as well, but it turns out that when you have tons of
> async tests, not being able to start new tests until either that timeout or
> result makes for a very slow test suite, so that's why most serious test
> frameworks are growing support for dealing with async code. Then all you
> want to be able to limit is work-in-progress (sloppily called parallelism)
>

I don't see any support in junit or testng for multi-threaded tests.  jtreg
uses the strategy of having a set of reusable JVM processes, each of which
is running only one test at a time, which provides "pretty good" isolation,
and seems to work well.


> main methods
>>
>
> That's a common scenario, but one that can be solved by having
> non-daemonic pooled worker threads.
>

Do you have docs for such a thread pool?


> and unix process reapers are all examples where it's reasonable to block
>> forever.
>>
>
> What about waitpid() + WNOHANG?
>

Are you suggesting periodic polling is better than blocking?


> S: "I think it's unreasonable to not provide this for users
>>> (especially when we can do so more efficiently)." <- If efficiency is
>>> desired then blocking is definitely not the right solution.
>>>
>>
>> What about efficiently providing isComplete?
>>
>
> In my experience isComplete is virtually useless without being able to
> extract the value, in which case you may as well introduce a non-blocking
> `Optional poll()`
>

Do you support adding the Polling methods from
http://www.scala-lang.org/api/2.12.0-RC1/scala/concurrent/Future.html
to CompletionStage, i.e. isDone and getNow?


> The result may already be available without actually blocking.  It may
>> even be known to be available immediately.  One would like to get the value
>> without additional allocation.
>>
>
> I've seen that use-case :), and it tends to either be a situation where
> the value is available by pure luck (or…scheduling artifacts) or when one
> is keeping CompletionStages where strict values could be kept instead
> (think rebinding a value on completion).
>
> Reading what your'e writing, may I dare propose that what you're after is
> something along the lines of a: PollableCompletionStage which sits in
> between CompletionStage and CompletableFuture?
>

I've been waffling!  Right now, I'm leaning towards having Java APIs return
fully mutable CompletableFutures as Benjamin and Pavel suggested upthread.
Why?  Because a completion stage can be thought of as all of:
- a consumer of an upstream value
- a subscription to the event that makes the upstream value available, and
- the producer of a possible value for downstream consumers.
Cancellation could be interpreted as a request to unsubscribe from upstream
producer or a notification to downstream consumers that no value will be
forthcoming, or both.

Here's a problem with jdk9 minimalCompletionStage: even if you are happy
with the minimality of the source stage (perhaps it's shared) you might
want downstream stages to be mutable, but the methods such as thenApply
also create minimal completion stages.  If you want to convert to a mutable
future, you can try calling toCompletableFuture, but there's no guarantee
that will work, even if it doesn't throw UOE.  You can hand-construct a
CompletableFuture which is then completed in a Runnable via thenRun, but
then this is opaque to the CompletableFuture implementation, and
implementation features such as stack overflow prevention will be
ineffective.

So right now I'm OK with e.g. Process#onExit simply returning
CompletableFuture.

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Attila Szegedi]

Not at all, you could just have a call to cancel() block until the future 
completes.

*ducks*

Attila.

> On 25 Sep 2016, at 16:34, Viktor Klang  wrote:
> 
> If that truely is the case then the only way of implementing a readonly
> Future is by throwing an exception from cancel...
> 
> -- 
> Cheers,
> √
> 
> On Sep 25, 2016 4:20 PM, "Joe Bowbeer"  wrote:
> 
>> This statement regarding what happens after cancel is called is correct:
>> 
>> "*After this method returns, subsequent calls to **isDone**() will always
>> return true*. Subsequent calls to isCancelled() will always return true
>> if this method returned true."
>> 
>> After cancel returns, the future is completed, hence isDone. If cancel
>> returns true, i.e. it was cancelled, then  isCancelled returns true. But,
>> for example if the future is already completed when cancel is called, then
>> cancel will return false and isCancelled will return false.
>> 
>> On Sep 25, 2016 6:49 AM, "David Holmes"  wrote:
>> 
>>> I think that was meant to read “After this method returns _*true*_,
>>> subsequent calls …”
>>> 
>>> 
>>> 
>>> David

Re: We need to add blocking methods to CompletionStage!

[Viktor Klang]

On Sat, Sep 24, 2016 at 10:41 PM, Martin Buchholz 
wrote:

> No one is suggesting we add cancel to CompletionStage - I agree that would
> break users, by making an immutable interface mutable.
>

+1


> This also means that CompletionStage cannot extend Future.
>

+1


> I also would not want to have a toFuture method that would return a
> j.u.c.Future, because of misfit Future.cancel.
>

Would you mind elaborating here? According to the cancel method spec on
Future it is completely fine for it to be a no-op which always returns
false:

"This attempt will fail if the task has already completed, has already been
cancelled, *or could not be cancelled for some other reason.*"

Source:
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Future.html



>   If we are adding cancel, then it will be to make a new interface, such
> as the suggested CancellableCompletionStage.
>
> I also agree that CompletionStage does *not* represent "a running task of
> sorts", j.u.c. Future specs are still confusing in that way due to
> FutureTask heritage.
>

+1


>
> On Thu, Sep 22, 2016 at 7:51 PM, James Roper  wrote:
>
>> For example, we often cache futures and return them from a libraries API,
>> if a client could cancel a future, that would break everything else that
>> received that future.
>
>
> On Fri, Sep 23, 2016 at 2:24 PM, Viktor Klang 
> wrote:
>
>> PPS: A misunderstanding is that CompletionStage represents a running task
>> of sorts, one that can be cancelled etc. This is not the case.
>>
>
>
>



-- 
Cheers,
√

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Alex Otenko]

> On 25 Sep 2016, at 22:49, Martin Buchholz  wrote:
> ...Say you are implementing some existing function with a traditional 
> synchronous API.  Your implementation is multi-threaded, but that's an 
> implementation detail.  Before you return to the caller, you must wait for 
> the various parts of your computation to complete (the "join" part of 
> fork/join).  It seems reasonable to wait forever.  If some part of your 
> computation is interacting with an unreliable external resource, then it 
> makes sense for that particular "wait" (which need not occupy a thread) to 
> have a timeout.  Test methods, main methods and unix process reapers are all 
> examples where it's reasonable to block forever.

More than that, cancelling the wait does not cancel the computation it was 
waiting for.

You need to time out / cancel the wait at the junctions where you have the 
power to arbitrate fair use of resource locked up in the caller, by the caller.

Alex

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Joe Bowbeer]

Cancellation: David, I can see your point. Future.cancel(true) was
discussed 8/27/05 and the extra text was added to make it clearer that the
state of Future after cancel is called is separate from the state of any
associated thread or task.

However, I think the added text corresponded too closely to the only
implementation of Future.cancel that existed at the time. Elsewhere the
spec tries to permit Future.cancel to fail for some reason other than
because the Future had already completed:

  "This attempt will fail if the task [...] could not be cancelled for some
other reason."

  "returns false if the task could not be cancelled, typically because it
has already completed normally"

Without the added text, my interpretation would be that if cancel returns
false and then isDone returns false, then cancel failed for some other
reason.

On Sun, Sep 25, 2016 at 5:39 PM, Martin Buchholz 
wrote:

>
>
> On Sun, Sep 25, 2016 at 2:22 PM, David Holmes 
> wrote:
>
>>
>> Yet we somehow added the clarification with no regard as to whether
>> cancel returned true or not. That seems wrong.
>>
>
> Yikes!  I had always assumed that cancel was not permitted to leave the
> Future incomplete, perhaps influenced by the wording,
>
> """After this method returns, subsequent calls to {@link #isDone} will
> always return {@code true}."""
>
> It's much more in the spirit of Java to throw an exception if the future
> cannot be completed.  It's never come up, I think.
>

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Joe Bowbeer]

This statement regarding what happens after cancel is called is correct:

"*After this method returns, subsequent calls to **isDone**() will always
return true*. Subsequent calls to isCancelled() will always return true if
this method returned true."

After cancel returns, the future is completed, hence isDone. If cancel
returns true, i.e. it was cancelled, then  isCancelled returns true. But,
for example if the future is already completed when cancel is called, then
cancel will return false and isCancelled will return false.

On Sep 25, 2016 6:49 AM, "David Holmes" <davidchol...@aapt.net.au> wrote:

> I think that was meant to read “After this method returns _*true*_,
> subsequent calls …”
>
>
>
> David
>
>
>
> *From:* Concurrency-interest [mailto:concurrency-interest-
> boun...@cs.oswego.edu] *On Behalf Of *Viktor Klang
> *Sent:* Sunday, September 25, 2016 9:03 PM
> *To:* Martin Buchholz <marti...@google.com>
> *Cc:* concurrency-interest <concurrency-inter...@cs.oswego.edu>;
> core-libs-dev <core-libs-dev@openjdk.java.net>
> *Subject:* Re: [concurrency-interest] We need to add blocking methods to
> CompletionStage!
>
>
>
>
>
>
>
> On Sun, Sep 25, 2016 at 12:34 PM, Viktor Klang <viktor.kl...@gmail.com>
> wrote:
>
>
>
>
>
> On Sat, Sep 24, 2016 at 10:41 PM, Martin Buchholz <marti...@google.com>
> wrote:
>
> No one is suggesting we add cancel to CompletionStage - I agree that would
> break users, by making an immutable interface mutable.
>
>
>
> +1
>
>
>
> This also means that CompletionStage cannot extend Future.
>
>
>
> +1
>
>
>
> I also would not want to have a toFuture method that would return a
> j.u.c.Future, because of misfit Future.cancel.
>
>
>
> Would you mind elaborating here? According to the cancel method spec on
> Future it is completely fine for it to be a no-op which always returns
> false:
>
>
>
> "This attempt will fail if the task has already completed, has already
> been cancelled, *or could not be cancelled for some other reason.*"
>
>
>
> Source: https://docs.oracle.com/javase/8/docs/api/java/util/
> concurrent/Future.html
>
>
>
> There's an interesting (read: weird) spec clause in cancel:
>
>
>
> "*After this method returns, subsequent calls to isDone() will always
> return true*. Subsequent calls to isCancelled() will always return true
> if this method returned true."
>
>
>
> That clause is in contradiction with the previously quoted line.
>
>
>
>
>
>
>
>
>
>   If we are adding cancel, then it will be to make a new interface, such
> as the suggested CancellableCompletionStage.
>
>
>
> I also agree that CompletionStage does *not* represent "a running task of
> sorts", j.u.c. Future specs are still confusing in that way due to
> FutureTask heritage.
>
>
>
> +1
>
>
>
>
>
> On Thu, Sep 22, 2016 at 7:51 PM, James Roper <ja...@lightbend.com> wrote:
>
> For example, we often cache futures and return them from a libraries API,
> if a client could cancel a future, that would break everything else that
> received that future.
>
>
>
> On Fri, Sep 23, 2016 at 2:24 PM, Viktor Klang <viktor.kl...@gmail.com>
> wrote:
>
> PPS: A misunderstanding is that CompletionStage represents a running task
> of sorts, one that can be cancelled etc. This is not the case.
>
>
>
>
>
>
>
>
>
> --
>
> Cheers,
>
> √
>
>
>
>
>
> --
>
> Cheers,
>
> √
>
> ___
> Concurrency-interest mailing list
> concurrency-inter...@cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>

Re: We need to add blocking methods to CompletionStage!

[Viktor Klang]

On Sun, Sep 25, 2016 at 12:34 PM, Viktor Klang 
wrote:

>
>
> On Sat, Sep 24, 2016 at 10:41 PM, Martin Buchholz 
> wrote:
>
>> No one is suggesting we add cancel to CompletionStage - I agree that
>> would break users, by making an immutable interface mutable.
>>
>
> +1
>
>
>> This also means that CompletionStage cannot extend Future.
>>
>
> +1
>
>
>> I also would not want to have a toFuture method that would return a
>> j.u.c.Future, because of misfit Future.cancel.
>>
>
> Would you mind elaborating here? According to the cancel method spec on
> Future it is completely fine for it to be a no-op which always returns
> false:
>
> "This attempt will fail if the task has already completed, has already
> been cancelled, *or could not be cancelled for some other reason.*"
>
> Source: https://docs.oracle.com/javase/8/docs/api/java/util/
> concurrent/Future.html
>

There's an interesting (read: weird) spec clause in cancel:

"*After this method returns, subsequent calls to isDone() will always
return true*. Subsequent calls to isCancelled() will always return true if
this method returned true."

That clause is in contradiction with the previously quoted line.


>
>
>
>>   If we are adding cancel, then it will be to make a new interface, such
>> as the suggested CancellableCompletionStage.
>>
>> I also agree that CompletionStage does *not* represent "a running task of
>> sorts", j.u.c. Future specs are still confusing in that way due to
>> FutureTask heritage.
>>
>
> +1
>
>
>>
>> On Thu, Sep 22, 2016 at 7:51 PM, James Roper  wrote:
>>
>>> For example, we often cache futures and return them from a libraries
>>> API, if a client could cancel a future, that would break everything else
>>> that received that future.
>>
>>
>> On Fri, Sep 23, 2016 at 2:24 PM, Viktor Klang 
>> wrote:
>>
>>> PPS: A misunderstanding is that CompletionStage represents a running
>>> task of sorts, one that can be cancelled etc. This is not the case.
>>>
>>
>>
>>
>
>
>
> --
> Cheers,
> √
>



-- 
Cheers,
√

Re: We need to add blocking methods to CompletionStage!

[Viktor Klang]

On Sun, Sep 25, 2016 at 11:49 PM, Martin Buchholz 
wrote:

>
>
> On Fri, Sep 23, 2016 at 2:24 PM, Viktor Klang 
> wrote:
>
>>
>> PS. As a sidenote, Martin, and in all friendliness, "actor purist API"?
>> C'mon, I know you're better than that! CompletionStage's design has nothing
>> to do with Actors and if Single Responsibility Principle is considered
>> purism then I'm not sure why we don't have a single interface with all
>> methods in it.
>> Let's try to keep things friendly.
>>
>
> Let's be best friends!
>

One step at a time… :)


>
> The programming profession is still learning how to write concurrent
> programs, fumbling in the dark, and that's particularly true for myself!
> There are competing approaches and we don't know which will prove
> successful.
>

I'm not sure that they are all that competing TBH! There's always the
tradeoff between utility and safety. The problem is that the JDK is a
single layer for ALL Java programmers. Sometimes I think it would be wiser
to have one layer of high-utility/unsafer for library developers and other
"low-level tasks" and have a lower-utility/safer layer for more App-level
development. (I.e. always choose the least powerful thing with the highest
safety which gets the job done.)


>
> PPPS: Adding blocking methods without mandatory timeouts has in practice
>> proven to be a recipe for disaster.
>>
>
> H... By design, Unix processes that create subprocesses should
> dedicate a thread for each subprocess, that waits indefinitely in waitpid.
> Linux kernel folks deliberately decided this was fine.  In the Real World,
> one needs to interface with Other Peoples' APIs.
>

If you expect the machine to be up until the end of the universe, I think
it's fine to put an unbounded wait.
In the Real World—nothing waits forever for anything. (Evolution and all
that…)


>
> Software that rarely blocks is a great idea.  Especially the folks who
> write network server infrastructure increasingly agree.  But it is truly a
> difficult paradigm shift; we will get there slowly.
>

Trust me—having spent the past 6+ years on that journey—I know exactly what
you're talking about!


>
> ---
>
> Say you are implementing some existing function with a traditional
> synchronous API.  Your implementation is multi-threaded, but that's an
> implementation detail.  Before you return to the caller, you must wait for
> the various parts of your computation to complete (the "join" part of
> fork/join).  It seems reasonable to wait forever.  If some part of your
> computation is interacting with an unreliable external resource, then it
> makes sense for that particular "wait" (which need not occupy a thread) to
> have a timeout.
>

If you're implementing some existing method which expects a synchronous
response, but are doing so in a deferred way, that is exactly the kids of
situations where that tends to come back haunt you after a while.
Especially if calls to said implementation by some chance ends up being
executed on said multithreaded implementation's thread pool. (Enqueue
"hilarious" production deadlock-forever)


> Test methods,
>

Yeah, I thought so as well, but it turns out that when you have tons of
async tests, not being able to start new tests until either that timeout or
result makes for a very slow test suite, so that's why most serious test
frameworks are growing support for dealing with async code. Then all you
want to be able to limit is work-in-progress (sloppily called parallelism)


> main methods
>

That's a common scenario, but one that can be solved by having non-daemonic
pooled worker threads.


> and unix process reapers are all examples where it's reasonable to block
> forever.
>

What about waitpid() + WNOHANG?


>
>
>> S: "I think it's unreasonable to not provide this for users
>> (especially when we can do so more efficiently)." <- If efficiency is
>> desired then blocking is definitely not the right solution.
>>
>
> What about efficiently providing isComplete?
>

In my experience isComplete is virtually useless without being able to
extract the value, in which case you may as well introduce a non-blocking
`Optional poll()`


>
> The result may already be available without actually blocking.  It may
> even be known to be available immediately.  One would like to get the value
> without additional allocation.
>

I've seen that use-case :), and it tends to either be a situation where the
value is available by pure luck (or…scheduling artifacts) or when one is
keeping CompletionStages where strict values could be kept instead (think
rebinding a value on completion).

Reading what your'e writing, may I dare propose that what you're after is
something along the lines of a: PollableCompletionStage which sits in
between CompletionStage and CompletableFuture?

-- 
Cheers,
√

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Viktor Klang]

On Sun, Sep 25, 2016 at 11:22 PM, David Holmes <davidchol...@aapt.net.au>
wrote:

> Joe,
>
>
>
> That is ignoring the error case. If the cancel fails then it is not
> complete and it is not cancelled. We added the extra wording back in August
> 2005. It is interesting to note that Martin’s initial query then only
> related to the state of the thread, but that it was clear about things only
> happening if cancel returned true:
>
>
>
> “My guess is that if cancel(true) returns true, then subsequent cals to
> isCancelled() and isDone() will return true, *even if the thread executing
> the task is still running*“
>
>
>
> Yet we somehow added the clarification with no regard as to whether cancel
> returned true or not. That seems wrong.
>

Agreed.


>
>
> David
>
>
>
> *From:* Concurrency-interest [mailto:concurrency-interest-
> boun...@cs.oswego.edu] *On Behalf Of *Joe Bowbeer
> *Sent:* Monday, September 26, 2016 12:20 AM
> *To:* David Holmes <dhol...@ieee.org>
> *Cc:* Martin Buchholz <marti...@google.com>; concurrency-interest <
> concurrency-inter...@cs.oswego.edu>; core-libs-dev <
> core-libs-dev@openjdk.java.net>
>
> *Subject:* Re: [concurrency-interest] We need to add blocking methods to
> CompletionStage!
>
>
>
> This statement regarding what happens after cancel is called is correct:
>
> "*After this method returns, subsequent calls to isDone() will always
> return true*. Subsequent calls to isCancelled() will always return true
> if this method returned true."
>
> After cancel returns, the future is completed, hence isDone. If cancel
> returns true, i.e. it was cancelled, then  isCancelled returns true. But,
> for example if the future is already completed when cancel is called, then
> cancel will return false and isCancelled will return false.
>
>
>
> On Sep 25, 2016 6:49 AM, "David Holmes" <davidchol...@aapt.net.au> wrote:
>
> I think that was meant to read “After this method returns _*true*_,
> subsequent calls …”
>
>
>
> David
>
>
>
> *From:* Concurrency-interest [mailto:concurrency-interest-
> boun...@cs.oswego.edu] *On Behalf Of *Viktor Klang
> *Sent:* Sunday, September 25, 2016 9:03 PM
> *To:* Martin Buchholz <marti...@google.com>
> *Cc:* concurrency-interest <concurrency-inter...@cs.oswego.edu>;
> core-libs-dev <core-libs-dev@openjdk.java.net>
> *Subject:* Re: [concurrency-interest] We need to add blocking methods to
> CompletionStage!
>
>
>
>
>
>
>
> On Sun, Sep 25, 2016 at 12:34 PM, Viktor Klang <viktor.kl...@gmail.com>
> wrote:
>
>
>
>
>
> On Sat, Sep 24, 2016 at 10:41 PM, Martin Buchholz <marti...@google.com>
> wrote:
>
> No one is suggesting we add cancel to CompletionStage - I agree that would
> break users, by making an immutable interface mutable.
>
>
>
> +1
>
>
>
> This also means that CompletionStage cannot extend Future.
>
>
>
> +1
>
>
>
> I also would not want to have a toFuture method that would return a
> j.u.c.Future, because of misfit Future.cancel.
>
>
>
> Would you mind elaborating here? According to the cancel method spec on
> Future it is completely fine for it to be a no-op which always returns
> false:
>
>
>
> "This attempt will fail if the task has already completed, has already
> been cancelled, *or could not be cancelled for some other reason.*"
>
>
>
> Source: https://docs.oracle.com/javase/8/docs/api/java/util/
> concurrent/Future.html
>
>
>
> There's an interesting (read: weird) spec clause in cancel:
>
>
>
> "*After this method returns, subsequent calls to isDone() will always
> return true*. Subsequent calls to isCancelled() will always return true
> if this method returned true."
>
>
>
> That clause is in contradiction with the previously quoted line.
>
>
>
>
>
>
>
>
>
>   If we are adding cancel, then it will be to make a new interface, such
> as the suggested CancellableCompletionStage.
>
>
>
> I also agree that CompletionStage does *not* represent "a running task of
> sorts", j.u.c. Future specs are still confusing in that way due to
> FutureTask heritage.
>
>
>
> +1
>
>
>
>
>
> On Thu, Sep 22, 2016 at 7:51 PM, James Roper <ja...@lightbend.com> wrote:
>
> For example, we often cache futures and return them from a libraries API,
> if a client could cancel a future, that would break everything else that
> received that future.
>
>
>
> On Fri, Sep 23, 2016 at 2:24 PM, Viktor Klang <viktor.kl...@gmail.com>
> wrote:
>
> PPS: A misunderstanding is that CompletionStage represents a running task
> of sorts, one that can be cancelled etc. This is not the case.
>
>
>
>
>
>
>
>
>
> --
>
> Cheers,
>
> √
>
>
>
>
>
> --
>
> Cheers,
>
> √
>
>
> ___
> Concurrency-interest mailing list
> concurrency-inter...@cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>
> ___
> Concurrency-interest mailing list
> concurrency-inter...@cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>


-- 
Cheers,
√

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Viktor Klang]

On Sun, Sep 25, 2016 at 10:01 PM, Martin Buchholz 
wrote:

>
>
> On Sun, Sep 25, 2016 at 7:34 AM, Viktor Klang 
> wrote:
>
>> If that truely is the case then the only way of implementing a readonly
>> Future is by throwing an exception from cancel...
>>
> We the maintainers of j.u.c.Future have always thought that canceling a
> Future will surely leave it completed.  Of course, implementers of any Java
> interface can choose to throw UOE for any method, but this is not intended
> for Future.cancel.  An interface without cancel probably should not be
> extending Future.
>

This seems to have been "addressed" further down this thread.


>
> ---
>
> Here's another way to think of the range of functionality between current
> CompletionStage and current CompletableFuture:
>
> - Add Polling methods from scala Future such as isCompleted
> 
> These are also discouraged, but less so than Await methods
> """Note: using this method yields nondeterministic dataflow programs."""
> Will adding these to CompletionStage be less controversial?
>
> - Add Await blocking methods (that in scala cannot be called directly,
> using the CanAwait mechanism)
>
> - Add cancellation
>
> - Add other methods to complete the future ("Promise")
>
> - Add the full CompletableFuture API, including the obtrude methods
>

> Cancellation is interesting, because it's a mutating method, and so cannot
> be used with a future shared with other users, but it also seems very
> reasonable as a "client" operation.  One can think of obtaining a
> non-shared future as a subscription to a one-time event, and that
> subscription takes up resources in the form of an object.  If the client is
> no longer interested in the event, then cancellation of the future can free
> up resources.  I'm still thinking of what Process.onExit
> 
> should return.  There's a tension between the various roles that
> CompletableFuture serves - as a container to write a value, as a possibly
> cancellable subscription to that value, and as a way to chain async
> computations together.
>

I'd rather suggest creating a 2.0 of FutureTask (i.e. represent the Task
part separately from the Future part)
Have something represents something being executed somewhere else, that can
support cancellation but also expose a read-only facet of itself.

I think you're right in that something is missing which would lie somewhere
in between CompletionStage and CompletableFuture.
What methods would you propose to add to such an interface and why?


>
> On Wed, Sep 21, 2016 at 2:25 PM, Benjamin Manes 
> wrote:
>
>> I've gradually come to terms using CF as part of an API and haven't
>> experienced a downside yet.
>>
>
> H I seem to be moving in that direction as well...
>



-- 
Cheers,
√

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Martin Buchholz]

On Sun, Sep 25, 2016 at 2:22 PM, David Holmes 
wrote:

>
> Yet we somehow added the clarification with no regard as to whether cancel
> returned true or not. That seems wrong.
>

Yikes!  I had always assumed that cancel was not permitted to leave the
Future incomplete, perhaps influenced by the wording,

"""After this method returns, subsequent calls to {@link #isDone} will
always return {@code true}."""

It's much more in the spirit of Java to throw an exception if the future
cannot be completed.  It's never come up, I think.

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Martin Buchholz]

On Sun, Sep 25, 2016 at 7:34 AM, Viktor Klang 
wrote:

> If that truely is the case then the only way of implementing a readonly
> Future is by throwing an exception from cancel...
>
We the maintainers of j.u.c.Future have always thought that canceling a
Future will surely leave it completed.  Of course, implementers of any Java
interface can choose to throw UOE for any method, but this is not intended
for Future.cancel.  An interface without cancel probably should not be
extending Future.

---

Here's another way to think of the range of functionality between current
CompletionStage and current CompletableFuture:

- Add Polling methods from scala Future such as isCompleted

These are also discouraged, but less so than Await methods
"""Note: using this method yields nondeterministic dataflow programs."""
Will adding these to CompletionStage be less controversial?

- Add Await blocking methods (that in scala cannot be called directly,
using the CanAwait mechanism)

- Add cancellation

- Add other methods to complete the future ("Promise")

- Add the full CompletableFuture API, including the obtrude methods

Cancellation is interesting, because it's a mutating method, and so cannot
be used with a future shared with other users, but it also seems very
reasonable as a "client" operation.  One can think of obtaining a
non-shared future as a subscription to a one-time event, and that
subscription takes up resources in the form of an object.  If the client is
no longer interested in the event, then cancellation of the future can free
up resources.  I'm still thinking of what Process.onExit

should return.  There's a tension between the various roles that
CompletableFuture serves - as a container to write a value, as a possibly
cancellable subscription to that value, and as a way to chain async
computations together.

On Wed, Sep 21, 2016 at 2:25 PM, Benjamin Manes  wrote:

> I've gradually come to terms using CF as part of an API and haven't
> experienced a downside yet.
>

H I seem to be moving in that direction as well...

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Viktor Klang]

If that truely is the case then the only way of implementing a readonly
Future is by throwing an exception from cancel...

-- 
Cheers,
√

On Sep 25, 2016 4:20 PM, "Joe Bowbeer" <joe.bowb...@gmail.com> wrote:

> This statement regarding what happens after cancel is called is correct:
>
> "*After this method returns, subsequent calls to **isDone**() will always
> return true*. Subsequent calls to isCancelled() will always return true
> if this method returned true."
>
> After cancel returns, the future is completed, hence isDone. If cancel
> returns true, i.e. it was cancelled, then  isCancelled returns true. But,
> for example if the future is already completed when cancel is called, then
> cancel will return false and isCancelled will return false.
>
> On Sep 25, 2016 6:49 AM, "David Holmes" <davidchol...@aapt.net.au> wrote:
>
>> I think that was meant to read “After this method returns _*true*_,
>> subsequent calls …”
>>
>>
>>
>> David
>>
>>
>>
>> *From:* Concurrency-interest [mailto:concurrency-interest-b
>> oun...@cs.oswego.edu] *On Behalf Of *Viktor Klang
>> *Sent:* Sunday, September 25, 2016 9:03 PM
>> *To:* Martin Buchholz <marti...@google.com>
>> *Cc:* concurrency-interest <concurrency-inter...@cs.oswego.edu>;
>> core-libs-dev <core-libs-dev@openjdk.java.net>
>> *Subject:* Re: [concurrency-interest] We need to add blocking methods to
>> CompletionStage!
>>
>>
>>
>>
>>
>>
>>
>> On Sun, Sep 25, 2016 at 12:34 PM, Viktor Klang <viktor.kl...@gmail.com>
>> wrote:
>>
>>
>>
>>
>>
>> On Sat, Sep 24, 2016 at 10:41 PM, Martin Buchholz <marti...@google.com>
>> wrote:
>>
>> No one is suggesting we add cancel to CompletionStage - I agree that
>> would break users, by making an immutable interface mutable.
>>
>>
>>
>> +1
>>
>>
>>
>> This also means that CompletionStage cannot extend Future.
>>
>>
>>
>> +1
>>
>>
>>
>> I also would not want to have a toFuture method that would return a
>> j.u.c.Future, because of misfit Future.cancel.
>>
>>
>>
>> Would you mind elaborating here? According to the cancel method spec on
>> Future it is completely fine for it to be a no-op which always returns
>> false:
>>
>>
>>
>> "This attempt will fail if the task has already completed, has already
>> been cancelled, *or could not be cancelled for some other reason.*"
>>
>>
>>
>> Source: https://docs.oracle.com/javase/8/docs/api/java/util/concurre
>> nt/Future.html
>>
>>
>>
>> There's an interesting (read: weird) spec clause in cancel:
>>
>>
>>
>> "*After this method returns, subsequent calls to isDone() will always
>> return true*. Subsequent calls to isCancelled() will always return true
>> if this method returned true."
>>
>>
>>
>> That clause is in contradiction with the previously quoted line.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>   If we are adding cancel, then it will be to make a new interface, such
>> as the suggested CancellableCompletionStage.
>>
>>
>>
>> I also agree that CompletionStage does *not* represent "a running task of
>> sorts", j.u.c. Future specs are still confusing in that way due to
>> FutureTask heritage.
>>
>>
>>
>> +1
>>
>>
>>
>>
>>
>> On Thu, Sep 22, 2016 at 7:51 PM, James Roper <ja...@lightbend.com> wrote:
>>
>> For example, we often cache futures and return them from a libraries API,
>> if a client could cancel a future, that would break everything else that
>> received that future.
>>
>>
>>
>> On Fri, Sep 23, 2016 at 2:24 PM, Viktor Klang <viktor.kl...@gmail.com>
>> wrote:
>>
>> PPS: A misunderstanding is that CompletionStage represents a running task
>> of sorts, one that can be cancelled etc. This is not the case.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Cheers,
>>
>> √
>>
>>
>>
>>
>>
>> --
>>
>> Cheers,
>>
>> √
>>
>> ___
>> Concurrency-interest mailing list
>> concurrency-inter...@cs.oswego.edu
>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
>>

Re: We need to add blocking methods to CompletionStage!

[Martin Buchholz]

No one is suggesting we add cancel to CompletionStage - I agree that would
break users, by making an immutable interface mutable.  This also means
that CompletionStage cannot extend Future. I also would not want to have a
toFuture method that would return a j.u.c.Future, because of misfit
Future.cancel.  If we are adding cancel, then it will be to make a new
interface, such as the suggested CancellableCompletionStage.

I also agree that CompletionStage does *not* represent "a running task of
sorts", j.u.c. Future specs are still confusing in that way due to
FutureTask heritage.

On Thu, Sep 22, 2016 at 7:51 PM, James Roper  wrote:

> For example, we often cache futures and return them from a libraries API,
> if a client could cancel a future, that would break everything else that
> received that future.


On Fri, Sep 23, 2016 at 2:24 PM, Viktor Klang 
wrote:

> PPS: A misunderstanding is that CompletionStage represents a running task
> of sorts, one that can be cancelled etc. This is not the case.
>

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Benjamin Manes]

My limited understanding is that the original API was
only CompletableFuture and that CompletionStage introduced as a compromise.
It did not appear to be an attempt to strictly follow an
interface-implementation separation, e.g. collections. As you said
#toCompletableFuture()
may throw an UOE, which means some use-cases can't rely on CompletionState
which limits its usefulness. In my case that would be an AsyncLoadingCache
with a synchronous LoadingCache view. I think having to code that the
resulting implementation would be worse if it called toCompletableFuture,
caught the exception, and then adapted as you said.

When the new future class was introduced it was stated,

"In other words, we (j.u.c) are not now in a position to dictate a common
interface for all SettableFuture, FutureValue, Promise, ListenableFuture,
etc like APIs. And as we've seen, different audiences want/need different
subsets of this API exposed as interfaces for their usages, and are in any
case unlikely to want change all their existing interfaces. However, what
we can do is provide a common underlying implementation that is as fast,
scalable, space-conserving, carefully-specified, and reliable as possible.
It should then be easy and attractive for others creating or reworking
higher-level APIs to relay all functionality to the CompletableFuture
implementation."  - Doug Lea, '12

I've gradually come to terms using CF as part of an API and haven't
experienced a downside yet.

On Wed, Sep 21, 2016 at 1:43 PM, Martin Buchholz 
wrote:

> (Sorry to re-open this discussion)
>
> The separation of a read-only CompletionStage from CompletableFuture is
> great.  I'm a fan of the scala style Promise/Future split as described in
> http://docs.scala-lang.org/overviews/core/futures.html, but: we need to
> re-add (safe, read-only) blocking methods like join.  Java is not Node.js,
> where there are no threads but there is a universal event loop.  Java
> programmers are used to Future, where the *only* way to use a future's
> value is to block waiting for it.  The existing CompletionStage methods are
> a better scaling alternative to blocking all the time, but blocking is
> almost always eventually necessary in Java.  For example, junit test
> methods that start any asynchronous computation need to block until the
> computation is done, before returning.
>
> As Viktor has pointed out, users can always implement blocking themselves
> by writing
>
> static  CompletableFuture toCompletableFuture(CompletionStage
> stage) {
> CompletableFuture f = new CompletableFuture<>();
> stage.handle((T t, Throwable ex) -> {
>  if (ex != null) f.completeExceptionally(ex);
>  else f.complete(t);
>  return null;
>  });
> return f;
> }
>
> static  T join(CompletionStage stage) {
> return toCompletableFuture(stage).join();
> }
>
> but unlike Viktor, I think it's unreasonable to not provide this for users
> (especially when we can do so more efficiently).  What is happening instead
> is API providers not using CompletionStage as return values in public APIs
> because of the lack of convenient blocking, and instead returning
> CompletableFuture, which is a tragic software engineering failure.
>
> Re-adding join is easy.  We discourage CompletionStage.toCompletableFuture
> from throwing UnsupportedOperationException, and implement join as:
>
> public default T join() { return toCompletableFuture().join(); }
>
> There is a risk of multiple-inheritance conflict with Future if we add
> e.g. isDone(), but there are no current plans to turn those Future methods
> into default methods, and even if we did in some future release, it would
> be only a source, not binary incompatibility, so far less serious.
>
> ___
> Concurrency-interest mailing list
> concurrency-inter...@cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>

Re: We need to add blocking methods to CompletionStage!

[Viktor Klang]

Hi Martin,

*Unsurprisingly*, I think it is a bad idea to pollute something which was
created as a non-blocking superset intended to provide maximum utility with
minimum of sharp edges.

However, I think you have a point in that toCompletableFuture throwing UOE
is rather unhelpful, and if some people are huge fans of parking threads
then let's see if we can come to terms on a solution which doesn't
compromise the solution for everyone else.

The subject of this thread, "We need to add blocking methods to
CompletionStage!", gave me a few questions: "who is 'we'?", "when do we
need blocking methods?" and "Why is CompletionStage the right place to add
this?"

I think it's great that you point out that Scala's Future (full disclosure:
I am a co-designer of that) extends the Awaitable[1] trait (think
interface).
It is worth pointing out that those methods are not callable on the
instance directly (notice the implicit evidence type) so all invocations
need to go through an external construct, Await, which makes it abundantly
clear that something different is going to happen.

I wrote a long explanation of the design process (at least parts of it)
here[2], but I'm including the relevant section on Await below:

There’s an underlying question that I think is worth answering: “Why does
Await.result exist, and why doesn’t it exist *on* Future?”

When we designed Scala’s Future one thing we based it on was the experience
with Akka Future (
http://doc.akka.io/api/akka/1.3.1/?_ga=1.21043707.1579561034.1353497989#akka.dispatch.Future
)

Note how it had `get` and `await` methods, which do similar things as the
C# `Result` method — block the currently executing Thread from progressing
until the value is available. Such a method is the counterpart to
asynchronous, it is synchronizing the current Thread with the Thread which
executes the Future — i.e. an API for synchronous programming.

Not only did we find that methods like that introduce performance problems
due to blocking Threads (delay until the value is available but also due to
Thread scheduler wakeup lag), but these methods also produce programs that
are difficult to reason about since they can deadlock and become
non-deterministic in their runtime behavior as things like GC-pauses etc
may cause either spurious failures (where timeouts are supplied) or
prolonged resource starvation due to unavailability of Threads to execute
other logic.

Having a single method for performing these potentially dangerous
operations and putting it outside of the Future API itself meant that it is
easy to spot where blocking is performed as well as easy to outlaw it (by
disallowing it to be present in source code). But we also took it a step
further, by creating the BlockContext mechanism we also made it possible to
the runtime to perform evasive manoeuvres in the presence of blocking. (
http://www.scala-lang.org/api/current/index.html#scala.concurrent.BlockContext
)






*Option 1:*

Now, if Java's type system was just a tad more powerful, it would support
intersection types and APIs expressing the problem of wanting to expose
more than CompletionStage but less than the concrete CompletableFuture type
could very easily return `*CompletionStage[T] with Future[T]*`.

Now we don't really have that, so it needs to be poorly emulated using
compound interfaces, becoming something like `*FutureCompletionStage[T]*`
which would have the `join`-method on Future. And that may actually not be
a bad idea at all. So let's call that *Option 1.*



*Option 2:*

Add a `toFuture` method to CompletionStage such that if you want to go and
use the blocking methods of Future, it is both shorter and does not expose
a concrete type in the signature.

*Futher proposals:*

* Adding a constructor to `CompletableFuture` which takes a
`CompletionStage` as parameter would make the boilerplate discussed
earlier[3] obsolete.

* I'd like to see if we could make the `toCompletableFuture` be a default
method which delegates to `new CompletableFuture<>(this)`, meaning that the
UOE problem would sort of solvable.



*Summary:*

* I think there is already a great place to host blocking methods:
j.u.c.Future

* We can get around the UOE by introducing the constructor on
CompletableFuture both as a fallback to things which do throw UOE on
toCompletableFuture, but also provides a terrific default implementation
for toCompletableFuture.

* We can introduce a toFuture-method with a default implementation which
calls `new CompletableFuture<>(this)`


Have a great weekend!



*Footnotes:*
1:
http://www.scala-lang.org/files/archive/api/2.11.8/index.html#scala.concurrent.Awaitable
2: https://medium.com/@viktorklang/hi-eef4acf316a8#.uesy1fqgo
3: static  CompletableFuture toCompletableFuture(CompletionStage
stage) { … }


PS. As a sidenote, Martin, and in all friendliness, "actor purist API"?
C'mon, I know you're better than that! CompletionStage's design has no

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Pavel Rappo]

On Wed, Sep 21, 2016 at 9:43 PM, Martin Buchholz  wrote:
> What is happening instead is API providers not using CompletionStage as
> return values in public APIs because of the lack of convenient blocking, and
> instead returning CompletableFuture, which is a tragic software engineering
> failure.

On Wed, Sep 21, 2016 at 10:25 PM, Benjamin Manes  wrote:
> I've gradually come to terms using CF as part of an API and haven't
> experienced a downside yet.

I agree with Benjamin and would like to hear more about how it's a
"tragic software engineering failure".

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[James Roper]

On 23 September 2016 at 07:27, Martin Buchholz  wrote:

> Thanks for the lesson, James!
>
> I took a look at the Scala/Akka/LightBend world.  Even there, blocking
> always remains a possibility, even if discouraged, e.g.
> scala.concurrent.Future extends Awaitable (!), and
> http://doc.akka.io/docs/akka/snapshot/general/actor-
> systems.html#Blocking_Needs_Careful_Management.
> And any general purpose Java API needs to be useful outside of a framework.
>

Yes absolutely - personally I'm not against putting join on CompletionStage
(I'm happy to leave the disagreement of this to Viktor), my main concern
regarding CS/CF is encapsulation - if a library performs an asynchronous
computation, that library should be able to safely return a future of that
(using the term future here in the general sense) to clients and remain in
control of it, without worrying about what the clients will do with it -
the future should be effectively "immutable" from the clients perspective,
that is to say, the client shouldn't be able to change it's behaviour at
all.  For example, we often cache futures and return them from a libraries
API, if a client could cancel a future, that would break everything else
that received that future.  We'd rather not have to be in a situation where
to build a resilient API, we have to "copy" every future before we return
it, like APIs currently have to do with the Java collections API.


>   But of course, the exception is junit tests, in that case, we encourage
>> the use of CompletionStage.toCompletableFuture to block.
>>
>
> We're currently fixing jdk9 
> CompletableFuture.minimalCompletionStage().toCompletableFuture()
> to be awaitable.
>
> To make toCompletableFuture().join() more reliable as a recommended way to
> block, I think we should remove the encouragement to throw UOE from:
> http://download.java.net/java/jdk9/docs/api/java/util/
> concurrent/CompletionStage.html#toCompletableFuture--
>
> It sounds like the Akka/LightBend World is happy with current
> CompletionStage API (may I call this the "actor purist API"?), while other
> users will want a bigger, more general CompletableFuture subset for
> consuming future values.  Frustratingly, some of them will want cancel(),
> some not; and we have no good names for any new interfaces; right now all I
> can think of is CompletionStage2 and CompletionStage3 !)
>
> The current implementation of CompletableFuture has a "memory leak
> problem" in that e.g. minimalStage().toCompletableFuture().isDone() will
> leak a Completion object until the source stage itself is collected.  Which
> doesn't happen if e.g. a direct isDone() is provided.
> JDK-8161600: Garbage retention when source CompletableFutures are never
> completed
> (but no one has ever complained!)
>



-- 
*James Roper*
*Software Engineer*

Lightbend  – Build reactive apps!
Twitter: @jroper 

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[James Roper]

On 22 September 2016 at 06:43, Martin Buchholz  wrote:

> What is happening instead is API providers not using CompletionStage as
> return values in public APIs because of the lack of convenient blocking,
> and instead returning CompletableFuture, which is a tragic software
> engineering failure.
>

Out of interest, which APIs are returning CompletableFuture rather than
CompletionStage?  In the Play Framework Java API, we have embraced
CompletionStage, we use it absolutely everywhere.  Likewise in Lagom
Framework and the Java API for Akka streams.

When it comes to blocking, we strongly advocate never blocking (in the
threading models of these projects, blocking on a future will make you very
prone to deadlocks).  But of course, the exception is junit tests, in that
case, we encourage the use of CompletionStage.toCompletableFuture to
block.  In these projects, you'll generally encounter two different
implementations of CompletionStage, one is CompletableFuture, the other is
a scala.concurrent.Future backed implementation.  Both implement
toCompletableFuture.  But a user that integrates a third party library with
its own CompletionStage implementation of course may encounter issues, that
said, I haven't yet seen many (or any?) libraries outside of our ecosystem
that are embracing CompletionStage or CompletableFuture in their public
APIs yet (this is probably due to my ignorance, not due to there not being
any), which is part of why I'm interested in knowing what libraries you
know of that are using it.

Re-adding join is easy.  We discourage CompletionStage.toCompletableFuture
> from throwing UnsupportedOperationException, and implement join as:
>
> public default T join() { return toCompletableFuture().join(); }
>
> There is a risk of multiple-inheritance conflict with Future if we add
> e.g. isDone(), but there are no current plans to turn those Future methods
> into default methods, and even if we did in some future release, it would
> be only a source, not binary incompatibility, so far less serious.
>
> ___
> Concurrency-interest mailing list
> concurrency-inter...@cs.oswego.edu
> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>
>


-- 
*James Roper*
*Software Engineer*

Lightbend  – Build reactive apps!
Twitter: @jroper 

Re: We need to add blocking methods to CompletionStage!

[Kasper Nielsen]

>
> Would not mind a CancellableCompletionStage interface
> CancellableCompletionStage extends CompletionStage {
>   isCancelled();
>   cancel(boolean)
> }
>
>
Just wanted to note. This is not just a question about functionality. It is
also a signal to users
about "hey this a computation you can cancel". For example, if you where to
retrofit all of
java.nio you would probably use CancellableCompletionStage (or whatever the
name)
for all methods. You do not really need to set an int or whatever the
read/write methods return
in 99% of all cases.

Best
  Kasper

Re: We need to add blocking methods to CompletionStage!

[Kasper Nielsen]

On 21 September 2016 at 22:43, Martin Buchholz  wrote:

> (Sorry to re-open this discussion)
>
> The separation of a read-only CompletionStage from CompletableFuture is
> great.  I'm a fan of the scala style Promise/Future split as described in
> http://docs.scala-lang.org/overviews/core/futures.html, but: we need to
> re-add (safe, read-only) blocking methods like join.


Just want to say that I agree. I have using CS/CF for APIs extensively
since Java 8. And all my usage basically boils down to 3 basic use cases.
And I think most others will end up with the same use cases.

1) Observing
The user receives a CS where he can query about the state of the future and
add follow up actions. I really would like to see the rest of the
non-mutating methods from CompletableFuture added to CompletionStage here
if possible.
get()
get(long, TimeUnit)
getNow()
isCompletedExceptionally
isDone()
isCompletedNormally() (isDone && !isCompletedExceptionally) <- new method

2) Cancellable
Tasks that can be cancelled by the user but where the user should not be
able to modify the result or set a custom exception.
For example, cancel connecting to a remote host or cancel some internal
computation.

Right not this is a bit painfull. I have to wrap CompletableFuture to allow
cancel but hide complete/obtrude

Would not mind a CancellableCompletionStage interface
CancellableCompletionStage extends CompletionStage {
  isCancelled();
  cancel(boolean)
}

3) Mutable
The user has complete control. Basically just these three additional
methods compared to CancellableCompletionStage.
  complete()
  completeExceptionally()
  obtrudeException()
I'm fine with returning CompletableFuture here.

Best
  Kasper

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Martin Buchholz]

Thanks for the lesson, James!

On Wed, Sep 21, 2016 at 3:57 PM, James Roper  wrote:

> On 22 September 2016 at 06:43, Martin Buchholz 
> wrote:
>
>> What is happening instead is API providers not using CompletionStage as
>> return values in public APIs because of the lack of convenient blocking,
>> and instead returning CompletableFuture, which is a tragic software
>> engineering failure.
>>
>
> Out of interest, which APIs are returning CompletableFuture rather than
> CompletionStage?  In the Play Framework Java API, we have embraced
> CompletionStage, we use it absolutely everywhere.  Likewise in Lagom
> Framework and the Java API for Akka streams.
>
> When it comes to blocking, we strongly advocate never blocking (in the
> threading models of these projects, blocking on a future will make you very
> prone to deadlocks).
>

I took a look at the Scala/Akka/LightBend world.  Even there, blocking
always remains a possibility, even if discouraged, e.g.
scala.concurrent.Future extends Awaitable (!), and
http://doc.akka.io/docs/akka/snapshot/general/actor-systems.html#Blocking_Needs_Careful_Management
.
And any general purpose Java API needs to be useful outside of a framework.


>   But of course, the exception is junit tests, in that case, we encourage
> the use of CompletionStage.toCompletableFuture to block.
>

We're currently fixing jdk9
CompletableFuture.minimalCompletionStage().toCompletableFuture() to be
awaitable.

To make toCompletableFuture().join() more reliable as a recommended way to
block, I think we should remove the encouragement to throw UOE from:
http://download.java.net/java/jdk9/docs/api/java/util/concurrent/CompletionStage.html#toCompletableFuture--

It sounds like the Akka/LightBend World is happy with current
CompletionStage API (may I call this the "actor purist API"?), while other
users will want a bigger, more general CompletableFuture subset for
consuming future values.  Frustratingly, some of them will want cancel(),
some not; and we have no good names for any new interfaces; right now all I
can think of is CompletionStage2 and CompletionStage3 !)

The current implementation of CompletableFuture has a "memory leak problem"
in that e.g. minimalStage().toCompletableFuture().isDone() will leak a
Completion object until the source stage itself is collected.  Which
doesn't happen if e.g. a direct isDone() is provided.
JDK-8161600: Garbage retention when source CompletableFutures are never
completed
(but no one has ever complained!)

Re: We need to add blocking methods to CompletionStage!

[Chris Hegarty]

Until now CS and CF have not appeared in Java SE API signatures,
outside of the j.u.c package. They are, however, currently being
proposed for use in API signatures for Java SE 9 [1][2], namely
j.l.Process[Handle]::onExit, and more extensively in the proposed new
HTTP Client.

CF was chosen, in some cases, mainly because it provides 'join'. While
some may not like it, a large number of developers still, at some point,
want to be able to block. It was felt that CF was more convenient,
rather than the CS::toCF dance.

In some cases CF provides too many knobs and is not quite suited. For
example, the java.net.http package description has the following
paragraph [3]:

 *  {@code CompletableFuture}s returned by this API will throw
 * {@link java.lang.UnsupportedOperationException} for their {@link
 * java.util.concurrent.CompletableFuture#obtrudeValue(Object) obtrudeValue}
 * and {@link java.util.concurrent.CompletableFuture#obtrudeException(Throwable)
 * obtrudeException} methods. Invoking the {@link
 * java.util.concurrent.CompletableFuture#cancel cancel} method on a
 * {@code CompletableFuture} returned by this API will not interrupt
 * the underlying operation, but may be useful to complete, exceptionally,
 * dependent stages that have not already completed.

At a minimum adding 'join' to CS would help ( and may cause the above
usages in JDK 9 to be revisited ). If this is not acceptable, or maybe
even separately, is there any appetite for a type between CS and CF,
that would expose a select set of methods ( which methods tbd ) that are
"more suited" [*] for use in the above kind of APIs, where the platform
or library is using them as a notification mechanism ( cancel may, or
may not, be useful to notify the platform / library that the operation /
result is no longer interesting, albeit somewhat of a hint ).

-Chris.

[1] 
http://download.java.net/java/jdk9/docs/api/java/util/concurrent/class-use/CompletableFuture.html
[2] 
http://download.java.net/java/jdk9/docs/api/java/util/concurrent/class-use/CompletionStage.html
[3] 
http://hg.openjdk.java.net/jdk9/sandbox/jdk/file/1fdd889687c8/src/java.httpclient/share/classes/java/net/http/package-info.java#l46
[*] for some definition of ...

> On 21 Sep 2016, at 21:43, Martin Buchholz  wrote:
> 
> (Sorry to re-open this discussion)
> 
> The separation of a read-only CompletionStage from CompletableFuture is 
> great.  I'm a fan of the scala style Promise/Future split as described in 
> http://docs.scala-lang.org/overviews/core/futures.html, but: we need to 
> re-add (safe, read-only) blocking methods like join.  Java is not Node.js, 
> where there are no threads but there is a universal event loop.  Java 
> programmers are used to Future, where the *only* way to use a future's value 
> is to block waiting for it.  The existing CompletionStage methods are a 
> better scaling alternative to blocking all the time, but blocking is almost 
> always eventually necessary in Java.  For example, junit test methods that 
> start any asynchronous computation need to block until the computation is 
> done, before returning.
> 
> As Viktor has pointed out, users can always implement blocking themselves by 
> writing
> 
> static  CompletableFuture toCompletableFuture(CompletionStage 
> stage) {
> CompletableFuture f = new CompletableFuture<>();
> stage.handle((T t, Throwable ex) -> {
>  if (ex != null) f.completeExceptionally(ex);
>  else f.complete(t);
>  return null;
>  });
> return f;
> }
> 
> static  T join(CompletionStage stage) {
> return toCompletableFuture(stage).join();
> }
> 
> but unlike Viktor, I think it's unreasonable to not provide this for users 
> (especially when we can do so more efficiently).  What is happening instead 
> is API providers not using CompletionStage as return values in public APIs 
> because of the lack of convenient blocking, and instead returning 
> CompletableFuture, which is a tragic software engineering failure.
> 
> Re-adding join is easy.  We discourage CompletionStage.toCompletableFuture 
> from throwing UnsupportedOperationException, and implement join as:
> 
> public default T join() { return toCompletableFuture().join(); }
> 
> There is a risk of multiple-inheritance conflict with Future if we add e.g. 
> isDone(), but there are no current plans to turn those Future methods into 
> default methods, and even if we did in some future release, it would be only 
> a source, not binary incompatibility, so far less serious.

Re: [concurrency-interest] We need to add blocking methods to CompletionStage!

[Martin Buchholz]

On Wed, Sep 21, 2016 at 2:38 PM, Pavel Rappo  wrote:

> On Wed, Sep 21, 2016 at 9:43 PM, Martin Buchholz 
> wrote:
> > What is happening instead is API providers not using CompletionStage as
> > return values in public APIs because of the lack of convenient blocking,
> and
> > instead returning CompletableFuture, which is a tragic software
> engineering
> > failure.
>
> On Wed, Sep 21, 2016 at 10:25 PM, Benjamin Manes 
> wrote:
> > I've gradually come to terms using CF as part of an API and haven't
> > experienced a downside yet.
>
> I agree with Benjamin and would like to hear more about how it's a
> "tragic software engineering failure".
>

Obviously I was exaggerating, but the software engineering benefits of
separating the producer and consumer users into separate types seems big
(like the Builder pattern for immutable collections).  Would you use
CompletionStage if it had the other methods consumers need?

We need to add blocking methods to CompletionStage!

[Martin Buchholz]

(Sorry to re-open this discussion)

The separation of a read-only CompletionStage from CompletableFuture is
great.  I'm a fan of the scala style Promise/Future split as described in
http://docs.scala-lang.org/overviews/core/futures.html, but: we need to
re-add (safe, read-only) blocking methods like join.  Java is not Node.js,
where there are no threads but there is a universal event loop.  Java
programmers are used to Future, where the *only* way to use a future's
value is to block waiting for it.  The existing CompletionStage methods are
a better scaling alternative to blocking all the time, but blocking is
almost always eventually necessary in Java.  For example, junit test
methods that start any asynchronous computation need to block until the
computation is done, before returning.

As Viktor has pointed out, users can always implement blocking themselves
by writing

static  CompletableFuture toCompletableFuture(CompletionStage
stage) {
CompletableFuture f = new CompletableFuture<>();
stage.handle((T t, Throwable ex) -> {
 if (ex != null) f.completeExceptionally(ex);
 else f.complete(t);
 return null;
 });
return f;
}

static  T join(CompletionStage stage) {
return toCompletableFuture(stage).join();
}

but unlike Viktor, I think it's unreasonable to not provide this for users
(especially when we can do so more efficiently).  What is happening instead
is API providers not using CompletionStage as return values in public APIs
because of the lack of convenient blocking, and instead returning
CompletableFuture, which is a tragic software engineering failure.

Re-adding join is easy.  We discourage CompletionStage.toCompletableFuture
from throwing UnsupportedOperationException, and implement join as:

public default T join() { return toCompletableFuture().join(); }

There is a risk of multiple-inheritance conflict with Future if we add e.g.
isDone(), but there are no current plans to turn those Future methods into
default methods, and even if we did in some future release, it would be
only a source, not binary incompatibility, so far less serious.