Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-25 Thread Martin Buchholz
Thanks all.

(I'm a picky reviewer myself, but sometimes I also approve changes that I
think are Meh)

On Tue, Mar 24, 2015 at 1:04 AM, David Holmes david.hol...@oracle.com
wrote:

 On 24/03/2015 5:55 PM, Paul Sandoz wrote:

 On Mar 24, 2015, at 12:27 AM, John Rose john.r.r...@oracle.com wrote:

 Meanwhile, Paul Sandoz is working to refactor these aspects, creating a
 large chunk of API that will be public, portable, and secure.  You've got
 to sympathize with him for not wanting to run through all the inevitable
 arguments about concurrency primitives once for Unsafe and once again later
 for VarHandles.


 Yes, that's it. As i said i am all for it, but sensing disagreement
 thought it might be best to hold off a bit. I don't wanna block if that
 disagreement is resolved. In fact i wanna take a swing at some other
 aspects of Unsafe to clear up the naming of putOrdered* as mooted in an
 email to hotspot-runtime.


 I will defer to John's wisdom on this. These discussions burn a lot of
 cycles with little concrete benefit to be gained.

 David

  Paul.




Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-24 Thread David Holmes

On 24/03/2015 5:55 PM, Paul Sandoz wrote:

On Mar 24, 2015, at 12:27 AM, John Rose john.r.r...@oracle.com wrote:

Meanwhile, Paul Sandoz is working to refactor these aspects, creating a large 
chunk of API that will be public, portable, and secure.  You've got to 
sympathize with him for not wanting to run through all the inevitable arguments 
about concurrency primitives once for Unsafe and once again later for 
VarHandles.


Yes, that's it. As i said i am all for it, but sensing disagreement thought it 
might be best to hold off a bit. I don't wanna block if that disagreement is 
resolved. In fact i wanna take a swing at some other aspects of Unsafe to clear 
up the naming of putOrdered* as mooted in an email to hotspot-runtime.


I will defer to John's wisdom on this. These discussions burn a lot of 
cycles with little concrete benefit to be gained.


David


Paul.



Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-24 Thread Paul Sandoz
On Mar 24, 2015, at 12:27 AM, John Rose john.r.r...@oracle.com wrote:
 Meanwhile, Paul Sandoz is working to refactor these aspects, creating a large 
 chunk of API that will be public, portable, and secure.  You've got to 
 sympathize with him for not wanting to run through all the inevitable 
 arguments about concurrency primitives once for Unsafe and once again later 
 for VarHandles.

Yes, that's it. As i said i am all for it, but sensing disagreement thought it 
might be best to hold off a bit. I don't wanna block if that disagreement is 
resolved. In fact i wanna take a swing at some other aspects of Unsafe to clear 
up the naming of putOrdered* as mooted in an email to hotspot-runtime.

Paul.


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-23 Thread Martin Buchholz
On Wed, Mar 11, 2015 at 2:12 AM, Paul Sandoz paul.san...@oracle.com wrote:


 I am all for clearing up the language here, but think it prudent to hold
 off doing much until the language for JMM revisions and VarHandle is
 produced, then we can revisit.


I'm disappointed we can't add such a small and safe doc fix now.

I agree the public docs for replacements for Unsafe methods will be much
harder (and more important) to get right.  Especially if they include
wording and API that help those familiar with C/C++11 atomics, as I would
like.


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-23 Thread John Rose
On Mar 23, 2015, at 1:24 PM, Martin Buchholz marti...@google.com wrote:
 
 On Wed, Mar 11, 2015 at 2:12 AM, Paul Sandoz paul.san...@oracle.com wrote:
 
 
 I am all for clearing up the language here, but think it prudent to hold
 off doing much until the language for JMM revisions and VarHandle is
 produced, then we can revisit.
 
 
 I'm disappointed we can't add such a small and safe doc fix now.
 
 I agree the public docs for replacements for Unsafe methods will be much
 harder (and more important) to get right.  Especially if they include
 wording and API that help those familiar with C/C++11 atomics, as I would
 like.

I have read this thread with interest.

If I understand it correctly, Martin is proposing that we document a semantics 
for Unsafe CAS that is weak enough to make it convenient to implement on many 
platforms, but is just strong enough to implement the (known) uses.

David is objecting that since Unsafe is just some native entry points to 
HotSpot internals (Atomic::cmpxchg), it should promise nothing other than the 
current behavior (operational semantics).  This behavior might sometimes be 
overly strong for known use cases, but it is what it is.

So there are standard APIs on the top and the JVM atomics at the bottom, and 
Unsafe in the middle, being pulled both ways.

Here's my view:  Unsafe exists to provide a reasonably understandable and 
platform-independent API that JDK libraries can code to.  It is not a fully 
public API in the sense of a JDK specification because although it is portable 
it cannot be be used securely by untrusted parties.  Of course, it is public 
in the sense of being well-known and widely-used, if not always wisely-used.

Meanwhile, Paul Sandoz is working to refactor these aspects, creating a large 
chunk of API that will be public, portable, and secure.  You've got to 
sympathize with him for not wanting to run through all the inevitable arguments 
about concurrency primitives once for Unsafe and once again later for 
VarHandles.

My position is this:  Since Unsafe is intended to be portable (though …unsafe) 
Martin's proposal makes sense.  It is good to firm up the semantics of Unsafe 
to some portable standard, as long as it doesn't box us into a corner.  
Eventually, we may wish (for performance reasons) to weaken some of its 
guarantees by adjusting its coupling to Atomic::* in HotSpot.

I support the cross-references to C11 semantics, as long as they are 
descriptive of conditions which are natural to the JMM.  If they are not, we 
need additional conversations about whether the JMM needs tweaking.  It is of 
course natural for Unsafe to make references to volatile variable semantics, 
since they are central to the JMM as it stands today.  And there may be some 
wiggle room matching volatiles to C11, if the fine print of the JMM allows 
volatiles to be implemented inconsistently from C11 memory_order_seq_cst.  In 
the end, if we refactor the way we do volatiles, I would expect juc.Atomic* and 
Unsafe to change at the same time, and it would not be harmful for Martin's 
javadoc to be in the mix, since we could make new Unsafe API points to reflect 
an enhanced understanding of what the JVM should do.

So, I think the present proposal makes sense, unless someone can point out some 
reason what we do (now or in the near future) is inconsistent with C11 
atomic_compare_exchange_strong.

And, I applaud Martin's long-running work to firm up the semantics of the 
mid-layer that Unsafe occupies.

Thanks!
— John

Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-11 Thread Paul Sandoz

On Mar 10, 2015, at 8:06 PM, Martin Buchholz marti...@google.com wrote:

 I agree with Andrew that the Unsafe API matters, and there are many 
 implementations of it (even multiple within openjdk itself!).
 
 I still know of no reason why my proposed spec is wrong.  If C11 support was 
 good and ubiquitous, I would recommend using atomic_compare_exchange_strong 
 or C++ equivalent in the hotspot implementation instead of  fences as is done 
 now.
 
 If you think you can write a better spec, please share!

I am all for clearing up the language here, but think it prudent to hold off 
doing much until the language for JMM revisions and VarHandle is produced, then 
we can revisit.

Paul.


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-10 Thread Martin Buchholz
I agree with Andrew that the Unsafe API matters, and there are many
implementations of it (even multiple within openjdk itself!).

I still know of no reason why my proposed spec is wrong.  If C11 support
was good and ubiquitous, I would recommend using atomic_compare_exchange_strong
or C++ equivalent in the hotspot implementation instead of fences as is
done now.

If you think you can write a better spec, please share!

---

It occurs to me that the
Memory Consistency Properties
section of the j.u.c. package specs only talk about happens-before, but
that should be strengthened to Synchronization actions which have a total
order.
http://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.4.4



On Tue, Mar 10, 2015 at 12:31 AM, Andrew Haley a...@redhat.com wrote:

 On 10/03/15 01:30, David Holmes wrote:
  On 10/03/2015 2:21 AM, Martin Buchholz wrote:
  On Mon, Mar 9, 2015 at 4:13 AM, David Holmes david.hol...@oracle.com
  mailto:david.hol...@oracle.com wrote:
 
  On 9/03/2015 7:27 PM, Andrew Haley wrote:
 
 
  But who is to say that is the specification for the Unsafe API?
 
 
  It's for us to say!
 
  I meant specifically the choice of the specification, not the right to
  suggest one.
 
  The public API's have a specification for what they provide to the
  programmer. On what basis are we assigning that same specification
  to Unsafe API? The Unsafe API has to meet the requirements of the
  public APIs that use it, but that doesn't  mean it necessarily
  should have the same specification as those APIs.
 
 
  We already have lots of code that implicitly depends on volatile
  read/write visibility, especially in j.u.c.
  Most obviously, how do you implement compareAndSet methods without the
  same visibility guarantees on the Unsafe equivalents?
 
  Obviously Unsafe has to be implemented in a way that satisfies the j.u.c
  routines that use it.
 
  My spec change only documents what everyone already knows to be true.
  Another way of looking at it is that the Unsafe methods are a low-level
  optimization of the equivalent j.u.c.a. objects without requiring memory
  allocation.
 
  The public API methods specify memory ordering semantics so that the
  end programmer can use those to reason about their use of j.u.c
  components.
 
  The implementation of j.u.c has to satisfy its specified semantics.
 
  The current implementation of j.u.c uses Unsafe operations that in
  turn utilize JVM operations. The JVM operations have memory
  synchronization affects that are stronger than required by j.u.c,
  and so satisfy them.
 
  So the question is: should Unsafe specify the expectations of its
  j.u.c client (what you propose); or should it specify what the JVM
  implementations provide; or should it not actually specify anything
  in this regard knowing that the implementation is required to
  correctly implement the public APIs?

 Unsafe is an API, so should be specified.  (Whether it's really a
 public API has been a matter of some controversy, and people certainly
 have treated it as though it were one!)  There is more than one
 implementation of some of these methods, and some architectures
 (AArch64 is the one I care about the most) have instructions which
 implement the JMM requirements perfectly but not anything stronger.
 If we were to adopt a stronger API for Unsafe that would slow down
 such platforms.

  I don't see much point in pushing down the spec from the public APIs
  to the Unsafe API. To me it suffices to say that the Unsafe methods
  must correctly implement the requirements of the public API without
  specifying that in specific terms of volatile read/writes (which it
  doesn't actually use) or full bi-directional fences (which it does
  use).

 It certainly can use volatile read/writes and I would very much like
 that to happen (at least on AArch64) before JDK9 is released.

  It's not as if Unsafe were a public API for which we might switch in
  implementations from different providers and hence we need a clear
  specification for those implementors to follow.

 Yes it is!  There are other Java VMs which use the OpenJDK class
 library, and they will probably have their own Unsafe builtins.

 Andrew.



Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-10 Thread Andrew Haley
On 10/03/15 01:30, David Holmes wrote:
 On 10/03/2015 2:21 AM, Martin Buchholz wrote:
 On Mon, Mar 9, 2015 at 4:13 AM, David Holmes david.hol...@oracle.com
 mailto:david.hol...@oracle.com wrote:

 On 9/03/2015 7:27 PM, Andrew Haley wrote:


 But who is to say that is the specification for the Unsafe API?


 It's for us to say!
 
 I meant specifically the choice of the specification, not the right to 
 suggest one.
 
 The public API's have a specification for what they provide to the
 programmer. On what basis are we assigning that same specification
 to Unsafe API? The Unsafe API has to meet the requirements of the
 public APIs that use it, but that doesn't  mean it necessarily
 should have the same specification as those APIs.


 We already have lots of code that implicitly depends on volatile
 read/write visibility, especially in j.u.c.
 Most obviously, how do you implement compareAndSet methods without the
 same visibility guarantees on the Unsafe equivalents?
 
 Obviously Unsafe has to be implemented in a way that satisfies the j.u.c 
 routines that use it.

 My spec change only documents what everyone already knows to be true.
 Another way of looking at it is that the Unsafe methods are a low-level
 optimization of the equivalent j.u.c.a. objects without requiring memory
 allocation.
 
 The public API methods specify memory ordering semantics so that the
 end programmer can use those to reason about their use of j.u.c
 components.
 
 The implementation of j.u.c has to satisfy its specified semantics.
 
 The current implementation of j.u.c uses Unsafe operations that in
 turn utilize JVM operations. The JVM operations have memory
 synchronization affects that are stronger than required by j.u.c,
 and so satisfy them.
 
 So the question is: should Unsafe specify the expectations of its
 j.u.c client (what you propose); or should it specify what the JVM
 implementations provide; or should it not actually specify anything
 in this regard knowing that the implementation is required to
 correctly implement the public APIs?

Unsafe is an API, so should be specified.  (Whether it's really a
public API has been a matter of some controversy, and people certainly
have treated it as though it were one!)  There is more than one
implementation of some of these methods, and some architectures
(AArch64 is the one I care about the most) have instructions which
implement the JMM requirements perfectly but not anything stronger.
If we were to adopt a stronger API for Unsafe that would slow down
such platforms.

 I don't see much point in pushing down the spec from the public APIs
 to the Unsafe API. To me it suffices to say that the Unsafe methods
 must correctly implement the requirements of the public API without
 specifying that in specific terms of volatile read/writes (which it
 doesn't actually use) or full bi-directional fences (which it does
 use).

It certainly can use volatile read/writes and I would very much like
that to happen (at least on AArch64) before JDK9 is released.

 It's not as if Unsafe were a public API for which we might switch in
 implementations from different providers and hence we need a clear
 specification for those implementors to follow.

Yes it is!  There are other Java VMs which use the OpenJDK class
library, and they will probably have their own Unsafe builtins.

Andrew.


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-10 Thread Andrew Haley
On 10/03/15 01:38, David Holmes wrote:
 PS. You state: Corresponds to C11 atomic_compare_exchange_strong. Yet 
 that would match with the full bi-directional fences of the VM 
 implementation (due to memory_order_seq_cst), not a volatile read 
 followed by a volatile write (which would be memory_order_acq_rel).

Sequential consistency between atomics does not require full
bidirectional fences with regard to other non-volatile memory
accesses.

Andrew.



Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-09 Thread David Holmes

On 9/03/2015 7:27 PM, Andrew Haley wrote:

On 09/03/15 00:34, David Holmes wrote:

On 7/03/2015 6:49 AM, Martin Buchholz wrote:

A wee code review fer ya:

https://bugs.openjdk.java.net/browse/JDK-8074578
http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Unsafe-CAS-spec/


Sorry Martin but this is neither accurate nor meaningful. It isn't
accurate because the actual Atomic::cmpxchg operations have full
bi-directional fences, and in the long case if locking is used you get
locking related barriers not volatile access barriers.


But the fact that an implementation is stronger than is strictly
required has no bearing on the specification.  We've discussed this at
some length on the concurrency-interest list, and the semantics of a
volatile read followed by a volatile write (if the copmpare is
successful) are what is required by the JMM.


But who is to say that is the specification for the Unsafe API? The 
public API's have a specification for what they provide to the 
programmer. On what basis are we assigning that same specification to 
Unsafe API? The Unsafe API has to meet the requirements of the public 
APIs that use it, but that doesn't  mean it necessarily should have the 
same specification as those APIs.


David


Andrew.



Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-09 Thread Andrew Haley
On 03/09/2015 11:13 AM, David Holmes wrote:
 On 9/03/2015 7:27 PM, Andrew Haley wrote:
 On 09/03/15 00:34, David Holmes wrote:
 On 7/03/2015 6:49 AM, Martin Buchholz wrote:
 A wee code review fer ya:

 https://bugs.openjdk.java.net/browse/JDK-8074578
 http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Unsafe-CAS-spec/

 Sorry Martin but this is neither accurate nor meaningful. It isn't
 accurate because the actual Atomic::cmpxchg operations have full
 bi-directional fences, and in the long case if locking is used you get
 locking related barriers not volatile access barriers.

 But the fact that an implementation is stronger than is strictly
 required has no bearing on the specification.  We've discussed this at
 some length on the concurrency-interest list, and the semantics of a
 volatile read followed by a volatile write (if the compare is
 successful) are what is required by the JMM.
 
 But who is to say that is the specification for the Unsafe API? The 
 public API's have a specification for what they provide to the 
 programmer. On what basis are we assigning that same specification to 
 Unsafe API? The Unsafe API has to meet the requirements of the public 
 APIs that use it, but that doesn't  mean it necessarily should have the 
 same specification as those APIs.

No, it doesn't, but there is no reason that the Unsafe API should
provide stronger guarantees than the public APIs, and to do so would,
to quote Hans Boehm, be too strong in ways that everybody pays for,
but that only benefit racy programs and experts who should know
better.

Andrew.


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-09 Thread Martin Buchholz
On Mon, Mar 9, 2015 at 4:13 AM, David Holmes david.hol...@oracle.com
wrote:

 On 9/03/2015 7:27 PM, Andrew Haley wrote:


 But who is to say that is the specification for the Unsafe API?


It's for us to say!


 The public API's have a specification for what they provide to the
 programmer. On what basis are we assigning that same specification to
 Unsafe API? The Unsafe API has to meet the requirements of the public APIs
 that use it, but that doesn't  mean it necessarily should have the same
 specification as those APIs.


We already have lots of code that implicitly depends on volatile read/write
visibility, especially in j.u.c.
Most obviously, how do you implement compareAndSet methods without the same
visibility guarantees on the Unsafe equivalents?
My spec change only documents what everyone already knows to be true.
Another way of looking at it is that the Unsafe methods are a low-level
optimization of the equivalent j.u.c.a. objects without requiring memory
allocation.


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-09 Thread Martin Buchholz
??
I mentally map Java volatile operations to C11 seq_cst because of the total
order over all operations on volatile variables.
Volatile reads and writes do not just have acquire/release semantics.
The java memory model does not (yet) have C11-style relaxed atomics (like
acquire or release)

On Mon, Mar 9, 2015 at 6:38 PM, David Holmes david.hol...@oracle.com
wrote:

 PS. You state: Corresponds to C11 atomic_compare_exchange_strong. Yet
 that would match with the full bi-directional fences of the VM
 implementation (due to memory_order_seq_cst), not a volatile read followed
 by a volatile write (which would be memory_order_acq_rel).



Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-09 Thread David Holmes

On 10/03/2015 3:04 PM, Martin Buchholz wrote:

??
I mentally map Java volatile operations to C11 seq_cst because of the
total order over all operations on volatile variables.
Volatile reads and writes do not just have acquire/release semantics.


You may be right. The synchronization order (not just operations on 
volatiles) is a total order. I tend to only think about the 
acquire/release semantics of volatile accesses - or more specifically 
the allowed reorderings with respect to other stores and loads. I 
confess I don't know what having a total order requires beyond the 
barriers we utilize for the volatile read/write actions ?? The fact we 
have two different orderings in C11 suggests something extra is required 
by the implementation to achieve seq_cst.


David


The java memory model does not (yet) have C11-style relaxed atomics
(like acquire or release)

On Mon, Mar 9, 2015 at 6:38 PM, David Holmes david.hol...@oracle.com
mailto:david.hol...@oracle.com wrote:

PS. You state: Corresponds to C11
atomic_compare_exchange___strong. Yet that would match with the
full bi-directional fences of the VM implementation (due to
memory_order_seq_cst), not a volatile read followed by a volatile
write (which would be memory_order_acq_rel).



Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-09 Thread David Holmes

On 10/03/2015 2:21 AM, Martin Buchholz wrote:

On Mon, Mar 9, 2015 at 4:13 AM, David Holmes david.hol...@oracle.com
mailto:david.hol...@oracle.com wrote:

On 9/03/2015 7:27 PM, Andrew Haley wrote:


But who is to say that is the specification for the Unsafe API?


It's for us to say!


I meant specifically the choice of the specification, not the right to 
suggest one.



The public API's have a specification for what they provide to the
programmer. On what basis are we assigning that same specification
to Unsafe API? The Unsafe API has to meet the requirements of the
public APIs that use it, but that doesn't  mean it necessarily
should have the same specification as those APIs.


We already have lots of code that implicitly depends on volatile
read/write visibility, especially in j.u.c.
Most obviously, how do you implement compareAndSet methods without the
same visibility guarantees on the Unsafe equivalents?


Obviously Unsafe has to be implemented in a way that satisfies the j.u.c 
routines that use it.



My spec change only documents what everyone already knows to be true.
Another way of looking at it is that the Unsafe methods are a low-level
optimization of the equivalent j.u.c.a. objects without requiring memory
allocation.


The public API methods specify memory ordering semantics so that the end 
programmer can use those to reason about their use of j.u.c components.


The implementation of j.u.c has to satisfy its specified semantics.

The current implementation of j.u.c uses Unsafe operations that in turn 
utilize JVM operations. The JVM operations have memory synchronization 
affects that are stronger than required by j.u.c, and so satisfy them.


So the question is: should Unsafe specify the expectations of its j.u.c 
client (what you propose); or should it specify what the JVM 
implementations provide; or should it not actually specify anything in 
this regard knowing that the implementation is required to correctly 
implement the public APIs? I don't see much point in pushing down the 
spec from the public APIs to the Unsafe API. To me it suffices to say 
that the Unsafe methods must correctly implement the requirements of the 
public API without specifying that in specific terms of volatile 
read/writes (which it doesn't actually use) or full bi-directional 
fences (which it does use). It's not as if Unsafe were a public API for 
which we might switch in implementations from different providers and 
hence we need a clear specification for those implementors to follow.


But if you think it important that these Unsafe methods say something 
about memory synchronization then the wording still needs to make clear 
what volatile is assumed to be being accessed. In particular the j.u.c 
usage always passes the offset of a variable that is volatile - hence 
getting volatile read/write semantics. But you can pass the Unsafe 
methods the offset for a variable that is not declared volatile - so 
what then? Should the specification state that the offset is 
assumed/expected to that for a volatile variable and hence it provides 
volatile read/write only if that variable is volatile? Or should we 
state something more generic as in:


This operation reads the Java variable at the given offset with 
volatile load semantics, and writes it with volatile store semantics.


That doesn't force the variable to be volatile but does require Unsafe 
to act as if it were. The wording is also similar to that used in 
getObjectVolatile/putObjectVolatile. (It also doesn't attempt to the 
address the debate as to whether there is any volatile write on failure 
- should it?)


Or should we say something else?

Cheers,
David


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-09 Thread David Holmes
PS. You state: Corresponds to C11 atomic_compare_exchange_strong. Yet 
that would match with the full bi-directional fences of the VM 
implementation (due to memory_order_seq_cst), not a volatile read 
followed by a volatile write (which would be memory_order_acq_rel).


http://en.cppreference.com/w/c/atomic/atomic_compare_exchange
http://en.cppreference.com/w/c/atomic/memory_order

David

On 10/03/2015 11:30 AM, David Holmes wrote:

On 10/03/2015 2:21 AM, Martin Buchholz wrote:

On Mon, Mar 9, 2015 at 4:13 AM, David Holmes david.hol...@oracle.com
mailto:david.hol...@oracle.com wrote:

On 9/03/2015 7:27 PM, Andrew Haley wrote:


But who is to say that is the specification for the Unsafe API?


It's for us to say!


I meant specifically the choice of the specification, not the right to
suggest one.


The public API's have a specification for what they provide to the
programmer. On what basis are we assigning that same specification
to Unsafe API? The Unsafe API has to meet the requirements of the
public APIs that use it, but that doesn't  mean it necessarily
should have the same specification as those APIs.


We already have lots of code that implicitly depends on volatile
read/write visibility, especially in j.u.c.
Most obviously, how do you implement compareAndSet methods without the
same visibility guarantees on the Unsafe equivalents?


Obviously Unsafe has to be implemented in a way that satisfies the j.u.c
routines that use it.


My spec change only documents what everyone already knows to be true.
Another way of looking at it is that the Unsafe methods are a low-level
optimization of the equivalent j.u.c.a. objects without requiring memory
allocation.


The public API methods specify memory ordering semantics so that the end
programmer can use those to reason about their use of j.u.c components.

The implementation of j.u.c has to satisfy its specified semantics.

The current implementation of j.u.c uses Unsafe operations that in turn
utilize JVM operations. The JVM operations have memory synchronization
affects that are stronger than required by j.u.c, and so satisfy them.

So the question is: should Unsafe specify the expectations of its j.u.c
client (what you propose); or should it specify what the JVM
implementations provide; or should it not actually specify anything in
this regard knowing that the implementation is required to correctly
implement the public APIs? I don't see much point in pushing down the
spec from the public APIs to the Unsafe API. To me it suffices to say
that the Unsafe methods must correctly implement the requirements of the
public API without specifying that in specific terms of volatile
read/writes (which it doesn't actually use) or full bi-directional
fences (which it does use). It's not as if Unsafe were a public API for
which we might switch in implementations from different providers and
hence we need a clear specification for those implementors to follow.

But if you think it important that these Unsafe methods say something
about memory synchronization then the wording still needs to make clear
what volatile is assumed to be being accessed. In particular the j.u.c
usage always passes the offset of a variable that is volatile - hence
getting volatile read/write semantics. But you can pass the Unsafe
methods the offset for a variable that is not declared volatile - so
what then? Should the specification state that the offset is
assumed/expected to that for a volatile variable and hence it provides
volatile read/write only if that variable is volatile? Or should we
state something more generic as in:

This operation reads the Java variable at the given offset with
volatile load semantics, and writes it with volatile store semantics.

That doesn't force the variable to be volatile but does require Unsafe
to act as if it were. The wording is also similar to that used in
getObjectVolatile/putObjectVolatile. (It also doesn't attempt to the
address the debate as to whether there is any volatile write on failure
- should it?)

Or should we say something else?

Cheers,
David


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-09 Thread Andrew Haley
On 09/03/15 00:34, David Holmes wrote:
 On 7/03/2015 6:49 AM, Martin Buchholz wrote:
 A wee code review fer ya:

 https://bugs.openjdk.java.net/browse/JDK-8074578
 http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Unsafe-CAS-spec/
 
 Sorry Martin but this is neither accurate nor meaningful. It isn't 
 accurate because the actual Atomic::cmpxchg operations have full 
 bi-directional fences, and in the long case if locking is used you get 
 locking related barriers not volatile access barriers.

But the fact that an implementation is stronger than is strictly
required has no bearing on the specification.  We've discussed this at
some length on the concurrency-interest list, and the semantics of a
volatile read followed by a volatile write (if the copmpare is
successful) are what is required by the JMM.

Andrew.


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-08 Thread David Holmes

On 7/03/2015 6:49 AM, Martin Buchholz wrote:

A wee code review fer ya:

https://bugs.openjdk.java.net/browse/JDK-8074578
http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Unsafe-CAS-spec/


Sorry Martin but this is neither accurate nor meaningful. It isn't 
accurate because the actual Atomic::cmpxchg operations have full 
bi-directional fences, and in the long case if locking is used you get 
locking related barriers not volatile access barriers. It isn't 
meaningful because you haven't said what variable the volatile semantics 
apply to (not that that would help in the lock-based case).


David


Re: RFR: JDK-8074578 Document memory visibility effects of some Unsafe methods

2015-03-08 Thread Martin Buchholz
I didn't expect this to be controversial.
The spec for regular compareAndSet has similar wording.
I don't think it matters whether implementation is via a lock, because the
lock itself entails at least a volatile read and write.
As to which variable is being read/written to, it should be obvious that
it's the location being CAS'ed (or equivalent).

Can we clarify existing spec for atomics?

http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/package-summary.html

   - compareAndSet and all other read-and-update operations such as
   getAndIncrement have the memory effects of both reading and writing
   volatile variables.


On Sun, Mar 8, 2015 at 5:34 PM, David Holmes david.hol...@oracle.com
wrote:

 On 7/03/2015 6:49 AM, Martin Buchholz wrote:

 A wee code review fer ya:

 https://bugs.openjdk.java.net/browse/JDK-8074578
 http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Unsafe-CAS-spec/


 Sorry Martin but this is neither accurate nor meaningful. It isn't
 accurate because the actual Atomic::cmpxchg operations have full
 bi-directional fences, and in the long case if locking is used you get
 locking related barriers not volatile access barriers. It isn't meaningful
 because you haven't said what variable the volatile semantics apply to (not
 that that would help in the lock-based case).

 David