Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[David Holmes]

Specdiffs now available again:

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang/java/lang/Class.html

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/java/lang/invoke/MethodHandle.html

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/java/lang/invoke/MethodHandles.Lookup.html

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/java/lang/reflect/AccessibleObject.html

http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/java/lang/reflect/Method.html

David

On 16/02/2018 7:20 PM, David Holmes wrote:

Hi Karen,

On 16/02/2018 8:01 AM, Karen Kinnear wrote:

David,

I think that is a much better solution. Let the description of each 
Lookup mode be precise, and you have already updated PRIVATE mode to

include nestmates.


Okay I've deleted that sentence.

Unfortunately something has broken specdiff so I can't regenerate the 
docs at the moment - and I've lost the ones I had generated.




I brought this concern up in the EG meeting  yesterday and wanted to 
clarify the difference
between handling of inner/outer classes for backward compatibility and 
general nestmate handling.


Many thanks for that detailed walk through.

David
-



Assumptions:
1. MethodHandle/VarHandle behavior is modeled on bytecode behavior.
2. Nestmates have the added capability of access to private members of 
their nestmates. Period.
3. In future we expect to use nestmates for more than inner/outer 
classes.
4. Inner/outer classes will continue to have the InnerClasses 
attribute, and starting in JDK11, javac

will also generate NestHost and NestMember attributes.
5. With Nestmates, javac will not generate the default (package) 
trampolines to allow inner/outer
classes to access each other’s private members. Note that today this 
is only done for members that

have compile time accesses.
(6. With Nestmates, bridges for protected members will still be 
generated unchanged.)


For nestmates in general, the modifications you have made below allow 
a nestmate to access private

members of their nestmates to match the bytecode behavior.

Prior to nestmates, there is a special workaround in 
MethodHandles.Lookup.in() to allow inner/outer
classes to access any member of any class that shares its top level 
class to emulate the generated trampolines.


https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#lookupModes-- 



In some cases, access between nested classes is obtained by the Java 
compiler by creating an wrapper method to access a private method of 
another class in the same top-level declaration. For example, a nested 
class |C.D| can access private members within other related classes 
such as |C|, |C.D.E|, or |C.B|, but the Java compiler may need to 
generate wrapper methods in those related classes. In such cases, a 
|Lookup| object on |C.E| would be unable to those private members. A 
workaround for this limitation is the |Lookup.in| 
 method, 
which can transform a lookup on |C.E| into one on any of those other 
classes, without special elevation of privilege.


This workaround will continue to be supported going forward explicitly 
for inner/outer classes.
A side-effect of this workaround is the ability of the returned Lookup 
to access not only private methods in
the “related” class, but also protected and inherited members of that 
class which are defined in other packages.
So this workaround will continue to work for inner/outer classes that 
are also nestmates for backward

compatibility.

Going forward, for nestmates in general, the goal is to provide access 
to private members, which
can be done via the access check to match bytecode behavior, and does 
not require special Lookup.in() workarounds.


If at some point in the future we decide we want increased access for 
nestmates, we can widen
this. Let’s just say that the complexity there is challenging and that 
it is better to err on the side of

starting out more restrictive.

Summary - I agree with David. We can leave the documentation as is, 
with the explicit changes to
access checking modified below for private members accessible to 
nestmates.


thanks David!
Karen

On Feb 14, 2018, at 8:36 PM, David Holmes > wrote:


Hi Karen,

Thanks for looking at this.

On 15/02/2018 1:16 AM, Karen Kinnear wrote:

David,
Re-reading these I had one suggestion:

- java/lang/invoke/MethodHandles.java
  * 
- * In some cases, access between nested classes is obtained by 
the Java compiler by creating

- * an wrapper method to access a private method of another class
- * in the same top-level declaration.
+ * Since JDK 11 the relationship between nested types can be 
expressed directly through the

+ * {@code NestHost} and {@code NestMembers} attributes.
+ * (See the Java 

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[David Holmes]

Hi Karen,

On 16/02/2018 8:01 AM, Karen Kinnear wrote:

David,

I think that is a much better solution. Let the description of each 
Lookup mode be precise, and you have already updated PRIVATE mode to

include nestmates.


Okay I've deleted that sentence.

Unfortunately something has broken specdiff so I can't regenerate the 
docs at the moment - and I've lost the ones I had generated.




I brought this concern up in the EG meeting  yesterday and wanted to 
clarify the difference
between handling of inner/outer classes for backward compatibility and 
general nestmate handling.


Many thanks for that detailed walk through.

David
-



Assumptions:
1. MethodHandle/VarHandle behavior is modeled on bytecode behavior.
2. Nestmates have the added capability of access to private members of 
their nestmates. Period.

3. In future we expect to use nestmates for more than inner/outer classes.
4. Inner/outer classes will continue to have the InnerClasses attribute, 
and starting in JDK11, javac

will also generate NestHost and NestMember attributes.
5. With Nestmates, javac will not generate the default (package) 
trampolines to allow inner/outer
classes to access each other’s private members. Note that today this is 
only done for members that

have compile time accesses.
(6. With Nestmates, bridges for protected members will still be 
generated unchanged.)


For nestmates in general, the modifications you have made below allow a 
nestmate to access private

members of their nestmates to match the bytecode behavior.

Prior to nestmates, there is a special workaround in 
MethodHandles.Lookup.in() to allow inner/outer
classes to access any member of any class that shares its top level 
class to emulate the generated trampolines.


https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#lookupModes--

In some cases, access between nested classes is obtained by the Java 
compiler by creating an wrapper method to access a private method of 
another class in the same top-level declaration. For example, a nested 
class |C.D| can access private members within other related classes such 
as |C|, |C.D.E|, or |C.B|, but the Java compiler may need to generate 
wrapper methods in those related classes. In such cases, a 
|Lookup| object on |C.E| would be unable to those private members. A 
workaround for this limitation is the |Lookup.in| 
 method, 
which can transform a lookup on |C.E| into one on any of those other 
classes, without special elevation of privilege.


This workaround will continue to be supported going forward explicitly 
for inner/outer classes.
A side-effect of this workaround is the ability of the returned Lookup 
to access not only private methods in
the “related” class, but also protected and inherited members of that 
class which are defined in other packages.
So this workaround will continue to work for inner/outer classes that 
are also nestmates for backward

compatibility.

Going forward, for nestmates in general, the goal is to provide access 
to private members, which
can be done via the access check to match bytecode behavior, and does 
not require special Lookup.in() workarounds.


If at some point in the future we decide we want increased access for 
nestmates, we can widen
this. Let’s just say that the complexity there is challenging and that 
it is better to err on the side of

starting out more restrictive.

Summary - I agree with David. We can leave the documentation as is, with 
the explicit changes to

access checking modified below for private members accessible to nestmates.

thanks David!
Karen

On Feb 14, 2018, at 8:36 PM, David Holmes > wrote:


Hi Karen,

Thanks for looking at this.

On 15/02/2018 1:16 AM, Karen Kinnear wrote:

David,
Re-reading these I had one suggestion:

- java/lang/invoke/MethodHandles.java
  * 
- * In some cases, access between nested classes is obtained by 
the Java compiler by creating

- * an wrapper method to access a private method of another class
- * in the same top-level declaration.
+ * Since JDK 11 the relationship between nested types can be 
expressed directly through the

+ * {@code NestHost} and {@code NestMembers} attributes.
+ * (See the Java Virtual Machine Specification, sections 4.7.28 
and 4.7.29.)
+ * In that case, the lookup class has direct access to private 
members of all its nestmates, and

+ * that is true of the associated {@code Lookup} object as well.
+ * Otherwise, access between nested classes is obtained by the 
Java compiler creating
+ * a wrapper method to access a private method of another class 
in the same nest.

  * For example, a nested class {@code C.D}

Updated the nested classes description to cover legacy approach and 
new nestmate approach.


- * {@code C.E} would be 

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Karen Kinnear]

David,

I think that is a much better solution. Let the description of each Lookup mode 
be precise,
and you have already updated PRIVATE mode to include nestmates.

I brought this concern up in the EG meeting  yesterday and wanted to clarify 
the difference
between handling of inner/outer classes for backward compatibility and general 
nestmate handling.

Assumptions:
1. MethodHandle/VarHandle behavior is modeled on bytecode behavior.
2. Nestmates have the added capability of access to private members of their 
nestmates. Period.
3. In future we expect to use nestmates for more than inner/outer classes.
4. Inner/outer classes will continue to have the InnerClasses attribute, and 
starting in JDK11, javac
will also generate NestHost and NestMember attributes.
5. With Nestmates, javac will not generate the default (package) trampolines to 
allow inner/outer
classes to access each other’s private members. Note that today this is only 
done for members that
have compile time accesses.
(6. With Nestmates, bridges for protected members will still be generated 
unchanged.)

For nestmates in general, the modifications you have made below allow a 
nestmate to access private
members of their nestmates to match the bytecode behavior.

Prior to nestmates, there is a special workaround in MethodHandles.Lookup.in() 
to allow inner/outer
classes to access any member of any class that shares its top level class to 
emulate the generated trampolines.

https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#lookupModes--

In some cases, access between nested classes is obtained by the Java compiler 
by creating an wrapper method to access a private method of another class in 
the same top-level declaration. For example, a nested class C.D can access 
private members within other related classes such as C, C.D.E, or C.B, but the 
Java compiler may need to generate wrapper methods in those related classes. In 
such cases, a Lookup object on C.E would be unable to those private members. A 
workaround for this limitation is the Lookup.in 

 method, which can transform a lookup on C.E into one on any of those other 
classes, without special elevation of privilege.

This workaround will continue to be supported going forward explicitly for 
inner/outer classes.
A side-effect of this workaround is the ability of the returned Lookup to 
access not only private methods in
the “related” class, but also protected and inherited members of that class 
which are defined in other packages.
So this workaround will continue to work for inner/outer classes that are also 
nestmates for backward
compatibility.

Going forward, for nestmates in general, the goal is to provide access to 
private members, which
can be done via the access check to match bytecode behavior, and does not 
require special Lookup.in() workarounds.

If at some point in the future we decide we want increased access for 
nestmates, we can widen
this. Let’s just say that the complexity there is challenging and that it is 
better to err on the side of
starting out more restrictive.

Summary - I agree with David. We can leave the documentation as is, with the 
explicit changes to
access checking modified below for private members accessible to nestmates.

thanks David!
Karen

> On Feb 14, 2018, at 8:36 PM, David Holmes  wrote:
> 
> Hi Karen,
> 
> Thanks for looking at this.
> 
> On 15/02/2018 1:16 AM, Karen Kinnear wrote:
>> David,
>> Re-reading these I had one suggestion:
>>> - java/lang/invoke/MethodHandles.java
>>>   * 
>>> - * In some cases, access between nested classes is obtained by the 
>>> Java compiler by creating
>>> - * an wrapper method to access a private method of another class
>>> - * in the same top-level declaration.
>>> + * Since JDK 11 the relationship between nested types can be expressed 
>>> directly through the
>>> + * {@code NestHost} and {@code NestMembers} attributes.
>>> + * (See the Java Virtual Machine Specification, sections 4.7.28 and 
>>> 4.7.29.)
>>> + * In that case, the lookup class has direct access to private members 
>>> of all its nestmates, and
>>> + * that is true of the associated {@code Lookup} object as well.
>>> + * Otherwise, access between nested classes is obtained by the Java 
>>> compiler creating
>>> + * a wrapper method to access a private method of another class in the 
>>> same nest.
>>>   * For example, a nested class {@code C.D}
>>> 
>>> Updated the nested classes description to cover legacy approach and new 
>>> nestmate approach.
>>> 
>>> - * {@code C.E} would be unable to those private members.
>>> + * {@code C.E} would be unable to access those private members.
>>> 
>>> Fixed typo: "access" was missing.
>>> 
>>>   * Discussion of private access:
>>>   * We say that a lookup has private access

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[David Holmes]

Hi Karen,

Thanks for looking at this.

On 15/02/2018 1:16 AM, Karen Kinnear wrote:

David,

Re-reading these I had one suggestion:


- java/lang/invoke/MethodHandles.java
   * 
- * In some cases, access between nested classes is obtained by the Java 
compiler by creating
- * an wrapper method to access a private method of another class
- * in the same top-level declaration.
+ * Since JDK 11 the relationship between nested types can be expressed 
directly through the
+ * {@code NestHost} and {@code NestMembers} attributes.
+ * (See the Java Virtual Machine Specification, sections 4.7.28 and 
4.7.29.)
+ * In that case, the lookup class has direct access to private members of 
all its nestmates, and
+ * that is true of the associated {@code Lookup} object as well.
+ * Otherwise, access between nested classes is obtained by the Java 
compiler creating
+ * a wrapper method to access a private method of another class in the 
same nest.
   * For example, a nested class {@code C.D}

Updated the nested classes description to cover legacy approach and 
new nestmate approach.


- * {@code C.E} would be unable to those private members.
+ * {@code C.E} would be unable to access those private members.

Fixed typo: "access" was missing.

   * Discussion of private access:
   * We say that a lookup has private access
   * if its {@linkplain #lookupModes lookup modes}
- * include the possibility of accessing {@code private} members.
+ * include the possibility of accessing {@code private} members
+ * (which includes the private members of nestmates).
   * As documented in the relevant methods elsewhere,
   * only lookups with private access possess the following capabilities:
   * 
- * access private fields, methods, and constructors of the lookup class
+ * access private fields, methods, and constructors of the lookup 
class and its nestmates

Clarify that private access includes nestmate access.

- *  access all members of the caller's class, all public types in the 
caller's module,
+ *  access all members of the caller's class and nestmates, all public 
types in the caller's module,

For the above, I would change this to
* access all members of the caller’s class, all private members of 
nestmates, all types in the caller’s package, all public …


Specifically, we are extended the PRIVATE mode as above to include 
access to all private members of nestmates
and this description is trying to summarize access when all possible 
bits are set.
None of the settings give you access to default (package-private) 
members that a nestmate inherits from a
different package - since the Lookup model is based on the JVMS/bytecode 
behavior.


True - you don't get access to a nestmates inherited protected members 
declared in a different package. I certainly didn't intend to somehow 
imply that.


I added the types in the caller’s package since PACKAGE gives you that 
but it was missing from the existing list.


Backing up ... in 8 this method doc simply said:

"A freshly-created lookup object on the caller's class has all possible 
bits set, since the caller class can access all its own members."


It doesn't try to say what a class can access, it just makes the obvious 
statement that it can access all its own members. If that was the 
current text I would not have needed to make any adjustment for nestmates.


But for 9/10 it states:

"A freshly-created lookup object on the caller's class has all possible 
bits set, except UNCONDITIONAL. The lookup can be used to access all 
members of the caller's class, all public types in the caller's module, 
and all public types in packages exported by other modules to the 
caller's module."


This is quite a different formulation as it now tries to enumerate the 
set of accessible things - or at least gives that impression to me! But 
it is not complete as it doesn't mention non-public types in the current 
package, nor does it mention package-accessible members of types 
(whether public or not).


With nestmates we have only expanded to include private member access, 
but the original text doesn't touch on nestmates directly at all. If we 
were to say only "all private members of nestmates" then we seem to 
suggest no access to the all the other members (public, directly 
declared protected, package). But if we say "nestmates" then that may 
imply more than intended as you point out. If we say nothing then we 
again imply by omission that there is no nestmate access.


This seems to be a bit of an unwravelling thread started by the changes 
in 9. I would suggest we simply delete this sentence altogether:


"The lookup can be used to access all members of the caller's class, all 
public types in the caller's module, and all public types in packages 
exported by other modules to the caller's module."


as each of the modes, together with the "Access Checking" section of the 
class 

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Feb 12, 2018, at 6:51 PM, David Holmes  wrote:
>>> But I, for one, prefer a "user pays" scheme over an "everyone pays" scheme 
>>> (which is what disallowing duplicates would also be).
>> It’s an awkward situation, experience suggests this type of thing bites back 
>> later on, so my inclination is to take the hit in the JDK and not return 
>> dups. The implementation might be able to cache information on the internal 
>> ReflectionData class.
> 
> Please see John's response on this:
> 
> http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000465.html
> 

Thanks for the context. You may be surprised to learn that i still don’t agree 
:-) so i will just state my objection more concisely and move on:

I don’t think as specified the method serves developers very well for the 
common use-case of iterating over the nest mates and performing actions with 
side-effects. A developer will have to write defensive code or, more likely, 
write buggy code that will blow up in the rare case a classfile contain 
duplicates e.g. generated by something other than javac. That rare case may be 
hard to track down and expensive to fix.

Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[David Holmes]

On 13/02/2018 12:51 PM, David Holmes wrote:

On 13/02/2018 12:39 PM, Paul Sandoz wrote:
On Feb 12, 2018, at 6:24 PM, David Holmes  
wrote:


On 13/02/2018 11:45 AM, Paul Sandoz wrote:
On Feb 12, 2018, at 1:55 PM, David Holmes  
wrote:

getNestMembers
—
"The list of nest members in the classfile is permitted to contain 
duplicates, or to explicitly include the nest host. It is not 
required that an implementation of this method removes these 
duplicates."
The "or to explicitly include the nest host” suggests it might not 
include the nest host, but a prior statement says it will be 
present in the zeroth element.


The "or" pertains to the list of nest members in the classfile - ie 
the contents of the NestMembers attribute. The returned array of 
nestmembers will always contain the nesthost as the zeroeth 
element, but may also contain it somewhere else if the classfile 
explicitly listed it in nestmembers.


I see, it’s easy to misread, well i did :-) Perhaps call out the 
attribute and provide a link to the JVMS?


I can add a link to JVMS if that is what we normally do. As for 
misreading ... the subject of the sentence is "The list of nest 
members in the classfile". ;-)




I know :-) most developers will not be thinking at the classfile level 
so some link for those that are interested or care is helpful i think.


Okay will see what I can reasonably add.


Added JVMS ref:

 * The list of nest members in the classfile (JVMS 4.1) is 
permitted to

 * contain duplicates, or to explicitly include the nest host.

David
-




Why the ambiguity over duplicates? is this motivated by 
performance? this may just push the cost to clients that have to 
always remove duplicates to function reliably and may be cause 
bugs if duplicates are rare and induced by certain relationships 
or loading patterns. My inclination would be for the returned 
array to not contain duplicates.


Yes performance. Having to check for duplicates adds a cost to 
every single well formed call to this API to account for something 
that the specification allows to happen but which we don't expect 
to happen and which javac will never produce. This has all been 
discussed previously.


Ok, it’s unfortunate that the cost will be placed on the developer 
who has to code defensively in case there might be duplicates i.e. 
the performance cost is pushed to an unbounded set of places (or 
places where bugs may lurk).


There's really no expectation that the developer will need to program 
defensively here as we don't expect compilers to produce such 
classfiles.


But the developer will not know that since they will be reading the 
JavaDoc.



But I, for one, prefer a "user pays" scheme over an "everyone pays" 
scheme (which is what disallowing duplicates would also be).


It’s an awkward situation, experience suggests this type of thing 
bites back later on, so my inclination is to take the hit in the JDK 
and not return dups. The implementation might be able to cache 
information on the internal ReflectionData class.


Please see John's response on this:

http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000465.html 



Thanks,
David


Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Feb 11, 2018, at 5:19 PM, David Holmes  wrote:
> 
> Hi Paul,
> 
> On 1/02/2018 2:48 AM, Paul Sandoz wrote:
>>> On Jan 30, 2018, at 7:22 PM, David Holmes  wrote:
>>> On 31/01/2018 12:24 PM, Paul Sandoz wrote:
> On Jan 30, 2018, at 1:55 AM, David Holmes  > wrote:
>  MethodHandle API Changes:
> 
> - java/lang/invoke/MethodHandle.java
>   * A non-virtual method handle to a specific virtual method 
> implementation
>   * can also be created.  These do not perform virtual lookup based on
>   * receiver type.  Such a method handle simulates the effect of
> - * an {@code invokespecial} instruction to the same method.
> + * an {@code invokespecial} instruction to the same non-private method;
> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to 
> the
> + * same private method (as applicable).
> 
> I tried to clarify that non-virtual invocations are not limited to 
> invokespecial - as private invocations via invokevirtual or 
> invokeinterface are also non-virtual.
> 
> 
 Why s/same method/same non-private method/ for the invokespecial?
 It’s possible to look up a private method within the same class using 
 Lookup.invokespecial and invoke it (and also look up a private constructor 
 and invoke it).
>>> 
>>> Yes you are right, but this text is not trying to describe how an 
>>> invokespecial might be used, but rather how "A non-virtual method handle to 
>>> a specific virtual method implementation can also be created.”
>>> 
>> Ok, i see now.
>>> But the notion of "specific virtual method implementation" is perhaps not 
>>> applicable to private methods in the first place.
>>> 
>>> Perhaps I just need to remove this change altogether and leave it as-is.
>>> 
>> I would be inclined just to leave it as is.
> 
> I decided to restore the original text and then add the following - as I do 
> want to make it clear that non-virtual is not restricted to invokespecial:
> 
> * A non-virtual method handle can also be created to simulate the effect
> * of an {@code invokevirtual} or {@code invokeinterface} instruction on
> * a private method (as applicable).
> 
> I have updated specdiffs for all of the changes here:
> 
> http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang/overview-summary.html

If a class has no nestmates is it implicitly a nest of itself? (is that 
considered something valid?)

  Class c = … // some class with no nest mates
  assert c.getNestHost() == c
  assert c.isNestmateOf(c)
  assert Stream.of(c.getNestMembers()).anyMatch(_c -> _c == c);

?

If class nh is a nest host with nest mates (in addition to itself):

  Class nh = … // some class that is a nest host

  assert nh.getNestHost() == c
  assert nh.isNestmateOf(nh)
  assert Stream.of(nh.getNestMembers()).anyMatch(_c -> _c == nh);
  assert Stream.of(nh.getNestMembers()).allMatch(_c -> _c.isNestmakeOf(nh));
  assert Stream.of(nh.getNestMembers()).allMatch(
_c -> Stream.of(nh.getNestMembers()).allMatch(__c -> __c.isNestmakeOf(_c)));

?

getNestMembers
—

"The list of nest members in the classfile is permitted to contain duplicates, 
or to explicitly include the nest host. It is not required that an 
implementation of this method removes these duplicates."

The "or to explicitly include the nest host” suggests it might not include the 
nest host, but a prior statement says it will be present in the zeroth element.

Why the ambiguity over duplicates? is this motivated by performance? this may 
just push the cost to clients that have to always remove duplicates to function 
reliably and may be cause bugs if duplicates are rare and induced by certain 
relationships or loading patterns. My inclination would be for the returned 
array to not contain duplicates.


> http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/overview-summary.html
> http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/overview-summary.html
> 

Looks good to me.

Paul.

> Please ignore the spurious "version" changes.
> 
> Thanks,
> David
> -
> 
>> Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Jan 30, 2018, at 7:22 PM, David Holmes  wrote:
> 
> Hi Paul,
> 
> On 31/01/2018 12:24 PM, Paul Sandoz wrote:
>>> On Jan 30, 2018, at 1:55 AM, David Holmes >> > wrote:
>>> 
>>> 
>>>  MethodHandle API Changes:
>>> 
>>> - java/lang/invoke/MethodHandle.java
>>>   * A non-virtual method handle to a specific virtual method implementation
>>>   * can also be created.  These do not perform virtual lookup based on
>>>   * receiver type.  Such a method handle simulates the effect of
>>> - * an {@code invokespecial} instruction to the same method.
>>> + * an {@code invokespecial} instruction to the same non-private method;
>>> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to 
>>> the
>>> + * same private method (as applicable).
>>> 
>>> I tried to clarify that non-virtual invocations are not limited to 
>>> invokespecial - as private invocations via invokevirtual or invokeinterface 
>>> are also non-virtual.
>>> 
>>> 
>> Why s/same method/same non-private method/ for the invokespecial?
>> It’s possible to look up a private method within the same class using 
>> Lookup.invokespecial and invoke it (and also look up a private constructor 
>> and invoke it).
> 
> Yes you are right, but this text is not trying to describe how an 
> invokespecial might be used, but rather how "A non-virtual method handle to a 
> specific virtual method implementation can also be created.”
> 

Ok, i see now.


> But the notion of "specific virtual method implementation" is perhaps not 
> applicable to private methods in the first place.
> 
> Perhaps I just need to remove this change altogether and leave it as-is.
> 

I would be inclined just to leave it as is.

Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[David Holmes]

Hi Paul,

On 31/01/2018 12:24 PM, Paul Sandoz wrote:
On Jan 30, 2018, at 1:55 AM, David Holmes > wrote:



  MethodHandle API Changes:

- java/lang/invoke/MethodHandle.java
   * A non-virtual method handle to a specific virtual method implementation
   * can also be created.  These do not perform virtual lookup based on
   * receiver type.  Such a method handle simulates the effect of
- * an {@code invokespecial} instruction to the same method.
+ * an {@code invokespecial} instruction to the same non-private method;
+ * or an {@code invokevirtual} or {@code invokeinterface} instruction to the
+ * same private method (as applicable).

I tried to clarify that non-virtual invocations are not limited to 
invokespecial - as private invocations via invokevirtual or 
invokeinterface are also non-virtual.





Why s/same method/same non-private method/ for the invokespecial?

It’s possible to look up a private method within the same class using 
Lookup.invokespecial and invoke it (and also look up a private 
constructor and invoke it).


Yes you are right, but this text is not trying to describe how an 
invokespecial might be used, but rather how "A non-virtual method handle 
to a specific virtual method implementation can also be created."


But the notion of "specific virtual method implementation" is perhaps 
not applicable to private methods in the first place.


Perhaps I just need to remove this change altogether and leave it as-is.

Thanks,
David


Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Daniel Heidinga]

Thanks David.
 
These changes all seem reasonable.  I was going to complain about changing 'will occur' to 'may occur' as 'may' make makes it difficult to determine when it will vs won't occur but I think it's correct in this case.
 
--Dan
 
- Original message -From: David Holmes Sent by: "valhalla-spec-experts" To: valhalla-spec-experts@openjdk.java.netCc:Subject: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate accessDate: Tue, Jan 30, 2018 4:55 AM 
I've gone through the API specifications for core reflection, MethodHandles and VarHandles to see what changes are needed to accommodate nestmates and the related invocation rule changes. Turns out there is very little needed and most of what there is is non-normative, just correcting or clarifying particular things. For VarHandle nothing is needed as it defers all access checking to MethodHandle.Lookup.
Bug: https://bugs.openjdk.java.net/browse/JDK-8191116
Webrev: http://cr.openjdk.java.net/~dholmes/8191116/webrev/
Core reflection changes:
These are minimal as core reflection already expresses all access control in terms of "Java language access control" which already allows for nestmate access.
- java/lang/reflect/AccessibleObject.java
  *  Java language access control prevents use of private members outside- * their class; package access members outside their package; protected members+ * their top-level class; package access members outside their package; protected members
This corrects the definition of private access in the Java language.
-  java/lang/reflect/Method.java
  * using dynamic method lookup as documented in The Java Language- * Specification, Second Edition, section 15.12.4.4; in particular,- * overriding based on the runtime type of the target object will occur.+ * Specification, section 15.12.4.4; in particular,+ * overriding based on the runtime type of the target object may occur.
Removed unnecessary reference to "Second Edition". Changed 'will occur' to 'may occur' to account for the different forms of invocation that may apply.
MethodHandle API Changes:
- java/lang/invoke/MethodHandle.java
  * A non-virtual method handle to a specific virtual method implementation  * can also be created.  These do not perform virtual lookup based on  * receiver type.  Such a method handle simulates the effect of- * an {@code invokespecial} instruction to the same method.+ * an {@code invokespecial} instruction to the same non-private method;+ * or an {@code invokevirtual} or {@code invokeinterface} instruction to the+ * same private method (as applicable).
I tried to clarify that non-virtual invocations are not limited to invokespecial - as private invocations via invokevirtual or invokeinterface are also non-virtual.
- java/lang/invoke/MethodHandles.java      * -     * In some cases, access between nested classes is obtained by the Java compiler by creating-     * an wrapper method to access a private method of another class-     * in the same top-level declaration.+     * Since JDK 11 the relationship between nested types can be expressed directly through the+     * {@code NestHost} and {@code NestMembers} attributes.+     * (See the Java Virtual Machine Specification, sections 4.7.28 and 4.7.29.)+     * In that case, the lookup class has direct access to private members of all its nestmates, and+     * that is true of the associated {@code Lookup} object as well.+     * Otherwise, access between nested classes is obtained by the Java compiler creating+     * a wrapper method to access a private method of another class in the same nest.      * For example, a nested class {@code C.D}
Updated the nested classes description to cover legacy approach and new nestmate approach.
-     * {@code C.E} would be unable to those private members.+     * {@code C.E} would be unable to access those private members.
Fixed typo: "access" was missing.
      * Discussion of private access:      * We say that a lookup has private access      * if its {@linkplain #lookupModes lookup modes}-     * include the possibility of accessing {@code private} members.+     * include the possibility of accessing {@code private} members+     * (which includes the private members of nestmates).      * As documented in the relevant methods elsewhere,      * only lookups with private access possess the following capabilities:      * -     * access private fields, methods, and constructors of the lookup class+     * access private fields, methods, and constructors of the lookup class and its nestmates
Clarify that private access includes nestmate access.
-     *  access all members of the caller's class, all public types in the caller's module,+     *  access all members of the caller's class and nestmates, all public types in the caller's module,
Ditto.
      * When called, the handle will treat the first argument as a receiver-     * and 

API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[David Holmes]

I've gone through the API specifications for core reflection, 
MethodHandles and VarHandles to see what changes are needed to 
accommodate nestmates and the related invocation rule changes. Turns out 
there is very little needed and most of what there is is non-normative, 
just correcting or clarifying particular things. For VarHandle nothing 
is needed as it defers all access checking to MethodHandle.Lookup.


Bug: https://bugs.openjdk.java.net/browse/JDK-8191116

Webrev: http://cr.openjdk.java.net/~dholmes/8191116/webrev/


 Core reflection changes:

These are minimal as core reflection already expresses all access 
control in terms of "Java language access control" which already allows 
for nestmate access.


- java/lang/reflect/AccessibleObject.java

  *  Java language access control prevents use of private members outside
- * their class; package access members outside their package; protected members
+ * their top-level class; package access members outside their package; 
protected members

This corrects the definition of private access in the Java language.

- java/lang/reflect/Method.java

  * using dynamic method lookup as documented in The Java Language
- * Specification, Second Edition, section 15.12.4.4; in particular,
- * overriding based on the runtime type of the target object will occur.
+ * Specification, section 15.12.4.4; in particular,
+ * overriding based on the runtime type of the target object may occur.

Removed unnecessary reference to "Second Edition". Changed 'will occur' 
to 'may occur' to account for the different forms of invocation that may 
apply.



 MethodHandle API Changes:

- java/lang/invoke/MethodHandle.java

  * A non-virtual method handle to a specific virtual method implementation
  * can also be created.  These do not perform virtual lookup based on
  * receiver type.  Such a method handle simulates the effect of
- * an {@code invokespecial} instruction to the same method.
+ * an {@code invokespecial} instruction to the same non-private method;
+ * or an {@code invokevirtual} or {@code invokeinterface} instruction to the
+ * same private method (as applicable).

I tried to clarify that non-virtual invocations are not limited to 
invokespecial - as private invocations via invokevirtual or 
invokeinterface are also non-virtual.


- java/lang/invoke/MethodHandles.java

  * 
- * In some cases, access between nested classes is obtained by the Java 
compiler by creating
- * an wrapper method to access a private method of another class
- * in the same top-level declaration.
+ * Since JDK 11 the relationship between nested types can be expressed 
directly through the
+ * {@code NestHost} and {@code NestMembers} attributes.
+ * (See the Java Virtual Machine Specification, sections 4.7.28 and 
4.7.29.)
+ * In that case, the lookup class has direct access to private members of 
all its nestmates, and
+ * that is true of the associated {@code Lookup} object as well.
+ * Otherwise, access between nested classes is obtained by the Java 
compiler creating
+ * a wrapper method to access a private method of another class in the 
same nest.
  * For example, a nested class {@code C.D}

Updated the nested classes description to cover legacy approach and new 
nestmate approach.


- * {@code C.E} would be unable to those private members.
+ * {@code C.E} would be unable to access those private members.

Fixed typo: "access" was missing.

  * Discussion of private access:
  * We say that a lookup has private access
  * if its {@linkplain #lookupModes lookup modes}
- * include the possibility of accessing {@code private} members.
+ * include the possibility of accessing {@code private} members
+ * (which includes the private members of nestmates).
  * As documented in the relevant methods elsewhere,
  * only lookups with private access possess the following capabilities:
  * 
- * access private fields, methods, and constructors of the lookup class
+ * access private fields, methods, and constructors of the lookup 
class and its nestmates

Clarify that private access includes nestmate access.

- *  access all members of the caller's class, all public types in the 
caller's module,
+ *  access all members of the caller's class and nestmates, all public 
types in the caller's module,

Ditto.

  * When called, the handle will treat the first argument as a receiver
- * and dispatch on the receiver's type to determine which method
+ * and, for non-private methods, dispatch on the receiver's type to 
determine which method
  * implementation to enter.
+ * For private methods the named method in {@code refc} will be invoked on 
the receiver.

Clarify dispatch process for private versus non-private.

  * @throws IllegalAccessException if access checking fails,
  *or if the method is {@code static},
- *