Re: Valhalla EG minutes 6/21/17

[John Rose]

On Jul 19, 2017, at 7:31 AM, Daniel Heidinga  wrote:
> 
> There are a couple of different terms that have been used to describe early 
> access features - incubator (jep 11), experiment, or optional.  At least for 
> me, these different terms result in different mental models for how this 
> should work for the VM.  Hopefully, we're all thinking about the same 
> behaviour semantics and are just using different terms but I think it's worth 
> clarifying.
>  
> 1) JEP 11 does a good job of describing incubator and makes it clear that it 
> refers to non-standard modules / java-level code.  Not VM features.

And what we want is "like JDK incubator code but for VM features", but defined 
rigorously.

> 2) An optional feature is something you can chose to implement (or not) but 
> must exist as part of the current JVM spec.  All of its JVM spec changes 
> would need to be in the spec and their enablement / enforcement would depend 
> on whether the VM chose to implement that optional feature.  Removal of the 
> optional feature from the JVM spec would basically be impossible - it may 
> fall into disuse through VMs not implementing it, but it would be hard to 
> remove.

Yes.  As with incubator code we include a proviso that the experimental 
features will be dropped.

> 3) An experimental feature on the other hand, is something that would be 
> allowed to have an appendix or set of experimental JVM spec changes that are 
> only enabled by command line option + present in classes with new minor 
> classfile versions.  These features would start life deprecated and 
> expectation is that classfiles with this particular minor version would no 
> longer be recognized by future VMs when the experimental features graduate to 
> the real JVM features, providing freedom to experiment without requiring the 
> VM to carry the support burden long term.

That sounds right.  We need features which customers can try out but which have 
a short shelf life.

Short means on the order of a single release and/or a year or two.  Shorter is 
better.

> My understanding has been that with the MVT prototype work, we've been aiming 
> for the 3rd case.  Does this match everyone's expectations?  Anyone think 
> we're aiming for some other point on the spectrum?

That's what I expect.


> One of my fears is that we're going to end up with the VM required to support 
> multiple ways of recognizing ValueCapableClasses / ValueTypes, especially if 
> there are spec changes between the different ways (think the mess that is 
> invokespecial and the ACC_Super flag) based on attribute vs annotation or 
> classfile version, etc.

Yes.

>  
> ...> - We could redesign this so that the VCC properties are checked during 
> loading/verification of Foo. I am concerned that, where Foo is version 54.0 
> and has attribute ValueCapableClass, this sort of error checking will violate 
> the Java SE 10 spec.
>  
> Doesn't the use of experimental features allow the new classfile versions to 
> define new behaviour?  I would expect the ValueCapableClass attribute to be 
> ignored in a v.54 classfile and only take affect in a v.54.1 classfile so 
> that the semantics can be changed in the future.

The VCC is a special case, because it must be compiled from standard source 
with no changes to javac.
Its class file doesn't directly use the new bytecodes (vload, etc.) or 
modifiers (ACC_VALUE).
This feature must be triggered by an annotation, not an attribute, nor a 
classfile version bit.

For the actual bytecodes (vload, etc.) a classfile version bit or attribute is 
the sort of thing
we need to enable the experimental feature.

> > 
> > Elaborating: we're presenting values as an optional feature of Java SE 10. 
> > For a JVM that does not implement the optional feature, JVMS 10 says that a 
> > ValueCapableClass attribute on a version 54.0 class file will be ignored. 
> > JVMS 4.7.1: "any attribute not defined as part of the class file 
> > specification must not affect the semantics of the class file. Java Virtual 
> > Machine implementations are required to silently ignore attributes they do 
> > not recognize." My interpretation of our mission, in adding an optional 
> > feature, is to provide new capabilities while having no impact on existing 
> > behavior. We can do new things where JVMS 10 specifies an error; we can't 
> > generate new errors where JVMS 10 specifies none.
>  
> New classfile version == new behaviour, right?

(Or new class file version + attribute.  Or new class file version + class file 
minor-version bit.  But yes.)

> 
> > 
> > - We could limit usage/interpretation of "ValueCapableClass" to 54.1 class 
> > files. Then eager error checks when loading Foo would be fine. But the 
> > ability to work with 54.0 ValueCapableClass class files is an important use 
> > case.
>  
> The model here is that using a JVM 10 javac, a classfile with the annotation 
> is v54 and on a VM with some 

Re: Valhalla EG minutes 6/21/17

[Remi Forax]

> De: "Daniel Heidinga" <daniel_heidi...@ca.ibm.com>
> À: "daniel smith" <daniel.sm...@oracle.com>
> Cc: "valhalla-spec-experts" <valhalla-spec-experts@openjdk.java.net>
> Envoyé: Mercredi 19 Juillet 2017 16:31:18
> Objet: Re: Valhalla EG minutes 6/21/17

> There are a couple of different terms that have been used to describe early
> access features - incubator (jep 11), experiment, or optional. At least for 
> me,
> these different terms result in different mental models for how this should
> work for the VM. Hopefully, we're all thinking about the same behaviour
> semantics and are just using different terms but I think it's worth 
> clarifying.
> 1) JEP 11 does a good job of describing incubator and makes it clear that it
> refers to non-standard modules / java-level code. Not VM features.
> 2) An optional feature is something you can chose to implement (or not) but 
> must
> exist as part of the current JVM spec. All of its JVM spec changes would need
> to be in the spec and their enablement / enforcement would depend on whether
> the VM chose to implement that optional feature. Removal of the optional
> feature from the JVM spec would basically be impossible - it may fall into
> disuse through VMs not implementing it, but it would be hard to remove.
> 3) An experimental feature on the other hand, is something that would be 
> allowed
> to have an appendix or set of experimental JVM spec changes that are only
> enabled by command line option + present in classes with new minor classfile
> versions. These features would start life deprecated and expectation is that
> classfiles with this particular minor version would no longer be recognized by
> future VMs when the experimental features graduate to the real JVM features,
> providing freedom to experiment without requiring the VM to carry the support
> burden long term.
> My understanding has been that with the MVT prototype work, we've been aiming
> for the 3rd case. Does this match everyone's expectations? Anyone think we're
> aiming for some other point on the spectrum?

I agree, it's an experimental feature, like invokedynamic was at its time, at 
least for the MVT. 

> One of my fears is that we're going to end up with the VM required to support
> multiple ways of recognizing ValueCapableClasses / ValueTypes, especially if
> there are spec changes between the different ways (think the mess that is
> invokespecial and the ACC_Super flag) based on attribute vs annotation or
> classfile version, etc.

Rémi 

> >> On Jul 5, 2017, at 8:12 AM, Karen Kinnear <karen.kinn...@oracle.com> wrote:

> >>> > Dan S: class loading in the proposed JVMS: if you see $Value
> >>> > 1) first derive the VCC name and see if already resolved
> >>> > 2) if not - load the VCC, check properties and derive
> >>> > (ed. note - if see VCC - lazily derive derived value class on touch)

> >>> It's not a requirement that the value class derivation is lazy, correct?
> >> Let’s double-check with Dan Smith at today’s meeting.

> >> The way I read 5.3 Creation and Loading in
>>> [ http://cr.openjdk.java.net/~dlsmith/values.html |
> >> http://cr.openjdk.java.net/~dlsmith/values.html ]
>>> it appears to allow lazy derivation as well as eager derivation, which I 
>>> think
> >> is what we both want
> >> since it allows implementations to optimize.
> >> Our current derivation is also eager.

>> Summary of today's discussion, supplemented with some reflection on what I 
>> see
> > as the requirements:

>> - The specification shouldn't care when the class is derived (though it must
>> occur, naturally, no later than resolution of "Foo$Value"); the specification
> > *might* care when error checking occurs, because that's observable.

>> - Current specification draft says error checking occurs when Foo$Value is
>> loaded (5.3), and that "Class or interface creation is triggered by another
>> class or interface D, which references C through its run-time constant pool."
>> So, as a test case: if my program has no references to Foo$Value (direct or
> > reflective), no VCC-related errors would occur.

>> - We could redesign this so that the VCC properties are checked during
>> loading/verification of Foo. I am concerned that, where Foo is version 54.0 
>> and
>> has attribute ValueCapableClass, this sort of error checking will violate the
> > Java SE 10 spec.
> Doesn't the use of experimental features allow the new classfile versions to
> define new behaviour? I would expect the ValueCapableClass attribute to b e
> ignored in a v.54 classfile and 

off-list Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

Hi,

Off-list as i am not sure i wanna commit to this publicly :-)

In the interest of moving this forward independent of MVT i can prototype some 
of this if you like and use the three usages of U.DAC in the JDK as use-cases 
(see also [1]) Perhaps surprisingly grepcode.com reports no external usages of 
U.DAC.

Some use-cases for U.DAC could be replaced by an isolated method feature, which 
makes me wonder if that is really the actual public feature we really want 
rather than anon/nameless classes?

My gut feeling is 2) can be implemented straightforwardly on top of Unsafe.DAC 
with appropriate checks, if we are ok with the current HS implementation at 
least as a starting point. To me the tricky bit is the spec gymnastics, it 
would have to be subset of the eventual behaviour and that can only expand in 
scope with refinement of terms. Is that a reasonably accurate assessment?

I believe we can also prototype the constant pool patching in an indirect 
manner as indicated by John and Remi.

That leaves the more general nested named class support which requires more 
fundamental hotspot work.

So in terms of progress order it could be: 2), 3), 1). But if we throw isolated 
methods into the mix then the plot thickens.

Paul.

[1]
https://bugs.openjdk.java.net/browse/JDK-8078602
Support j.l.i.BoundMethodHandle$Species_* classes unloading
https://bugs.openjdk.java.net/browse/JDK-8168848
Too many anonymous classes that aren't unloaded


> On 11 Jul 2017, at 14:42, Karen Kinnear  wrote:
> 
> Paul,
> 
> We are working hard on getting the nest mates requirements clarified.
> I would like to use that to support the Lookup.defineClass and not do a 
> Quick
> in advance for MVT . I think we should stick with the reduced restrictions
> for withfield for early access. I think we should put our energy into getting
> nest mates and Lookup.defineClass ready.
> 
> We have three parts to Lookup.defineClass.
> 1) PRIVATE mode handling - which we believe we can support with nest mates
> - we do not want to do any versions of this based on unsafe.DAC current 
> behavior,
> - we are looking at cleaner behavior for nestmates
> 2) “temporary” name - i.e. find doesn’t work
> 3) CP patching/user data
> 
> My mental model is that we don’t have to add all of these at the same time,
> although we will want user feedback on when to remove 
> unsafe.DefineAnonymousClass.
> In fact, we would love more examples of current use cases, to help guide our
> design.
> 
> thanks,
> Karen
> 
>> On Jul 5, 2017, at 2:43 PM, John Rose  wrote:
>> 
>> On Jul 5, 2017, at 11:39 AM, Paul Sandoz  wrote:
>>> 
>>> 
>>> I was unsure if we require a new method L.defineAnonClass or could leverage 
>>> the existing L.defineClass. IIUC for expediency the current hooks in the VM 
>>> lean towards anon classes, since there is already code to defer to the host 
>>> class, whereas the general defineClass case will likely require more work, 
>>> although i can potentially see some short cuts if we focus on VCC/DVT.
>>> 
>> 
>> Good point.  Yes, that part isn't designed yet.  It may manifest as an extra 
>> argument or two to the L.dC.
>> 
>> At least two degrees of freedom apply there:  1. suppressing the name (no 
>> system dictionary update),
>> 2. providing some sort of "user data" to bind to the class (can be a single 
>> ref.,  replaces CP patching).
>> 
>> For MVT we can just generate a throwaway name, like DVT.getName()+":29348".
>> 
>> — John
> 

Re: Valhalla EG minutes 6/21/17

[Karen Kinnear]

Paul,

We are working hard on getting the nest mates requirements clarified.
I would like to use that to support the Lookup.defineClass and not do a 
Quick
in advance for MVT . I think we should stick with the reduced restrictions
for withfield for early access. I think we should put our energy into getting
nest mates and Lookup.defineClass ready.

We have three parts to Lookup.defineClass.
1) PRIVATE mode handling - which we believe we can support with nest mates
- we do not want to do any versions of this based on unsafe.DAC current 
behavior,
- we are looking at cleaner behavior for nestmates
2) “temporary” name - i.e. find doesn’t work
3) CP patching/user data

My mental model is that we don’t have to add all of these at the same time,
although we will want user feedback on when to remove 
unsafe.DefineAnonymousClass.
In fact, we would love more examples of current use cases, to help guide our
design.

thanks,
Karen

> On Jul 5, 2017, at 2:43 PM, John Rose  wrote:
> 
> On Jul 5, 2017, at 11:39 AM, Paul Sandoz  > wrote:
>> 
>> 
>> I was unsure if we require a new method L.defineAnonClass or could leverage 
>> the existing L.defineClass. IIUC for expediency the current hooks in the VM 
>> lean towards anon classes, since there is already code to defer to the host 
>> class, whereas the general defineClass case will likely require more work, 
>> although i can potentially see some short cuts if we focus on VCC/DVT.
>> 
> 
> Good point.  Yes, that part isn't designed yet.  It may manifest as an extra 
> argument or two to the L.dC.
> 
> At least two degrees of freedom apply there:  1. suppressing the name (no 
> system dictionary update),
> 2. providing some sort of "user data" to bind to the class (can be a single 
> ref.,  replaces CP patching).
> 
> For MVT we can just generate a throwaway name, like DVT.getName()+":29348".
> 
> — John

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

> On 6 Jul 2017, at 15:58, John Rose  wrote:
> 
> On Jul 6, 2017, at 1:02 PM, Paul Sandoz  > wrote:
>> 
>> In terms of what we have today we could easily do:
>> 
>>  // lookup must have private access to the lookup class, which becomes the 
>> “host” class
>>  Class defineAnonymousClass(byte[] data)
>> 
>> is that ending gaining too much?
> 
> Sure, that's OK, or else an 'isAnonymous' optional argument.
> The difficult part here is specifying exactly what is a "host class".
> 

Agreed, that’s the tricky bit.


>> That still leaves the possibility of another method in the future say:
>> 
>>  Class defineClass(boolean isAnon, byte[] data, Object constant)
>> 
>> That’s a little fuzzy since it’s not clear to me how the generated class 
>> locates the constant (synthetic static final field of known name? substitute 
>> the last entry in the CP if appropriately defined in the class bytes as 
>> substitutable?).
> 
> On Jul 6, 2017, at 1:12 PM, Remi Forax  > wrote:
>> 
>> Lookup.getConstant() with a private Lookup ?
> 
> Exactly, an ad hoc API point like that.  Would fit nicely with a BSM and
> CONSTANT_ConstantDynamic and ldc.  Or a private static final and
> code in .

I like that.

Perhaps those associated constants could be squirrelled away in a ClassValue, 
and any class could participate and receive an ad-hoc constant.


> 
> If you need several of them, a Map would be your
> friend.  This doesn't need to be baked into the Lookup API, just a
> design pattern, supported nicely by a slightly different BSM.
> 

And there could be a condy for extracting a value for a given key from the Map 
constant.

Paul.

Re: Valhalla EG minutes 6/21/17

[John Rose]

On Jul 6, 2017, at 1:02 PM, Paul Sandoz  wrote:
> 
> In terms of what we have today we could easily do:
> 
>  // lookup must have private access to the lookup class, which becomes the 
> “host” class
>  Class defineAnonymousClass(byte[] data)
> 
> is that ending gaining too much?

Sure, that's OK, or else an 'isAnonymous' optional argument.
The difficult part here is specifying exactly what is a "host class".

> That still leaves the possibility of another method in the future say:
> 
>  Class defineClass(boolean isAnon, byte[] data, Object constant)
> 
> That’s a little fuzzy since it’s not clear to me how the generated class 
> locates the constant (synthetic static final field of known name? substitute 
> the last entry in the CP if appropriately defined in the class bytes as 
> substitutable?).

On Jul 6, 2017, at 1:12 PM, Remi Forax  wrote:
> 
> Lookup.getConstant() with a private Lookup ?

Exactly, an ad hoc API point like that.  Would fit nicely with a BSM and
CONSTANT_ConstantDynamic and ldc.  Or a private static final and
code in .

If you need several of them, a Map would be your
friend.  This doesn't need to be baked into the Lookup API, just a
design pattern, supported nicely by a slightly different BSM.

— John

Re: Valhalla EG minutes 6/21/17

[Remi Forax]

- Mail original -
> De: "Paul Sandoz" <paul.san...@oracle.com>
> À: "John Rose" <john.r.r...@oracle.com>
> Cc: valhalla-spec-experts@openjdk.java.net
> Envoyé: Jeudi 6 Juillet 2017 22:02:39
> Objet: Re: Valhalla EG minutes 6/21/17

> In terms of what we have today we could easily do:
> 
>  // lookup must have private access to the lookup class, which becomes the 
> “host”
>  class
>  Class defineAnonymousClass(byte[] data)
> 
> is that ending gaining too much?
> 
> That still leaves the possibility of another method in the future say:
> 
>  Class defineClass(boolean isAnon, byte[] data, Object constant)
> 
> That’s a little fuzzy since it’s not clear to me how the generated class 
> locates
> the constant (synthetic static final field of known name? substitute the last
> entry in the CP if appropriately defined in the class bytes as 
> substitutable?).

Lookup.getConstant() with a private Lookup ?

> 
> Paul.

Rémi

> 
>> On 5 Jul 2017, at 11:43, John Rose <john.r.r...@oracle.com> wrote:
>> 
>> On Jul 5, 2017, at 11:39 AM, Paul Sandoz <paul.san...@oracle.com> wrote:
>>> 
>>> 
>>> I was unsure if we require a new method L.defineAnonClass or could leverage 
>>> the
>>> existing L.defineClass. IIUC for expediency the current hooks in the VM lean
>>> towards anon classes, since there is already code to defer to the host 
>>> class,
>>> whereas the general defineClass case will likely require more work, 
>>> although i
>>> can potentially see some short cuts if we focus on VCC/DVT.
>>> 
>> 
>> Good point.  Yes, that part isn't designed yet.  It may manifest as an extra
>> argument or two to the L.dC.
>> 
>> At least two degrees of freedom apply there:  1. suppressing the name (no 
>> system
>> dictionary update),
>> 2. providing some sort of "user data" to bind to the class (can be a single
>> ref.,  replaces CP patching).
>> 
>> For MVT we can just generate a throwaway name, like DVT.getName()+":29348".
>> 
> > — John

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

In terms of what we have today we could easily do:

  // lookup must have private access to the lookup class, which becomes the 
“host” class
  Class defineAnonymousClass(byte[] data)

is that ending gaining too much?

That still leaves the possibility of another method in the future say:

  Class defineClass(boolean isAnon, byte[] data, Object constant)

That’s a little fuzzy since it’s not clear to me how the generated class 
locates the constant (synthetic static final field of known name? substitute 
the last entry in the CP if appropriately defined in the class bytes as 
substitutable?).

Paul.

> On 5 Jul 2017, at 11:43, John Rose  wrote:
> 
> On Jul 5, 2017, at 11:39 AM, Paul Sandoz  wrote:
>> 
>> 
>> I was unsure if we require a new method L.defineAnonClass or could leverage 
>> the existing L.defineClass. IIUC for expediency the current hooks in the VM 
>> lean towards anon classes, since there is already code to defer to the host 
>> class, whereas the general defineClass case will likely require more work, 
>> although i can potentially see some short cuts if we focus on VCC/DVT.
>> 
> 
> Good point.  Yes, that part isn't designed yet.  It may manifest as an extra 
> argument or two to the L.dC.
> 
> At least two degrees of freedom apply there:  1. suppressing the name (no 
> system dictionary update),
> 2. providing some sort of "user data" to bind to the class (can be a single 
> ref.,  replaces CP patching).
> 
> For MVT we can just generate a throwaway name, like DVT.getName()+":29348".
> 
> — John

Re: Valhalla EG minutes 6/21/17

[John Rose]

On Jul 5, 2017, at 12:13 PM, Dan Smith  wrote:
> Summary of today's discussion, supplemented with some reflection on what I 
> see as the requirements:

Thanks!

> - The specification shouldn't care when the class is derived (though it must 
> occur, naturally, no later than resolution of "Foo$Value"); the specification 
> *might* care when error checking occurs, because that's observable.

Even more specifically:  "when errors are reported".  Checking can happen long 
before reporting.

> - Current specification draft says error checking occurs when Foo$Value is 
> loaded (5.3), and that "Class or interface creation is triggered by another 
> class or interface D, which references C through its run-time constant pool." 
> So, as a test case: if my program has no references to Foo$Value (direct or 
> reflective), no VCC-related errors would occur.

I would prefer this in MVT, but it's not a showstopper.

In the long term this might be a good move, relative to operations that resolve 
Q-Foo (not Foo$Value), if Foo is supposed to be "merely an object" until 
suddenly its value aspect comes into play.  In the usual case where some 
ACC_VALUE bit is present in the class file, there is no cause to delay error 
processing.

> - We could redesign this so that the VCC properties are checked during 
> loading/verification of Foo. I am concerned that, where Foo is version 54.0 
> and has attribute ValueCapableClass, this sort of error checking will violate 
> the Java SE 10 spec.
> 
> Elaborating: we're presenting values as an optional feature of Java SE 10. 
> For a JVM that does not implement the optional feature, JVMS 10 says that a 
> ValueCapableClass attribute on a version 54.0 class file will be ignored. 
> JVMS 4.7.1: "any attribute not defined as part of the class file 
> specification must not affect the semantics of the class file. Java Virtual 
> Machine implementations are required to silently ignore attributes they do 
> not recognize." My interpretation of our mission, in adding an optional 
> feature, is to provide new capabilities while having no impact on existing 
> behavior. We can do new things where JVMS 10 specifies an error; we can't 
> generate new errors where JVMS 10 specifies none.

This is a good point.  It's the right way to manage the spec.

OTOH, it's hard to construct examples where this matters for MVT.  If I forget 
to use a VM that supports DVTs (or forget to flip the On switch), I am 
practically certain to get an error if I run my example MVT code.  The error 
should occur as soon as the DVT is extracted.  But it might also occur if the 
JVM falls over (due to some bug) when it loads the VCC and asks itself a 
premature question.  Practically speaking, I will never expect (in MVT) to use 
a VCC as a POJO with DVTs turned off.

> 
> - We could limit usage/interpretation of "ValueCapableClass" to 54.1 class 
> files. Then eager error checks when loading Foo would be fine. But the 
> ability to work with 54.0 ValueCapableClass class files is an important use 
> case.

Confirmed.  This is why "Shady" speaks of mainly of an annotation to enable 
VCCs.  (Over-cautiously, it also says "perhaps an attribute".)  I should have 
mentioned this earlier.

> - Actually, examining that use case more closely, we discussed how attributes 
> are not the right tool at all. We want clients of MVT to be able to generate 
> interesting programs using a vanilla Java SE 10 compiler. The best mechanism 
> to communicate "I'm a value-capable class" using a vanilla Java compiler is 
> annotations, not attributes.
> 
> Tentatively, the spec should be revised so that:
> - The "ValueCapableClass" attribute no longer exists. Delete changes to 4.7 
> and delete new section 4.7.25.
> - A "value capable class" is a class whose RuntimeVisibleAnnotations includes 
> a ValueCapableClass annotation (package TBD; whether the referenced class 
> must be loaded TBD)
> - As usual, this annotation doesn't impact behavior of class Foo
> - When loading Foo$Value, as specified in 5.3, we'll check that Foo is a 
> value capable class and that it has the right properties

Also:
- The annotation is ignored if the VCC does not have permission to access the 
annotation.  (It's filtered like @Contended, which is not in the spec.)

This check is complex to implement in general, since it interacts with the 
bootstrapping of the module system runtime.  But in fact it only needs to work 
for the specific @ValueCapableClass annotation.  That means a hardwired check 
to is likely to work, by embedded into the JVM which modules have access to the 
@VCC annotation.  For @Contended it works because the JVM "just knows" that 
@Contended lives in an unexported package of java.base.

— John

Re: Valhalla EG minutes 6/21/17

[Dan Smith]

> On Jul 5, 2017, at 8:12 AM, Karen Kinnear  wrote:
> 
>> > Dan S: class loading in the proposed JVMS: if you see $Value
>> >1) first derive the VCC name and see if already resolved
>> >2) if not - load the VCC, check properties and derive
>> >(ed. note - if see VCC - lazily derive derived value class on touch)
>>  
>> It's not a requirement that the value class derivation is lazy, correct?
> Let’s double-check with Dan Smith at today’s meeting.
> 
> The way I read 5.3 Creation and Loading in
> http://cr.openjdk.java.net/~dlsmith/values.html
> it appears to allow lazy derivation as well as eager derivation, which I 
> think is what we both want
> since it allows implementations to optimize.
> Our current derivation is also eager.

Summary of today's discussion, supplemented with some reflection on what I see 
as the requirements:

- The specification shouldn't care when the class is derived (though it must 
occur, naturally, no later than resolution of "Foo$Value"); the specification 
*might* care when error checking occurs, because that's observable.

- Current specification draft says error checking occurs when Foo$Value is 
loaded (5.3), and that "Class or interface creation is triggered by another 
class or interface D, which references C through its run-time constant pool." 
So, as a test case: if my program has no references to Foo$Value (direct or 
reflective), no VCC-related errors would occur.

- We could redesign this so that the VCC properties are checked during 
loading/verification of Foo. I am concerned that, where Foo is version 54.0 and 
has attribute ValueCapableClass, this sort of error checking will violate the 
Java SE 10 spec.

Elaborating: we're presenting values as an optional feature of Java SE 10. For 
a JVM that does not implement the optional feature, JVMS 10 says that a 
ValueCapableClass attribute on a version 54.0 class file will be ignored. JVMS 
4.7.1: "any attribute not defined as part of the class file specification must 
not affect the semantics of the class file. Java Virtual Machine 
implementations are required to silently ignore attributes they do not 
recognize." My interpretation of our mission, in adding an optional feature, is 
to provide new capabilities while having no impact on existing behavior. We can 
do new things where JVMS 10 specifies an error; we can't generate new errors 
where JVMS 10 specifies none.

- We could limit usage/interpretation of "ValueCapableClass" to 54.1 class 
files. Then eager error checks when loading Foo would be fine. But the ability 
to work with 54.0 ValueCapableClass class files is an important use case.

- Actually, examining that use case more closely, we discussed how attributes 
are not the right tool at all. We want clients of MVT to be able to generate 
interesting programs using a vanilla Java SE 10 compiler. The best mechanism to 
communicate "I'm a value-capable class" using a vanilla Java compiler is 
annotations, not attributes.

Tentatively, the spec should be revised so that:
- The "ValueCapableClass" attribute no longer exists. Delete changes to 4.7 and 
delete new section 4.7.25.
- A "value capable class" is a class whose RuntimeVisibleAnnotations includes a 
ValueCapableClass annotation (package TBD; whether the referenced class must be 
loaded TBD)
- As usual, this annotation doesn't impact behavior of class Foo
- When loading Foo$Value, as specified in 5.3, we'll check that Foo is a value 
capable class and that it has the right properties

—Dan

Re: Valhalla EG minutes 6/21/17

[John Rose]

On Jul 5, 2017, at 11:39 AM, Paul Sandoz  wrote:
> 
> 
> I was unsure if we require a new method L.defineAnonClass or could leverage 
> the existing L.defineClass. IIUC for expediency the current hooks in the VM 
> lean towards anon classes, since there is already code to defer to the host 
> class, whereas the general defineClass case will likely require more work, 
> although i can potentially see some short cuts if we focus on VCC/DVT.
> 

Good point.  Yes, that part isn't designed yet.  It may manifest as an extra 
argument or two to the L.dC.

At least two degrees of freedom apply there:  1. suppressing the name (no 
system dictionary update),
2. providing some sort of "user data" to bind to the class (can be a single 
ref.,  replaces CP patching).

For MVT we can just generate a throwaway name, like DVT.getName()+":29348".

— John

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

> On 5 Jul 2017, at 11:06, John Rose  wrote:
> 
> On Jul 5, 2017, at 10:59 AM, Paul Sandoz  > wrote:
>> 
>> I strongly suspect we can specify a safe version of 
>> Unsafe.defineAnonymousClass (minus constant pool patching) independent of 
>> nest mates. I believe that is desirable on it’s own as part of the replace 
>> unsafe functionality and if that can help MVT then even better!
> 
> We have an API point already for this, with a pre-planned "hole" for the 
> functionality you want.
> 
> If the VCC says "L=MethodHandles.lookup().in(DVT.class)" the Lookup should 
> retain the PRIVATE access mode.
> 

Yes.


> Then, L.defineClass(...) is specified to throw an exception when it sees the 
> PRIVATE access mode,

Right, i was just looking at that, currently scoped for package access.


> but what is supposed to happen in the future is that the newly defined class 
> is inserted into the nest of the Lookup.lookupClass.
> 

I was unsure if we require a new method L.defineAnonClass or could leverage the 
existing L.defineClass. IIUC for expediency the current hooks in the VM lean 
towards anon classes, since there is already code to defer to the host class, 
whereas the general defineClass case will likely require more work, although i 
can potentially see some short cuts if we focus on VCC/DVT.

Paul.

> In the very specific case of a DVT, it would be reasonable to allow the 
> injection you want, Paul, without throwing the required exception.  After 
> all, the DVT's are going away, so there's no compatibility risk for future 
> changes to the API.
> 
> The challenge to provide this for general use is to specify what happens when 
> the Lookup has PRIVATE access mode.  The best way to spec. this (IMO) is as a 
> forcible injection into the nest of the Lookup.lookupClass.  That requires 
> nestmates to be specified *first*, and *then* we can specify what it means to 
> inject a new (dynamically defined) nestmate.
> 
> — John

Re: Valhalla EG minutes 6/21/17

[Karen Kinnear]

Agree with John’s clarification - yes we are planning longterm for nest mate 
access.
And your proposal of using a safe replacement for Unsafe.defineAnonymousClass 
with
appropriate access to add into the nest makes sense.

At this time we are building an Early Access that needs to go out sooner than 
nest mates.
We should re-evaluate adding a nest mate dependency when we get closer to 
deadlines and
see if that works for our partners and our own timing.

thanks,
Karen

> On Jul 5, 2017, at 12:28 PM, Paul Sandoz  wrote:
> 
> 
>> On 5 Jul 2017, at 07:26, Karen Kinnear  wrote:
>> 
>> Paul,
>> 
>> What we were discussing was the ability to use the byte code itself - not 
>> the ValueType.findWither API.
>> John’s longer term plan is that ultimately the byte code can only be 
>> executed in the value class itself, and
>> since the derived value class has no methods, we need a temporary approach.
>> 
>> Did I misunderstand what you were saying?
>> 
> 
> No, i was missing aspect that you were referring to byte code generation.
> 
> What if we added a safe replacement for:
> 
>  Unsafe.defineAnonymousClass
>  https://bugs.openjdk.java.net/browse/JDK-8171335
> 
> (Which we anyway have to do.)
> 
> ?

> 
> Then the VCC or DVT could be used as the host class. However, i dunno if that 
> would be sufficient to cover the use-cases of byte code generation.

> 
> Paul.
> 
>> thanks,
>> Karen
>> 
>>> On Jun 26, 2017, at 2:52 PM, Paul Sandoz  wrote:
>>> 
>>> 
 On 23 Jun 2017, at 13:33, Karen Kinnear  wrote:
>>> 
 VWithfield - propose for MVT - allow package private access - since there 
 are no methods on the derived value class
 and the value capable class can’t have any methods with vbytecodes since 
 generated by javac
 - plan to make private when we add factory methods to value classes with a 
 compiler (and we have nest support)
 
>>> 
>>> I am unsure if it’s necessary for MVT purposes to dial back the 
>>> accessibility then dial it up again later on.
>>> 
>>> ValueType.findWither can be used in conjunction with 
>>> MethodHandle.privateLookupIn. It’s a little odd but works. What am i 
>>> missing?
>>> 
>>> Paul.
>>> 
>>> 
>>> 
>>> 
>> 
> 

Re: Valhalla EG minutes 6/21/17

[John Rose]

On Jul 5, 2017, at 7:26 AM, Karen Kinnear  wrote:
> 
> John’s longer term plan is that ultimately the byte code can only be executed 
> in the value class itself

Slight correction:  ultimately the byte code requires private access to the 
value class itself.
This means that nestmates can use that bytecode also.  Access rules are as if 
it were a private method.
We don't want to require bridge methods to use withers within a compilation 
unit.

Re: Valhalla EG minutes 6/21/17

[Karen Kinnear]

Paul,

What we were discussing was the ability to use the byte code itself - not the 
ValueType.findWither API.
John’s longer term plan is that ultimately the byte code can only be executed 
in the value class itself, and
since the derived value class has no methods, we need a temporary approach.

Did I misunderstand what you were saying?

thanks,
Karen

> On Jun 26, 2017, at 2:52 PM, Paul Sandoz  wrote:
> 
> 
>> On 23 Jun 2017, at 13:33, Karen Kinnear  wrote:
> 
>> VWithfield - propose for MVT - allow package private access - since there 
>> are no methods on the derived value class
>>   and the value capable class can’t have any methods with vbytecodes since 
>> generated by javac
>>   - plan to make private when we add factory methods to value classes with a 
>> compiler (and we have nest support)
>> 
> 
> I am unsure if it’s necessary for MVT purposes to dial back the accessibility 
> then dial it up again later on.
> 
> ValueType.findWither can be used in conjunction with 
> MethodHandle.privateLookupIn. It’s a little odd but works. What am i missing?
> 
> Paul.
> 
> 
> 
> 

Re: Valhalla EG minutes 6/21/17

[Karen Kinnear]

Bjorn,

Thank you for the corrections.
> On Jul 5, 2017, at 5:10 AM, Bjorn B Vardal  wrote:
> 
> > Bjorn: single representation for both value capable class and derived value 
> > class
>  
> We basically rely on single class data structure for the VCC and DVT, but we 
> still expose these as separate java/lang/Class objects.
>  
>  
> > I think you said “treat ;Q” as the name for the derived value class 
> > rather than as an escape character  (feel free to correct my notes)
>  
> I meant to say the opposite of that: Since our DVT and VCC have the same name 
> (because they rely on the same class data structure),
> we would simply treat ";Q" as an escape character that switches to DVT mode, 
> instead of as part of the name of the value class.
That makes more sense.
>  
>  
> > Bjorn: could hack using the different name as two views of the same thing
> >e.g. vbox/vunbox would need to swap names
> >(ed. note - please let us know if this is doable without heroic efforts)
>  
> I can make this work as long as there's a predictable prefix and suffix, so 
> ";Q$value" works.
Thank you - very much appreciated.
>  
>  
> > Dan S: class loading in the proposed JVMS: if you see $Value
> >1) first derive the VCC name and see if already resolved
> >2) if not - load the VCC, check properties and derive
> >(ed. note - if see VCC - lazily derive derived value class on touch)
>  
> It's not a requirement that the value class derivation is lazy, correct?

Let’s double-check with Dan Smith at today’s meeting.

The way I read 5.3 Creation and Loading in
http://cr.openjdk.java.net/~dlsmith/values.html
it appears to allow lazy derivation as well as eager derivation, which I think 
is what we both want
since it allows implementations to optimize.
Our current derivation is also eager.

thanks,
Karen

>  
>  
>  
> - Original message -
> From: Karen Kinnear 
> Sent by: "valhalla-spec-experts" 
> 
> To: valhalla-spec-experts@openjdk.java.net
> Cc:
> Subject: Valhalla EG minutes 6/21/17
> Date: Fri, Jun 23, 2017 10:33 PM
>  
> attendees: Remi, Bjorn, Dan H, Dan S, John, Maurizio, Frederic, Lois, Karen
>  
> AIs:
> All: review Dan Smith’s proposals
>MVT JVMS: Specification for Value Classes: 
> http://cr.openjdk.java.net/~dlsmith/values.html 
>  - initial proposal
> *** let’s pin this down ASAP so we - Remi for ASM, IBM and Oracle can deliver 
> early binaries for early adopters to try
>  
>incremental proposals for post early access (or maybe post - MVT TBD)
>Direct Value Class: Specification for Value Classes with Explicit 
> Declarations: http://cr.openjdk.java.net/~dlsmith/values-declaration.html 
> 
>Specification for Value Classes with CONSTANT_ClassType: 
> http://cr.openjdk.java.net/~dlsmith/values-classtype.html#values-classtype-4.1
>  
> 
> All: review John Rose’s proposals:
>   ConstantDynamic JVMS changes: 
> http://cr.openjdk.java.net/~jrose/jvm/condy-jvms-2017-0620.html 
> 
> note: this is orthogonal to MVT
>   java/lang/Class.java makeSecondaryClass: 
> http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-June/000286.html
>  
> 
> post EA
>  
> Hotspot and IBM:
>   what could be available for early access  for early adopters to experiment 
> with?
>   revisit early access timing - if we were to set expectations that
>  - model would be to deliver binaries and periodic binary updates to 
> match the source builds. This is not a one shot delivery.
>  - limit functionality (platforms, reflection behavior unspecified, no 
> JVMTI, …)
>  - performance improvements are not yet there, expect to come in 
> incrementally
>  - maybe verifier isn’t ready?
>  - stability issues
>  
> Potential early adopters: Ian Graves? Doug Lea? Others?
> So - would you be willing to start experimenting with minimal value types 
> even with the restrictions above?
>  
> We would find it helpful to get your feedback on
>- the basic conceptual model
>- usage model
>- use cases - so we can optimize what you care about
>- required features we missed 
> - so that when we ship this experimentally it is much closer to what you 
> need
>  
> Model of usage:
> 1) Value-capable-class: created in java with annotation, javac generates a 
> regular classfile with no new constant pool entries
> or bytecodes
> 2) MethodHandles and ValueType APIs
>  - this is the default model of usage
> 3) generated byte codes
> - you can generate your own byte codes to work on value 

Re: Valhalla EG minutes 6/21/17

[Bjorn B Vardal]

> Bjorn: single representation for both value capable class and derived value class
 
We basically rely on single class data structure for the VCC and DVT, but we still expose these as separate java/lang/Class objects.
 
 
>     I think you said “treat ;Q” as the name for the derived value class rather than as an escape character  (feel free to correct my notes)
 
I meant to say the opposite of that: Since our DVT and VCC have the same name (because they rely on the same class data structure),
we would simply treat ";Q" as an escape character that switches to DVT mode, instead of as part of the name of the value class.
 
 
> Bjorn: could hack using the different name as two views of the same thing
>    e.g. vbox/vunbox would need to swap names
>    (ed. note - please let us know if this is doable without heroic efforts)
 
I can make this work as long as there's a predictable prefix and suffix, so ";Q$value" works.
 
 
> Dan S: class loading in the proposed JVMS: if you see $Value
>    1) first derive the VCC name and see if already resolved
>    2) if not - load the VCC, check properties and derive
>    (ed. note - if see VCC - lazily derive derived value class on touch)
 
It's not a requirement that the value class derivation is lazy, correct?
 
 
 
- Original message -From: Karen Kinnear Sent by: "valhalla-spec-experts" To: valhalla-spec-experts@openjdk.java.netCc:Subject: Valhalla EG minutes 6/21/17Date: Fri, Jun 23, 2017 10:33 PM attendees: Remi, Bjorn, Dan H, Dan S, John, Maurizio, Frederic, Lois, Karen
 
AIs:
All: review Dan Smith’s proposals
   MVT JVMS: Specification for Value Classes: http://cr.openjdk.java.net/~dlsmith/values.html - initial proposal
*** let’s pin this down ASAP so we - Remi for ASM, IBM and Oracle can deliver early binaries for early adopters to try
 
   incremental proposals for post early access (or maybe post - MVT TBD)
       Direct Value Class: Specification for Value Classes with Explicit Declarations: http://cr.openjdk.java.net/~dlsmith/values-declaration.html
       Specification for Value Classes with CONSTANT_ClassType: http://cr.openjdk.java.net/~dlsmith/values-classtype.html#values-classtype-4.1
All: review John Rose’s proposals:
      ConstantDynamic JVMS changes: http://cr.openjdk.java.net/~jrose/jvm/condy-jvms-2017-0620.html
note: this is orthogonal to MVT
      java/lang/Class.java makeSecondaryClass: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-June/000286.html
                post EA
 
Hotspot and IBM:
  what could be available for early access  for early adopters to experiment with?
  revisit early access timing - if we were to set expectations that
     - model would be to deliver binaries and periodic binary updates to match the source builds. This is not a one shot delivery.
     - limit functionality (platforms, reflection behavior unspecified, no JVMTI, …)
     - performance improvements are not yet there, expect to come in incrementally
     - maybe verifier isn’t ready?
     - stability issues
 
Potential early adopters: Ian Graves? Doug Lea? Others?
    So - would you be willing to start experimenting with minimal value types even with the restrictions above?
 
    We would find it helpful to get your feedback on
       - the basic conceptual model
       - usage model
       - use cases - so we can optimize what you care about
       - required features we missed 
    - so that when we ship this experimentally it is much closer to what you need
 
Model of usage:
1) Value-capable-class: created in java with annotation, javac generates a regular classfile with no new constant pool entries
or bytecodes
2) MethodHandles and ValueType APIs
     - this is the default model of usage
3) generated byte codes
    - you can generate your own byte codes to work on value types
    - at this point you can’t generate your own value type class this way (until we get to Direct Value Class support)
 
Constant Pool changes for Early Access:
1) proposal for a value class is:
      part I: CONSTANT_Class_info: UTF8: “;Q;” // i.e. this would be a descriptor using a UTF8 string to speed up implementation
      part II: hotspot requests that we use a different name for the value type than for the value capable class
           implementation request - today we need unique strings to identify unique runtime types and this is baked in multiple places
 
     So propose: CONSTANT_Class_info: UTF8:”QFoo$
 
Where is the name exposed?
   in the constant pool when you generate your own bytecodes - in descriptors and class names
 
Dan S: Longer term: will declare a value class directly. The box first is a temporary approach in which we derive the value class.
Propose we not spend a lot of time here blurring the difference and needing to hide the derived value class.
 
Bjorn: single representation for both value capable class and derived value class
    I think you said “treat ;Q” as