subject:"Re\: Draft JVMS changes for Nestmates"


One follow up to a new comment John made ...

On 20/04/2017 10:09 AM, John Rose wrote:

On Apr 19, 2017, at 12:56 AM, David Holmes mailto:david.hol...@oracle.com>> wrote:



I don't see how that is a significant concern.  The symbol bodies are
hot in cache
at the point we would check prefixes, since they are already being
scanned for other
purposes, such as initial interning, and also syntax checking.  Existing
processing
is exactly as deep (or shallow) as the checks I want.


I don't follow that. If I'm loading a nest-top class and validating
its NM entries none of those entries need have been loaded yet and so
none will be in the cache.


I see what the problem is:  The only checking I am proposing
is syntax checking on the names.  If by "deep checking" you
mean loading classes from the names, I agree, we don't want
to do that.  The MemberOfNest attribute is passive.  It does
not need to initiate loads; it just needs to validate claims
of nest membership by other classes.

(I even doubt Dan's assertion that in some corner cases,
when the nest-host is trying out a private access, there
must be some loading done to validate the access.  Probably
we can get away with no loading at all, just name checks
against already-loaded classes.)


If the access is legitimate then no loading is needed as the common 
nest-host is already loaded. Currently the implementation will load the 
purported nest-host if needed, so in the case where they are not 
actually nest-mates this would load the "foreign" nest-host class. But 
when the initiator is the nest-host the check can be inverted by 
searching NestMembers for the target class instead. I see one quirky 
aspect of this in that in the unlikely case that the target class is in 
NestMembers but does not itself list the nest-host in its MemberOfNest, 
the access check will succeed, but before the invocation can actually 
happen the target class must be linked and at that point validation 
would fail (assuming we adopt the fail-fast model).


David


— John

Re: Draft JVMS changes for Nestmates

Limited response as issues have been covered elsewhere, and Dan has 
revisited suggested spec wording. But for completeness ...


On 20/04/2017 10:32 AM, John Rose wrote:

On Apr 18, 2017, at 11:40 PM, David Holmes mailto:david.hol...@oracle.com>> wrote:


4.7.26 The MemberOfNest Attribute

"the class is implicitly a member of its own nest"

I suggest changing "a" to "the sole". Otherwise "a" implies there may
be other members. As these are static properties being defined I don't
think we need to be accounting for some future dynamic expansion of a
nest.


I think you're missing the point of the spec. logic here.  If there is a
non-empty NestMembers attribute, then in fact there *are* other members.
Therefore "a" is correct, and "the sole" would be incorrect.


Yes another gaseous brain expulsion on my part. I was thinking only of 
the case where neither attributes are present.



Separately, we do need to leave wiggle room for dynamic injection.
Dan, I think that would be done by allowing a dynamic injection
operation to implicitly extend the NestMembers list to make a
"ticket" for the injected class, and implicitly add (if necessary)
a MemberOfNest to the injected class.

...


4.10.1 Verification by Type Checking

Does the order in the rules mandate the order in which the VM must
actually check things? classHasValidNest seems much more complex than
the actual runtime process of validation (though it may just be
exposing distinct steps that are lumped together at runtime).


Agree.  I think this logic can be simplified.  (Will take that to a
separate sub-thread.)


---

4.10.1.8 Type Checking for Restricted Member References

I'm not very good at reading or understanding these rules, but I'm
surprised that the invokespecial rules seem to make no mention of
nestmates at all:

"A method reference is allowed by an invokespecial instruction if it
is not a restricted invokespecial reference, ..."

This seems to open things up to all private method references??


Well, the whole point of nestmates is to extend, in a regular way,
all access rules that pertain to self-access and privacy.
If the extension is truly regular, it is not surprising that the spec.
changes are subtle like this.  We're not trying to do an ad hoc
patch on some set of particular behaviors, but rather extend
pre-existing notions so they apply in more conditions, including
invokespecial restrictions.


I was expecting to see the restrictions have a dependency on nestmates 
actually being involved - which is how I've currently implemented it. I 
can throw away some of the verifier changes if I only need to check for 
private access. But I'm unclear exactly where (in the spec) we enforce 
that only nestmates get this right to use invokespecial on private 
methods? Is it the access check applied during method resolution?


Cheers,
David
-



---

5.4.4 Access Control

I don't think you need to restate:

"A class with a MemberOfNest attribute belongs to the nest hosted by
the referenced host class.

A class without a MemberOfNest attribute implicitly belongs to a nest
hosted by the class itself. (If the class also lacks a NestMembers
attribute, then the nest has only one member.)"

The rule is simply stated as is: "belonging to the same nest as D" -
belonging to the same nest is, or should be, already defined elsewhere.


Actually, there is no "elsewhere"; this is where the condition is defined.
We don't need a special section for "nests"; we don't have a special
section for "packages" either.  So that part reads fine to me.


I'm avoiding commenting on the protected access changes but do want to
raise a concern that there are no changes to Method resolution
(5.4.3.3) yet resolution relies on the definition of access control to
prune/discard inaccessible methods. It appears now that we're allowing
more potential successful resolutions, then using the additional rules
on the invoke* bytecodes to try and discard any undesirable results.

---

6.5 getfield (and others)

You note "These rules are redundant: verification already guarantees
them " but this brings me back to an area I keep raising concerns
about: the split between static verification based checks and dynamic
runtime checks. Yes the verifier precludes certain things but if we
run without verification the rules expressed for the bytecodes are
considered to be still required at runtime. If you delete them because
they overlap with verifier rules we have no way to tell which rules
must be enforced regardless of verification status.

---

6.5 invokespecial

You note regarding the IllegalAccessError:

"This replaces a VerifyError previously specified by 4.9.2. The check
must be delayed until after resolution in order to determine whether
the referenced method is private."

We know the access flags for the referenced method at verification
time - shouldn't that be the sole basis for the verifier check?
Afterall it is verifying the static properties of the classfile and
bytecode. If the ac

Re: Draft JVMS changes for Nestmates


On 20/04/2017 7:58 AM, Dan Smith wrote:

On Apr 19, 2017, at 1:21 PM, Dan Smith mailto:daniel.sm...@oracle.com>> wrote:



5.4.4 Access Control

I don't think you need to restate:

"A class with a MemberOfNest attribute belongs to the nest hosted by
the referenced host class.

A class without a MemberOfNest attribute implicitly belongs to a nest
hosted by the class itself. (If the class also lacks a NestMembers
attribute, then the nest has only one member.)"

The rule is simply stated as is: "belonging to the same nest as D" -
belonging to the same nest is, or should be, already defined elsewhere.


My intent is that this *is* the definition. But sounds like you're
expecting that definition to be in 4.7.25/4.7.26, which is probably
more intuitive. I'll change that.


Thanks. Yes I was expecting a definition somewhere else, up front.


Okay, here's another attempt at defining the attributes and the "nest"
concept.

I've been uneasy about `NestMembers` being presented as the
"declaration" of the nest, since it's not validated. Instead, I see
`MemberOfNest` as really where nests are defined (as a cumulative result
of many class declarations—like packages), and `NestMembers` as just a
help for validation.


I can't decide whether that understates the importance of the 
bi-directionality of membership, or not - both parties must agree on 
membership for that membership to exist. It may not have any practical 
significance.



With that in mind, `MemberOfNest` ought to come first:



**4.7.25 The `MemberOfNest` Attribute**
---

**The `MemberOfNest` attribute is a fixed-length attribute in the
`attributes` table of a `ClassFile` structure ([4.1]).**

**A _nest_ is a set of classes and interfaces that share access to their
`private` members ([5.4.4]).**


Aside: it is this basic definition of "nest" that I expected to appear 
somewhere earlier in the spec. JVMS doesn't have any upfront "concepts 
and definitions" section, so not sure where it would go.



**A class or interface with a `MemberOfNest` attribute belongs to the
nest hosted by a designated _host class_. The host class authorizes
membership in the nest with a corresponding entry in its `NestMembers`
attribute ([4.7.26], [4.10]).**


Only nit is I would say "claims membership of" rather than "belongs to". 
That emphasises that membership must be ratified by the "host class", 
and matches the following "authorizes membership" text.



**A class or interface without a `MemberOfNest` attribute belongs to the
nest hosted by itself. (Often, this nest is a singleton consisting only
of the class itself.)**

**There may be at most one `MemberOfNest` attribute in the `attributes`
table of a `ClassFile` structure.**

**The `MemberOfNest` attribute has the following format:**

...


**4.7.26 The `NestMembers` Attribute**
--

**The `NestMembers` attribute is a variable-length attribute in the
`attributes` table of a `ClassFile` structure ([4.1]). It authorizes an
enumerated set classes and interfaces to claim membership in a nest
hosted by the current class or interface.**

**There may be at most one `NestMembers` attribute in the `attributes`
table of a `ClassFile` structure.**

**The `attributes` table of a `ClassFile` structure must not contain
both a `MemberOfNest` attribute and a `NestMembers` attribute.**


**This rule prevents a host class from claiming membership in a

different nest. It is implicitly a member of the nest that it hosts.**

**The `NestMembers` attribute has the following format:**

...


5.4.4 Access Control


A class or interface _C_ is _accessible_ to a class or interface _D_ if
and only if either of the following is true:

...

- _R_ is `private` and is declared ~~in _D_~~ **by a class belonging to
the same nest as _D_ ([4.7.25])**.




And now to the bikeshedding game:


:)


`MemberOfNest`, as the key thing, could just be called `Nest` (imagine a
`Package` attribute—we wouldn't call it `MemberOfPackage`). Or, since
it's a reference to a class, `NestHost ` / `NestTop` / `NestMother` /
`NestWhatever`.


I like to distinguish the name of the attribute from the entity the 
attribute represents. Given the names Maurizio chose are not terribly 
wrong, and nothing else is significantly better, I'm quite content with 
them.



`NestMembers` isn't horrible, but we could highlight its validation role
with something like `AuthorizedNestMembers`.


To me the "authorized" is implicit. By calling it out explicitly it 
suggests to me there are also UnauthorizedNestMembers. But to me this is 
an oxymoron - if you are unauthorized you are not an 
unauthorized-member, you are not any kind of member at all.


Cheers,
David


—Dan

Re: Draft JVMS changes for Nestmates


On 20/04/2017 2:47 AM, Dan Smith wrote:



On Apr 19, 2017, at 8:12 AM, Brian Goetz  wrote:


Yep. Also, multiple classes can claim the same nest member class in their 
NestMembers attributes. Not a problem as long as the MemberOfNest attribute (if 
any) of the member class points to a host class that claims it.



At the risk of bikeshedding, I find the names NestMembers and MemberOfNest 
confusing; I keep having to think about the directionality, since they sound so 
similar.  Would be good to find names that have obviously opposite 
directionality.

A not-very-good suggestion (but which illustrates the directionality thing I'm going for) would be 
"NestDeclaration" and "NestUse".  It's pretty clear which is which without 
thinking about it.

Perhaps: NestHost and NestMember ?


I also don't love that "NestMembers" seems more authoritative than it really 
is. (Doesn't get validated, doesn't suggest that other classes can be dynamically added.)

Something like "AllowedInNest" would better convey the actual meaning of that 
attribute.


I don't see how AllowedInNest is any less authoritative sounding than 
NestMembers. Neither suggests other classes may be added dynamically - 
nor should they in my opinion. These are static attributes of a 
classfile defining a relationship as it was known to exist when the 
classfile was created. Any dynamic means to add to a nest at runtime 
would not be modifying the classfile NestMembers attribute, but the 
runtime representation thereof. I suppose InitialNestMembers may capture 
this more accurately - though it does then beg the question as to how to 
add the to set later.


Aside: when first discussed in the context of generic specialization I 
assumed the NestMembers entry would be a "wildcard" which indicated that 
not only was the named generic type a nestmember, but all 
specializations thereof (which would presumably have a name derived from 
the base generic type). But in the context of Lookup.defineClass I have 
no idea how it is/was expected that nest membership would expand - or 
how the validity of doing so would be established.


David


—Dan

Re: Draft JVMS changes for Nestmates


Hi Dan,

On 20/04/2017 2:06 AM, Dan Smith wrote:

On Apr 19, 2017, at 12:45 AM, John Rose mailto:john.r.r...@oracle.com>> wrote:

 - NMs must be non-empty (degenerate nest is never explicit)
 - NMs may not contain duplicates (cf. treatment of ClassFile.interfaces)
 - NMs may not contain the current class (i.e., an index to a class
with the same name as this_class)
 - NMs may contain only package siblings (ditto)
 - NMs and MoN may not refer to array classes (this is probably
implied by the package prefix checks)


These are all do-able, but I'm not sure they're consistent with the
spirit of JVMS in its treatment of attributes. We don't usually assert
that lists are non-empty, don't contain duplicates, etc. (Granted, most
attributes are not relevant to JVM execution.) You mention `interfaces`,
but I don't see any such assertion in 4.1.


The NestMembers duplicate check exists because I copied the spec and 
code from the InnerClasses attribute. 4.7.6 states the classes[] "must 
have exactly one corresponding entry ..." - hence a check for duplicates 
is done. I used the same language when I defined NestMembers. So the 
precedent does exist.



You can identify some package mismatches by looking at names, but not
all (names with the same "package" name may be handled by different
class loaders). I worry that making a partial effort here will give a
false sense of security and lead someone in the future to see that as a
bug and try to fully validate the package restriction.


I concur. We need the actual package check at nest-top validation time 
anyway, so checking this at parsing time seems pure overhead.



I hate array "class names". Ugh. But, again, we're not generally
concerned with that kind of hygiene in attributes.


 - MoN may not contain the current class (ditto)
 - MoN may contain only a package sibling (i.e., a referenced class
name must have the same package prefix as this_class)
 - NMs and MoN may not refer to array classes (this is probably
implied by the package prefix checks)


If MoN names the current class, verification will fail (there's no
matching NM attribute).

If MoN names a class in a different package, verification will fail (the
rule checks "samePackageName").

If MoN names an array type, verification will fail (either due to the
"samePackageName" check or because there's no matching NM attribute).

I don't see the benefit in making these "syntax" checks in addition to
verification checks. (Even a class naming itself as its own superclass
isn't a syntax error—it gets checked later.)


David's prototype has the duplication check, and some of the other
checks happen later.  I think they should all happen during class loading.


Something you might like is if we moved the MoN verification check out
of verification (which is mainly concerned with Code attributes anyway)
and into a late step of the class loading process (5.3.5)? Something like:

1) Get the bits
2) Parse the class
3) Load & validate superclasses
4) Load & validate interfaces
5) Load & validate MemberOfNest


At present I do (5) during "linking" just prior to verification (in case 
verification  is disabled). It happens after the superclass and 
superinterfaces are themselves linked. But note that nest-top validation 
only requires the nest-top to be loaded, not linked.


David
-


—Dan

Re: Draft JVMS changes for Nestmates


Correction below ...

On 19/04/2017 5:56 PM, David Holmes wrote:

On 19/04/2017 5:51 PM, John Rose wrote:

On Apr 19, 2017, at 12:14 AM, David Holmes mailto:david.hol...@oracle.com>> wrote:


Some of these are a lot more awkward to do during classfile parsing
and will require symbol comparisons. I was wanting to avoid "deep
validation" of NMs as it penalizes all the good code. Having a "bad"
NM entry seems harmless as these entries are only used to validate the
initial claim of nest membership. If an entry is "bad" then by
definition it will not match with any claimee.


I don't see how that is a significant concern.  The symbol bodies are
hot in cache
at the point we would check prefixes, since they are already being
scanned for other
purposes, such as initial interning, and also syntax checking.  Existing
processing
is exactly as deep (or shallow) as the checks I want.


I don't follow that. If I'm loading a nest-top class and validating its
NM entries none of those entries need have been loaded yet and so none
will be in the cache.


Sorry John - brain fart on my part. We're talking about the symbols that 
have just been parsed and loaded into the constant pool, not the classes 
those symbols name.


David
-


David
-


If we turn off the "verify" flag for class loading, then maybe we can
buy something
by dropping those (and all the other) constraint checks.

By syntax checking, I mean that if I mention a CONSTANT_Class in the CP,
and its corresponding CONSTANT_Utf8 has a broken syntax (e.g., "."
instead
of "/", or two "//" in a row) the JVMS mandates an error.  But those
are the
same bytes I want to look at when they are referenced by a MoN or NMs
attribute.  It will all be in cache, and the cycles to do the checks
will be
undetectable.

— John

Re: Draft JVMS changes for Nestmates

On Apr 18, 2017, at 11:40 PM, David Holmes  wrote:
> 
> 4.7.26 The MemberOfNest Attribute
> 
> "the class is implicitly a member of its own nest"
> 
> I suggest changing "a" to "the sole". Otherwise "a" implies there may be 
> other members. As these are static properties being defined I don't think we 
> need to be accounting for some future dynamic expansion of a nest.

I think you're missing the point of the spec. logic here.  If there is a
non-empty NestMembers attribute, then in fact there *are* other members.
Therefore "a" is correct, and "the sole" would be incorrect.

Separately, we do need to leave wiggle room for dynamic injection.
Dan, I think that would be done by allowing a dynamic injection
operation to implicitly extend the NestMembers list to make a
"ticket" for the injected class, and implicitly add (if necessary)
a MemberOfNest to the injected class.

...
> 
> 4.10.1 Verification by Type Checking
> 
> Does the order in the rules mandate the order in which the VM must actually 
> check things? classHasValidNest seems much more complex than the actual 
> runtime process of validation (though it may just be exposing distinct steps 
> that are lumped together at runtime).

Agree.  I think this logic can be simplified.  (Will take that to a separate 
sub-thread.)

> ---
> 
> 4.10.1.8 Type Checking for Restricted Member References
> 
> I'm not very good at reading or understanding these rules, but I'm surprised 
> that the invokespecial rules seem to make no mention of nestmates at all:
> 
> "A method reference is allowed by an invokespecial instruction if it is not a 
> restricted invokespecial reference, ..."
> 
> This seems to open things up to all private method references??

Well, the whole point of nestmates is to extend, in a regular way,
all access rules that pertain to self-access and privacy.
If the extension is truly regular, it is not surprising that the spec.
changes are subtle like this.  We're not trying to do an ad hoc
patch on some set of particular behaviors, but rather extend
pre-existing notions so they apply in more conditions, including
invokespecial restrictions.

> 
> ---
> 
> 5.4.4 Access Control
> 
> I don't think you need to restate:
> 
> "A class with a MemberOfNest attribute belongs to the nest hosted by the 
> referenced host class.
> 
> A class without a MemberOfNest attribute implicitly belongs to a nest hosted 
> by the class itself. (If the class also lacks a NestMembers attribute, then 
> the nest has only one member.)"
> 
> The rule is simply stated as is: "belonging to the same nest as D" - 
> belonging to the same nest is, or should be, already defined elsewhere.

Actually, there is no "elsewhere"; this is where the condition is defined.
We don't need a special section for "nests"; we don't have a special
section for "packages" either.  So that part reads fine to me.

> I'm avoiding commenting on the protected access changes but do want to raise 
> a concern that there are no changes to Method resolution (5.4.3.3) yet 
> resolution relies on the definition of access control to prune/discard 
> inaccessible methods. It appears now that we're allowing more potential 
> successful resolutions, then using the additional rules on the invoke* 
> bytecodes to try and discard any undesirable results.
> 
> ---
> 
> 6.5 getfield (and others)
> 
> You note "These rules are redundant: verification already guarantees them " 
> but this brings me back to an area I keep raising concerns about: the split 
> between static verification based checks and dynamic runtime checks. Yes the 
> verifier precludes certain things but if we run without verification the 
> rules expressed for the bytecodes are considered to be still required at 
> runtime. If you delete them because they overlap with verifier rules we have 
> no way to tell which rules must be enforced regardless of verification status.
> 
> ---
> 
> 6.5 invokespecial
> 
> You note regarding the IllegalAccessError:
> 
> "This replaces a VerifyError previously specified by 4.9.2. The check must be 
> delayed until after resolution in order to determine whether the referenced 
> method is private."
> 
> We know the access flags for the referenced method at verification time - 
> shouldn't that be the sole basis for the verifier check? Afterall it is 
> verifying the static properties of the classfile and bytecode. If the actual 
> resolved method has different access to the referenced method then that may 
> lead to ICCE (depending on the exact nature of the change - a private method 
> made public may not be an issue for example).
> 
> ---
> 
> 5.4.3.3 Method resolution
> 
> You did not make any changes here, but as per my comment in the bug report we 
> do require, IMHO, further tightening here to ensure nestmate invokespecial 
> invocations do not resolve to method implementations that they should not. 
> The example here is a hierarchy of nestmates (C extends B extends A) where A 
> and C both declare a private metho

Re: Draft JVMS changes for Nestmates

On Apr 18, 2017, at 11:40 PM, David Holmes  wrote:
> 
> I have not yet seen a follow up from John as to whether we require or just 
> allow, an empty NestMembers attribute to indicate a singleton nest member.

I think we should neither require nor allow an empty NestMembers attribute.

Dan may prefer to minimize syntax checking, in which case an empty
NestMembers attribute would be allowed.

Nobody has proposed a reason to require it.  A nest of one member is
(depending on your POV) either the overwhelmingly common trivial case,
or a useless degree of freedom.  Taking the POV (from Brian and me)
hat every class has a nest (just as every class has a package), an
empty NestMembers attribute is just a waste of bits stating the obvious,
about a stand-alone class.

(I'd prefer to disallow the empty NMs attribute, lest it accidentally acquire
some additional meaning.  But as long as we specify that an empty one
conveys the same condition as no attributes at all, we are safe from accident.)

— John

Re: Draft JVMS changes for Nestmates

On Apr 19, 2017, at 12:56 AM, David Holmes  wrote:
> 
>> I don't see how that is a significant concern.  The symbol bodies are
>> hot in cache
>> at the point we would check prefixes, since they are already being
>> scanned for other
>> purposes, such as initial interning, and also syntax checking.  Existing
>> processing
>> is exactly as deep (or shallow) as the checks I want.
> 
> I don't follow that. If I'm loading a nest-top class and validating its NM 
> entries none of those entries need have been loaded yet and so none will be 
> in the cache.

I see what the problem is:  The only checking I am proposing
is syntax checking on the names.  If by "deep checking" you
mean loading classes from the names, I agree, we don't want
to do that.  The MemberOfNest attribute is passive.  It does
not need to initiate loads; it just needs to validate claims
of nest membership by other classes.

(I even doubt Dan's assertion that in some corner cases,
when the nest-host is trying out a private access, there
must be some loading done to validate the access.  Probably
we can get away with no loading at all, just name checks
against already-loaded classes.)

— John

Re: Draft JVMS changes for Nestmates

On Apr 19, 2017, at 2:10 PM, David Holmes  wrote:
> 
> I find these names very good - granted I've been working with them more than 
> anyone else (but didn't define them). The plurality makes it clear to me.

+1 for plurality; that does it for me too, although it is a one-letter hint.

I'm happy with the names, though I have a little problem (that Dan is trying to 
fix)
with making clear the distinction between a Nest and the Host (or Top or Root) 
of
a Nest.  Things get messy when conflate them, which terminology can help avoid.

— John

P.S. Can't resist, so OK, in the spirit of "crib", how about throwing a private 
"party"?

Background:  The asymmetry of the relation is essential to the design of the 
attributes.

But it does not come from one class being somehow semantically or syntactically
distinguished in any way, other than one:  The special "nest host" (or "nest 
top")
is responsible for inviting any number of classes (including zero) to join its 
nest.

(The nest host is like the admin of a skype conversation, in which all 
conversants
are otherwise equal.  And like skype conversations, the group has only one 
purpose.
For nestmates, the purpose is to broaden the definition of ACC_PRIVATE.)

Under the theory that the nest host invites the other nestmates to the party,
we could emphasize that role in the attribute names:

NestMembers => NestInvitations (=> NestGrants since it is checked reactively)
MemberOfNest => RequestNest (=> Nest, or NestHost—requesting is a given when we 
do access control)

(Hmm…  NMs => PartyInvitations, PartyRoster; MoN => RSVPToParty, JoinParty.)

So, at this point, the best I change I can suggest to strengthen the asymmetry,
but keep the nicest metaphor (nest) is NestGrants / Nest or NestHost.

Why isn't "Nest" ambiguous?  Doesn't it define a nest?  Well, no, if you look
at the way attributes are named in the JVMS, they refer to an "attribute"
(duh) of the containing object.  Thus, if ClassFile has a Nest attribute, it
means that the ClassFile *has* an attribute of type Nest, not that the ClassFile
is defining another entity of type Nest *outside* of the ClassFile.  So I claim
that "Nest" is perfectly unambiguous:  It means that the ClassFile is asserting
the identity of its Nest, using the convention of naming the nest-host.
Choosing the more specific term "NestHost" would make it more clear
that the Nest is being named by mentioning its host.

The word "host" is a good fit (better than top or root) if we are emphasizing
the fact that the host's job is to grant entry into the nest.

Although reusing "host" is a feature rather than a bug, it could also be
NestSecretary, NestDoorman, NestMaitreDeHotel, NestBouncer, etc.

OK, I'm stopping.

Re: Draft JVMS changes for Nestmates

One other topic to add to this thread, which I forgot to mention: compatibility.

Sometimes JVMS changes need to be guarded by rules that preserve old behavior 
for old class file version numbers, in order to prevent changes in behavior 
when an old class is run on a new JVM. My hope is that there's no need for such 
logic here.

Here's my analysis. Let me know if you can think of any other cases, or if you 
feel that any of these changes are concerning.

---

- The MemberOfNest and NestMembers attributes are illegal prior to 54.0. So 
well-formed older classes always belong to singleton nests—they can't claim 
membership in a different nest, and can't act as hosts.

- Access control for private members declared in or called from old class files 
then degenerates to the old rule: the private member must be declared by the 
calling class.

- The verification-time checks on the referenced class of an invokespecial have 
been moved to resolution time. Some cases:

- Referencing a public method in a disallowed class used to be a 
VerifyError*; now it's a IAE at resolution time

- Referencing a private method in a disallowed class used to be a 
VerifyError; now it's an IAE at resolution time

- Referencing a private method in a subclass that resolves to a member of 
the current class used to be a VerifyError; now it's allowed.

- Referencing a nonexistent method in a disallowed class used to be a 
VerifyError; now it's a NSME at resolution time

- Referencing a static method in a disallowed class used to be a 
VerifyError; now it's a ICCE at resolution time

(*Confirmed with some testing, although I found that referencing an 
indirect superinterface caused an ICCE, apparently at resolution time.)

- The verification-time restriction on the target reference stack type of an 
invokespecial is only enforced if the referenced class is a superclass and the 
resolved method is non-private. Only change here (besides those outlined above 
due to a reference to a non-superclass) is that a private method referenced in 
a superclass that used to get a VerifyError (due to a disallowed stack type) 
will now get an IAE at resolution time.

- Verification is more eager to resolve classes and methods referenced by 
`invokespecial`. The change in class loading only affects programs that would 
have had VerifyErrors before (for a reference to a disallowed 
class—superclasses are already loaded). The change in method resolution may be 
visible, depending on how the implementation chooses to handle errors. But 
flexibility for these sorts of changes is granted by 5.4.

- As described in the comment in 5.4.4, tweaking the applicability of the 
`protected` access referenced class restriction causes some IAEs to become 
ICCEs or NSMEs.

- The runtime checks added to `invokespecial` introduce some new ICCEs when the 
current class or referenced class are interfaces.

Summary: almost all of these are simply changes in the error produced, and 
perhaps in the timing of errors being reported, depending on when resolution 
happens. There's one case where the new rules for private invokespecial allow 
something that was an error before (fine). There's also one case where new 
errors are introduced, but that would be a deliberate choice in response to a 
bug report.

—Dan

Re: Draft JVMS changes for Nestmates

> On Apr 19, 2017, at 1:21 PM, Dan Smith  wrote:
> 
>> 
>> 5.4.4 Access Control
>> 
>> I don't think you need to restate:
>> 
>> "A class with a MemberOfNest attribute belongs to the nest hosted by the 
>> referenced host class.
>> 
>> A class without a MemberOfNest attribute implicitly belongs to a nest hosted 
>> by the class itself. (If the class also lacks a NestMembers attribute, then 
>> the nest has only one member.)"
>> 
>> The rule is simply stated as is: "belonging to the same nest as D" - 
>> belonging to the same nest is, or should be, already defined elsewhere.
> 
> My intent is that this *is* the definition. But sounds like you're expecting 
> that definition to be in 4.7.25/4.7.26, which is probably more intuitive. 
> I'll change that.

Okay, here's another attempt at defining the attributes and the "nest" concept.

I've been uneasy about `NestMembers` being presented as the "declaration" of 
the nest, since it's not validated. Instead, I see `MemberOfNest` as really 
where nests are defined (as a cumulative result of many class declarations—like 
packages), and `NestMembers` as just a help for validation.

With that in mind, `MemberOfNest` ought to come first:

**4.7.25 The `MemberOfNest` Attribute**
---

**The `MemberOfNest` attribute is a fixed-length attribute in the `attributes` 
table of a `ClassFile` structure ([4.1]).**

**A _nest_ is a set of classes and interfaces that share access to their 
`private` members ([5.4.4]).**

**A class or interface with a `MemberOfNest` attribute belongs to the nest 
hosted by a designated _host class_. The host class authorizes membership in 
the nest with a corresponding entry in its `NestMembers` attribute ([4.7.26], 
[4.10]).**

**A class or interface without a `MemberOfNest` attribute belongs to the nest 
hosted by itself. (Often, this nest is a singleton consisting only of the class 
itself.)**

**There may be at most one `MemberOfNest` attribute in the `attributes` table 
of a `ClassFile` structure.**

**The `MemberOfNest` attribute has the following format:**

...

**4.7.26 The `NestMembers` Attribute**
--

**The `NestMembers` attribute is a variable-length attribute in the 
`attributes` table of a `ClassFile` structure ([4.1]). It authorizes an 
enumerated set classes and interfaces to claim membership in a nest hosted by 
the current class or interface.**

**There may be at most one `NestMembers` attribute in the `attributes` table of 
a `ClassFile` structure.**

**The `attributes` table of a `ClassFile` structure must not contain both a 
`MemberOfNest` attribute and a `NestMembers` attribute.**

> **This rule prevents a host class from claiming membership in a different 
> nest. It is implicitly a member of the nest that it hosts.**

**The `NestMembers` attribute has the following format:**

...

5.4.4 Access Control

A class or interface _C_ is _accessible_ to a class or interface _D_ if and 
only if either of the following is true:

...

- _R_ is `private` and is declared ~~in _D_~~ **by a class belonging to the 
same nest as _D_ ([4.7.25])**.

And now to the bikeshedding game:

`MemberOfNest`, as the key thing, could just be called `Nest` (imagine a 
`Package` attribute—we wouldn't call it `MemberOfPackage`). Or, since it's a 
reference to a class, `NestHost ` / `NestTop` / `NestMother` / `NestWhatever`.

`NestMembers` isn't horrible, but we could highlight its validation role with 
something like `AuthorizedNestMembers`.

—Dan

Re: Draft JVMS changes for Nestmates


On 20/04/2017 12:12 AM, Brian Goetz wrote:

Yep. Also, multiple classes can claim the same nest member class in
their NestMembers attributes. Not a problem as long as the
MemberOfNest attribute (if any) of the member class points to a host
class that claims it.



At the risk of bikeshedding, I find the names NestMembers and
MemberOfNest confusing; I keep having to think about the directionality,
since they sound so similar.  Would be good to find names that have
obviously opposite directionality.


I find these names very good - granted I've been working with them more 
than anyone else (but didn't define them). The plurality makes it clear 
to me.


David


A not-very-good suggestion (but which illustrates the directionality
thing I'm going for) would be "NestDeclaration" and "NestUse".  It's
pretty clear which is which without thinking about it.

Perhaps: NestHost and NestMember ?

Re: Draft JVMS changes for Nestmates

2017-04-19 Thread Brian Goetz


Nest host
Nest master
Nest top
Nest owner
Nest dictator-for-life
...


On 4/19/2017 3:21 PM, Dan Smith wrote:

I dislike "host class" because that is already used for the quite different case of 
VMACs. I can live with other names but I think they should incorporate "nest" eg. 
nest-holder, nest-host. But I think nest-top is an ideal match. (And no I didn't invent this 
terminology:)  ).

Okay. Probably best to sit on this for awhile and circle back to it, see how 
we're all feeling.

Re: Draft JVMS changes for Nestmates

Thanks a lot for the feedback! My comments below:

> On Apr 19, 2017, at 12:40 AM, David Holmes  wrote:
> 
>> JEP name
>> 
>> I don't know how permanent JEP names are supposed to be, but I'd prefer a 
>> different name at this point. Something like: "Expanded JVM Access to 
>> Private Members"—shorter, focused on the feature itself rather than its 
>> relationship to the Java language. Or maybe "Class Nests for Access to 
>> Private Members".
> 
> The intent of the name is to aid in the contextual positioning of this JEP ie 
> address why do we even need this. The JEP addresses an inconsistency between 
> access controls in the Java programming language and access controls in the 
> JVM. The primary goal here is to align the two, while at the same time 
> allowing for additional usage of nestmates in the future.

Yeah, but I think explaining the motivation in the title makes for a long title 
and limits the imagination to one particular use case.

>> The JEP uses "nest top" to describe the class that nest members reference; I 
>> prefer "host class", which better describes the class's role and isn't tied 
>> to the Java "top level class" concept. I know we use "host class" internally 
>> in Hotspot, perhaps when working with anonymous classes (of the JVM flavor), 
>> but I think in that context it will ultimately mean the same thing? Are we 
>> comfortable repurposing the term in this way?
> 
> Given that a nest-top type must be a top-level type, I think nest-top fits 
> perfectly.

This is a good example of my point about the title: the JVM has no concept* of 
"top level type", and doesn't need one. We just need a specific class to act as 
the reference point by which a group of classes can talk about a nest.

For example, I can imagine some language introducing a class 
file—foo.bar.Nest$$001—solely for the purpose of providing a hook for this 
reference point. javac's choice to designate a top-level class as the 
nest-top/nest-host is a compilation strategy.

(*Yes, there's an InnerClasses attribute, and Signature attributes, and lots of 
other stuff to facilitate compilation in the Java language. But those represent 
auxiliary metadata, not core JVM concepts.)

> I dislike "host class" because that is already used for the quite different 
> case of VMACs. I can live with other names but I think they should 
> incorporate "nest" eg. nest-holder, nest-host. But I think nest-top is an 
> ideal match. (And no I didn't invent this terminology :) ).

Okay. Probably best to sit on this for awhile and circle back to it, see how 
we're all feeling.

>> I follow Brian's model when it comes to nest membership (5.4.4): every class 
>> belongs to a nest, possibly (in the absence of MemberOfNest) the nest hosted 
>> by itself. Many nests are singletons without any associated explicit 
>> attributes.
> 
> I have not yet seen a follow up from John as to whether we require or just 
> allow, an empty NestMembers attribute to indicate a singleton nest member.

If in our model *every* class has a nest, then we definitely don't want to 
require every class to have a NestMembers attribute.

In my spec (5.4.4), the presence of a NestMembers attribute is irrelevant. A 
class without a MemberOfNest attribute belongs to its own nest, and there's no 
need to validate anything about NestMembers.

>> Verification of MemberOfNest
>> 
>> I include a discussion block about different options of validating 
>> MemberOfNest. I think the consensus, and my preference, is to do it during 
>> verification of the member class. (NestMembers, on the other hand, is never 
>> validated, except as a side-effect of checking MemberOfNest.)
> 
> Given verification is optional the current approach is to validate during 
> linking just prior to the verification step. Whether that can conceptually be 
> considered part of verification I am unsure.

I think this is the source of a lot of anomalies—spec treats verification as an 
essential part of linking, while Hotspot treats it as an optional add-on.

You should be okay, for some definition of "just prior". These are the 
constraints, as I read them:
- "A class or interface is completely loaded before it is linked." (5.4)
- Loading includes structural checks (CFE), version checks (UCVE), and 
validation and loading of superclasses and superinterfaces (5.3.5)
- "Errors detected during linkage are thrown at a point in the program where 
some action is taken by the program that might, directly or indirectly, require 
linkage to the class or interface involved in the error." (5.4)

That last one is pretty mysterious to me (what does "some action" mean? what 
about "indirectly"?), but seems to grant pretty wide latitude on the timing.

>> Verification of invokespecial
>> 
>> Allowing invokespecial to refer to classes other than the current class and 
>> its supers is a significant change. I noticed and relied heavily on the 
>> parallel with invokevirtual making protected method calls. So I tried to 
>> mak

Re: Draft JVMS changes for Nestmates

On Apr 19, 2017, at 9:47 AM, Dan Smith  wrote:
> 
> Something like "AllowedInNest" would better convey the actual meaning of that 
> attribute.

It's true that this is what the attribute does, which is a plus. But it also is 
less directional than NMs. (Am I claiming I am allowed in some nest or am I 
granting nest privileges to others?  Can't tell from the name.) To me at least 
the plural "s" makes it clear who is in charge of the list of "grantees".

Re: Draft JVMS changes for Nestmates


> On Apr 19, 2017, at 8:12 AM, Brian Goetz  wrote:
> 
>> Yep. Also, multiple classes can claim the same nest member class in their 
>> NestMembers attributes. Not a problem as long as the MemberOfNest attribute 
>> (if any) of the member class points to a host class that claims it.
> 
> 
> At the risk of bikeshedding, I find the names NestMembers and MemberOfNest 
> confusing; I keep having to think about the directionality, since they sound 
> so similar.  Would be good to find names that have obviously opposite 
> directionality.  
> 
> A not-very-good suggestion (but which illustrates the directionality thing 
> I'm going for) would be "NestDeclaration" and "NestUse".  It's pretty clear 
> which is which without thinking about it.
> 
> Perhaps: NestHost and NestMember ?

I also don't love that "NestMembers" seems more authoritative than it really 
is. (Doesn't get validated, doesn't suggest that other classes can be 
dynamically added.)

Something like "AllowedInNest" would better convey the actual meaning of that 
attribute.

—Dan

Re: Draft JVMS changes for Nestmates

> On Apr 19, 2017, at 1:38 AM, John Rose  wrote:
> 
> "Nest" also connotes a home-like place and therefore privacy.
> (Could also have said "home" or "family"—those are also private spaces.  Or 
> "crib"?)

Sigh. Now I will forever wish we had called this feature "cribs".
https://www.youtube.com/watch?v=cAb54qbMF_k

> Here's a corner case we need to cover:  I have to load my supers before I can 
> load myself.
> If one (or more) of my supers are in the same nest as me, the checking of MoN 
> must succeed
> no matter who is the host class of the nest.
> 
> This could get embarrassing if the host class is the subtype, if the super 
> then needs to
> recursively load the subtype (which is waiting on the supertype to load) 
> before it can
> validate its MoN attribute.  If verification (linking) is started after all 
> of the classes are
> loaded, things are OK, since the MoN checks happen only after everything is 
> loaded.

Yes, I think the earliest we want to be loading MemberOfNest is after 
superclasses and superinterfaces are loaded (5.3.5 step 4). That avoids the 
cycle.

> I was surprised to see invokespecial occasionally mentioned cases where it 
> only works on
> private members.  Is that a current constraint, or a new one?
> 
> I thought a class could invokespecial any of its methods, willy-nilly, and 
> this permission
> would seem to extend naturally to nestmates.  (I think I'm forgetting 
> something here.)

The old constraint is that (non-) invokespecial can only reference the 
current class, a superclass, or a direct superinterface.

The new constraint is that (non-) invokespecial must either reference a 
private method, or must reference the current class, a superclass, or a direct 
superinterface.

The purpose of this check is to force outsiders to use virtual dispatch, rather 
than jumping into the middle of an override chain. But private methods can't be 
overridden, so an "outsider" from the same nest won't be defeating any 
restrictions.

(We could grant all members of a nest the right to call invokespecial on all 
nest members' non-private members and supers, but that's a new feature. Would 
be similar to granting access to inherited protected members.)

>> When invokespecial involves interface types, the verifier can't guarantee 
>> that receiver objects actually implement that interface (JDK-8134358). It's 
>> an open question whether this is really a problem, but I included a 
>> tentative fix in the invokespecial runtime rules.
> 
> I am leaning towards putting this check in.  If we put it in, it should go in 
> everywhere,
> including invokespecial of default methods, both in and out of the caller's 
> nest.
> I think your language might have allowed an untyped loophole for nestmates.
> 
> My goal in this is simplicity of effect:  There are no loopholes, no 
> exceptions
> to typing of the incoming receiver (L0) of a default method.  If we are going 
> to
> check any of these invokespecial receivers, we must check them all.

The verifier attempts to ensure that:
- For non- invokespecial, the type of the objectref is a subtype of the 
referenced class/interface
- For non-, non-private invokespecial, the type of the objectref is a 
subtype of the current class/interface

The proposed rules repeat these two assertions, but with a dynamic check. 
Private methods are treated differently because they aren't subject to the same 
constraints. (It could simplify our lives considerably if we had a dedicated 
`invokeprivate` method, but alas. That's how you should think about it, anyway.)

—Dan

Re: Draft JVMS changes for Nestmates

> On Apr 19, 2017, at 12:45 AM, John Rose  wrote:
> 
>  - NMs must be non-empty (degenerate nest is never explicit)
>  - NMs may not contain duplicates (cf. treatment of ClassFile.interfaces)
>  - NMs may not contain the current class (i.e., an index to a class with the 
> same name as this_class)
>  - NMs may contain only package siblings (ditto)
>  - NMs and MoN may not refer to array classes (this is probably implied by 
> the package prefix checks)

These are all do-able, but I'm not sure they're consistent with the spirit of 
JVMS in its treatment of attributes. We don't usually assert that lists are 
non-empty, don't contain duplicates, etc. (Granted, most attributes are not 
relevant to JVM execution.) You mention `interfaces`, but I don't see any such 
assertion in 4.1.

You can identify some package mismatches by looking at names, but not all 
(names with the same "package" name may be handled by different class loaders). 
I worry that making a partial effort here will give a false sense of security 
and lead someone in the future to see that as a bug and try to fully validate 
the package restriction.

I hate array "class names". Ugh. But, again, we're not generally concerned with 
that kind of hygiene in attributes.

>  - MoN may not contain the current class (ditto)
>  - MoN may contain only a package sibling (i.e., a referenced class name must 
> have the same package prefix as this_class)
>  - NMs and MoN may not refer to array classes (this is probably implied by 
> the package prefix checks)

If MoN names the current class, verification will fail (there's no matching NM 
attribute).

If MoN names a class in a different package, verification will fail (the rule 
checks "samePackageName").

If MoN names an array type, verification will fail (either due to the 
"samePackageName" check or because there's no matching NM attribute).

I don't see the benefit in making these "syntax" checks in addition to 
verification checks. (Even a class naming itself as its own superclass isn't a 
syntax error—it gets checked later.)

> David's prototype has the duplication check, and some of the other checks 
> happen later.  I think they should all happen during class loading.

Something you might like is if we moved the MoN verification check out of 
verification (which is mainly concerned with Code attributes anyway) and into a 
late step of the class loading process (5.3.5)? Something like:

1) Get the bits
2) Parse the class
3) Load & validate superclasses
4) Load & validate interfaces
5) Load & validate MemberOfNest

—Dan

Re: Draft JVMS changes for Nestmates


On 19/04/2017 5:51 PM, John Rose wrote:

On Apr 19, 2017, at 12:14 AM, David Holmes mailto:david.hol...@oracle.com>> wrote:


Some of these are a lot more awkward to do during classfile parsing
and will require symbol comparisons. I was wanting to avoid "deep
validation" of NMs as it penalizes all the good code. Having a "bad"
NM entry seems harmless as these entries are only used to validate the
initial claim of nest membership. If an entry is "bad" then by
definition it will not match with any claimee.


I don't see how that is a significant concern.  The symbol bodies are
hot in cache
at the point we would check prefixes, since they are already being
scanned for other
purposes, such as initial interning, and also syntax checking.  Existing
processing
is exactly as deep (or shallow) as the checks I want.


I don't follow that. If I'm loading a nest-top class and validating its 
NM entries none of those entries need have been loaded yet and so none 
will be in the cache.


David
-


If we turn off the "verify" flag for class loading, then maybe we can
buy something
by dropping those (and all the other) constraint checks.

By syntax checking, I mean that if I mention a CONSTANT_Class in the CP,
and its corresponding CONSTANT_Utf8 has a broken syntax (e.g., "." instead
of "/", or two "//" in a row) the JVMS mandates an error.  But those are the
same bytes I want to look at when they are referenced by a MoN or NMs
attribute.  It will all be in cache, and the cycles to do the checks will be
undetectable.

— John

Re: Draft JVMS changes for Nestmates

Hi Dan,

First, thanks for getting to this so quickly! Much appreciated.

On 19/04/2017 4:42 AM, Dan Smith wrote:

Hi, all.

I've uploaded a draft of JVMS changes for JEP 181 "Align JVM Checks with Java
Language Rules for Nested Classes" to:
http://cr.openjdk.java.net/~dlsmith/private-access.html

Some comments below on my thinking in drafting the spec text, and on the JEP
generally.

JEP name

I don't know how permanent JEP names are supposed to be, but I'd prefer a different name at this
point. Something like: "Expanded JVM Access to Private Members"—shorter, focused on the
feature itself rather than its relationship to the Java language. Or maybe "Class Nests for
Access to Private Members".

The intent of the name is to aid in the contextual positioning of this
JEP ie address why do we even need this. The JEP addresses an
inconsistency between access controls in the Java programming language
and access controls in the JVM. The primary goal here is to align the
two, while at the same time allowing for additional usage of nestmates
in the future.

Terminology

The term "nest" is nice because it's short and mostly unspoiled by overloading in this
context (I think?); it's not great because it's informal and doesn't mean anything the first time
you hear it. I thought about something more clinical like "access control context", but
I'm not convinced that's an improvement. How do others feel?

I think John has been talking about "nests" for at least 4 years now. :)
It fits nicely with the existing terminology of "nested types" - it
seems quite natural that "nested types" form a "nest". I have no problem
with it being "informal" in some sense, but we are using it to define a
specific relationship between certain types.

The JEP uses "nest top" to describe the class that nest members reference; I prefer "host class",
which better describes the class's role and isn't tied to the Java "top level class" concept. I know we use
"host class" internally in Hotspot, perhaps when working with anonymous classes (of the JVM flavor), but I
think in that context it will ultimately mean the same thing? Are we comfortable repurposing the term in this way?

Given that a nest-top type must be a top-level type, I think nest-top
fits perfectly. I dislike "host class" because that is already used for
the quite different case of VMACs. I can live with other names but I
think they should incorporate "nest" eg. nest-holder, nest-host. But I
think nest-top is an ideal match. (And no I didn't invent this
terminology :) ).

I follow Brian's model when it comes to nest membership (5.4.4): every class
belongs to a nest, possibly (in the absence of MemberOfNest) the nest hosted by
itself. Many nests are singletons without any associated explicit attributes.

I have not yet seen a follow up from John as to whether we require or
just allow, an empty NestMembers attribute to indicate a singleton nest
member.

Verification of MemberOfNest

I include a discussion block about different options of validating
MemberOfNest. I think the consensus, and my preference, is to do it during
verification of the member class. (NestMembers, on the other hand, is never
validated, except as a side-effect of checking MemberOfNest.)

Given verification is optional the current approach is to validate
during linking just prior to the verification step. Whether that can
conceptually be considered part of verification I am unsure.

Verification of invokespecial

Allowing invokespecial to refer to classes other than the current class and its
supers is a significant change. I noticed and relied heavily on the parallel
with invokevirtual making protected method calls. So I tried to make the two as
similar as possible. In a few places, the treatment of protected methods
doesn't seem ideal, and rather than trying to mirror that with non-private
invokespecial, I modified the protected method treatment.

This concerns me because protected-access is a somewhat complex/messy
issue so aligning private-access with it seems to go in the wrong
direction. Cleaning up the protected access rules, while perhaps
desirable, is technically out-of-scope for this JEP in my opinion.

The "protected check" of verification, in particular, was a mess before, and I
think I've made it a lot more manageable, and compatible with a parallel rule for
invokespecial. I could use some feedback on exactly what Hotspot's verifier has been
doing here, though, since I'm pretty sure it didn't match the old specified rules.

Might be better to take this up separately.

MethodHandle resolution

The spec (5.4.3.5) is vague about what errors can occur during MethodHandle
resolution. I assume any linkage error specified for the instruction can also
occur via MethodHandle resolution, and that will include failures due to
invokespecial improperly referencing a class.

This is an area I have least understanding of, but this seems to cover it:

Re: Draft JVMS changes for Nestmates


Hi John,

Aside: my replies to the experts list are held up for moderation.

On 19/04/2017 4:45 PM, John Rose wrote:

On Apr 18, 2017, at 1:49 PM, Dan Smith mailto:daniel.sm...@oracle.com>> wrote:




On Apr 18, 2017, at 2:12 PM, Remi Forax mailto:fo...@univ-mlv.fr>> wrote:


Verification of MemberOfNest

I include a discussion block about different options of validating
MemberOfNest.
I think the consensus, and my preference, is to do it during
verification of
the member class. (NestMembers, on the other hand, is never
validated, except
as a side-effect of checking MemberOfNest.)


It means that a NestMembers can contains Class that are not yet/never
defined.
It's not a problem for me.


Yep. Also, multiple classes can claim the same nest member class in
their NestMembers attributes. Not a problem as long as the
MemberOfNest attribute (if any) of the member class points to a host
class that claims it.


We need to specify more structural constraints on the new attributes.
If we make them more strongly normalizing, there will be fewer chances
for semantic bugs on unexpected inputs.  Also, making them structural
constraints means we check them early, during class file loading.

These are good, I think:

 - NMs must be non-empty (degenerate nest is never explicit)
 - NMs may not contain duplicates (cf. treatment of ClassFile.interfaces)
 - NMs may not contain the current class (i.e., an index to a class with
the same name as this_class)
 - MoN may not contain the current class (ditto)
 - MoN may contain only a package sibling (i.e., a referenced class name
must have the same package prefix as this_class)
 - NMs may contain only package siblings (ditto)
 - NMs and MoN may not refer to array classes (this is probably implied
by the package prefix checks)

You already covered:
 - a class may have exactly one or zero of NMs or MoN (no double dipping
of any kind)
 - NMs and MoN may only have CONSTANT_Class attributes (implies no
nulls, well-formed names)

David's prototype has the duplication check, and some of the other
checks happen later.  I think they should all happen during class loading.


Some of these are a lot more awkward to do during classfile parsing and 
will require symbol comparisons. I was wanting to avoid "deep 
validation" of NMs as it penalizes all the good code. Having a "bad" NM 
entry seems harmless as these entries are only used to validate the 
initial claim of nest membership. If an entry is "bad" then by 
definition it will not match with any claimee.


David
-


— John

Re: Draft JVMS changes for Nestmates

2017-04-19 Thread Brian Goetz

Yep. Also, multiple classes can claim the same nest member class in 
their NestMembers attributes. Not a problem as long as the 
MemberOfNest attribute (if any) of the member class points to a host 
class that claims it.



At the risk of bikeshedding, I find the names NestMembers and 
MemberOfNest confusing; I keep having to think about the directionality, 
since they sound so similar.  Would be good to find names that have 
obviously opposite directionality.


A not-very-good suggestion (but which illustrates the directionality 
thing I'm going for) would be "NestDeclaration" and "NestUse".  It's 
pretty clear which is which without thinking about it.


Perhaps: NestHost and NestMember ?

Re: Draft JVMS changes for Nestmates

On Apr 19, 2017, at 12:14 AM, David Holmes  wrote:
> 
> Some of these are a lot more awkward to do during classfile parsing and will 
> require symbol comparisons. I was wanting to avoid "deep validation" of NMs 
> as it penalizes all the good code. Having a "bad" NM entry seems harmless as 
> these entries are only used to validate the initial claim of nest membership. 
> If an entry is "bad" then by definition it will not match with any claimee.

I don't see how that is a significant concern.  The symbol bodies are hot in 
cache
at the point we would check prefixes, since they are already being scanned for 
other
purposes, such as initial interning, and also syntax checking.  Existing 
processing
is exactly as deep (or shallow) as the checks I want.

If we turn off the "verify" flag for class loading, then maybe we can buy 
something
by dropping those (and all the other) constraint checks.

By syntax checking, I mean that if I mention a CONSTANT_Class in the CP,
and its corresponding CONSTANT_Utf8 has a broken syntax (e.g., "." instead
of "/", or two "//" in a row) the JVMS mandates an error.  But those are the
same bytes I want to look at when they are referenced by a MoN or NMs
attribute.  It will all be in cache, and the cycles to do the checks will be
undetectable.

— John

Re: Draft JVMS changes for Nestmates