Re: invokespecial-super-init

[John Rose]

I, not generated by Groovy
interface WC { } // marker type for wrapper constructors
class C {  // generated by Groovy
  final char p, q;
  private static class Finals { final char p, q; }
  private C(WC ig, int x, MethodHandle finals) { super(x); Finals f = 
finals.invokeExact(this); this.p=finals.p; this.q=finals.q; }
  private C(WC ig, String y, MethodHandle finals) { super(y); Finals f = 
finals.invokeExact(this); this.p=finals.p; this.q=finals.q; }
  public C(int x, boolean z) { this(null, x, MH.bindTo(z)); }
  public C(String y, boolean z) { this(null, y, MH.bindTo(z)); }
  // public C(DynamicArgList dynamicArgs) { this(no can do!); super(this 
neither!); }
}

This pattern is approximately as general as random bytecodes inside 
constructors, is reasonably compact, and does not require new method handle 
types or verifier rules.

(Note that the MH "finals" is able to "see" the UI under the type C.  It is 
supposed to treat it reasonably, just like constructor code is supposed to.  
Since the wrapper constructors are marked private, it is impossible for 
untrusted parties to inject malicious MH code.  The MH could be replaced by a 
private instance method, if there is no need to have a different MH at 
different construction sites.)

What do you think?  Is this close to the workarounds you already use?

— John

> bye jochen
> 
> 
> Am 29.08.2015 03:40, schrieb John Rose:
>> The invokespecial-super-init dance is the thing MH's can't quite do, the 
>> "super" call every constructor (except Object.).
>> 
>> It very hard to secure this pattern; just ask anybody who has worked on 
>> (de-)serialization security.
>> 
>> But, we can look at it from a more limited point of view which might improve 
>> your use case, Jochen.
>> 
>> A method handle is supposed to be a fully competent replacement for 
>> hardwired bytecodes, and it is, except for invokespecial-super from a 
>> constructor.  The reason this is hard is that there is no way to constrain 
>> such a method handle, once constructed, to operate inside a constructor.  
>> And superclasses have a right to expect that you are running their 
>> constructor as a unique, non-repeatable part of creating a subclass object.  
>> (By "have a right" I really mean "it would be wrong to do the unexpected" by 
>> which I also mean "attack surfaces are likely to open up if we do this.)
>> 
>> So, is there a way to package up a method handle so that it can only be used 
>> as as unique, non-repeatable part of creating a subclass object?  Yes, it 
>> can:  Wire in an unconditional "new instance" operation, and immediately run 
>> the "invokespecial super" on the new thing.
>> 
>> Now the problem reduces to:  Your class (just like its super) has a right to 
>> expect that constructor code will be run on every newly-created instance 
>> (after the super constructor), before the new object is made available to 
>> other code.  Can we package up the previous new-invokespecial-super method 
>> handle so it can only be used in this way?  Well, no, since every 
>> constructor *also* has a hardwired call to invokespecial; we are back to the 
>> pre-existing new-invokespecial type of MH.
>> 
>> There are several possible ways out, but the problem is delicate.  The 
>> purpose of constructors is to statically mark code that must be executed 
>> before any (normally published) reference to an object is reachable by 
>> non-class code.  If there were a way to statically mark code as 
>> "post-super-init" (""?), we could make an agreement with a 
>> class that such a method would serve as the equivalent of a constructor, but 
>> it would be the caller's responsibility to allocate the new instance *and* 
>> call the super init.  Allowing bytecode to call this stuff would require a 
>> bunch of new verifier rules, in a place where the verifier is already hard 
>> to understand.  Perhaps a method handle could be allowed to operate where 
>> normal bytecode cannot, but you see the problem:  Method handles are 
>> designed to give a dynamic alternative to things you can already do in 
>> bytecode.
>> 
>> The "post-super-init" convention can be a private convention within a class, 
>> in the special case of Groovy, since Groovy is responsible for generating 
>> the whole class, and can trust itself to invoke all necessary initialization 
>> code on each new instance.  So if you had an new-invokespecial-super MH in a 
>> private context within a Groovy-generated class, you could use it to create 
>> a "mostly blank" instance, and then fill it in before sharing it with 
&g

Re: invokespecial-super-init

[John Rose]

On Sep 17, 2015, at 10:10 AM, Michael Haupt  wrote:
> 
> ummm ... this seems to imply I can remove the findSuperConstructor() proposal 
> from the Indy JEP. Incidentally, it's on my list for this week - and less 
> work is always good. ;-) Even if, in this case, it leads to disappointment. I 
> agree with you in that opening up the MH API like this will introduce several 
> trapdoors and additional complication.
> 
> Please let me know ...

Hi Michael.  See previous message.  It looks like (in most cases, mostly) the 
burden can be put back on the bytecode generators, like Groovy.

In a couple of cases we might want to add API for these use cases:

1. The workarounds are so complex and error-prone that a convenience function 
is needed.  (As with PICs, etc., requires a matured notion of what the 
convenience should be.)

2. We might still need some sort of multi-way super. call, but I think 
there are bytecode-level workarounds for this also, that verify today.  (The 
multi-way super comes is visible in the "no can do!" comment of my POC code.)

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: invokespecial-super-init

[Jochen Theodorou]

hi John,

thanks for replying...

After having read that, I think part of the problem actually comes from 
this new-invokespecial-super being split in two bytecodes. It means 
there can be a lot of things in between, including different paths. This 
makes the Verifier difficult.


The other part is that I need to react to runtime types. Currently this 
is only possible by using a generic handle, that will install the real 
target later on... With the problem, that the first call of the target 
is done from inside the generic handle, instead of the callsite. In 
terms of object creation, this means I will have access to the object, 
and in case of super-init-calls it would mean me having access to a not 
fully initialized class and potentially doing bad things here.


And that is even though I don't even need a handle that returns 
something. But since there is no real connection between slot 0 of the 
constructor I am in and the generic handle


But I wonder if there is really no way around that. Let me construct 
something crazy here... What would be if we had a dummy object instead? 
Let us call it GenericInstance for now. Generic Instance is internally 
connected to the partially generated class, but has no fields or methods 
offering access to it. The only way to create a GenericInstance would be 
by a factory method, from the indy API, like findSpecialConstructor or 
such. I would define the signature that it returns the GenericInstance. 
The handle itself is supposed to realize a new-transform 
arguments-invokeSpecial kind of sequence. The Verifier thus needs to 
acknowledge it to do that. And there needs to be code, that takes the 
result of the GenericInstance and then places the real instance in 
variable slot 0.


Since it is a two fold mechanism I cannot programatically do anything 
with the GenericInstance object, but to reach it through. Only the part 
unwrapping it can access the real instance (and also check the class to 
be sure) and that would be VM code.


I think this way splitting the method or have a constructor equivalent 
is not required... but I am not sure something like GenericInstance can 
be done. In pure Java probably not



bye jochen


Am 29.08.2015 03:40, schrieb John Rose:

The invokespecial-super-init dance is the thing MH's can't quite do, the super call 
every constructor (except Object.init).

It very hard to secure this pattern; just ask anybody who has worked on 
(de-)serialization security.

But, we can look at it from a more limited point of view which might improve 
your use case, Jochen.

A method handle is supposed to be a fully competent replacement for hardwired bytecodes, and it is, 
except for invokespecial-super from a constructor.  The reason this is hard is that there is no way to 
constrain such a method handle, once constructed, to operate inside a constructor.  And superclasses 
have a right to expect that you are running their constructor as a unique, non-repeatable part of 
creating a subclass object.  (By have a right I really mean it would be wrong to do 
the unexpected by which I also mean attack surfaces are likely to open up if we do this.)

So, is there a way to package up a method handle so that it can only be used as as unique, 
non-repeatable part of creating a subclass object?  Yes, it can:  Wire in an unconditional 
new instance operation, and immediately run the invokespecial super on the 
new thing.

Now the problem reduces to:  Your class (just like its super) has a right to 
expect that constructor code will be run on every newly-created instance (after 
the super constructor), before the new object is made available to other code.  
Can we package up the previous new-invokespecial-super method handle so it can 
only be used in this way?  Well, no, since every constructor *also* has a 
hardwired call to invokespecial; we are back to the pre-existing 
new-invokespecial type of MH.

There are several possible ways out, but the problem is delicate.  The purpose of constructors is to 
statically mark code that must be executed before any (normally published) reference to an object is 
reachable by non-class code.  If there were a way to statically mark code as post-super-init 
(postsuperinit?), we could make an agreement with a class that such a method would 
serve as the equivalent of a constructor, but it would be the caller's responsibility to allocate the new 
instance *and* call the super init.  Allowing bytecode to call this stuff would require a bunch of new 
verifier rules, in a place where the verifier is already hard to understand.  Perhaps a method handle 
could be allowed to operate where normal bytecode cannot, but you see the problem:  Method handles are 
designed to give a dynamic alternative to things you can already do in bytecode.

The post-super-init convention can be a private convention within a class, in the special case of 
Groovy, since Groovy is responsible for generating the whole class, and can trust itself

invokespecial-super-init

[John Rose]

The invokespecial-super-init dance is the thing MH's can't quite do, the 
super call every constructor (except Object.init).

It very hard to secure this pattern; just ask anybody who has worked on 
(de-)serialization security.

But, we can look at it from a more limited point of view which might improve 
your use case, Jochen.

A method handle is supposed to be a fully competent replacement for hardwired 
bytecodes, and it is, except for invokespecial-super from a constructor.  The 
reason this is hard is that there is no way to constrain such a method handle, 
once constructed, to operate inside a constructor.  And superclasses have a 
right to expect that you are running their constructor as a unique, 
non-repeatable part of creating a subclass object.  (By have a right I really 
mean it would be wrong to do the unexpected by which I also mean attack 
surfaces are likely to open up if we do this.)

So, is there a way to package up a method handle so that it can only be used as 
as unique, non-repeatable part of creating a subclass object?  Yes, it can:  
Wire in an unconditional new instance operation, and immediately run the 
invokespecial super on the new thing.

Now the problem reduces to:  Your class (just like its super) has a right to 
expect that constructor code will be run on every newly-created instance (after 
the super constructor), before the new object is made available to other code.  
Can we package up the previous new-invokespecial-super method handle so it can 
only be used in this way?  Well, no, since every constructor *also* has a 
hardwired call to invokespecial; we are back to the pre-existing 
new-invokespecial type of MH.

There are several possible ways out, but the problem is delicate.  The purpose 
of constructors is to statically mark code that must be executed before any 
(normally published) reference to an object is reachable by non-class code.  If 
there were a way to statically mark code as post-super-init 
(postsuperinit?), we could make an agreement with a class that such a 
method would serve as the equivalent of a constructor, but it would be the 
caller's responsibility to allocate the new instance *and* call the super init. 
 Allowing bytecode to call this stuff would require a bunch of new verifier 
rules, in a place where the verifier is already hard to understand.  Perhaps a 
method handle could be allowed to operate where normal bytecode cannot, but you 
see the problem:  Method handles are designed to give a dynamic alternative to 
things you can already do in bytecode.

The post-super-init convention can be a private convention within a class, in 
the special case of Groovy, since Groovy is responsible for generating the 
whole class, and can trust itself to invoke all necessary initialization code 
on each new instance.  So if you had an new-invokespecial-super MH in a private 
context within a Groovy-generated class, you could use it to create a mostly 
blank instance, and then fill it in before sharing it with anybody else.  Such 
an invokespecial-super MH could be adequately protected from other users by 
requiring that Lookup.findSpecialConstructor can only work on full-powered 
lookups, regardless of the accessibility of the super constructor.

There are two further problems with this, though.  First, constructors have a 
unique ability and obligation to initialize blank final variables (the 
non-static ones).  So the Lookup.findSpecialConstructor MH has to take an 
argument, not just for its super-constructor, but also for *each* final 
variable in the *current* class.  (Note that Lookup.findSetter will *not* allow 
finals to be set, because it cannot prove that the caller is somehow inside a 
constructor, and, even if inside it, is trustably acting on behalf of it.)  
There are other ways to go, but you can see this problem too:  The 
new-invokespecial operator has to take responsibility for working with the 
caller to fill in the blank finals.

The second further problem is even more delicate.  The JVM enforces rules of 
calling init even (sometimes) against the wishes of people who generate class 
files.  We don't fully understand the practical effects of relaxing these 
rules.  Proofs of assertions (such as type correctness and security) require 
strong premises, and the rigid rules about init help provide such premises.  
An example of a proof-failure would be somebody looking at a class, ensuring 
that all instances are secure based on the execution of init methods, but 
then fail to notice that the class *also* runs some instances through an 
alternate path, using new-invokespecial-super, which invalidates the proof by 
failing to run some crucial setup code.

With all that said, there is still wiggle room.  For example, one *possible* 
solution that might help Groovy, while being restrictive enough to avoid the 
problems above, would be to split init methods and sew them together again 
with method handles.

Suppose there were a reliable way