Re: Isolated Methods JEP

[Michael Haupt]

Hi John,

> Am 13.08.2016 um 01:48 schrieb John Rose :
> This does lead me to an extra thought:  I think IM's could be given the 
> capability to have multiple entry points.  This would allow a number of 
> interesting applications, starting with exception handling that appeals to a 
> helper object, instead of a handler table and some special verifier rules.
> 
> Immediately beyond that, although now I'm being a Mission Creep, if an IM has 
> multiple entry points for exception handling, it could also use these entry 
> points for frame resumption, as described by Ron Pressler in a previous 
> JVMLS, and Andrey Breslav this year.  (That leads to the question of how an 
> IM can yield to an suspension point, a very good question.  And something 
> that a helper object can help with.  And so on.  This might help us do fibers 
> experiments more quickly.)

multiple entry points would also, FWIW, enable OSR.

How to invoke a method with a specific entry point? Via Indy, with an extra 
argument?

The handler table that would go away if exception handling was conveniently 
pushed to a helper object would make a proud reappearance in the form of a 
valid entry point table. We don't want arbitrary jumps into the middle of 
things. :-)

Best,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 
München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 
3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
  Oracle is committed to developing 
practices and products that help protect the environment

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Michael Haupt]

Hi Jochen,

> Am 13.08.2016 um 12:21 schrieb Jochen Theodorou :
> 
> On 13.08.2016 00:54, John Rose wrote:
> [...]
>> Probably the IM is just a faster, more compact way to make something you
>> could also have made by spinning MH's together.  If someone figures out
>> what an IM can do that you can't also do with MethodHandles.* methods,
>> I'm very eager to hear about it.  (And I'll probably want to take it
>> away, or else add a new MHs API point.)  Mainly, an IM gives you control
>> over packaging and backtrace structure.
> 
> how would throwing and catching exceptions behave there? Can I do these 
> without showing the IM on the trace? Because... unless it got changed in the 
> meantime... using a MethodHandle that does catch Exceptions introduces a 
> trace element (and quite the speed penalty)


by default IMs should have an appearance in the stack trace. However, it may 
make sense to suppress their appearance in cases similar to those that can be 
used for LambdaForms (the @Hidden annotation comes to mind). This would depend 
strongly on the use case, and exposing this feature might be risky.

Best,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 
München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 
3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
  Oracle is committed to developing 
practices and products that help protect the environment

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Michael Haupt]

Hi Mark,

some degree of bytecode generator framework usage will be required with IMs all 
the same. The instructions have to come from somewhere. However, as John 
pointed out, the code may be substantially simpler in structure, which may lead 
to an overall more lightweight code installation process.

Regarding Smalltalk block implementation, I wonder how you'd deal with closing 
over elements of the lexical context. VarHandles, included in an IM's constants 
array, might play a role here.

Best,

Michael

> Am 15.08.2016 um 01:07 schrieb Mark Roos :
> 
> My expectations would be for faster load time and a simpler implementation. 
> In my application 
> I do think that the need to use a tool like ASM to handle the complexity of 
> class file generation is 
> the primary bottleneck. 
> 
> Getting rid of computing the various data structures and placing them in the 
> correct location was 
> the primary reason to not even start looking at writing the class file 
> directly.  Separating out the 
> bytecodes in nice, but for me the constants are the most difficult part.  In 
> my case this 
> is due to the constant pool being a combination of values used by the jvm as 
> parameters along with 
> values used directly as constants in my program.  The former is handled by 
> ASM as needed by the 
> jvm while the latter my compiler generates as a serialized representation of 
> a Smalltalk constant. 
> It would be nice if these were separable ( and of course bytes not Strings 
> for my constants) 
> 
> I do use line numbers but I also place them in the BSMs for my debugger.  The 
> only place which 
> not having them may matter is within some optimizations which cause back 
> jumps. I could recreate 
> these if I knew the byte code index anyway.   
> 
> Since Smalltalk uses blocks quite a bit I do have more than one method in 
> each of my class files. 
> Each block within the method creates an additional static method.  The idea 
> of helper methods 
> sounds intriguing.  Since different methods can have different stack frames 
> having a view method 
> would be handy as would the source code lookup and debugger helpers. 
> 
> I use exceptions to handle the Smalltalk return in block ( [^foo] and 
> Smalltalk thrown exceptions.   
> I do like the fiber ideas.  It would be interesting to try to make the fiber 
> idea a standard 
> feature.  This seems like a way to get stack manipulation as well even 
> without reification. 
> 
> Ah yes, mission creep.  I would be happy with a fast defineClass with the GC 
> improvements John 
> mentioned. 
> 
> regards 
> mark

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 
München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 
3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
  Oracle is committed to developing 
practices and products that help protect the environment

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Mark Roos]

John mentioned
        There's a fiber frame
system struggling to emerge here.

This is interesting.  The approaches
I have looked at involve lots of code
rewriting especially to capture the
stack as its unwound.  Are you thinking
of some approach where the exception
could capture the stack as it was
unwound?  I am sure this is a major
security issue but it would be useful
to build debuggers with.  Perhaps
a cmd line flag.

mark

OF course I would like to inspect copy
and modify the stack -)
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Mark Roos]

My expectations would be for faster load
time and a simpler implementation. In my application
I do think that the need to use a tool
like ASM to handle the complexity of class file generation is
the primary bottleneck.

Getting rid of computing the various
data structures and placing them in the correct location was
the primary reason to not even start
looking at writing the class file directly.  Separating out the
bytecodes in nice, but for me the constants
are the most difficult part.  In my case this
is due to the constant pool being a
combination of values used by the jvm as parameters along with
values used directly as constants in
my program.  The former is handled by ASM as needed by the
jvm while the latter my compiler generates
as a serialized representation of a Smalltalk constant.
It would be nice if these were separable
( and of course bytes not Strings for my constants)

I do use line numbers but I also place
them in the BSMs for my debugger.  The only place which
not having them may matter is within
some optimizations which cause back jumps. I could recreate
these if I knew the byte code index
anyway.  

Since Smalltalk uses blocks quite a
bit I do have more than one method in each of my class files.
Each block within the method creates
an additional static method.  The idea of helper methods
sounds intriguing.  Since different
methods can have different stack frames having a view method
would be handy as would the source code
lookup and debugger helpers.

I use exceptions to handle the Smalltalk
return in block ( [^foo] and Smalltalk thrown exceptions.  
I do like the fiber ideas.  It
would be interesting to try to make the fiber idea a standard 
feature.  This seems like a way
to get stack manipulation as well even without reification.

Ah yes, mission creep.  I would
be happy with a fast defineClass with the GC improvements John
mentioned.

regards
mark
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Jochen Theodorou]

On 13.08.2016 00:54, John Rose wrote:
[...]

Probably the IM is just a faster, more compact way to make something you
could also have made by spinning MH's together.  If someone figures out
what an IM can do that you can't also do with MethodHandles.* methods,
I'm very eager to hear about it.  (And I'll probably want to take it
away, or else add a new MHs API point.)  Mainly, an IM gives you control
over packaging and backtrace structure.


how would throwing and catching exceptions behave there? Can I do these 
without showing the IM on the trace? Because... unless it got changed in 
the meantime... using a MethodHandle that does catch Exceptions 
introduces a trace element (and quite the speed penalty)


bye Jochen

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Peter Levart]

Hi Michael, John,


On 08/13/2016 12:46 AM, John Rose wrote:
So there will be no class hosting this method? Not even the "caller" 
class of the Lookup instance that loadCode was invoked upon? What 
will be reported by Reflection.getCallerClass() invoked in a 
@CallerSensitive method invoked from the isolated method?


That last question is easy:  When you create a MH on a @CS method, via 
the Lookup API, the caller is permanently bound to the caller class in 
the Lookup object.  (If you are wondering about security, note that 
you can't get a MH on a @CS method without a full-power Lookup 
object.)  After that, it is irrelevant where and how the MH is 
eventually invoked, including from an isolated method.



On 08/13/2016 12:46 AM, John Rose wrote:
A can see now that defining an isolated method in the context of a 
lookup class can only be allowed when the Lookup instance contains 
ALL_MODES in the mode bits as such method would have access to any 
members that a normal method defined in the lookup class has.


Not sure. I think this suggestion implies something like class 
membership, when in fact the IM should have the access rights defined 
by the Lookup, without belonging to the lookupClass. I may be 
misunderstanding your point.


Since access rights are permanently bound into the MH's stored in the 
isolated constant pool, there should be little or no reason to ask 
about access rights for the IM itself.  Access rights are checked 
during symbolic resolution of class, method, and field references. 
 (Class object access rights are a tricky case, since classes are easy 
to grab.)  Inside the IM constant pool, all references are "live", and 
previously resolved.  There's no reason for the IM to re-resolve them.



...Ah, I can see it now that I have once more looked at the 
specification about the types of entries in constant pool array:


CONSTANT_Methodref_info: a java.lang.invoke.DirectMethodHandle of 
the right kind, obtained via the appropriate API in 
java.lang.invoke.MethodHandles.Lookup


So all methods will be pre-resolved. There will be no symbolic 
references in the isolated methods. The name "isolated" has a clear 
meaning now. Thanks.



Regards, Peter

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[John Rose]

On Aug 12, 2016, at 2:56 PM, Michael Haupt  wrote:
> 
> Hi Mark,
> 
>> Am 05.08.2016 um 12:13 schrieb Mark Roos > >:
>> Good to see this.
> 
> thanks. :-)
> 
>> What is the philosophy for code annotations, line numbers and stack maps? 
>> 
>> Would they be some form of constant entry? 
> 
> I recall you had asked this question at JVMLS. I think it definitely makes 
> sense to add support for these. How would you think they should be 
> represented?
> 
> Here are some thoughts.
> 
> First, stack maps. Leave their generation to the bytecode generation 
> framework responsible for producing the bytecode array (whichever one that 
> may be), allow for easy extraction of the map from its generation result, and 
> for passing that on to the loadCode implementation. This will require a 
> library-level representation of stack maps (inspired by Sec. 4.7.4 of the JVM 
> Spec), unless the plain class file format for them is used - which may not be 
> desirable given the convenience attempts the API is making with the constants 
> array. To maintain convenience, stack map generation could be the 
> responsibility of the loadCode implementation. This would duplicate the work 
> of generating the map, which seems rather unwelcome.

I think the right answer will involve making the verification of IM bytecodes 
simpler.  For example, if there is no invokespecial, all the nasty rules for 
that instruction go away (along with those for "new").  Frame maps are designed 
to support one-pass checking of bytecodes.  There are restrictions we can put 
on IM bytecodes that will recover the ability to perform one-pass checking, 
with implicitly derived frame maps.  In a nutshell, the first type-state wins, 
and any other type-states (e.g., from backward branches) need to match.  This 
makes it very slightly harder to generate IM bytecodes (with sometimes an extra 
checkcast for merge points), but in exchange nobody needs to mess with frame 
maps (except maybe VM internals).

This does lead me to an extra thought:  I think IM's could be given the 
capability to have multiple entry points.  This would allow a number of 
interesting applications, starting with exception handling that appeals to a 
helper object, instead of a handler table and some special verifier rules.

Immediately beyond that, although now I'm being a Mission Creep, if an IM has 
multiple entry points for exception handling, it could also use these entry 
points for frame resumption, as described by Ron Pressler in a previous JVMLS, 
and Andrey Breslav this year.  (That leads to the question of how an IM can 
yield to an suspension point, a very good question.  And something that a 
helper object can help with.  And so on.  This might help us do fibers 
experiments more quickly.)

> Annotations can be passed as a map from annotation classes to lists of 
> values. Line numbers can be passed as an int array of pairs of numbers, where 
> even indices represent start PCs, and odd indices, line numbers.

> Adding to your list, a source file association can be represented as a URL or 
> String. (Cream on top: a multiline String containing the IM's actual source 
> code.)

I hope all that logic can be wrapped up in a helper object, to be executed only 
when needed.  I.e., load quickly *with no paperwork*, and have the helper 
object figure out what happened later on, if we get asked.

interface IsolatedMethod { // reflected view of raw IM
  // recovered arguments to Lookup.loadCode:
  Lookup lookup();
  String name();
  MethodType type();
  ByteSequence bytes();  // NOTE: R/O view; same type for CONSTANT_Bytes
  List constants();

  // this has to be stuck in somehow:
  //Inspector inspector();

  interface Inspector {  // helper for "paperwork"
String sourceFile(IsolatedMethod m);
String className(IsolatedMethod m);
int mapLineNumber(IsolatedMethod m, int bci);
List mapLocalVariables(IsolatedMethod m, int bci);
List mapExceptionHandlers(IsolatedMethod m, int bci);
List mapFrameState(IsolatedMethod m, int bci);
...
//?MethodHandleInfo methodHandleInfo(IsolatedMethod m);
  }
}

> As a further addition, exceptions can be passed as a simple array or list, 
> and an exception handler table can be passed as a tuple (not in the Valhalla 
> sense, yet) of an int array of triplets of numbers representing start, end, 
> and handler PCs and of a list or array of exception classes.

Yes, exception handling is the tricky part (along with verification).  Maybe we 
could use an Inspector to help extract the details:

interface Inspector { …
  int exceptionContinuationPoint(IsolatedMethod m, int bci, Class exc);
}

The idea would be to call the IM inspector for a continuation point.  Of 
course, that thing might throw an exception, which makes the model muddy, 
though not impossible.  Verification of the CP could be lazy, or the CP's could 
be marked up front and checked during loadCode.

There's a fiber frame system struggling to

Re: Isolated Methods JEP

[John Rose]

On Aug 12, 2016, at 2:56 PM, Michael Haupt  wrote:
> 
> Hi Martijn,
> 
>> Am 09.08.2016 um 00:15 schrieb Martijn Verburg > >:
>> Love the overall idea.  A few thoughts (feel free to ignore as I'm by no 
>> means an expert in the VM).
> 
> thank you. As this JEP spans the VM and the libraries, with a definitely 
> client-facing part in MethodHandles.Lookup, user questions are important. :-)
> 
>> * Is there a risk that Gargantuan can expand to infinite size?
> 
> There is, as much as there is a risk that infinitely many classes are loaded 
> dynamically (e.g., via Unsafe.defineAnonymousClass()). Collecting IMs (and 
> the constants they introduce) is not out of the question, though. Once the 
> method handle representing an IM can be collected, the IM itself can be, too; 
> and its constants can unless they are shared.

Indeed, special GC work is one of the costs of IMs, since the normal 
class-scale storage management does not apply.  The benefit is you can load 
bits of code with low latency and low footprint, and they can go away as soon 
as you stop using them.  (The MH produced by the Lookup call is the only 
user-visible GC root to the "stuff" of the IM.  When that MH goes dead, the JVM 
can edit the big G class to reclaim space, at the JVM's option.)

>> * I'm really nervous about the security implications of being able to load 
>> an arbitrary byte[] that can be executed as a method. I'm hoping that the 
>> use of the internal Unsafe + Jigsaw's module boundaries makes this secure. 
>> Lots of really, really smart people have looked at the security implications 
>> of this, right? :-)
> 
> Does Unsafe.defineAnonymousClass() worry you? It allows just this: loading 
> some byte[] that can be executed as a method - albeit with the added 
> complexity of it being wrapped in a complete class file.
> 
> The choice of MethodHandles.Lookup as the place for loadCode() is deliberate. 
> A Lookup instance defines a context for visibility and access to other 
> elements, and the instructions in the byte[] array will be subject to the 
> scrutiny imposed by the Lookup instance at hand.
> 
> People smarter than me will say more about the security implications. This is 
> a JEP draft. There is no implementation yet, not even a prototype, and there 
> is a lot to be learned along the way.

See also my previous points about MH factories handling access checks, using 
pre-existing Lookup APIs.

Probably the IM is just a faster, more compact way to make something you could 
also have made by spinning MH's together.  If someone figures out what an IM 
can do that you can't also do with MethodHandles.* methods, I'm very eager to 
hear about it.  (And I'll probably want to take it away, or else add a new MHs 
API point.)  Mainly, an IM gives you control over packaging and backtrace 
structure.

>> * Gargantuan - Appreciate it's a placeholder but I can't say I like the 
>> name, it doesn't tell me what it does on the tin.
> 
> It's a literary reference that's supposed to imply "large" [1] - could've 
> used "Gulliver" but he's one of us. ;-)
> 
> Best,
> 
> Michael
> 
> [1] https://en.wikipedia.org/wiki/Gargantua_and_Pantagruel 
> 
I love the Online Etymology Dictionary for this sort of stuff:
  http://etymonline.com/index.php?term=gargantuan

— John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[John Rose]

On Aug 12, 2016, at 2:57 PM, Michael Haupt  wrote:
> 
> Hi Peter,
> 
> thanks for your feedback! I'm responding to both your messages in one.
> 
>> Am 09.08.2016 um 05:05 schrieb Peter Levart > >:
>> In the JEP's description you write:
>> 
>> "The loadCode method creates a method from the passed bytecode instructions 
>> and constants and returns a MethodHandle that can be used to call the 
>> method. The implementation of loadCode will take care of verification of the 
>> code to load."
>> 
>> That's clear.
>> 
>> "This method is isolated from any class and behaves largely like a static 
>> method. The method handle resulting from a loadCode invocation is of the 
>> REF_static kind. It cannot be cracked via 
>> MethodHandles.Lookup.revealDirect()."
>> 
>> So there will be no class hosting this method? Not even the "caller" class 
>> of the Lookup instance that loadCode was invoked upon? What will be reported 
>> by Reflection.getCallerClass() invoked in a @CallerSensitive method invoked 
>> from the isolated method?

That last question is easy:  When you create a MH on a @CS method, via the 
Lookup API, the caller is permanently bound to the caller class in the Lookup 
object.  (If you are wondering about security, note that you can't get a MH on 
a @CS method without a full-power Lookup object.)  After that, it is irrelevant 
where and how the MH is eventually invoked, including from an isolated method.

>> Perhaps the following is a hint...
>> 
>> "The context for a method defined in this way is determined by the Lookup 
>> instance receiving the loadCode call. In case the lookup privileges are not 
>> sufficient, an exception will be thrown."
>> 
>> The "context" meaning the caller context, including the caller class that 
>> appears in the stack trace of a call originating from an isolated method and 
>> is important for security decisions?
> 
> In all of this, I think it is important to not let the idea creep in that IMs 
> are members of some class. This is not to be the case.
> 
> With that out of the way ;-) let me stress that this is a very good question, 
> and there have in fact been discussions about this. The notion of "host" has 
> been floating around, and it currently seems that the lookupClass - to which 
> you refer as the "caller" - is a meaningful candidate for this.
> 
> Does this answer your question?

In general, we won't duplicate all the class-file infrastructure around the 
IM's.  So the surrounding class, local-variable tables, etc., etc., won't be 
allowed to "creep into" the IM design.  This makes some bits tricky, like "what 
does it look like on the back trace", and "where are the exception handler 
BCI's stored".  The general answer will be some combination of "there are 
simplified rules for IMs", and "some reflective tasks can be delegated to a 
helper object or a BSM-like helper method", and finally "use a full class file 
if you want more control".

>> On 08/09/2016 02:05 PM, Peter Levart wrote:
>>> In which case would lookup privileges be insufficient to define an isolated 
>>> method? I would expect the privileges of the code in the isolated method to 
>>> be checked when such method is executed and not when defining such method - 
>>> lazily, like for normal methods.
>> 
>> A can see now that defining an isolated method in the context of a lookup 
>> class can only be allowed when the Lookup instance contains ALL_MODES in the 
>> mode bits as such method would have access to any members that a normal 
>> method defined in the lookup class has.
> 
> 
> Not sure. I think this suggestion implies something like class membership, 
> when in fact the IM should have the access rights defined by the Lookup, 
> without belonging to the lookupClass. I may be misunderstanding your point.

Since access rights are permanently bound into the MH's stored in the isolated 
constant pool, there should be little or no reason to ask about access rights 
for the IM itself.  Access rights are checked during symbolic resolution of 
class, method, and field references.  (Class object access rights are a tricky 
case, since classes are easy to grab.)  Inside the IM constant pool, all 
references are "live", and previously resolved.  There's no reason for the IM 
to re-resolve them.

>> In which case it is important which "caller" class the privileges are check 
>> against. This would most naturally be the "caller" class of the Lookup 
>> instance used to define the method. In all respects the code of such 
>> isolated method would appear to be defined in the lookup class except it 
>> would not be denotable symbolically - only through a MethodHandle. Analogous 
>> to VM-anonymous classes. So why not call such methods "anonymous methods" 
>> instead of isolated methods?
> 
> 
> Most aspects of this JEP draft, including the name, are open for discussion. 
> :-)
> 
> I would not want to call an IM "anonymous" because it should be given a name. 
> If it is supposed

Re: Isolated Methods JEP

[Michael Haupt]

Hi Peter,

thanks for your feedback! I'm responding to both your messages in one.

> Am 09.08.2016 um 05:05 schrieb Peter Levart  >:
> In the JEP's description you write:
> 
> "The loadCode method creates a method from the passed bytecode instructions 
> and constants and returns a MethodHandle that can be used to call the method. 
> The implementation of loadCode will take care of verification of the code to 
> load."
> 
> That's clear.
> 
> "This method is isolated from any class and behaves largely like a static 
> method. The method handle resulting from a loadCode invocation is of the 
> REF_static kind. It cannot be cracked via 
> MethodHandles.Lookup.revealDirect()."
> 
> So there will be no class hosting this method? Not even the "caller" class of 
> the Lookup instance that loadCode was invoked upon? What will be reported by 
> Reflection.getCallerClass() invoked in a @CallerSensitive method invoked from 
> the isolated method? Perhaps the following is a hint...
> 
> "The context for a method defined in this way is determined by the Lookup 
> instance receiving the loadCode call. In case the lookup privileges are not 
> sufficient, an exception will be thrown."
> 
> The "context" meaning the caller context, including the caller class that 
> appears in the stack trace of a call originating from an isolated method and 
> is important for security decisions?

In all of this, I think it is important to not let the idea creep in that IMs 
are members of some class. This is not to be the case.

With that out of the way ;-) let me stress that this is a very good question, 
and there have in fact been discussions about this. The notion of "host" has 
been floating around, and it currently seems that the lookupClass - to which 
you refer as the "caller" - is a meaningful candidate for this.

Does this answer your question?

> On 08/09/2016 02:05 PM, Peter Levart wrote:
>> In which case would lookup privileges be insufficient to define an isolated 
>> method? I would expect the privileges of the code in the isolated method to 
>> be checked when such method is executed and not when defining such method - 
>> lazily, like for normal methods.
> 
> A can see now that defining an isolated method in the context of a lookup 
> class can only be allowed when the Lookup instance contains ALL_MODES in the 
> mode bits as such method would have access to any members that a normal 
> method defined in the lookup class has.


Not sure. I think this suggestion implies something like class membership, when 
in fact the IM should have the access rights defined by the Lookup, without 
belonging to the lookupClass. I may be misunderstanding your point.

> In which case it is important which "caller" class the privileges are check 
> against. This would most naturally be the "caller" class of the Lookup 
> instance used to define the method. In all respects the code of such isolated 
> method would appear to be defined in the lookup class except it would not be 
> denotable symbolically - only through a MethodHandle. Analogous to 
> VM-anonymous classes. So why not call such methods "anonymous methods" 
> instead of isolated methods?


Most aspects of this JEP draft, including the name, are open for discussion. :-)

I would not want to call an IM "anonymous" because it should be given a name. 
If it is supposed to appear in a stack trace, that would be most helpful. The 
class to be displayed in this case can be the "host", but it should be 
displayed differently than usual to avoid the notion of class membership.

Best,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 
München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 
3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
  Oracle is committed to developing 
practices and products that help protect the environment

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Michael Haupt]

Hi Mark,

> Am 05.08.2016 um 12:13 schrieb Mark Roos :
> Good to see this.

thanks. :-)

> What is the philosophy for code annotations, line numbers and stack maps? 
> 
> Would they be some form of constant entry? 

I recall you had asked this question at JVMLS. I think it definitely makes 
sense to add support for these. How would you think they should be represented?

Here are some thoughts.

First, stack maps. Leave their generation to the bytecode generation framework 
responsible for producing the bytecode array (whichever one that may be), allow 
for easy extraction of the map from its generation result, and for passing that 
on to the loadCode implementation. This will require a library-level 
representation of stack maps (inspired by Sec. 4.7.4 of the JVM Spec), unless 
the plain class file format for them is used - which may not be desirable given 
the convenience attempts the API is making with the constants array. To 
maintain convenience, stack map generation could be the responsibility of the 
loadCode implementation. This would duplicate the work of generating the map, 
which seems rather unwelcome.

Annotations can be passed as a map from annotation classes to lists of values. 
Line numbers can be passed as an int array of pairs of numbers, where even 
indices represent start PCs, and odd indices, line numbers.

Adding to your list, a source file association can be represented as a URL or 
String. (Cream on top: a multiline String containing the IM's actual source 
code.) As a further addition, exceptions can be passed as a simple array or 
list, and an exception handler table can be passed as a tuple (not in the 
Valhalla sense, yet) of an int array of triplets of numbers representing start, 
end, and handler PCs and of a list or array of exception classes.

Best,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 
München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 
3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
  Oracle is committed to developing 
practices and products that help protect the environment

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Michael Haupt]

Hi Martijn,

> Am 09.08.2016 um 00:15 schrieb Martijn Verburg :
> Love the overall idea.  A few thoughts (feel free to ignore as I'm by no 
> means an expert in the VM).

thank you. As this JEP spans the VM and the libraries, with a definitely 
client-facing part in MethodHandles.Lookup, user questions are important. :-)

> * Is there a risk that Gargantuan can expand to infinite size?

There is, as much as there is a risk that infinitely many classes are loaded 
dynamically (e.g., via Unsafe.defineAnonymousClass()). Collecting IMs (and the 
constants they introduce) is not out of the question, though. Once the method 
handle representing an IM can be collected, the IM itself can be, too; and its 
constants can unless they are shared.

> * I'm really nervous about the security implications of being able to load an 
> arbitrary byte[] that can be executed as a method. I'm hoping that the use of 
> the internal Unsafe + Jigsaw's module boundaries makes this secure. Lots of 
> really, really smart people have looked at the security implications of this, 
> right? :-)

Does Unsafe.defineAnonymousClass() worry you? It allows just this: loading some 
byte[] that can be executed as a method - albeit with the added complexity of 
it being wrapped in a complete class file.

The choice of MethodHandles.Lookup as the place for loadCode() is deliberate. A 
Lookup instance defines a context for visibility and access to other elements, 
and the instructions in the byte[] array will be subject to the scrutiny 
imposed by the Lookup instance at hand.

People smarter than me will say more about the security implications. This is a 
JEP draft. There is no implementation yet, not even a prototype, and there is a 
lot to be learned along the way.

> * Gargantuan - Appreciate it's a placeholder but I can't say I like the name, 
> it doesn't tell me what it does on the tin.

It's a literary reference that's supposed to imply "large" [1] - could've used 
"Gulliver" but he's one of us. ;-)

Best,

Michael

[1] https://en.wikipedia.org/wiki/Gargantua_and_Pantagruel

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992 
München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 
3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
  Oracle is committed to developing 
practices and products that help protect the environment

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Peter Levart]

Hi Michael,


On 08/09/2016 02:05 PM, Peter Levart wrote:
In which case would lookup privileges be insufficient to define an 
isolated method? I would expect the privileges of the code in the 
isolated method to be checked when such method is executed and not 
when defining such method - lazily, like for normal methods.


A can see now that defining an isolated method in the context of a 
lookup class can only be allowed when the Lookup instance contains 
ALL_MODES in the mode bits as such method would have access to any 
members that a normal method defined in the lookup class has.


Regards, Peter

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Peter Levart]

Hi Michael,

In the JEP's description you write:

   /"The loadCode method creates a method from the passed bytecode
   instructions and constants and returns a MethodHandle that can be
   used to call the method. The implementation of loadCode will take
   care of verification of the code to load.//"/

//
That's clear.

//

   /"This method is isolated from any class and behaves largely like a
   static method. The method handle resulting from a loadCode
   invocation is of the REF_static kind. It cannot be cracked via
   MethodHandles.Lookup.revealDirect().//"/


So there will be no class hosting this method? Not even the "caller" 
class of the Lookup instance that loadCode was invoked upon? What will 
be reported by Reflection.getCallerClass() invoked in a @CallerSensitive 
method invoked from the isolated method? Perhaps the following is a hint...


//

   /"The context for a method defined in this way is determined by the
   Lookup instance receiving the loadCode call. In case the lookup
   privileges are not sufficient, an exception will be thrown.//"

   /

The "context" meaning the caller context, including the caller class 
that appears in the stack trace of a call originating from an isolated 
method and is important for security decisions?


In which case would lookup privileges be insufficient to define an 
isolated method? I would expect the privileges of the code in the 
isolated method to be checked when such method is executed and not when 
defining such method - lazily, like for normal methods. In which case it 
is important which "caller" class the privileges are check against. This 
would most naturally be the "caller" class of the Lookup instance used 
to define the method. In all respects the code of such isolated method 
would appear to be defined in the lookup class except it would not be 
denotable symbolically - only through a MethodHandle. Analogous to 
VM-anonymous classes. So why not call such methods "anonymous methods" 
instead of isolated methods?



Regards, Peter


On 08/05/2016 06:33 PM, Michael Haupt wrote:

Dear all,

during the Indy workshop at JVMLS, a JEP draft supposed to enable the 
loading of methods in isolation was mentioned and discussed. This JEP 
draft is now public, and I'd like to invite The Interested Parties 
(you know who you are) to tear it apart in a constructive manner. :-)


https://bugs.openjdk.java.net/browse/JDK-8158765

Thanks,

Michael

--

Oracle 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
OracleJava Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 
Potsdam, Germany


ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, 
D-80992 München

Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 
163/167, 3543 AS Utrecht, Niederlande

Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
Green Oracle  	Oracle is committed 
to developing practices and products that help protect the environment





___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Martijn Verburg]

Hi Michael,

Love the overall idea.  A few thoughts (feel free to ignore as I'm by no
means an expert in the VM).

* Is there a risk that Gargantuan can expand to infinite size?

* I'm really nervous about the security implications of being able to load
an arbitrary byte[] that can be executed as a method. I'm hoping that the
use of the internal Unsafe + Jigsaw's module boundaries makes this secure.
Lots of really, really smart people have looked at the security
implications of this, right? :-)

* Gargantuan - Appreciate it's a placeholder but I can't say I like the
name, it doesn't tell me what it does on the tin.

Cheers,
Martijn

On 5 August 2016 at 17:33, Michael Haupt  wrote:

> Dear all,
>
> during the Indy workshop at JVMLS, a JEP draft supposed to enable the
> loading of methods in isolation was mentioned and discussed. This JEP draft
> is now public, and I'd like to invite The Interested Parties (you know who
> you are) to tear it apart in a constructive manner. :-)
>
> https://bugs.openjdk.java.net/browse/JDK-8158765
>
> Thanks,
>
> Michael
>
> --
>
> [image: Oracle] 
> Dr. Michael Haupt | Principal Member of Technical Staff
> Phone: +49 331 200 7277 | Fax: +49 331 200 7561
> Oracle Java Platform Group | LangTools Team | Nashorn
> Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam,
> Germany
>
> ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstraße 25, D-80992
> München
> Registergericht: Amtsgericht München, HRA 95603
>
> Komplementärin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering
> 163/167, 3543 AS Utrecht, Niederlande
> Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
> Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
> [image: Green Oracle]  Oracle is
> committed to developing practices and products that help protect the
> environment
>
>
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
>
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[Mark Roos]

Good to see this.

What is the philosophy for code annotations,
line numbers and stack maps?

Would they be some form of constant
entry?

regards
mark
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev