Re: guardIfConstant, the constant sniffer combinator

[John Rose]

This is very similar to a "growable switch" combinator,
which would call the provider for each distinct selector
value.  A key difference is that non-constant values
go through the fallback, where a "growable switch"
doesn't need a fallback, since the provider MH is
always free to just return a standard fallback as
its result (as is the case with your combinator).

I wonder if the concepts can be combined somehow?

Here's a test question:  When the JIT, after heroic
effort, discovers that an argument is constant, can
it run the provider (in the compiler thread???) to
determine a custom handler for the newly discovered
branch of constant code?  The answer is probably,
"sorry, no", although the JIT might set up an uncommon
trap (and/or an execution counter) that can revisit
the question at some point.

Here's a close point of correspondence between the
idea of a growable switch and your combinator:  Both
need a memory.  Both want to remember the appearance
of constants, so a later optimization phase can use the
full historical knowledge.

— John

> On Apr 10, 2019, at 2:47 PM, Remi Forax  wrote:
> 
> The problem is the following,
> with the java compiler intrinsic of amber, String.format() is optimized using 
> invokedynamic in the case the format (the first argument) is constant (and 
> some other conditions on the format), this is great, perf are like 25x in 
> simple benchmarks, and that all because in a lot of code, the format is not 
> constant for the Java compiler. 
> 
> By example,
>  class Logger {
>public static void log(String format, String message) {
>  System.err.println(String.format(format, message));
>}
>  }
>  ...
>  logger.log("%s", "hello");
> 
> The format is not a constant inside Logger::log for the Java compiler but 
> when the code is JITed, due to inlining, logger.log("hello") calls 
> String.format() with a constant.
> 
> 
> I propose a way to fix that, by providing a method handle combiner 
> (guardIfConstant) that detects if an argument is a constant and do something 
> different if it's a constant or not.
> It's a little more complex than that, we don't only want to have a separate 
> path if the argument is a constant, we also want to be able to build a method 
> handle tree depending on the value of that constant.
> 
>  MethodHandle guardIfConstant(int argument, MethodHandle targetProvider, 
> MethodHandle fallback)
> 
> the semantics: if the nth argument is a constant, the target provider is 
> called with that argument and the return value, a method handle, is called 
> with all the arguments, otherwise the fallback is called.
> 
> in term of method type:
>  - the method type of the return value of guardIfConstant is the same as 
> fallback
>  - the method type of targetProvider returns a MethodHandle and takes a 
> single parameter which is the nth parameter type of the fallback method type,
>the returned method handle as to have the same method type as the fallback.
> 
> Rémi
> 
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> https://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
https://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Strange observation: MethodHandle.invokeWithArguments() would not work, whereas Method.invoke() would with the very same arguments

[John Rose]

On Feb 12, 2018, at 11:59 AM, Rony G. Flatscher  wrote:
> 
> While testing a rather complex one (an adaption of the JavaFX address book 
> example enhanced with a
> BarChart, [1]), that exhibits a very strange behavior: when setting the 
> values for the CategoryAxis
> supplying an ObservableList of the month names in the current Locale, using a 
> MethodHandle and
> invoking it with invokeWithArguments() would yield (debug output):

I just happened to see your message about your adventures with method handle:

http://mail.openjdk.java.net/pipermail/jigsaw-dev/2018-February/013603.html 


This isn't really a jigsaw question, so I'm replying to mlvm-dev.

It looks like you are mixing or interconverting arrays of strings
with lists of strings.  The print statements and CCE show that
you are passing an array of strings into a place which expects
a single string, and the print statements suggest you are
in fact passing a list containing a string array into a place
which expects a list of strings.  Either way there are too
many brackets in your actual argument.

The prime suspect when the number of brackets is off by one
is varargs.  You code might be failing because of surprises
in the overload resolution of invokeWA, which accepts
a varargs Object array *and* a single List.

Is your runtime invoke mechanism treating invokeWA as an ordinary
method?  It is rather extraordinary, and may warrant a second look.

String str;
Object obj;
Object[] aobj;
String[] astr;
List lst;

plain single arity invocations:

1 mh.invokeWithArguments(str) => new Object[] { str }
2 mh.invokeWithArguments(obj) => new Object[] { obj }

and yet:

3 mh.invokeWithArguments(aobj) => aobj (multiple args)
4 mh.invokeWithArguments(astr) => astr (multiple args again!)
5 mh.invokeWithArguments(lst) => lst.toArray() (multiple args again!)

but again, a cast removes varargs:

6 mh.invokeWithArguments((Object) aobj) => new Object[] { aobj }
7 mh.invokeWithArguments((Object) astr) => new Object[] { astr }
8 mh.invokeWithArguments((Object) lst) => new Object[] { lst }

Your bug looks like a confusion between two of these,
perhaps 5 and 8.

Regards,
— John




___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Performance of non-static method handles

[John Rose]

Vladimir Ivanov did some work a few years ago on MH customization for hot MH 
instances. It’s in the system. That should get better results than what you 
show. I wonder why it isn’t kicking in. You are using invokeExact right?

> On Feb 2, 2018, at 1:26 PM, Charles Oliver Nutter  wrote:
> 
> Hey folks!
> 
> I'm running some simple benchmarks for my FOSDEM handles talk and wanted to 
> reopen discussion about the performance of non-static-final method handles.
> 
> In my test, I just try to call a method that adds given argument to a static 
> long. The numbers for reflection and static final handle are what I'd expect, 
> with the latter basically being equivalent to a direct call:
> 
> Direct: 0.05ns/call
> Reflected: 3ns/call
> static final Handle: 0.05ns/call
> 
> If the handle is coming from an instance field or local variable, however, 
> performance is only slightly faster than reflection. I assume the only real 
> improvement in this case is that it doesn't box the long value I pass in.
> 
> local var Handle: 2.7ns/call
> 
> What can we do to improve the performance of non-static method handle 
> invocation?
> 
> - Charlie
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Switch vs if ?

[John Rose]

On Jan 19, 2018, at 8:52 AM, fo...@univ-mlv.fr wrote:
> 
> 
> The underlying storage of the MH switcher should use a
> @Stable array of per-case MH's, allowing good inlining.
> For a growable switch, the array could either grow or split
> out into sub-arrays as appropriate, as a growing HAMT.
> 
> two questions, what is the semantics in case of concurrent updates, i suppose 
> it's like calling setTarget() from multiple threads ?

This is ranging out of the field of hotspot-compiler.  Moving to mlvm-dev.
(See below for previous message.)

My take:

It's even more like resolving a condy constant or indy call site:  You can have
races, and the JVM picks a winner and makes it visible forever everywhere.

This pattern differs from the classic mutex-protected variable, because
there is only one state transition, not many.  Java's static initializer
initialization also has a mutex, with one state transition.  Leaving out
the mutex allows multiple threads to perform BSM linkage with less
synchronization (at the cost of more parallel work, which is OK).

What we are modeling is code with a frontier which can grow.  Once
the code grows at some point, it is static forever afterward.  You could
model it as if the code has always been there statically but had to be
"discovered" by the bootstrap method.

Actual mutable code (where the code has one shape to start and
then changes to a semantically different shape) is a very special
feature supported by SwitchPoint and CallSite.setTarget.  If you want
that in your switch, you can build it into the switch by composition.

> and how to specify a switch on type (not only a switch on int/long) ?

I think the combinator should be able to operate over any key type
whatever (like HashMap).  It can have special sub-implementations
for integral types (esp. if the actual traffic is dense) or for Comparables
(using TreeMap).

I suppose the harder question is how to deal with key weakness.
Maybe the combinator is given a Map class or factory?

— John



On Jan 19, 2018, at 8:52 AM, fo...@univ-mlv.fr wrote:
> 
> 
> 
> De: "John Rose" <john.r.r...@oracle.com>
> À: "Rémi Forax" <fo...@univ-mlv.fr>
> Cc: "Roland Westrelin" <rwest...@redhat.com>, "Laurent Bourgès" 
> <bourges.laur...@gmail.com>, "hotspot compiler" 
> <hotspot-compiler-...@openjdk.java.net>
> Envoyé: Vendredi 19 Janvier 2018 02:03:26
> Objet: Re: Switch vs if ?
> On Jan 17, 2018, at 2:42 PM, Remi Forax <fo...@univ-mlv.fr 
> <mailto:fo...@univ-mlv.fr>> wrote:
> 
> next we need a method handle combinator for the integer switch for amber.
> 
> With its own profiling, of course.
> 
> yes, that the part which is not easy to simulate without a combinator
> 
> 
> Also, method handle switch combinators have the potential to
> grow dynamically, unlike static switches in bytecode.
> 
> yes.
> 
> 
> This is a trick of PyPy, which "grows" basic blocks as
> they are executed; each polymorphic dispatch point works
> like an open-ended switch whose cases are all types
> *encountered so far* at that point by the program.
> 
> So the default branch of a MH switcher could act like
> an indy call site, and spin new cases whenever the old
> ones don't match.
> 
> yes, it means that the switcher has to be its own class like the SwitchPoint 
> because it provide a method to add a new case.
> 
> 
> The underlying storage of the MH switcher should use a
> @Stable array of per-case MH's, allowing good inlining.
> For a growable switch, the array could either grow or split
> out into sub-arrays as appropriate, as a growing HAMT.
> 
> two questions, what is the semantics in case of concurrent updates, i suppose 
> it's like calling setTarget() from multiple threads ?
> and how to specify a switch on type (not only a switch on int/long) ?
> 
> 
> — John
> 
> Rémi
> 

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Writing a compiler to handles, but filter seems to executed in reverse

[John Rose]

Thanks, IBM!!

Filed:  https://bugs.openjdk.java.net/browse/JDK-8194554

On Jan 3, 2018, at 12:04 PM, Remi Forax <fo...@univ-mlv.fr> wrote:
> 
> IBM implementation uses the left to right order !
> I've just tested with the latest Java 8 available.
> 
> Java(TM) SE Runtime Environment (build 8.0.5.7 - 
> pxa6480sr5fp7-20171216_01(SR5 FP7))
> IBM J9 VM (build 2.9, JRE 1.8.0 Linux amd64-64 Compressed References 
> 20171215_373586 (JIT enabled, AOT enabled)
> OpenJ9   - 5aa401f
> OMR  - 101e793
> IBM  - b4a79bf)
> 
> so it's an implementation bug, #2 seems to be the right solution.
> 
> Rémi
> 
> De: "John Rose" <john.r.r...@oracle.com>
> À: "Da Vinci Machine Project" <mlvm-dev@openjdk.java.net>
> Envoyé: Mercredi 3 Janvier 2018 20:37:42
> Objet: Re: Writing a compiler to handles, but filter seems to executed in 
> reverse
> On Jan 2, 2018, at 12:35 PM, Charles Oliver Nutter <head...@headius.com 
> <mailto:head...@headius.com>> wrote:
> 
> Is there a good justification for doing it this way, rather than having
> filterArguments start with the *last* filter nearest the target?
> 
> No, it's a bug.  The javadoc API spec. does not emphasize the ordering
> of the filter invocations, but the pseudocode makes it pretty clear what
> order things should come in.  Certainly the spec. does not promise the
> current behavior.  When I wrote the spec. I intended the Java argument
> evaluation order to apply, and the filters to be executed left-to-right.
> And then, when I wrote the code, I used an accumulative algorithm
> with a for-each loop, leading indirectly to reverse evaluation order.
> Oops.
> 
> There are two ways forward:
> 
> 1. Declare the spec. ambiguous, and document the current behavior
> as the de facto standard.
> 
> 2. Declare the spec. unambiguous, change the behavior to left-to-right
> as a bug fix, and clarify the spec.
> 
> I think we can try for #2, on the grounds that multiple filters are a rare
> occurrence.  The risk is that existing code that uses multiple filters *and*
> has side effect ordering constraints between the filters will break.
> 
> Question:  What does the IBM JVM do?  I think they have a very
> different implementation, and they are supposed to follow the spec.
> 
> — John
> 
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Writing a compiler to handles, but filter seems to executed in reverse

[John Rose]

On Jan 2, 2018, at 12:35 PM, Charles Oliver Nutter  wrote:
> 
> Is there a good justification for doing it this way, rather than having
> filterArguments start with the *last* filter nearest the target?

No, it's a bug.  The javadoc API spec. does not emphasize the ordering
of the filter invocations, but the pseudocode makes it pretty clear what
order things should come in.  Certainly the spec. does not promise the
current behavior.  When I wrote the spec. I intended the Java argument
evaluation order to apply, and the filters to be executed left-to-right.
And then, when I wrote the code, I used an accumulative algorithm
with a for-each loop, leading indirectly to reverse evaluation order.
Oops.

There are two ways forward:

1. Declare the spec. ambiguous, and document the current behavior
as the de facto standard.

2. Declare the spec. unambiguous, change the behavior to left-to-right
as a bug fix, and clarify the spec.

I think we can try for #2, on the grounds that multiple filters are a rare
occurrence.  The risk is that existing code that uses multiple filters *and*
has side effect ordering constraints between the filters will break.

Question:  What does the IBM JVM do?  I think they have a very
different implementation, and they are supposed to follow the spec.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Writing a compiler to handles, but filter seems to executed in reverse

[John Rose]

On Jan 2, 2018, at 12:36 PM, Charles Oliver Nutter  wrote:
> 
> An alternative workaround: I do the filters myself, manually, in the order
> that I want them to executed. Also gross.

(Under the theory that the spec. is ambiguous, leading to implementor's
choice, this would be your only option.)

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

[BSM-RETURNS-MH] allow invokedynamic BSM to return MethodHandle instead of CallSite

[John Rose]

As we use BSMs more and more, we will want to dispense with the CallSite
infrastructure, in many cases.  I filed an RFE to track this.  If we can agree
on specification, it will be fairly easy to implement.

(I am CC-ing this to valhalla-spec-experts, for visibility to non-OpenJDK 
people.)

— John

https://bugs.openjdk.java.net/browse/JDK-8188790
allow invokedynamic BSM to return MethodHandle instead of CallSite

Invokedynamic is defined to return a CallSite in order to allow a CallSite 
object to support per-call-site mutability, which enables certain advanced 
optimizations based on argument type speculation (aka. inline caching).

Also, for the same reason, invokedynamic instructions are individually linked, 
unlike other instructions which refer to the constant pool.  This allows each 
invokedynamic to operate on a separate "history" of type speculations, when the 
call site is mutable.

However, most uses of invokedynamic do not use these advanced degrees of 
freedom; most uses return a ConstantCallSite, and return the same call site (or 
an equivalent one) for each distinct occurrence of an invokedynamic.

This simpler use case can be referred to as "same CCS every time".  By 
contrast, the fully general case may be referred to as "possibly a different 
MCS each time".

We can reduce link overheads and footprint, as well as classfile complexity, if 
we support this simpler use case direct, instead of as an under-the-covers 
optimization of the full use case.

We can compatibly extend the JVM Specification, specifically as documented in 
the "package-info.java" javadoc for java.lang.invoke, by allowing a new 
behavior from a bootstrap method (BSM) which links an invokedynamic 
instruction.  This new behavior is simply to return a reference to a 
MethodHandle instead of a CallSite.  This is currently an illegal response, so 
it would be a compatible extension.

Specifically, if a MethodHandle is returned from a BSM call in response to 
linking an invokedynamic instruction, then the linkage of that dynamic call 
site would directly use the method handle.  In terms of the current 
specification, it would be as if the BSM had returned a ConstantCallSite 
wrapping the given method handle.  (Today's JVM can easily perform this 
optimization under the covers, also, discarding the CCS and just using the MH 
that it wraps.)

Also, if a BSM for a given CONSTANT_InvokeDynamic constant ever returns a 
MethodHandle (instead of a CallSite), then, as a new behavior, the JVM records 
the method handle permanently as the result of linking that constant pool 
constant.  Future linkage events that refer to the same constant pool entry (of 
type CONSTANT_InvokeDynamic) must then return the identical method handle, with 
no further calls to the BSM associated with that constant.

(Corner case:  In the case of race conditions, the present specification says 
that the JVM is obligated to pick one race winner and publish it as the 
overriding answer to all racing BSM invocations.  With respect to the proposed 
change, a race condition may propose a mix of MHs and CSs as potential winners. 
The JVM must choose a winner, as before, and the new semantics of permanent 
linkage apply if and only if the winner is an MH.  In such a case any competing 
CS results would be discarded; if a CS wins any competing MH results would be 
ignored.  Better luck next time.  If a given BSM always returns an MH, then 
races might occur but only one winner will apply for all call sites; this will 
be the common case.)

A side benefit of this extension would be bringing "indy" and "condy" (dynamic 
constants) closer together, in their linkage behavior.  It would be easier to 
share BSMs between the two instructions (invokedynamic and ldc).  Perhaps it 
would make it easier to converge the two facilities further in the future.
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: constant-dynamic specification, updated

[John Rose]

On Jun 25, 2017, at 3:00 PM, Per Bothner  wrote:
> 
> That will be difficult as long as constant pool indexes and various
> other fields are limited to 16 bits.

Yep; that's a challenge, until we break the glass and make a
completely new class file format that integrates a var-int format.
(My favorite is UNSIGNED5.)

But we can (and should) put off the day of glass-breaking
by tactical moves like CONSTANT_Group, which will allow
you to express a single sequence of constant tokens of
any length, without (necessarily) using up indexes in the
main CP.

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: constant-dynamic specification, updated

[John Rose]

On Jun 25, 2017, at 12:47 PM, Mark Roos  wrote:
> 
> I was looking for boot strap args to allow byte[] constants from the 
> constantPool. 
> 
> Is that planned? 

Yes; there are two more CP types planned:

1. CONSTANT_Data = arbitrary blob of octets
2. CONSTANT_Group = series of constant specifiers outside of main CP

That would seem to complete the CP story.  You can build constants
by calling BSMs that take a flexible mix of random data octets and
references to constants.  You can't mix them together seamlessly,
but you can at least put all of your octets in one bag (Data) and all
of your constant references in another (Group).

We could think to use the CP to nest additional class syntax inside a CP entry.
E.g., CONSTANT_ClassDeclaration which would carry the bytes of a class-file
structure, *except* the constant pool prelude itself.  That might be a natural
way to allow nest several classes into one classfile (the old MClass idea).

(Why not just use CONSTANT_Data?  Because a class file parser
often needs to recursively descend into all parts of a class file which
may contain indexes into the constant pool, or detect other relations
between subparts of the file.  An opaque Data block defeats this.)

But it seems more likely that we would use attributes as envelopes for
the nested structure.  CP entries work better if they are of limited size.
That's why we put part of the payload of CONSTANT_InvokeDynamic
into the BootstrapMethods attribute instead of directly in the CP.
So think in terms of a ClassDefinitions attribute with a series of
definitions; then if we need CONSTANT_ClassDeclaration, it just
carries a single index into the attribute.

The CONSTANT_Data guy probably will carry payload in a segmented
ConstantDataBlocks attribute, and the CONSTANT_Group guy will
probably carry payload in a ConstantGroups attribute.  The structure
of both attributes will probably allow random access to variably-sized
large blobs of data.  The CGs will allow cheap nesting, for things like
ASTs.

At least, that's how I'm thinking about it.  My overall goal is to keep
the classfile alive as an efficient medium for more and more advanced
language runtimes.

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Fwd: constant-dynamic specification, updated

[John Rose]

Date: June 24, 2017 at 4:52:04 PM PDT
To: valhalla-spec-expe...@openjdk.java.net

I have updated the javadoc API that is relevant to the proposed
JVM features for dynamic constants and more powerful bootstrap
methods.

http://cr.openjdk.java.net/~jrose/jvm/condy-jvms-2017-0620.html

Here is a rough draft of the corresponding JVMS changes:

http://cr.openjdk.java.net/~jrose/jvm/specdiff-condy-2017-0624.zip

Please enjoy and comment.

— John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: the End of History in the constant pool

[John Rose]

On May 18, 2017, at 6:55 PM, Mark Roos  wrote:
> 
> Does this allow the use of a byteArray as one or more of the bootstrap 
> constants? 
> 
> I seem to recall that was something I wished for when I did a similar effort 
> with 
> constant call sites. 

Yes, it does.  The byte array would be its own condy node, which could then
be used as a BSM argument.  This is why condy is more than just a slightly
better way to do indy-of-no-arguments.

Also, by tweaking MH.invokeWithArguments to support thousands of arguments,
the set of possible BSM argument lists is much more interesting (limited to 64k
at present, until we get Groups).

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: the End of History in the constant pool

[John Rose]

On May 18, 2017, at 2:13 PM, fo...@univ-mlv.fr wrote:
> 
> uses invokeinterface then the spec was rebooted, the second version has it's 
> own bytecode 186,

There was another in between that used invokeinterface and reified a CallSite 
with
patchable state.  I did a really nasty design for this w/o MHs, but it was like 
cooking
blindfolded.
— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: the End of History in the constant pool

[John Rose]

On May 18, 2017, at 12:55 PM, John Rose <john.r.r...@oracle.com> wrote:
> 
>  I'll post some info about that in a moment.

Here's what we are thinking about doing with condy in the short term.
Comments are welcome.  This work is landing in branches of the Amber
project.

— John

Just as the linkage of an invokedynamic call site involves an upcall
from the JVM to Java-based linkage logic, we can apply this same trick
to the resolution of a constant pool entry. A CONSTANT_Dynamic
constant pool entry encodes the bootstrap method to perform the
resolution (a MethodHandle), the type of the constant (a Class), and
any static bootstrap arguments (an arbitrary sequence of constants,
barring cycles in the CP between dynamic constants.)

We add a new constant pool form, CONSTANT_Dynamic (new constant tag
17), which has two components following its tag byte: the index of a
bootstrap method, in the same format as the index found in a
CONSTANT_InvokeDynamic, and a CONSTANT_NameAndType) which encodes the
expected type.

Behaviorally, a CONSTANT_Dynamic constant is resolved by executing its
bootstrap method on the following parameters: 1. a local Lookup
object, 2. the String representing the name component of the constant,
3. the Classrepresenting the expected constant type, and 4. any
remaining bootstrap arguments. As with invokedynamic, multiple threads
can race to resolve, but a unique winner will be chosen and any other
contending answers discarded. Instead of returning a CallSite object
(as the invokedynamicinstruction requires) the bootstrap method would
return a value which would be immediately converted to the required
type.

As with invokedynamic, the name component is an additional channel,
besides the type, for passing expression information to the bootstrap
method. It is expected that just as invokedynamic instructions find
uses for the name component (e.g., a method name or some ad hoc
descriptor) dynamic constants will also find uses for the name (e.g.,
the name of a enum constant or the spelling of a symbolic
constant). Putting the CONSTANT_NameAndTypein both places makes for a
more regular design. In effect, a CONSTANT_Methodref and
CONSTANT_Fieldref constants are used to refer to named members of
classes, while the analogous CONSTANT_InvokeDynamic and
CONSTANT_Dynamic constants are used to refer to named entities with
user-programmed bootstraps.

The type component of the constant, with both invokedynamic
andCONSTANT_Dynamic, determines the effective type of the call site or
constant (respectively). The bootstrap method does not contribute or
constrain this type information, so that bootstrap methods may be (and
often are) weakly typed whereas the bytecodes themselves are always
strongly typed.

To relax length restrictions on bootstrap specifiers, the language
which defines the invocation of bootstrap methods will be adjusted
(with complete backward compatibility) to allow variable arity
(ACC_VARARGS) bootstrap methods to absorb, into their trailing
arguments, all remaining static arguments, even if there are 2^16-1 of
them. (The classfile format already allows this, although there is no
way to read over-long bootstrap argument lists.) For consistency, the
invokeWithArguments methods of MethodHandle will also be expanded in
this way, if the target method has variable arity. In this way the
invocation of bootstrap methods can be specified in terms of the
weakly typed methods invokeWithArguments and invoke, just as today it
is specified in terms of invoke alone.



___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

the End of History in the constant pool

[John Rose]

(I'm moving an amber-dev conversation about condy to mlvm-dev.)

We are working on a condy JEP and spec. as well as a prototype, which is
good progress.  I'll post some info about that in a moment.

On May 18, 2017, at 12:16 PM, Remi Forax  wrote:
> 
> I would prefer 17 to 21, because 21 is already used internally by ASM :)

I don't think anyone is objecting to 17 for CONSTANT_ConstantDynamic,
and I expect 17 is what we will land on.  It's been held open for 
(approximately)
this purpose.

Fun facts from History:  CONSTANT_17 was used by a prototype version of
invokedynamic which was discarded.  (The EG wisely discarded a couple of 
designs,
including a design without method handles!)  The prototype in that case 
overloaded
the invokeinterface instruction, in ways which are useless to recall.  In order 
to make
a clean break, we helped ourselves to another constant tag.  Soon after that,
I realized a future need for expression-like structures threaded through the 
constant
pool and bootstrap specifiers, although it was a bridge too far at the time.  So
we made no effort to "compact" our constant pool tag usage, knowing there might
be followup work in the future.

Also:  From the primordial days of Java there is CONSTANT_Unicode (tag 2)
which AFAIK has never been used from JDK 1 forward.  I think modern "take" on
character sets is to have one format for text (usually UTF8) and one for binary
octets.  (This is exemplified, for example, in CBOR.)  I expect some day to use
the constant tag 2 for such a purpose.  Basically, it would amount to giving 
class
files the power to "swallow" resource files (or smaller random byte snippets).
It has an obvious multiplicative effect on condy, but we don't need it yet, so
we are going with the minimal proposal.

I think the Ultimate, End-of-History CP tags are CONSTANT_ConstantDynamic,
CONSTANT_Data, and CONSTANT_Group.

The Group is simply a subsequence of CP values (probably of limited set of 
types).
It would be used for packing array constants and other aggregate types.
Today we use bootstrap specifiers, which can be as long as 2^16-1 items,
so there's no immediate motivation for a new grouping construct.  But the main 
point
of a Group would be to lift the restriction that all CP constants are defined 
in one
space of 2^16-1 code points.  Instead, a group would contain serialized CP
entries that have no absolute CP index, but rather are loaded as part of the 
group.

The group's size limit could also be raised to a u4 from u2.  I think the octet 
data
size limit should be u8 but that requires further API work in the JDK.  My hope 
is that
both Data and Group can serve at a wide range of length scales, O(10) to 
O(10^10).

In the interests of incremental delivery, the forthcoming JEP only deals with a 
limited
subset of this stuff.   The bug JDK-8161256 is a "kitchen sink" description of 
proposals
(both live and abandoned) for futures in this direction.

— John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[John Rose]

On May 5, 2017, at 5:44 AM, Vladimir Ivanov  
wrote:
> 
> In other words, LF specialization influence how many classes for compiled LFs 
> are loaded, but doesn't change what actually happen during MH invocation. (No 
> inlining on bytecode level is needed during specialization. JIT will already 
> do that during compilation. No need to help it.)

Not that hidden frames are terribly important, but it does strike me as odd, at 
this moment, that stack traces for customized MHs and "raw" MHs look the pretty 
much the same.

Idea:  Have two levels of hiding:  Hide always, and hide if inlined into some 
caller.  That way a successfully optimized MH will show fewer stack frames on 
close inspection.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[John Rose]

On May 3, 2017, at 9:37 PM, Wenlei Xie  wrote:
> 
> Thank you Vladimir for the help ! I see the point why MH.bindTo() is not a 
> good fit for implementing lambda capturing. 

A simple rule for using MHs is that they are designed to be another form of 
code.
Creating many of them at a high rate is likely to stress JVM in ways similar
to loading many small classes at a high rate.

So bindTo is really code customization, which is not the same thing as data 
capture.
The MH before bindTo is an algorithm with a variable "hole" in it, where the MH 
after
bindTo is a customized version of the algorithm, with the hole filed by a 
constant.
It's a little like a C++ template instance.

I'd like high-count bindTo to be cheaper, of course, but it's not the design 
center,
and it's not where we are investing optimization effort.  Maybe in the future.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Ephemerons for Java

[John Rose]

On Jan 9, 2017, at 5:27 PM, John Rose <john.r.r...@oracle.com> wrote:

> Thanks!  That's a great start.  Should be a JEP.  — John

P.S.  You got me.  I filed https://bugs.openjdk.java.net/browse/JDK-8172476 to 
help track this.

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Leaking LambdaForm classes?

[John Rose]

Could be LF caching gone rogue. Does the user create many LF shapes, e.g. one 
per datum?

– John

> On Jan 6, 2017, at 9:32 AM, Charles Oliver Nutter  wrote:
> 
> Anyone else encountered this?
> 
> https://github.com/jruby/jruby/issues/4391
> 
> We have a user reporting metaspace getting filled up with LambdaForm classes 
> that have no instances. I would not expect this to happen given that they're 
> generated via AnonymousClassloader and we would need to hold a reference to 
> them to keep them alive.
> 
> I'm trying to get a heap dump from this user. If anyone has other 
> suggestions, feel free to comment on the issue.
> 
> - Charlie
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: EXT: Re: series of switchpoints or better

[John Rose]

On Oct 5, 2016, at 12:45 PM, Charles Oliver Nutter  wrote:
>  
> It is also good to hear that the old "once invalidated, it will not optimized 
> again - ever" is no longer valid.
> 
> And hopefully it will stay that way as long as we keep making noise :-)

Go ahead, be that way!___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: series of switchpoints or better

[John Rose]

On Oct 5, 2016, at 7:00 AM, Charles Oliver Nutter  wrote:
> 
> I will say that using SwitchPoints is FAR better than our alternative 
> mechanism: pinging the (meta)class each time and checking a serial number.

This makes my day!  That's exactly what SwitchPoints are designed to deliver.  
They are intended to hook into the same mechanism the JVM uses to invalidate 
code that has out-of-date devirtualized methods.  (I.e., when you load the 
second definition overriding a method, you might cause some call sites to 
recompile.  That stuff is too good not to share with language implementors.)
 
> Or I can do one switchpoint for all methodhandles in the system, which makes 
> me wonder if after a meta class change the callsite ever gets Jitted again. 
> The later performance penalty is actually also not very attractive to me.
> 
> We have fought to keep the JIT from giving up on us, and I believe that as of 
> today you can invalidate call sites forever and the JIT will still recompile 
> them (within memory, code cache, and other limits of course).

I'm glad of that.  It has been a fight to tuned everything up just so.  Can you 
say (or blog) more about what you had to tweak to keep the JIT happen?  That 
may help other users, and/or help us make the JIT less irritable.

> However, you'll be invalidating every call site for every modification. If 
> the system eventually settles, that's fine. If it doesn't, you're going to be 
> stuck with cold call site performance most of the time.
>  
> So what is the way to go here? Or is there an even better way?
> 
> I strongly recommend the switchpoint-per-class granularity (or finer, like 
> switchpoint-per-class-and-method-name, which I am playing with now).

For the classic devirtualization trick, the JVM uses something that works like 
a cross between a switchpoint per super-class and a switchpoint per method:  
When you load a new sub-class, each of its supers S is walked, causing a search 
for any devirtualized methods S.m in any compiled code.  A compiled code blob 
("nmethod") contains a list of dependencies which might (under suitable 
conditions) require it to be discarded and recompiled.  This list can contain 
something like "S.m was devirtualized and hasn't been overridden yet", or 
something else like "switchpoint x has not been triggered yet".

http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/08492e67bf32/src/share/vm/code/codeCache.cpp#l1180
 


The logic which handles switchpoints is (as Remi said) built on top of mutable 
call sites, which are handled in the JVM here:

http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/08492e67bf32/src/share/vm/code/dependencies.cpp#l1740
 


I'm always glad when this stuff works like it's supposed to.  It emboldens me 
to try more!

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: A fast JSON serializer

[John Rose]

On Sep 14, 2016, at 3:03 PM, Remi Forax  wrote:
> 
> Another idea from the JVM Summit,
> because the JDK9 now uses fast string concatenation by default, a JSON 
> serializer based on reflection can be outperformed by a hand written code.  
> 
> The following code is a small JSON serializer that use the 
> StringConcatFactory of JDK9
>  https://gist.github.com/forax/76124b69631475105f87ddd2205a4d91 
> 
> 

Very cute.  Lots of ideas in there.  Cool use of permuteArgs.  And a MIC!

Doesn't handle strings with quotes or escapes, but that's easy to fix if you 
don't care about performance of that case.

Re-entrance detection is too strict, I think; looks like it would fail if the 
Car had a Wheel.

You could build a templating engine by factoring out the middle parts of it.  
That could then be used to create the JSON dumper.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[John Rose]

On Aug 12, 2016, at 2:56 PM, Michael Haupt  wrote:
> 
> Hi Martijn,
> 
>> Am 09.08.2016 um 00:15 schrieb Martijn Verburg > >:
>> Love the overall idea.  A few thoughts (feel free to ignore as I'm by no 
>> means an expert in the VM).
> 
> thank you. As this JEP spans the VM and the libraries, with a definitely 
> client-facing part in MethodHandles.Lookup, user questions are important. :-)
> 
>> * Is there a risk that Gargantuan can expand to infinite size?
> 
> There is, as much as there is a risk that infinitely many classes are loaded 
> dynamically (e.g., via Unsafe.defineAnonymousClass()). Collecting IMs (and 
> the constants they introduce) is not out of the question, though. Once the 
> method handle representing an IM can be collected, the IM itself can be, too; 
> and its constants can unless they are shared.

Indeed, special GC work is one of the costs of IMs, since the normal 
class-scale storage management does not apply.  The benefit is you can load 
bits of code with low latency and low footprint, and they can go away as soon 
as you stop using them.  (The MH produced by the Lookup call is the only 
user-visible GC root to the "stuff" of the IM.  When that MH goes dead, the JVM 
can edit the big G class to reclaim space, at the JVM's option.)

>> * I'm really nervous about the security implications of being able to load 
>> an arbitrary byte[] that can be executed as a method. I'm hoping that the 
>> use of the internal Unsafe + Jigsaw's module boundaries makes this secure. 
>> Lots of really, really smart people have looked at the security implications 
>> of this, right? :-)
> 
> Does Unsafe.defineAnonymousClass() worry you? It allows just this: loading 
> some byte[] that can be executed as a method - albeit with the added 
> complexity of it being wrapped in a complete class file.
> 
> The choice of MethodHandles.Lookup as the place for loadCode() is deliberate. 
> A Lookup instance defines a context for visibility and access to other 
> elements, and the instructions in the byte[] array will be subject to the 
> scrutiny imposed by the Lookup instance at hand.
> 
> People smarter than me will say more about the security implications. This is 
> a JEP draft. There is no implementation yet, not even a prototype, and there 
> is a lot to be learned along the way.

See also my previous points about MH factories handling access checks, using 
pre-existing Lookup APIs.

Probably the IM is just a faster, more compact way to make something you could 
also have made by spinning MH's together.  If someone figures out what an IM 
can do that you can't also do with MethodHandles.* methods, I'm very eager to 
hear about it.  (And I'll probably want to take it away, or else add a new MHs 
API point.)  Mainly, an IM gives you control over packaging and backtrace 
structure.

>> * Gargantuan - Appreciate it's a placeholder but I can't say I like the 
>> name, it doesn't tell me what it does on the tin.
> 
> It's a literary reference that's supposed to imply "large" [1] - could've 
> used "Gulliver" but he's one of us. ;-)
> 
> Best,
> 
> Michael
> 
> [1] https://en.wikipedia.org/wiki/Gargantua_and_Pantagruel 
> 
I love the Online Etymology Dictionary for this sort of stuff:
  http://etymonline.com/index.php?term=gargantuan

— John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Isolated Methods JEP

[John Rose]

On Aug 12, 2016, at 2:57 PM, Michael Haupt  wrote:
> 
> Hi Peter,
> 
> thanks for your feedback! I'm responding to both your messages in one.
> 
>> Am 09.08.2016 um 05:05 schrieb Peter Levart > >:
>> In the JEP's description you write:
>> 
>> "The loadCode method creates a method from the passed bytecode instructions 
>> and constants and returns a MethodHandle that can be used to call the 
>> method. The implementation of loadCode will take care of verification of the 
>> code to load."
>> 
>> That's clear.
>> 
>> "This method is isolated from any class and behaves largely like a static 
>> method. The method handle resulting from a loadCode invocation is of the 
>> REF_static kind. It cannot be cracked via 
>> MethodHandles.Lookup.revealDirect()."
>> 
>> So there will be no class hosting this method? Not even the "caller" class 
>> of the Lookup instance that loadCode was invoked upon? What will be reported 
>> by Reflection.getCallerClass() invoked in a @CallerSensitive method invoked 
>> from the isolated method?

That last question is easy:  When you create a MH on a @CS method, via the 
Lookup API, the caller is permanently bound to the caller class in the Lookup 
object.  (If you are wondering about security, note that you can't get a MH on 
a @CS method without a full-power Lookup object.)  After that, it is irrelevant 
where and how the MH is eventually invoked, including from an isolated method.

>> Perhaps the following is a hint...
>> 
>> "The context for a method defined in this way is determined by the Lookup 
>> instance receiving the loadCode call. In case the lookup privileges are not 
>> sufficient, an exception will be thrown."
>> 
>> The "context" meaning the caller context, including the caller class that 
>> appears in the stack trace of a call originating from an isolated method and 
>> is important for security decisions?
> 
> In all of this, I think it is important to not let the idea creep in that IMs 
> are members of some class. This is not to be the case.
> 
> With that out of the way ;-) let me stress that this is a very good question, 
> and there have in fact been discussions about this. The notion of "host" has 
> been floating around, and it currently seems that the lookupClass - to which 
> you refer as the "caller" - is a meaningful candidate for this.
> 
> Does this answer your question?

In general, we won't duplicate all the class-file infrastructure around the 
IM's.  So the surrounding class, local-variable tables, etc., etc., won't be 
allowed to "creep into" the IM design.  This makes some bits tricky, like "what 
does it look like on the back trace", and "where are the exception handler 
BCI's stored".  The general answer will be some combination of "there are 
simplified rules for IMs", and "some reflective tasks can be delegated to a 
helper object or a BSM-like helper method", and finally "use a full class file 
if you want more control".

>> On 08/09/2016 02:05 PM, Peter Levart wrote:
>>> In which case would lookup privileges be insufficient to define an isolated 
>>> method? I would expect the privileges of the code in the isolated method to 
>>> be checked when such method is executed and not when defining such method - 
>>> lazily, like for normal methods.
>> 
>> A can see now that defining an isolated method in the context of a lookup 
>> class can only be allowed when the Lookup instance contains ALL_MODES in the 
>> mode bits as such method would have access to any members that a normal 
>> method defined in the lookup class has.
> 
> 
> Not sure. I think this suggestion implies something like class membership, 
> when in fact the IM should have the access rights defined by the Lookup, 
> without belonging to the lookupClass. I may be misunderstanding your point.

Since access rights are permanently bound into the MH's stored in the isolated 
constant pool, there should be little or no reason to ask about access rights 
for the IM itself.  Access rights are checked during symbolic resolution of 
class, method, and field references.  (Class object access rights are a tricky 
case, since classes are easy to grab.)  Inside the IM constant pool, all 
references are "live", and previously resolved.  There's no reason for the IM 
to re-resolve them.

>> In which case it is important which "caller" class the privileges are check 
>> against. This would most naturally be the "caller" class of the Lookup 
>> instance used to define the method. In all respects the code of such 
>> isolated method would appear to be defined in the lookup class except it 
>> would not be denotable symbolically - only through a MethodHandle. Analogous 
>> to VM-anonymous classes. So why not call such methods "anonymous methods" 
>> instead of isolated methods?
> 
> 
> Most aspects of this JEP draft, including the name, are open for discussion. 
> :-)
> 
> I would not want to call an IM "anonymous" 

Re: FTR: JSR 292 RFEs from yesterday's Indy Support Group

[John Rose]

On Aug 4, 2016, at 9:11 AM, Mark Roos  wrote:
> 
> With respect to general data in constant pools 
> 
> Adding raw bytes as a constant type would be a good thing.  I currently 
> have to encode my object serialization as hex utf8 which is not nice. 

+1

> As for the use of a MH to instantiate the constant.  Would this be done 
> at load time or would it just insert a ConstantCallSite to instantiate at 
> run time ( lazy )?  My two concerns would be that not everything is in 
> place at load time and the creation of lots of large constants which are 
> never used. 

The constant would be expanded when it is resolved, on first use.
That's the usual semantics for constants (like CONSTANT_MethodHandle).
So unused constants won't take up any space beyond their dormant
symbolic references.

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Threaded interpreter in Java

[John Rose]

On Aug 3, 2016, at 1:22 PM, Remi Forax  wrote:
> 
> Charles ask if we can make a fast register interpreter in Java,
> here is my take on a threaded-like interpreter
> 
> https://gist.github.com/forax/f38b533e089217cfc4d0ae3c6e2de9c9

Nicely done.  We were talking last night at dinner about making
bytecode interpreters go fast, and this is helpful.

I think there may be a combinator here, for bytecode dispatch loops.
(This feels like the PIC combinator, both fundamental and tricky to get right.)
A "bytecode" dispatch combinator might provide a pattern like the following:

MethodHandle init, pred, step, fini, index;
@NonNull MethodHandle[] tab;
R dispatch(A… a) {
   V v = init(a…);  // one here; there might be many v…
   while (pred(a…)) {
  v = step(v, a…);
  // execute one "instruction":
  int i = index(v, a…);
  tab[i].invokeExact(v, a…);
   }
   return fini(V, a…);
}

The explicit table would be recopied internally to an @Stable array for 
constant folding.
The various index values could be tracked and turned into traces (or some other 
profile
driven structure) which would be compiled together, in specialized segments of 
the unrolled
interpreter loop.

Or maybe just the table lookup part alone makes a good combinator, to be paired
with the loop combinators?

Comments welcome…

— John

P.S. Another new RFE:
experimental hook for creating heisenboxes
https://bugs.openjdk.java.net/browse/JDK-8163133
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

FTR: good old paper on quasi-quotation

[John Rose]

Related to some discussion at the JVMLS about quasi-constants
(templates, constants with holes), here is the early history of
quasi-quoting in computing, as reported by someone who was there.

Alan Bawden, Quasiquotation in Lisp (1999)
http://repository.readscheme.org/ftp/papers/pepm99/bawden.pdf

Also regarding the history of quasiquotes, I found this little gem,
showing one of our own community, in his tender years, innocently
confounding the Boffins of Quotation.

Boolos & Jeffrey, Logic, Logic, Logic (1999), p. 393
https://books.google.com/books?id=2BvlvetSrlgC=PA393=PA393=bl=sYbY0uDPnJ=HWnkaAHgQOG45MGn5_KyYbPicsk=en=X=0ahUKEwjEpL_33KPOAhVK7GMKHdU0BbwQ6AEISTAL#v=onepage=false

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

FTR: JSR 292 RFEs from yesterday's Indy Support Group

[John Rose]

some frameworks need access delegation via MethodHandles.lookup
https://bugs.openjdk.java.net/browse/JDK-8162494

some frameworks want to advise method handle optimization
https://bugs.openjdk.java.net/browse/JDK-8163006

CallSites should support polymorphic inline caches
https://bugs.openjdk.java.net/browse/JDK-8163007
(THIS IS A PLACEHOLDER)

MethodHandles should have synchronization combinator
https://bugs.openjdk.java.net/browse/JDK-8163004

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: JVMLS indy support group provisional agenda

[John Rose]

Thanks, to those who were present, for a very lively conversation!

Here are the slides as presented today:

  http://cr.openjdk.java.net/~jrose/pres/201608-IndySupportGroup.pdf

I would like to add notes to the end of the deck, to summarize what we actually 
discussed.

I could use some help with that.  So, please, send your recollections to help 
reconstruct our very interesting discussion.

Very rough notes are fine; so are one-liners; send 1-1 or reply on-list at your 
option.

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: JVMLS indy support group provisional agenda

[John Rose]

Thanks, I added those points.

On Jul 31, 2016, at 10:34 PM, Vladimir Ivanov <vladimir.x.iva...@oracle.com> 
wrote:
> 
> Thanks a lot for preparing the agenda, John!
> 
> I'd like to nominate a topic into API section: enable super constructor 
> lookups in j.l.i.
> 
> FTR it was discussed on the list before [1].
> 
> Also, Charlie Nutter asked (at JFokus 2015?) for lightweight safepointing 
> solution, but there was no progress on it yet. Maybe it's worth to reiterate 
> the discussion as well?
> 
> Best regards,
> Vladimir Ivanov
> 
> [1] http://mail.openjdk.java.net/pipermail/mlvm-dev/2015-February/006292.html
> 
> 
> On 7/31/16 10:13 PM, John Rose wrote:
>> Tomorrow at the JVMLS we will have an hour-long workshop to discuss indy
>> and MHs.
>> I plan to run this as a Q/A and (if we are in good form) a
>> brain-storming session.
>> Here are the discussion topics that pop quickly to the top of my list.
>> Any more suggestions?
>> — John
>> 
>> Provisional Agenda: API
>> §API changes in java.lang.invoke for Java 9
>> –VarHandles ({find{Static},unreflect}VarHandle)
>> –Loop combinators, tryFinally, various incremental conveniences
>> –Module-aware Lookups, access-checked Class lookups
>> §What are the use cases for unsafe Lookups?
>> §What about polymorphic inline caches?
>> Provisional Agenda: Performance
>> §Performance changes
>> –MH caching, MH customization, profile injection
>> –Indified string concatenation
>> §Native method handles
>> §Low level performance stuff
>> –@Stable, @{Dont,Force}Inline, @PolymorphicSignature
>> §Who wants to nominate their polymorphic inline cache?
>> 
>> 
>> 
>> ___
>> mlvm-dev mailing list
>> mlvm-dev@openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>> 
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: InvokeDynamic PIC Slowdown (deopt issue?) need advice

[John Rose]

On Jul 24, 2016, at 3:41 PM, Mark Roos  wrote:
> 
>  A few questions on implementation. 
> 
> My old prototype looks like: 
> private RtObject[] _mDicts = new RtObject[8]; // array of 
> method dicts 
> private MethodHandle[] _methods = new MethodHandle[8]; // the 
> code MH 
> MethodHandle lookupSelf(RtObject rcvr, RtCallSite site) 
> RtObject methodDict = rcvr.classField(); 
> if(_mDicts[0] == methodDict) return _methods[0]; 
> if(_mDicts[1] == methodDict) return _methods[1]; 
> 
> If I fill the arrays I treat them as a circular buffer and overwrite the 
> existing contents. 
> 
> You mention the need for the last element in the arrays to be null.  Why? 

It doesn't; I was thinking of a doubtful loop shape improvement.
In you code, there's no loop at all.

> If the arrays are declared 'Stable'  does this mean that when I fill them I 
> cannot reuse them?  Would 
> I just replace the arrays with new ones? 

The documentation for Stable (which is not a standard feature yet, and may not 
become one)
says you are only allowed to set a stable array (any level of structure, from 
root to leaf) exactly
once.  The JIT avoids constant folding the stable array structure until it sees 
a non-default value.
If you then change a non-default value to another value, you've broken the 
rules.

Perhaps the real rule is that you have to be willing to live with any of the 
(non-default) values
that the JIT chooses to constant-fold.  In any case, that's operationally what 
the JIT provides
for you if you use Stable values.

The above set of rules or non-rules is only half-baked.  That's why Stable is 
not a feature.
It's merely an internal optimization.

> You also mention a concern if the test items are Objects vs MethodHandles.  
> My test MH does the same 
> object reference compare and I was trying to avoid executing the 
> rcvr.classField() for each test.  Would 
> I be better off if I used a test MH instead? 

If you have a clean token you can use with comparisons, that is fine.

The advantage of MH's is simple:  They get inlined vigorously.

– John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: InvokeDynamic PIC Slowdown (deopt issue?) need advice

[John Rose]

On May 31, 2016, at 12:41 PM, Mark Roos  wrote:
> 
> It looks like, from some fine timing, that each time the Smalltalk class 
> changes there is a large amount 
> of time added to the call.  Which I would expect if there was a deopt 
> whenever a different GWT triggered. 
> There are 6 GWTs in this chain ( idleValue can be one of six Smalltalk 
> classes). 

Has anybody on this list played with using a short @Stable array to represent a 
PIC?
Editing the PIC would involve changing the array instead of recompiling a call 
site.

The effect would be to preserve existing inlinings of the PIC site (unless they
self-invalidate) but allow the PIC to update itself over time as new cases 
arise.
Previously compiled uses of the PIC would stay optimized as-is.

The @Stable array would always have a series of zero or more non-null entries,
followed by at least one null entry.  The search in the @Stable array would 
inline
and short-circuit over irrelevant PIC entries, if the pattern-matching logic 
were
inlinable.  Entries could be as simple as @Stable 2-arrays of guard MH and 
target MH.
(If they are objects, some care with TrustFinalFields would also be needed.)

Using this technique would probably lead to fewer deopts.  The @Stable array 
could
also be shared by several PIC sites, if that helps with footprint.

class PIC {
  @Stable final MethodHandle[][] cache = new MethodHandle[PIC_SIZE+1][];
  // cache[0] = new MethodHandle[] { guard, target }, etc.
  // cache[cache.length-1] is either null or some permanent catch-all logic
}

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

binding to indy with an annotation-driven reweaver

[John Rose]

Inspired in part by a recent exchange between Remi and Charlie[1],
I've been thinking recently, again, about binding Java APIs to indy.

[1]: https://groups.google.com/forum/#!topic/jvm-languages/IjIEzDc_d3U

I think I have a way to make it work, and (what is more)
I think the end result looks pretty good.  Even better,
along the way we can create a mechanism for naturally
constant-folding selected method calls (like List.of)
at link-time.  (Library defined constants ahead!)

There are two things which make all this hard.
First, indy calls its BSM at link time (for the particular
indy instruction, since the JVM is lazily linked),
but the Java language does not expose link-time operations,
except very indirectly (in  code, for example).

Second, many good indy use cases are at least partly
signature-polymorphic, just like method handles. 
But the Java language does not allow you to generify over
method type signatures (e.g., argument types of (), (int),
(int,int), (String), (String,int), etc., all in one API point).
Even Valhalla only lets you generify over one value at
a time.  (One step at a time!)

Note that these two hard points might occur at the same
time, since some interesting BSMs are in fact signature
polymorphic.  (Well, at least they are varargs methods,
and varargs is an OK substitute for S-P, at link time.)

OK, so we are reduced to baking special handling for those
sorts of things into the language (as MethodHandle and
VarHandle do), or passing some sort of smoke signal
through the language and reweaving the bytecodes
(as Remi is so good at).

How would we signal a BSM call, though Java, in a way
that a bytecode reweaver could recognize it?

Here's an answer, maybe the simplest answer:  Mark
some API points as BSMs (in disguise; don't let javac
know!).  When the reweaver encounters a BSM call in
bytecode, it has to collect all the operands and ensure
that they are constants.  If they are constants, then
the reweaver collects the whole call, and sticks it
into an auxiliary static method (or pattern-matches
the whole call into an indy bootstrap specifier, if
possible).  If they are not constants, it is a reweaving
error.

class IndyTrickster {
   enum StringIndexerKind { LENGTH, HASHCODE, CHAR0, PARSEINT };
   @IndyTricks.AtLinkTime
   static ToIntFunction indexString(StringIndexerKind k) { … }
}

class IndyTrickUser {
   int foo(String s) {
 return indexString(LENGTH).applyAsInt(s);  // returns s.length()
   }
   int bad(String s, StringIndexerKind k) {
 return indexString(k).applyAsInt(s);  // ERROR in reweaver?
   }
}

This is really just a mechanism for materializing link-time constants,
which all by itself is pretty interesting.

I've chosen enums here, because (a) they are constants, but
(b) they cannot be directly supported (today) as indy static
arguments.  So the reweaver has to dump some code in an
auxiliary method somewhere, at least to materialize the enum.

(And see JDK-8161256, "general data in constant pools", for
a better way forward.)

We can stop here and use this trick to create APIs that materialize
constant values of type List, Map, etc.  Put the AtLinkTime
annotation on List.of, for example, et voila.

This raises the question, what happens if an operand fails to be
a link-time constant?  Should the reweaver silently keep the
call as-is, so that it runs every time, instead of just once at
link time?  Probably yes, but what happens when somebody
is expecting link-time folding, and wants to hear if it fails?

One answer:  An optional argument to the AtLinkTime annotation,
to determine what to do if the folding fails (error/warn/allow).
The List.of guys would just silently allow the non-folding uses,
since that is what they are used for now.

It would seem that link-time constants can only be produced by
static methods, but that would be wrong:  You can have link-time
computations that call non-static methods also, as long as the
receiver of each such method is itself a link-time constant.

(All of this suggests that the Java language should just have
a first-class notion of link-time constant, as it already has
compile-time constants.  But as we all know, Java grows
slowly and deliberately.  Experimenting first outside the JLS,
such as suggested here with reweavers, is a great way to
add weight to the case for change.)

What's next?  Well, so far the thing returned from the
constant-producing BSM has a type fully determined
by the static declaration of the BSM method.

You could grab some S-P magic from MethodHandles
to follow the BSM call by an S-P call, like this:

class IndyTrickster {
   @IndyTricks.AtLinkTime
   static MethodHandle doubler(Class type) {
  return makeDoublerMH(type);
   }
   static MethodHandle makeDoublerMH(Class type) {
  … // returns (type x) -> (String) Arrays.asList(x, x).toString()
   }
}

class IndyTrickUser {
   String foo1(String s) {
 return (String) doubler(String.class).invoke(s);
   }
   String foo2(int n) {
 return 

Re: RFR [9] 8150829: Enhanced drop-args, identity and default constant, varargs adjustment

[John Rose]

Quick comment.  In this: 
>MethodHandle h0= constant(boolean.class, true);


…there should a space between h0 and =.

> On Apr 13, 2016, at 5:12 AM, shilpi.rast...@oracle.com wrote:
> 
> Thank You Paul for the suggestions.
> 
> Please review the updated webrev 
> http://cr.openjdk.java.net/~srastogi/8150829/webrev.05 
> 
> 
> Thanks,
> Shilpi
> 
> On 4/12/2016 7:32 PM, Paul Sandoz wrote:
>> Hi,
>> 
>> Just minor comments, below.
>> 
>> Paul.
>> 
>> 
>> MethodHandle
>> —
>> 
>>  972 /**
>>  973   * Adapts this method handle to be {@linkplain #asVarargsCollector 
>> variable arity}
>>  974   * if the boolean flag is true, else {@linkplain #asFixedArity 
>> fixed arity}.
>>  975   * If the method handle is already of the proper arity mode, it is 
>> returned
>>  976   * unchanged.
>>  977   * This method is sometimes useful when adapting a method 
>> handle that
>>  978   * may be variable arity, to ensure that the resulting adapter is 
>> also
>>  979   * variable arity if and only if the original handle was.  For 
>> example,
>>  980   * this code changes the first argument of a handle
>> 
>> , {@code mh},
>> 
>> to {@code int} without
>>  981   * disturbing its variable arity property:
>>  982   * {@code 
>> mh.asType(mh.type().changeParameterType(0,int.class)).withVarargs(mh.isVarargsCollector())}
>> 
>> The above paragraph can be an @apiNote.
>> 
>> Also can you format the code block over two lines to emphasise the last 
>> call., otherwise i think it is harder to read.
>> 
>> 
>>  983   * @param makeVarargs true if the return method handle should have 
>> variable arity behavior
>>  984   * @return a method handle of the same type, with possibly 
>> adjusted variable arity behavior
>>  985   * @throws IllegalArgumentException if {@code makeVarargs} is true 
>> and
>>  986   * this method handle does not have a trailing array 
>> parameter
>>  987   * @since 9
>> 
>> Add
>> 
>> @see #asVarargsCollector
>> @see #asFixedArity
>> 
>> ?
>> 
>>  988  */
>>  989  public MethodHandle withVarargs(boolean makeVarargs) {
>> 
>> 
>> 
>> MethodHandles
>> —
>> 
>> 2387 /** Produces a constant method handle of the requested return type 
>> which
>> 
>> new line after ‘/**'
>> 
>> 
>> 2388  * returns the default value for that type every time it is invoked.
>> 2389  * The resulting constant method handle will have no side effects.
>> 2390  * The returned method handle is equivalent to {@code 
>> empty(methodType(type))}.
>> 2391  * It is also equivalent to {@code 
>> explicitCastArguments(constant(Object.class, null), methodType(type))},
>> 2392  * since {@code explicitCastArguments} converts {@code null} to 
>> default values.
>> 
>> Is this method more efficient than the other two?
> It is existing method made it public and renamed it to zero from zeroHandle.
>> 
>> 
>> 2393  * @param type the expected return type of the desired method handle
>> 2394  * @return a constant method handle that takes no arguments and 
>> returns the default value of the given type (or void, if the type is void)
>> 
>> Can you format to be a little more consistent and not so long on the line 
>> length, as it becomes really tricky read.
>> 
>> 
>> 2395  * @throws NullPointerException if the argument is null
>> 2396  * @see MethodHandles#constant
>> 2397  * @see MethodHandles#empty
>> 2398  * @since 9
>> 2399  */
>> 2400 public static  MethodHandle zero(Class type) {
>> 2401 Objects.requireNonNull(type);
>> 2402 return type.isPrimitive() ?  
>> zero(Wrapper.forPrimitiveType(type), type) : zero(Wrapper.OBJECT, type);
>> 2403 }
>> 2404
>> 
>> 
>> 2409 /**
>> 2410  * Produces a method handle of the requested type which ignores any 
>> arguments, does nothing,
>> 2411  * and returns a suitable default depending on the return type.
>> 2412  * That is, it returns a zero primitive value, a {@code null}, or 
>> {@code void}.
>> 2413  * The returned method handle is equivalent to
>> 2414  * {@code dropArguments(zero(type.returnType()), 0, 
>> type.parameterList())}.
>> 2415  * 
>> 2416  * Example:  Given a predicate and target, a useful "if-then" 
>> construct can be constructed as
>> 
>> s/Example:/@apiNote (same applies to the method dropArgumentsToMatch)
>> 
>> s/constructed/produced
>> 
>> 
>> 2417  * {@code guardWithTest(pred, target, empty(target.type())}.
>> 2418  * @param type the type of the desired method handle
>> 2419  * @return a constant method handle of the given type, which 
>> returns a default value of the given return type
>> 2420  * @throws NullPointerException if the argument is null
>> 2421  * @see MethodHandles#zero
>> 2422  * @see MethodHandles#constant
>> 2423  * @since 9
>> 2424  */
>> 2425 public static  MethodHandle empty(MethodType type) {

Re: Packages and source paths.

[John Rose]

On Mar 30, 2016, at 5:17 AM, MacGregor, Duncan (GE Energy Connections) 
 wrote:
> 
> I’ve been doing some work on getting useful coverage information from Jacoco 
> for Magik, and some of the same issues probably apply to other JVM language 
> implementations such as JRuby and Nashorn, so I thought I’d ask a couple of 
> things here. I’m attempting to get some changes into Jacoco to better handle 
> source files which are not organised in the standard java way, either because 
> they are organised according to other existing conventions (as in our case), 
> or in an extremely ad hoc fashion (javascript/nashorn).
> 
> 
>  1.  What are people recording as the source file for their generated class 
> files? Does it include an absolute or relative path (is it relative to some 
> well defined root)?
>  2.  What package are your classes in?
> 
> The three current proposals for changing jacoco are to
> 
>  1.  Ignore the package when a source file contains directory separators
>  2.  Add an “ignore package” configuration option for source lookup
>  3.  Add configuration options to ignore packages for specified file 
> extensions
> 
> I think any of these three options would work in my case, but I’d like to 
> make this change as widely applicable as possible.

(Sounds like they need something like the javac -sourcepath option, plus maybe 
stuff like "patch -pN" to elide path components.  Too many degrees of freedom.)

For java.lang.invoke code, most generated classes are in 'java.lang.invoke' 
(surprise!).
Most are anonymous classes (which means there is an illegal slash "/" in the 
Class.getName.)
Somewhat perversely, we store a SourceFile attribute of "LambdaForm$something",
where the something is vaguely informative on the theory that it will soothe 
irritated
readers of backtraces.

PDH (probably doesn't help)

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (S): 7177745: JSR292: Many Callsite relinkages cause target method to always run in interpreter mode

[John Rose]

 that not only the root method is always 
> interpreted, but all bound method handle chains are broken into numerous 
> per-LF nmethods (see JDK-8069591 for some details).
> 
> MLVM folks, I'd like to hear your opinion about what kind of behavior do you 
> expect from JVM w.r.t. mutable call sites.
> 
> There are valid use-cases when JVM shouldn't throttle the recompilation 
> (e.g., long-running application with indy-based dynamic tracing). Maybe 
> there's a place for a new CallSite flavor to clearly communicate application 
> expectations to the JVM? Either always recompile (thus eventually reaching 
> peak performance) or give up and generate less efficient machine code, but 
> save on possible recompilations.
> 
> Best regards,
> Vladimir Ivanov
> 
> On 1/20/16 2:37 AM, John Rose wrote:
>> On Jan 18, 2016, at 4:54 AM, Vladimir Ivanov
>> <vladimir.x.iva...@oracle.com <mailto:vladimir.x.iva...@oracle.com>> wrote:
>>> 
>>> The fix is to avoid updating recompilation count when corresponding
>>> nmethod is invalidated due to a call site target change.
>> 
>> Although I'm not vetoing it (since it seems it will help customers in
>> the short term), I'm uncomfortable with this fix because it doesn't
>> scale to large dyn. lang. applications with many unstable call sites.
>>  Put another way, it feels like we are duct-taping down a failsafe
>> switch (against infinite recompilation) in order to spam a
>> micro-benchmark:  a small number mega-mutable call sites for which we
>> are willing to spend (potentially) all of the JIT resources, including
>> those usually allocated to application performance in the steady state.
>>  Put a third way:  I am not comfortable with unthrottled infinite
>> recompilation as a performance strategy.
>> 
>> I've commented on the new RFE (JDK-8147550) where to go next, including
>> the following sentiments:
>> 
>>> There is a serious design tension here, though: Some users apparently
>>> are willing to endure an infinite series of recompilations as part of
>>> the cost of doing business; JDK-7177745 addresses this need by turning
>>> off the fail-safe against (accidental, buggy) infinite recompilation
>>> for unstable CSs. Other users might find that having a percentage of
>>> machine time devoted to recompilation is a problem. (This has been the
>>> case in the past with non-dynamic languages, at least.) The code shape
>>> proposed in this bug report would cover all simple unstable call
>>> sites (bi-stable, for example, would compile to a bi-morphic call),
>>> but, in pathological cases (infinite sequence of distinct CS targets)
>>> would "settle down" into a code shape that would be sub-optimal for
>>> any single target, but (as an indirect MH call) reasonable for all the
>>> targets together.
>>> 
>>> In the absence of clear direction from the user or the profile, the
>>> JVM has to choose infinite recompilation or a good-enough final
>>> compilation. The latter choice is safer. And the
>>> infinite recompilation is less safe because there is no intrinsic
>>> bound on the amount of machine cycles that could be diverted to
>>> recompilation, given a dynamic language application with
>>> enough mega-mutable CSs. Settling down to a network of indirect calls
>>> has a bounded cost.
>>> 
>>> Yes, one size-fits-all tactics never please everybody. But the JVM
>>> should not choose tactics with unlimited downsides.
>> 
>> — John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (S): 7177745: JSR292: Many Callsite relinkages cause target method to always run in interpreter mode

[John Rose]

On Jan 20, 2016, at 4:13 AM, Remi Forax  wrote:
> 
> I understand that having the VM that may always recompile may be seen as a 
> bug,
> but having the VM that bailout and stop recompiling, or more generally change 
> the compilation strategy is a bug too.

As you can guess from my previous message, I agree with this, except for
"change the compilation strategy".  The JVM earns its way in the world by
routinely changing compilation strategy.  The reason most people don't
notice is the strategy changes are profile-driven and self-correcting.

Nothing in the 292 world promises a particular strategy, just a best effort
to create and execute great code, assuming stable application behavior.

When an optimization breaks, the JVM's strategy may also fail to adjust
correctly.  One symptom of that is infinite recompilation, usually because
one line of code is being handled badly, but which creates huge bloat
in the code cache for thousands of lines of code that happen to be
inlined nearby.  We try hard to avoid this.

We also try hard to detect this problem.  That is the true meaning of those
strange cutoffs.  Nobody things falling into the interpreter is a good idea,
except that it, on balance, is a better idea than (a) throwing an assertion
error, or (b) filling the CPU with JIT jobs and the code cache with discards.
The third choice (c) run offending method in the interpreter at least
preserves a degree of forward progress, while allowing the outraged
user to report a bug.

The correct fix to the bug, IMO, is never to jump from (c) to (a) or (b).
It is to find and fix the problem with the compilation strategy, and the
profile-driven gating logic for it.

If your car's transmission gets a bug (now that they are computers,
they can), what would you prefer?
(a) stop the car immediately,
(b) run the car in first gear at full speed, or
(c) slow the car to a defined speed limit (25mph).
Detroit prefers, and the JVM implements, option (c).

> The problem here is that there is no way from the point of view of a dyn lang 
> runtime to know what will be the behavior of the VM for a callsite if the VM 
> decide to stop to recompile, decide to not inline, decide to inline some part 
> of the tree, etc.

Yes.  And it usually doesn't matter; the issue doesn't come up until something 
breaks,
or we find a performance pothole.  The current problem is (in my mind) a break, 
not
a performance pothole that needs tuning.  If we fix the break, people shouldn't 
need
to worry about this stuff, usually.

> Said differently, using an invokedynamic allows to create code shapes that 
> will change dynamically, if the VM behavior also changes dynamically, it's 
> like building a wall on moving parts, the result is strange dynamic behaviors 
> that are hard to diagnose and reproduce.

JVMs have always been like that, because of dynamic class loading, but with indy
it is more so, since it's much easier to "override" some previously fixed 
behavior.

> The recompilation behavior of the VM should be keep simple and predicatable, 
> basically, the VM should always recompile the CS with no failsafe switch.

We agree that the failsafe should not trip.  Just like we agree that
the circuit breakers in our building should not trip.  We disagree,
perhaps, about what to do when they trip.  I don't want to duct-tape
them back into the "on" position; do you?

> If dyn lang runtime devs have trouble with that, they can already use an 
> exactInvoker to simulate an indirect mh call and we can even provide new 
> method handle combiners to gracefully handle multi-stable CS.

That's all true.  The new combiners might have some sort
of handshake with the JVM to self-adjust their code shape.

But I claim the baseline behavior that I have called for is
the most generally useful, since it is able to amortize
recompilation resources over multiple CS misses,
put global limits on total recompilation effort, and
preserve reasonable forward progress executing
good-enough code.

(Having a CS change force a reoptimization is tantamount to
adding a JIT control API, as Compiler.recompile(cs) like
System.gc().  But just for CS-bearing methods.  We are
a long way from understanding how to work such an API.)

Idea:  Perhaps CS's should have a callback which says,
"Hey, CS, the JIT has mispredicted you a bunch of times;
would you like to nominate an alternative representation?"
The call would be made asynchronously, outside the JIT.
The default behavior would be to say "nope" with the results
given above, but the CS could also return a MH (perhaps
the CS.dynamicInvoker, or perhaps some more elaborate logic),
which the JVM would slide into place over the top of the CS.
Despite the fact that CS bindings are final, the new binding
would take its place.  And it would be the user's choice
whether that binding pointed to the old CS or a new CS or
some combination of both.

— John
___
mlvm-dev mailing list

Re: MethodHandles.Lookup and modules

[John Rose]

FYI, we just had a useful conversation about how the new readability
restrictions from Jigsaw modules should affect the MHs.Lookup API.
Specifically, we need a couple more access modes to represent the new
layers of type readability (interacts with lookup and accessibility),
plus a special behavior (another access mode bit) for publicLookup,
to represent the fact that there is no longer a true global scope.

Given the increased complexity of access layers we also want to
add a (trivial) API point to directly drop access mode bits from a Lookup.

See http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-December/005829.html
(copied below) and nearby messages.

The Jigsaw guys have a charming name for what Lookup.in does:
It "teleports" a lookup to another location.  Teleporting outside a module
tends to shut down all access rights (to preserve monotonicity of privilege),
except for publicLookup which retains its special globalized viewpoint.

This is a refinement on the current Lookup enhancement in the Jigsaw repo.
As one consequence, the publicLookup.lookupClass can go back to
being Object, and is usefully teleportable elsewhere, so that (in turn)
it is broadly useful with the new API point Lookup.findClass, which
uses the lookupClass as a starting point for name resolution.

— John

Begin forwarded message:

From: John Rose <john.r.r...@oracle.com>
Subject: Re: MethodHandles.Lookup and modules
Date: December 18, 2015 at 12:20:08 AM PST
To: Alex Buckley <alex.buck...@oracle.com>
Cc: jigsaw-...@openjdk.java.net

On Dec 17, 2015, at 6:01 PM, John Rose <john.r.r...@oracle.com> wrote:
> 
> I think I would prefer case 2.  The user model is PUBLIC is the weakest 
> (non-empty) access
> mode available to bytecode behaviors.  As such it respects the LC's position 
> in the module
> graph, and excludes module-private, package-private, and class-private.  
> UNCONDITIONAL
> is the special thing provided by publicLookup, which ignores the module 
> graph.  Then
> PACKAGE opens up the LC's package, MODULE opens up the LC's module, and 
> PRIVATE
> opens up the LC itself (plus its nestmates).  Feels pretty good, especially 
> since MODULE
> and PACKAGE continue to have a parallel sense of restriction.
> 
> What do you think?

So I caught you in the hall and we talked, and this seems agreeable to us both,
perhaps with a name change to UNCONDITIONAL, and also a distinction between
PUBLIC and QUALIFIED (as you originally proposed).

To try and tease out some symmetry here:
- Always, any type T is accessible to itself, when T = LC.
- PACKAGE mode: Any type T is accessible (within its own package), when 
PACKAGE(T) = PACKAGE(LC).
- MODULE mode: A public type T is accessible (within or beyond its package), 
when MODULE(T) = MODULE(LC).
- QUALIFIED mode: A public type T is accessible beyond its module, when 
IS_CE(T, LC),
   where IS_CE(T, LC) = IS_CONDITIONALLY_EXPORTED(PACKAGE(T), MODULE(LC)) and 
MODULE(LC) READS MODULE(T).
- PUBLIC mode: A public type T is accessible beyond its module friends when 
IS_UE(T, LC),
   where IS_UE(T, LC) = IS_UNCONDITIONALLY_EXPORTED(PACKAGE(T)) and MODULE(LC) 
READS MODULE(T).

These conditions can be tested independently.  PACKAGE implies MODULE, but 
everything else is disjoint.

Also:
- UNCONDITIONAL: In this mode, a type T is accessible if 
IS_UNCONDITIONALLY_EXPORTED(PACKAGE(T)), regardless of LC.
- PRIVATE/PROTECTED: These protection modes apply only to non-types (JVM does 
not enforce "private" on classes).
- NOACCESS: This is not a mode but the absence of any combination of modes; no 
access is allowed.

The publicLookup should have UNCONDITIONAL and PUBLIC set.
An original full-power lookup does *not* have UNCONDITIONAL set, just PUBLIC.
The purpose of UNCONDITIONAL is to allow publicLookup to be unconcerned
(as documented) about its LC.  We can restore LC to be java.lang.Object.

The distinction between QUALIFIED and PUBLIC is present simply because of
the logical fact (as you point out) that, if you teleport to a new module, you
must lose your qualified imports, but you shouldn't lose your unconditional 
ones.

The distinction between PUBLIC and UNCONDITIONAL is present in order
to capture the differing behaviors of lookups derived from publicLookup and
those derived from full-power lookups.

The presence of MODULE captures the larger but package-like scope of
a module's internal names.

About "mode stripping":

You suggested (which sounds OK) that there is no need to "validate" bit masks 
of lookup objects.
Just have excludeModes clear some bits and continue.  This means there can be 
lookups which
are able to read (say) using PACKAGE mode but not PUBLIC mode.  (Today's 
lookups can
have PUBLIC without PACKAGE but not vice versa.)  Each mode bit enables a single
line in the above logic, and (as you can see) the lines can be applied 
independently.
Some implications follow.

Re: RFR(L): 8139885: implement JEP 274: enhanced method handles

[John Rose]

On Nov 13, 2015, at 8:39 AM, Michael Haupt  wrote:
> 
> Dear all,
> 
> please review this change.
> RFE: https://bugs.openjdk.java.net/browse/JDK-8139885
> Corresponding JEP: https://bugs.openjdk.java.net/browse/JDK-8130227
> Webrev: http://cr.openjdk.java.net/~mhaupt/8139885/webrev.00/ 
> 

Looks great.  One bug:

+static boolean isVoid(Class t) {
+return t == void.class || t == Void.class;
+}

I think there's nothing in the spec that calls for java.lang.Void to get 
special processing for try/finally.
If I'm right, get rid of isVoid; just check for void.class not Void.class.

I'd prefer to see reuse of misMatchedTypes() or newIllegalArgumentException() 
instead of err().  I would prefer to push formatting should into the 
error-producing subroutine.  This can be cleaned up later.  OTOH, the 
complexity of string concat instructions will be going down with Aleksey's 
work, so maybe I shouldn't be so sensitive to inline concat stuff.

My usual practice, though, is to move argument checking and error reporting 
code out of line, away from the main logic.  I think this is good style, and it 
gives the JIT a little bit (a very little bit) of help.

In that vein, the argument checking function foldArgumentChecks should stay 
private, shouldn't it?

I'm impressed to see how useful the Stream API is for doing the loop combinator.

Testing is good.  I'm glad to see new BigArity cases.

One wishful item:  It would be nice if the javadoc examples could be integrated 
into JavaDocExamplesTest.java.  I see that is messy since we are now using 
TestNG instead of JUnit.  The point of JavaDocExamplesTest was to make it easy 
to "sync" the javadoc with the examples, by having one place to copy-and-paste 
the javadoc.

Anyway, reviewed, conditional on that one bug fix.

— John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: JFokus VM Tech Day 2016

[John Rose]

On Nov 11, 2015, at 12:35 AM, Marcus Lagergren  wrote:
> 
> bare silicone magic

That extra E moves the venue from Santa Clara to Las Vegas.___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Proposed API for JEP 259: Stack-Walking API

[John Rose]

On Oct 31, 2015, at 11:50 AM, Remi Forax  wrote:
> 
> I think most of the runtime language developers, myself included will kill to 
> have this feature included into the JDK.
> There are several features of dynamic languages that are currently hard to 
> implement like type specialization, stack reification, as you said fibers, 
> and more generally de-optimization that will be far easier to implement with 
> that capability (currently only Nashorn can afford to implement some of these 
> optimizations).

Oh, yes, we've been thinking about it, eagerly, even though other things 
(modules, true polymorphism, value types, native interconnect) keep pushing to 
the top of the list.

Most recently, at the JavaOne "Ask the Architects", we were asked "what 
big-ticket item would you add to Java that is not already on the map?"  Mark 
immediately said, "coroutines".

— John

P.S. ...Which relieved me of the duty to say "coroutines", allowing me to talk 
about a group of overlapping DSL-like features like object construction 
expressions, complex constants, builders and complex array selection 
expressions.   Those things all fit together, you know.

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Proposed API for JEP 259: Stack-Walking API

[John Rose]

On Oct 31, 2015, at 12:51 PM, John Rose <john.r.r...@oracle.com> wrote:
> 
> modules, true polymorphism, value types, native interconnect

P.S.  Project refs:  Jigsaw, Valhalla (List), Valhalla, Panama.___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Proposed API for JEP 259: Stack-Walking API

[John Rose]

On Oct 31, 2015, at 12:54 PM, Vitaly Davidovich  wrote:
> John, is there a link for more info on "true polymorphism"

It's what's in Valhalla, if you ignore the value types.

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: invokespecial-super-init

[John Rose]

I, not generated by Groovy
interface WC { } // marker type for wrapper constructors
class C {  // generated by Groovy
  final char p, q;
  private static class Finals { final char p, q; }
  private C(WC ig, int x, MethodHandle finals) { super(x); Finals f = 
finals.invokeExact(this); this.p=finals.p; this.q=finals.q; }
  private C(WC ig, String y, MethodHandle finals) { super(y); Finals f = 
finals.invokeExact(this); this.p=finals.p; this.q=finals.q; }
  public C(int x, boolean z) { this(null, x, MH.bindTo(z)); }
  public C(String y, boolean z) { this(null, y, MH.bindTo(z)); }
  // public C(DynamicArgList dynamicArgs) { this(no can do!); super(this 
neither!); }
}

This pattern is approximately as general as random bytecodes inside 
constructors, is reasonably compact, and does not require new method handle 
types or verifier rules.

(Note that the MH "finals" is able to "see" the UI under the type C.  It is 
supposed to treat it reasonably, just like constructor code is supposed to.  
Since the wrapper constructors are marked private, it is impossible for 
untrusted parties to inject malicious MH code.  The MH could be replaced by a 
private instance method, if there is no need to have a different MH at 
different construction sites.)

What do you think?  Is this close to the workarounds you already use?

— John

> bye jochen
> 
> 
> Am 29.08.2015 03:40, schrieb John Rose:
>> The invokespecial-super-init dance is the thing MH's can't quite do, the 
>> "super" call every constructor (except Object.).
>> 
>> It very hard to secure this pattern; just ask anybody who has worked on 
>> (de-)serialization security.
>> 
>> But, we can look at it from a more limited point of view which might improve 
>> your use case, Jochen.
>> 
>> A method handle is supposed to be a fully competent replacement for 
>> hardwired bytecodes, and it is, except for invokespecial-super from a 
>> constructor.  The reason this is hard is that there is no way to constrain 
>> such a method handle, once constructed, to operate inside a constructor.  
>> And superclasses have a right to expect that you are running their 
>> constructor as a unique, non-repeatable part of creating a subclass object.  
>> (By "have a right" I really mean "it would be wrong to do the unexpected" by 
>> which I also mean "attack surfaces are likely to open up if we do this.)
>> 
>> So, is there a way to package up a method handle so that it can only be used 
>> as as unique, non-repeatable part of creating a subclass object?  Yes, it 
>> can:  Wire in an unconditional "new instance" operation, and immediately run 
>> the "invokespecial super" on the new thing.
>> 
>> Now the problem reduces to:  Your class (just like its super) has a right to 
>> expect that constructor code will be run on every newly-created instance 
>> (after the super constructor), before the new object is made available to 
>> other code.  Can we package up the previous new-invokespecial-super method 
>> handle so it can only be used in this way?  Well, no, since every 
>> constructor *also* has a hardwired call to invokespecial; we are back to the 
>> pre-existing new-invokespecial type of MH.
>> 
>> There are several possible ways out, but the problem is delicate.  The 
>> purpose of constructors is to statically mark code that must be executed 
>> before any (normally published) reference to an object is reachable by 
>> non-class code.  If there were a way to statically mark code as 
>> "post-super-init" (""?), we could make an agreement with a 
>> class that such a method would serve as the equivalent of a constructor, but 
>> it would be the caller's responsibility to allocate the new instance *and* 
>> call the super init.  Allowing bytecode to call this stuff would require a 
>> bunch of new verifier rules, in a place where the verifier is already hard 
>> to understand.  Perhaps a method handle could be allowed to operate where 
>> normal bytecode cannot, but you see the problem:  Method handles are 
>> designed to give a dynamic alternative to things you can already do in 
>> bytecode.
>> 
>> The "post-super-init" convention can be a private convention within a class, 
>> in the special case of Groovy, since Groovy is responsible for generating 
>> the whole class, and can trust itself to invoke all necessary initialization 
>> code on each new instance.  So if you had an new-invokespecial-super MH in a 
>> private context within a Groovy-generated class, you could use it to create 
>> a "mostly blank" instance, and then fill it in before sharing it with 
&g

Re: invokespecial-super-init

[John Rose]

On Sep 17, 2015, at 10:10 AM, Michael Haupt  wrote:
> 
> ummm ... this seems to imply I can remove the findSuperConstructor() proposal 
> from the Indy JEP. Incidentally, it's on my list for this week - and less 
> work is always good. ;-) Even if, in this case, it leads to disappointment. I 
> agree with you in that opening up the MH API like this will introduce several 
> trapdoors and additional complication.
> 
> Please let me know ...

Hi Michael.  See previous message.  It looks like (in most cases, mostly) the 
burden can be put back on the bytecode generators, like Groovy.

In a couple of cases we might want to add API for these use cases:

1. The workarounds are so complex and error-prone that a convenience function 
is needed.  (As with PICs, etc., requires a matured notion of what the 
convenience should be.)

2. We might still need some sort of multi-way super. call, but I think 
there are bytecode-level workarounds for this also, that verify today.  (The 
multi-way super comes is visible in the "no can do!" comment of my POC code.)

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

invokespecial-super-init

[John Rose]

The invokespecial-super-init dance is the thing MH's can't quite do, the 
super call every constructor (except Object.init).

It very hard to secure this pattern; just ask anybody who has worked on 
(de-)serialization security.

But, we can look at it from a more limited point of view which might improve 
your use case, Jochen.

A method handle is supposed to be a fully competent replacement for hardwired 
bytecodes, and it is, except for invokespecial-super from a constructor.  The 
reason this is hard is that there is no way to constrain such a method handle, 
once constructed, to operate inside a constructor.  And superclasses have a 
right to expect that you are running their constructor as a unique, 
non-repeatable part of creating a subclass object.  (By have a right I really 
mean it would be wrong to do the unexpected by which I also mean attack 
surfaces are likely to open up if we do this.)

So, is there a way to package up a method handle so that it can only be used as 
as unique, non-repeatable part of creating a subclass object?  Yes, it can:  
Wire in an unconditional new instance operation, and immediately run the 
invokespecial super on the new thing.

Now the problem reduces to:  Your class (just like its super) has a right to 
expect that constructor code will be run on every newly-created instance (after 
the super constructor), before the new object is made available to other code.  
Can we package up the previous new-invokespecial-super method handle so it can 
only be used in this way?  Well, no, since every constructor *also* has a 
hardwired call to invokespecial; we are back to the pre-existing 
new-invokespecial type of MH.

There are several possible ways out, but the problem is delicate.  The purpose 
of constructors is to statically mark code that must be executed before any 
(normally published) reference to an object is reachable by non-class code.  If 
there were a way to statically mark code as post-super-init 
(postsuperinit?), we could make an agreement with a class that such a 
method would serve as the equivalent of a constructor, but it would be the 
caller's responsibility to allocate the new instance *and* call the super init. 
 Allowing bytecode to call this stuff would require a bunch of new verifier 
rules, in a place where the verifier is already hard to understand.  Perhaps a 
method handle could be allowed to operate where normal bytecode cannot, but you 
see the problem:  Method handles are designed to give a dynamic alternative to 
things you can already do in bytecode.

The post-super-init convention can be a private convention within a class, in 
the special case of Groovy, since Groovy is responsible for generating the 
whole class, and can trust itself to invoke all necessary initialization code 
on each new instance.  So if you had an new-invokespecial-super MH in a private 
context within a Groovy-generated class, you could use it to create a mostly 
blank instance, and then fill it in before sharing it with anybody else.  Such 
an invokespecial-super MH could be adequately protected from other users by 
requiring that Lookup.findSpecialConstructor can only work on full-powered 
lookups, regardless of the accessibility of the super constructor.

There are two further problems with this, though.  First, constructors have a 
unique ability and obligation to initialize blank final variables (the 
non-static ones).  So the Lookup.findSpecialConstructor MH has to take an 
argument, not just for its super-constructor, but also for *each* final 
variable in the *current* class.  (Note that Lookup.findSetter will *not* allow 
finals to be set, because it cannot prove that the caller is somehow inside a 
constructor, and, even if inside it, is trustably acting on behalf of it.)  
There are other ways to go, but you can see this problem too:  The 
new-invokespecial operator has to take responsibility for working with the 
caller to fill in the blank finals.

The second further problem is even more delicate.  The JVM enforces rules of 
calling init even (sometimes) against the wishes of people who generate class 
files.  We don't fully understand the practical effects of relaxing these 
rules.  Proofs of assertions (such as type correctness and security) require 
strong premises, and the rigid rules about init help provide such premises.  
An example of a proof-failure would be somebody looking at a class, ensuring 
that all instances are secure based on the execution of init methods, but 
then fail to notice that the class *also* runs some instances through an 
alternate path, using new-invokespecial-super, which invalidates the proof by 
failing to run some crucial setup code.

With all that said, there is still wiggle room.  For example, one *possible* 
solution that might help Groovy, while being restrictive enough to avoid the 
problems above, would be to split init methods and sew them together again 
with method handles.

Suppose there were a reliable way 

Re: What can we improve in JSR292 for Java 9?

[John Rose]

On Aug 26, 2015, at 6:06 AM, Rémi Forax fo...@univ-mlv.fr wrote:
 
 it's less powerful but it seems you can encode all your examples of loops too.

No, I don't think so.  Multiple pred/fini pairs implement multiple loop exits.
Any loop with an if-return in it has an exit distinct from the loop's main 
test.
And each exit requires a different fini function, to encode a special result.

The match loop has this multi-exit character:

  for (int i; i  a.len; i++) {
Object e = a[i];
if (ok(e))  return e;  // first fini function; could also be e.payload
  }
  return null;  // second fini function; could also be throwNotFound or 
Optional.none

Sometimes even  type predicate logic expands into multi-exit code, if the 
loop needs to return a value that encodes which sub-predicate caused the exit.

— John


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: What can we improve in JSR292 for Java 9?

[John Rose]

On Feb 25, 2015, at 6:29 PM, John Rose john.r.r...@oracle.com wrote:
 
 Maybe this is general enough:
 
MHs.loop(init, predicate, body)(*a)
= { let i = init(*a); while (predicate(i, *a)) { i = body(i, *a); } 
 return i; }
 
 ...where the type of i depends on init, and if init returns void then you
 have a classic side-effect-only loop.

FTR, I've been thinking more about this one, with help from Michael Haupt.

The above loop is pretty good, except for a couple of things: 'i' ought to be 
tuple of loop-varying values, and the exit path needs a more general 
post-processing step, a finalizer that corresponds to the initializers that 
set up the loop variables.

A generalized (linear) loop has two kinds of values flowing within it: 
loop-invariant and loop-varying. (The control predicate has a special role, but 
may be classed as loop-varying. The finalizer is also special, and 
loop-invariant.) Loop-varying values can be inputs, control values (array 
indices, etc.), and outputs, but can all, I think, be modeled as step 
functions using a common signature.

This takes us to something like the C for-loop, but with a sprinkle of tuples, 
with a finishing bow around it to tie it up:
  for ((v1,v2,v3,…) = inits; predicate(v1…); (v1…) = step(v1…))  { … };  return 
finish(v1…);

Loop-local but invariant values can be modeled as non-updated loop variables, 
as in this C idiom:
  for (int i, limit = computeLimit(); i  limit; i++) …

Pure side effects can be modeled as void-valued step functions, so we want to 
allow void as a corner case for loop variable types.  (I.e., where there is a 
step but not a value.)  I don't want to make any more special provision for 
pure side effects (such as a unique Runnable-like body for a loop), both 
because it isn't necessary, and because it encourages sloppy side-effect-driven 
loops.

If we allow each loop variable to have its own predicate and finalizer, we can 
properly model the effects of ordered guards and special early-exit paths.

loop({init*}, {pred*}, {step*}, {fini*}) := lambda (arg*) {
  let var* = init*(arg*)
  for (;;) {
for ((v,p,s,f) in (var*, pred*, step*, fini*))
  if (!p(var*, arg*))
return f(var*, arg*)
  v = s(var*, arg*) // all vars help update each var
}
  }
}

All invocations are exact.

As a bonus we can easily derive post-checked loops (do-while), and time-shifted 
index variables (p++) simply by adding more variables and/or steps.

Constraints on parameter and return types: 
- There must be the same number of inits, preds, steps, and finis.
- There must be at least one input function (e.g., a pred, so that the loop has 
a chance to exit).
- Parameter lists of the init functions must all be the same.
- Parameter lists of the pred, step, and fini functions must all be the same.
- Parameter lists of the step functions must be a suffix of the parameter lists 
of the init functions.
- Parameters of the step functions must be the same as the return types of the 
init functions (with voids removed), followed by the common suffix.
- Return types of the predicates must all be boolean.
- Return types of the finalizers must all be the same (and can be void).
- The return type of each init function must match the return type of the 
corresponding step function (a var type or void).

The result, a looping method handle, has a return type taken from the finalizer 
function(s), and argument types taken from the init functions.  The loop has 
one iteration variable for each non-void init/step function pair.  Each 
step/pred/fini call takes all of the iteration variable values, followed by all 
the external (loop-invariant) arguments.  No throws are caught by the loop; 
throws cause the loop to terminate abnormally.

In some loops, such as find first match, the predicate naturally follows the 
step function, because it needs to evaluate the result of the step.  To model 
post-checks (and the classic do/while shape), a second void-valued loop 
variable is required:

loop(...) := lambda (arg*) {
  let (var, _) = (null, void)
  for (;;) {
if (_) return _ // trivial predicate: exit branch never taken
var = s(_, arg*)  // pick up next search key
if (!p(var, arg*))  // did we find it?
  return f(var, arg*)  // extract value from search key as needed
_ = _(var, arg*) // void value, no side effect
  }
}

This variable-doubling could be compressed out if there were a further boolean 
(per pred/step pair) which says which order to run them in.  To me this seems 
like too much ad hoc structure; such structure tends to creep over time as we 
discover other important patterns.  I think it would be better to support 
optionality for common no op loop components, as follows:

Optionalities:
- If an init is null, a constant zero (or null/false/void) function is supplied 
by inspecting the return type of the corresponding step function.
- If a pred is null, a constant true function is supplied.
- If a step is null, a suitable

Re: ClassLoader leak in MethodHandle.asType()

[John Rose]

On Apr 24, 2015, at 3:24 PM, Peter Levart peter.lev...@gmail.com wrote:
 
 Anyway. The inexact invoke() always transforms a specific MH to a generic one 
 (Object, Object, ...)Object, right?

Yes.

 So using inexact invoke() on any MH can't trigger the leak. It's just that if 
 someone attempts to transform a generic MH to some more concrete one to be 
 able to connect it with further transformations does the danger of the leak 
 appear.

Good point, thanks.

This suggests that the right answer is to keep the 1-element cache as-is,
but don't fill it with any mh1=mh0.asType(...) for which mh1.type introduces
a class loader not already present in mh0.type.

I.e., don't use a weak reference for the 1-element cache, but don't fill it
with anything that might require a weak reference.  Use a backup cache
(if needed) to handle exotic type changes.

 So maybe, just for the purpose of inexact invoke(), the caching could be 
 performed just in case when asType transformed method type is of the form 
 (Object, Object, ...)Object. This would also prevent accidental thrashing of 
 the cache when inexact invoke() is intermixed with other asType() invocations.


The 1-element cache thrashes for other reasons already, and
I think we need a multi-way (multi-MH) backup cache of some sort.
The backup cache would have weak references.

Having a 1-element front-side cache would cover inefficiencies
in the backup cache, which in turn would cover inefficiencies 
for generating new conversion LFs and/or MHs.

One interesting point:  If we change the 1-element cache to use
a weak reference, the cost of reloading the cache will be higher,
since each reload must create a new WeakReference instance.
This is a possible reason to hold off on the suggested fix.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (XS): 8078290: Customize adapted MethodHandle in MH.invoke() case

[John Rose]

Reviewed. Nice find. 

– John

 On Apr 21, 2015, at 10:37 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
 wrote:
 
 http://cr.openjdk.java.net/~vlivanov/8078290/webrev.00/
 https://bugs.openjdk.java.net/browse/JDK-8078290
 
 Customization logic introduced in [1] doesn't work as expected for 
 MH.invoke() when asType() conversion happens and cached MH is used.
 
 Generic invoker LambdaForm looks like:
 
 invoke_002_MT=Lambda(a0:L,a1:L,a2:L)={
t3:L=Invokers.checkGenericType(a0:L,a2:L);
t4:V=Invokers.checkCustomized(a0:L);
t5:L=MethodHandle.invokeBasic(t3:L,a1:L);t5:L}
 }
 
 Original MH(a0) is customized, but adapted MH (t3) is used for invocation. a0 
 != t3 when MH.asType() conversion happens in Invokers.checkGenericType().
 
 Invoker LambdaForm should have the following structure:
 
 invoke_002_MT=Lambda(a0:L,a1:L,a2:L)={
t3:L=Invokers.checkGenericType(a0:L,a2:L);
t4:V=Invokers.checkCustomized(t3:L);
t5:L=MethodHandle.invokeBasic(t3:L,a1:L);t5:L}
 }
 
 Testing: manual (verified invoker LF), microbenchmark, 
 jdk/test/java/lang/invoke, hotspot/test/compiler/jsr292, nashorn
 
 Best regards,
 Vladimir Ivanov
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: comment from James Gosling on multi-language VM

[John Rose]

If you look closely at the date of the question, you'll see that it is almost 
exactly 20 years old, as is James' answer.

— John

 On Apr 17, 2015, at 10:40 AM, Fernando Cassia fcas...@gmail.com wrote:
 
 
 On Thu, Apr 16, 2015 at 8:19 PM, John Rose john.r.r...@oracle.com 
 mailto:john.r.r...@oracle.com wrote:
 Has anyone looked at the possibility of compiling languages other than
 Java to the Java bytecodes?
 
 xRuby compiles early Ruby to Java Bytecode
 https://code.google.com/p/xruby/ https://code.google.com/p/xruby/
...___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

comment from James Gosling on multi-language VM

[John Rose]

James made some relevant comments on the JVM as a multi-language engine.

It was just a little while ago, as I noticed going through some old files.

Folks are still working on specialized allocation (aka. value types) and 
continuations (aka. coroutines  tailcall).

Also, C is no longer right out; see Project Panama.

— John

From: owner-hotjava-interest-dig...@java.sun.com
To: hotjava-interest-dig...@java.sun.com
Subject:   hotjava-interest-digest V1 #24
Date: Sat, 1 Apr 1995 17:34:17 -0800
Reply-To: hotjava-inter...@java.sun.com
X-Info: To unsubscribe, send 'unsubscribe' to 
hotjava-interest-digest-requ...@java.sun.com

hotjava-interest-digestSunday, 2 April 1995Volume 01 : Number 
024


--

From: rich...@cogsci.ed.ac.uk
Date: Fri, 31 Mar 95 13:30:04 BST
Subject: How general is the Java bytecode language?

Has anyone looked at the possibility of compiling languages other than
Java to the Java bytecodes?  In particular, there seem to be many people
interested in using Scheme as a scripting language, and the ability
to download Scheme programs into HotJava would be nice.

- -- Richard

--
...

--


From: j...@scndprsn.eng.sun.com (James Gosling)
Date: Fri, 31 Mar 1995 20:34:24 +0800
Subject: Re: How general is the Java bytecode language?

 Has anyone looked at the possibility of compiling languages other than
 Java to the Java bytecodes?  In particular, there seem to be many people
 interested in using Scheme as a scripting language, and the ability
 to download Scheme programs into HotJava would be nice.

It's reasonably general, although the security restrictions make some
languages hard.  C and C++ with general pointer arithmetic are right
out.  Fortran's pass-by-reference call semantics and common are
tough.  Pascal is pretty straightforward.  Scheme is easy or hard,
depending on how good you want the performance to be: the invocation
model in Java is very C like with no provision for continuations.
Continuations can be handled as classes  the key trick in a compiler
would be to decide when a full-blown continuation can be optimized
away.  The story is similar with datatypes like numbers: in general,
every number would be an instance of some class  the trick is to
optimize that into direct machine instructions.  Using the Java
bytecodes is likely to be somewhat less than optimal because things
like continuations and general numbers need to be done as classes, the
underlying engine doesn't have things like specialized allocators for
boxed integers.


Note to Sun employees: this is an EXTERNAL mailing list!
Info: send 'help' to hotjava-interest-requ...@java.sun.com

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (XXS): 8077054: DMH LFs should be customizeable

[John Rose]

Looks good.  Would you please also check the number of distinct LFs compiled 
when running standard benchmarks?  (With and without the change.)  The number 
should go up, but only slightly.

Thanks!
— John

On Apr 9, 2015, at 11:44 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 http://cr.openjdk.java.net/~vlivanov/8077054/webrev.00/
 https://bugs.openjdk.java.net/browse/JDK-8077054
 
 It's a followup fix for JDK-8069591 [1].
 
 As Charlie observed, it's important to customize DirectMethodHandles as well. 
 Otherwise, additional frame for DMH LambdaForm is created and it causes 
 observable peak performance regression in some situations (e.g. JRuby9K).
 
 Testing: manual (microbenchmark, compilation log inspection).
 
 Thanks!
 
 Best regards,
 Vladimir Ivanov
 
 [1] https://bugs.openjdk.java.net/browse/JDK-8069591
 Customize LambdaForms which are invoked using
  MH.invoke/invokeExact
 ___
 mlvm-dev mailing list
 mlvm-dev@openjdk.java.net
 http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8057967: CallSite dependency tracking scales devastatingly poorly

[John Rose]

On Apr 2, 2015, at 9:17 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 
 I recommend putting CONTEXT_OFFSET into CallSite, not the nested class.
 For one thing, your getDeclaredField call will fail (I think) with a 
 security manager installed.
 You can load it up where TARGET_OFFSET is initialized.
 Since I removed DependencyContext, I moved CONTEXT_OFFSET to CallSite.
 
 BTW why do you think security manager was the problem? (1) 
 Class.getDeclaredField() is caller-sensitive; and (2) DependencyContext was 
 eagerly initialized with CallSite (see UNSAFE.ensureClassInitialized() in 
 original version).

CallSite$DependencyContext and CallSite are distinct classes.
At the JVM level they cannot access each others' private members.
So if DependencyContext wants to reflect a private field from CallSite,
there will be extra security checks.  These sometimes fail, as in:

https://bugs.openjdk.java.net/browse/JDK-7050328

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8057967: CallSite dependency tracking scales devastatingly poorly

[John Rose]

On Apr 1, 2015, at 1:56 PM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 http://cr.openjdk.java.net/~vlivanov/8057967/webrev.00/hotspot/
 http://cr.openjdk.java.net/~vlivanov/8057967/webrev.00/jdk/
 https://bugs.openjdk.java.net/browse/JDK-8057967

Impressive work.

Question:  How common is state 2 (context-free CS) compared to state 3 
(indy-bound CS)?

And is state 2 well tested by Box2D?

I recommend putting CONTEXT_OFFSET into CallSite, not the nested class.
For one thing, your getDeclaredField call will fail (I think) with a security 
manager installed.
You can load it up where TARGET_OFFSET is initialized.

I haven't looked at the JVM changes yet, and I don't understand the cleaner, 
yet.

Can a call site target class change as a result of LF recompiling or 
customization?
If so, won't that cause a risk of dropped dependencies?

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: The curious case of MHS.Lookup.unreflect on MethodHandle.invoke/invokeExact

[John Rose]

On Mar 18, 2015, at 3:59 AM, Paul Sandoz paul.san...@oracle.com wrote:
 
 Perhaps such method handles were originally cached to avoid an explosion 
 (deliberate or otherwise) of class generation of LFs, but now there is more 
 sophisticated LF caching in place this is not necessary.?

That's probably correct.  Per-MT caching is overkill for all but the most 
crucial MHs.  — John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (S): 8074548: Never-taken branches cause repeated deopts in MHs.GWT case

[John Rose]

Good fix, thanks.  — John

P.S.  I noticed a typo:  it's (contraction of it is) should be its 
(genitive of it).

On Mar 16, 2015, at 11:26 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 http://cr.openjdk.java.net/~vlivanov/8074548/webrev.00/
 https://bugs.openjdk.java.net/browse/JDK-8074548
 
 MethodHandleImpl::profileBoolean doesn't update never-taken branch count when 
 hitting a deopt on it. As a result, for rarely taken branches consequent 
 compilations consider them as never-taken and prune them again causing 
 repeated deopts. It severely affects peak performance.
 
 The fix is to update MHI::profileBoolean intrinsic to insert a guard and 
 uncommon trap w/ reexecute bit set for never-seen value. Once previously 
 never seen value is encountered, the execution resumes after deopt in 
 MHI::profileBoolean and corresponding count becomes non-zero.
 
 The guard doesn't add any additional overhead, since it dominates all value 
 usages and all branches on the same value are eliminated.
 
 Testing: java/lang/invoke, nashorn, octane

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (S): 8075263: MHI::checkCustomized isn't eliminated for inlined MethodHandles

[John Rose]

Reviewed.  — John

On Mar 16, 2015, at 11:47 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 http://cr.openjdk.java.net/~vlivanov/8075263/webrev.00/hotspot
 http://cr.openjdk.java.net/~vlivanov/8075263/webrev.00/jdk
 https://bugs.openjdk.java.net/browse/JDK-8075263


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: What can we improve in JSR292 for Java 9?

[John Rose]

On Mar 6, 2015, at 5:30 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 * try/finally as a core atom of MethodHandles API.
 
 Libraries like invokebinder provide a shortcut API To generating the
 large tree of handles needed for try/finally, but the JVM may not be
 able to optimize that tree as well as a purpose-built adapter.
 
 I agree there.  We should put this in.
 
MHs.tryFinally(target, cleanup)(*a)
  = { try { return target(*a); } finally { cleanup(*a); } }
 
 (Even here there are non-universalities; what if the cleanup
 wants to see the return value and/or the thrown exception?
 Should it take those as one or two leading arguments?)
 Probably, it's too much. Result-agnostic cleanup logic looks like the 
 cleanest solution from API perspective.
 
 There are catchException and foldArguments/filterReturnValue to combine with. 
 Or am I missing something?

As Charlie has explained, try/finally is *just barely* doable with the existing 
API points, as you note.  I think we want a composite API point which can be 
readily compiled down to the natural JVM operations, instead of requiring the 
user to emit a complex pattern.  The complex pattern will probably not be 
recognizable, and so won't be compiled down to the simple JVM operations.

 
 * Implementation-specific inspection API.
 
 I know there are different ways to express a MH tree on different JVMs
 (e.g. J9) but it would still be a big help for me if there were a good
 way to get some debug-time structural information about a handle I'm
 using. Hidden API would be ok if it's not too hidden :-)
 
 Idea of the day:  An ASM-like library for method handles.
 Make a MethodHandleReader which can run a visitor over the MH.
 The ops of the visitor would be a selection of public MH operations
 like filter, collect, spread, lookup, etc.
 Also ASM-like, the library would have a MethodHandleWriter
 would could be hooked up with the reader to make filters.
 Not sure how useful it would be for an API user. I find it hard to recover MH 
 transformation chains from MethodHandle trees, since the representation we 
 use is quite low-level. They end up as trees of BoundMethodHandles and you 
 can guess the intention only by carefully inspecting LambdaForms.

Yes, that's probably the downside.  But (remaining hopeful) even if decoding 
the LF will not recover the original MH transformation chain, it may recover an 
*equivalent* one.

 You can try it yourself by turning on 
 -Djava.lang.invoke.MethodHandle.DEBUG_NAMES=true and calling 
 MethodHandle.toString() on a method handle of your choice.


That's the first thing for a customer to try.

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: What can we improve in JSR292 for Java 9?

[John Rose]

On Mar 4, 2015, at 2:47 PM, Charles Oliver Nutter head...@headius.com wrote:
 
 Busy week, finally circling back to this thread...
 
 On Wed, Feb 25, 2015 at 8:29 PM, John Rose john.r.r...@oracle.com wrote:
 * A loop handle :-)
 
 Given a body and a test, run the body until the test is false. I'm
 guessing there's a good reason we don't have this already.
 
 A few reasons:   1. You can code your own easily.
 
 I can't code one that will specialize for every call path, though,
 unless I generate a new loop body for every call path.

Good point; if we had an intrinsic we could specialize this more readily, 
across more degrees of freedom.

OTOH:  What you are proposing is a workaround for a basic weakness in our 
system which we must eventually address.
There is no reason why you shouldn't be able to write a fully generic algorithm 
(generic across calling sequences) and get good performance for each distinct 
instance of the algorithm.  (Cue rants about the Loop Customization Problem.)

But, as with try/finally, if there is a common use case which can only be 
expressed in a strained fashion using existing API points, we should provide a 
more natural notation for that use case; both users and JVM will find it easier 
to DTRT.

 2. There's no One True Loop the way there is a One True If.
 The run until test is false model assumes all the real work is
 done with side-effects, which are off-center from the MH model.
 
 This I can appreciate. My mental model of MHs started to trend toward
 a general-purpose IR, and I believe if I had some sort of backward
 branch it could be that. But I understand if that's the wrong
 conceptual model, and I realize now that MHs are basically call stack
 adapters with a bit of forward branching thrown in.

Good; you are hitting me where it counts, in the bytecodes.

 It does feel like there's a need for better representation of
 branch-joining or phi or whatever you want to call it, though.

More details on this, please?  The tail-part of a GWT is a phi.  Or are you 
talking about phis on back-edges?

I agree we need a loop, if only because we want to compile them to (the 
equivalent of) backwards-branching bytecodes.  It's the same argument as for 
try/finally.

 3. A really clean looping mechanism probably needs a sprinkle
 of tail call optimization.
 
 I'm not saying that loops should never have side effects, but I
 am saying that a loop mechanism should not mandate them.
 
 Maybe this is general enough:
 
MHs.loop(init, predicate, body)(*a)
= { let i = init(*a); while (predicate(i, *a)) { i = body(i, *a); } 
 return i; }
 
 ...where the type of i depends on init, and if init returns void then you
 have a classic side-effect-only loop.
 
 Ahh yes, this makes sense. If it were unrolled, it would simply be a
 series of folds and drops as each iteration through the body modified
 the condition in some way. So then we just need it to work without
 unrolling.

Glad you like it.  This looks like a candidate, then.

(I wish we had a similar candidate for invokespecial/super.  That is badly 
twisted around the verifier.)

 My silly use case for this would be to emit simple expressions
 entirely as a MH chain, so we'd get the benefit of MH optimizations
 without generating our own bytecode (and with forced inlining and
 perhaps a richer semantic representation than just bytecode). It's not
 a very compelling case, of course, since I could just emit bytecode
 too.

If it's a common use case, even if you could do it with bytecode spinning, it 
is reasonable to support a MH version, to help users avoid the friction of 
switching notations between MHs and bytecodes.

 
 * try/finally as a core atom of MethodHandles API.
 
 Libraries like invokebinder provide a shortcut API To generating the
 large tree of handles needed for try/finally, but the JVM may not be
 able to optimize that tree as well as a purpose-built adapter.
 
 I agree there.  We should put this in.
 
   MHs.tryFinally(target, cleanup)(*a)
 = { try { return target(*a); } finally { cleanup(*a); } }
 
 (Even here there are non-universalities; what if the cleanup
 wants to see the return value and/or the thrown exception?
 Should it take those as one or two leading arguments?)
 
 In InvokeBinder, the finally is expected to require no additional
 arguments compared to the try body, since that was the use case I
 needed.

Good to know.  We start from what we know we need.

 You bring up a good point...and perhaps the built-in JSR292
 tryFinally should take *two* handles: one for the exceptional path
 (with exception in hand) and one for the non-exceptional path (with
 return value in hand)? The exceptional path would be expected to
 return the same type as the try body or re-raise the exception. The
 non-exceptional path would be expected to return void.

Yup.  Or, the exception path could be a single MH that takes a union argument 
of U(normal-value, exception) and returns the expected value.  In the JVM

Re: Lost perf between 8u40 and 9 hs-comp

[John Rose]

On Mar 3, 2015, at 8:23 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 Suggestion:  Instead of have the intrinsic expand to nothing, have it expand 
 to an uncommon trap (on the slow path), with the uncommon trap doing the 
 profile update operation (as currently coded).
 Right now, VM doesn't care about profiling logic at all. The intrinsic is 
 used only to inject profile data and all profiling happens in Java code. Once 
 MHI.profileBoolean is intrinsified (profile is injected), no profiling 
 actions are performed.

What I'm thinking is that an uncommon trap could re-run the interpreter 
definition of MHI.profileBoolean using Action_reinterpret.
That would update the state, wouldn't it?  Then the compiler would recompile 
(after a little time) and see the updated state.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: What can we improve in JSR292 for Java 9?

[John Rose]

On Feb 25, 2015, at 4:02 PM, Charles Oliver Nutter head...@headius.com wrote:
 
 After talking with folks at the Jfokus VM Summit, it seems like
 there's a number of nice-to-have and a few need-to-have features we'd
 like to see get into java.lang.invoke. Vladimir suggested I start a
 thread on these features.
 
 A few from me:
 
 * A loop handle :-)
 
 Given a body and a test, run the body until the test is false. I'm
 guessing there's a good reason we don't have this already.

A few reasons:   1. You can code your own easily.
2. There's no One True Loop the way there is a One True If.
The run until test is false model assumes all the real work is
done with side-effects, which are off-center from the MH model.
3. A really clean looping mechanism probably needs a sprinkle
of tail call optimization.

I'm not saying that loops should never have side effects, but I
am saying that a loop mechanism should not mandate them.

Maybe this is general enough:

MHs.loop(init, predicate, body)(*a)
= { let i = init(*a); while (predicate(i, *a)) { i = body(i, *a); } return 
i; }

...where the type of i depends on init, and if init returns void then you
have a classic side-effect-only loop.

 * try/finally as a core atom of MethodHandles API.
 
 Libraries like invokebinder provide a shortcut API To generating the
 large tree of handles needed for try/finally, but the JVM may not be
 able to optimize that tree as well as a purpose-built adapter.

I agree there.  We should put this in.

   MHs.tryFinally(target, cleanup)(*a)
 = { try { return target(*a); } finally { cleanup(*a); } }

(Even here there are non-universalities; what if the cleanup
wants to see the return value and/or the thrown exception?
Should it take those as one or two leading arguments?)

 * Argument grouping operations in the middle of the argument list.
 
 JRuby has many signatures that vararg somewhere other than the end of
 the argument list, and the juggling required to do that logic in
 handles is complex: shift to-be-boxed args to end, box them, shift box
 back.

We now have MHs.collectArguments.  Do you want MHs.spreadArguments
to reverse the effect?  Or is there something else I'm missing?

 Another point about these more complicated forms: they're ESPECIALLY
 slow early in execution, before LFs have been compiled to bytecode.
 
 * Implementation-specific inspection API.
 
 I know there are different ways to express a MH tree on different JVMs
 (e.g. J9) but it would still be a big help for me if there were a good
 way to get some debug-time structural information about a handle I'm
 using. Hidden API would be ok if it's not too hidden :-)

Idea of the day:  An ASM-like library for method handles.
Make a MethodHandleReader which can run a visitor over the MH.
The ops of the visitor would be a selection of public MH operations
like filter, collect, spread, lookup, etc.
Also ASM-like, the library would have a MethodHandleWriter
would could be hooked up with the reader to make filters.

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: What can we improve in JSR292 for Java 9?

[John Rose]

On Feb 25, 2015, at 5:09 PM, Mark Roos mr...@roos.com wrote:
 
 I would like to see some form of PIC (polymorphic inline cache ) support that 
 jits (inlines)  well and transitions from mono to bi to multi(4,5) to mega 
 nicely. 

What would be the starting points for such a thing?

It seems to be like Charlie's MH builder, something that would be a library 
class.

But maybe (like tryFinally) there are points where the library class has a hard
time making its intentions clear to the JVM, because of API deficiencies.

(Sorry this isn't more specific.)

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: FYI: LambdaForm-based java.lang.invoke implementation overview

[John Rose]

Oops, as it turns out, that was for FTPR = For The Public Record.
I neglected to Reply-Single; apologies for the noise, everyone.
— John

On Feb 21, 2015, at 3:54 PM, John Rose john.r.r...@oracle.com wrote:
 
 Here's copy of my Skype comments to you, FTR.  — John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: FYI: LambdaForm-based java.lang.invoke implementation overview

[John Rose]

Here's copy of my Skype comments to you, FTR.  — John

Great slide deck!
Error on slide 12: invokespecial *has* a receiver.  While I'm tweaking, I 
suppose you could say single dispatch (via table) vs. single dispatch (via 
search), but that's a matter of debate.
Slide 13: optional static arguments are probably TMI on the first slide showing 
a BSM; they might confuse people into thinking they are dynamic (normal) 
arguments.  Suggest a slide 13.5 which introduces static and dynamic arguments 
at the same time.
Slide 49: /vmindex from symbolic link/s/link/reference/ (symbolic link is not 
a spec. term)
Slide 54: The javadoc is messy; you quote it correctly.  But, while prim/ref 
conversion is casting (5.5), it is *also* assignment (5.2); the non-error 
actions are the same in the prim/ref case.  (MHs.explicitCastArguments uses 
additional types of casting.)  I think the only reason we mention 5.5 here 
instead of 5.2 is an accident during spec creation; we may have briefly an 
erroneous ref-to-ref cast after the boxing operation.  Probably it's worth a 
footnote, saying something like assignment (5.2)*  ...  *spec says casting 
(5.5) which mandates the same behavior for the prim/ref case.
Slide 64: Informative rendering of an LF to bytecodes.  Wouldn't it be cool to 
have have a slide 64.5 of a representative MH compiled to MH-customized native 
code...  (That opens up the whole big question of share-vs-customized, but 
still would be nice to see object code.)
Slide 78: invokeExact_000_MT should renumber the temps; you apparently inserted 
a2/a3 but did not change t2/t3 to t4/t5.
Slide 85: s/invokeGeneric linkage/invokeExact linkage/  (there is no 
invokeGeneric in the spec, though there was at one point)
I really like the Grand Plan slide!
Slides 86-89:  You have just C frames, which don't give enough context.  
Suggest appending younger frames to the stack trace.
86 - MHN.linkCallSite - MHN.linkCallSiteImpl - CallSite.makeSite - 
MH.invoke (on BSM)
87 - MHN.linkMethodImpl - Invokers.methodHandleInvokeLinkerMethod - 
Invokers.invokeHandleForm (cf. MethodHandles.exactInvoker)
88 - Lookup.linkMethodHandleConstant - MHs.getDirectMethodForConstant - 
MHs.getDirectMethodNoSecurityManager - getDirectMethodCommon (cf. 
Lookup.findVirtual)
89 - MHN.findMethodHandleType - MethodType.makeImpl (cf. 
MethodType.methodType)
Slide 90: Maybe give a use case for each annotation?
@Hidden : MethodHandleImpl.guardWithCatch
@ForceInline : DirectMethodHandle.internalMemberName
@DontInline  : LambdaForm.interpretWithArguments
@Compiled : output of InvokerBytecodeGenerator.generateCustomizedCode

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Invokedynamic and recursive method call

[John Rose]

On Jan 7, 2015, at 8:13 AM, Remi Forax fo...@univ-mlv.fr wrote:
 
 But if fibo is called through an invokedynamic, instead of emitting a direct 
 call to fibo,
 the JIT generates a code that push the method handle on stack and execute it
 like if the metod handle was not constant
 (the method handle is constant because the call at depth=1 is inlined !).

Invocation of non-constant MH's had a performance regression with the LF-based 
implementation.
As of JDK-8069591 they should be no slower and sometimes faster than the old 
implementation.
— John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8063137: Never-taken branches should be pruned when GWT LambdaForms are shared

[John Rose]

On Jan 28, 2015, at 1:00 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 I polished the change a little according to your comments (diff against v03):
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.03-04/hotspot 
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.03-04/hotspot

+1  Glad to see the AndI folds up easily; thanks for the cleanup.___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (XXS): 8071788: CountingWrapper.asType() is broken

[John Rose]

Good.

On Jan 28, 2015, at 9:22 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 The fix is to use adapted MethodHandle to construct LambdaForm.

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8063137: Never-taken branches should be pruned when GWT LambdaForms are shared

[John Rose]

Looking very good, thanks.  Ship it!

Actually, can you insert a comment why the injected counts are not scaled?  (Or 
perhaps they should be??)

Also, we may need a followup bug for the code with this comment:
  // Look for the following shape: AndI (ProfileBoolean) (ConI 1))

Since profileBoolean returns a TypeInt::BOOL, the AndI with (ConI 1) should 
fold up.
So there's some work to do in MulNode, which may allow that special pattern 
match to go away.
But I don't want to divert the present bug by a possibly complex dive into 
fixing AndI::Ideal.

(Generally speaking, pattern matching should assume strong normalization of its 
inputs.  Otherwise you end up duplicating pattern match code in many places, 
inconsistently.  Funny one-off idiom checks like this are evidence of 
incomplete IR normalization.  See http://en.wikipedia.org/wiki/Rewriting for 
some background on terms like normalization and confluence which are 
relevant to C2.)

— John

On Jan 27, 2015, at 8:05 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 Thanks for the feedback, John!
 
 Updated webrev:
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.03/jdk
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.03/hotspot
 
 Changes:
  - renamed MHI::profileBranch to MHI::profileBoolean, and ProfileBranchNode 
 to ProfileBooleanNode;
  - restructured profile layout ([0] = false_cnt, [1] = true_cnt)
  - factored out profile injection in a separate function 
 (has_injected_profile() in parse2.cpp)
  - ProfileBooleanNode stores true/false counts instead of taken/not_taken 
 counts
  - matching from value counts to taken/not_taken happens in 
 has_injected_profile();
  - added BoolTest::ne support
  - sharpened test for AndI case: now it checks AndI (ProfileBoolean) (ConI 1) 
 shape
 
 Best regards,
 Vladimir Ivanov

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8063137: Never-taken branches should be pruned when GWT LambdaForms are shared

[John Rose]

On Jan 26, 2015, at 8:41 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 What do you think about the following version?
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.02 
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.02
 
 As you suggested, I reified MHI::profileBranch on LambdaForm level and 
 removed @LambdaForm.Shared. My main concern about removing @Sharen was that 
 profile pollution can affect the code before profileBranch call (akin to 
 8068915 [1]) and it seems it's the case: Gbemu (at least) is sensitive to 
 that change (there's a 10% difference in peak performance between @Shared and 
 has_injected_profile()).
 
 I can leave @Shared as is for now or remove it and work on the fix to the 
 deoptimization counts pollution. What do you prefer?

Generic advice here:  It's better to leave it out, if in doubt.  If it has a 
real benefit, and we don't have time to make it clean, put it in and file a 
tracking bug to clean it up.

I re-read the change.  It's simpler and more coherent now.

I see one more issue which we should fix now, while we can.  It's the sort of 
thing which is hard to clean up later.

The two fields of the profileBranch array have obscure and inconsistent 
labelings.  It took me some hard thought and the inspection of three files to 
decide what taken and not taken mean in the C2 code that injects the 
profile.  The problem is that, when you look at profileBranch, all you see is 
an integer (boolean) argument and an array, and no clear indication about which 
array element corresponds to which argument value.  It's made worse by the fact 
that taken and not taken are not mentioned at all in the JDK code, which 
instead wires together the branches of selectAlternative without much comment.

My preferred formulation, for making things clearer:  Decouple the idea of 
branching from the idea of profile injection.  Name the intrinsic (yes, one 
more bikeshed color) profileBoolean (or even injectBooleanProfile), and use 
the natural indexing of the array:  0 (Java false) is a[0], and 1 (Java true) 
is a[1].  We might later extend this to work with booleans (more generally, 
small-integer flags), of more than two possible values, klasses, etc.

This line then goes away, and 'result' is used directly as the profile index:
+int idx = result ? 0 : 1;

The ProfileBooleanNode should have an embedded (or simply indirect) array of 
ints which is a simple copy of the profile array, so there's no doubt about 
which count is which.

The parsing of the predicate that contains profileBoolean should probably be 
more robust, at least allowing for 'eq' and 'ne' versions of the test.  (C2 
freely flips comparison senses, in various places.)  The check for Op_AndI must 
be more precise; make sure n-in(2) is a constant of the expected value (1).  
The most robust way to handle it (but try this another time, I think) would be 
to make two temp copies of the predicate, substituting the occurrence of 
ProfileBoolean with '0' and '1', respectively; if they both fold to '0' and '1' 
or '1' and '0', then you take the indicated action.

I suggest putting the new code in Parse::dynamic_branch_prediction, which 
pattern-matches for injected profiles, into its own subroutine.  Maybe:
   bool use_mdo = true;
   if (has_injected_profile(btest, test, taken, not_taken)) {
 use_mdo = false;
   }
   if (use_mdo) { ... old code

I see why you used the opposite order in the existing code:  It mirrors the 
order of the second and third arguments to selectAlternative.  But the JVM 
knows nothing about selectAlternative, so it's just confusing when reading the 
VM code to know which profile array element means what.

— John

P.S.  Long experience with byte-order bugs in HotSpot convinces me that if you 
are not scrupulously clear in your terms, when working with equal and opposite 
configuration pairs, you will have a long bug tail, especially if you have to 
maintain agreement about the configurations through many layers of software.  
This is one of those cases.  The best chance to fix such bugs is not to allow 
them in the first place.  In the case of byte-order, we have first vs. 
second, MSB vs. LSB, and high vs. low parts of values, for values in 
memory and in registers, and all possible misunderstandings about them and 
their relation have probably happened and caused bugs.___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8069591: Customize LambdaForms which are invoked using MH.invoke/invokeExact

[John Rose]

On Jan 22, 2015, at 9:56 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 Remi, John, thanks for review!
 
 Updated webrev:
 http://cr.openjdk.java.net/~vlivanov/8069591/webrev.01/
 
 This time I did additional testing (COMPILE_THRESHOLD  0) and spotted a 
 problem with MethodHandle.copyWith(): a MethodHandle can inherit customized 
 LambdaForm this way. I could have added LambdaForm::uncustomize() call in 
 evey Species_*::copyWith() method, but I decided to add it into MethodHandle 
 constructor. Let me know if you think it's too intrusive.

It's OK to put it there.

Now I'm worried that the new customization logic will defeat code sharing for 
invoked MHs, since uncustomize creates a new LF that is a duplicate of the 
original LF.  That breaks the genetic link for children of the invoked MH, 
doesn't it?  (I like the compileToBytecode call, if it is done on the 
original.)  In fact, that is also a potential problem for the first version of 
your patch, also.

Suggestion:  Have every customized LF contain a direct link to its uncustomized 
original.  Have uncustomize just return that same original, every time.  Then, 
when using LF editor operations to derive new LFs, always have them extract the 
original before making a derivation.

(Alternatively, have the LF editor caches be shared between original LFs and 
all their customized versions.  But that doesn't save all the genetic links.)

 Also, I made DirectMethodHandles a special-case, since I don't see any 
 benefit in customizing them.

The overriding method in DHM should be marked @Override, so that we know all 
the bits fit together.

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8063137: Never-taken branches should be pruned when GWT LambdaForms are shared

[John Rose]

On Jan 20, 2015, at 11:09 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 What I'm mainly poking at here is that 'isGWT' is not informative about
 the intended use of the flag.
 I agree. It was an interim solution. Initially, I planned to introduce 
 customization and guide the logic based on that property. But it's not there 
 yet and I needed something for GWT case. Unfortunately, I missed the case 
 when GWT is edited. In that case, isGWT flag is missed and no annotation is 
 set.
 So, I removed isGWT flag and introduced a check for selectAlternative 
 occurence in LambdaForm shape, as you suggested.

Good.

I think there is a sweeter spot just a little further on.  Make profileBranch 
be an LF intrinsic and expose it like this:
  GWT(p,t,f;S) := let(a=new int[3]) in lambda(*: S) { 
selectAlternative(profileBranch(p.invoke( *), a), t, f).invoke( *); }

Then selectAlternative triggers branchy bytecodes in the IBGen, and 
profileBranch injects profiling in C2.
The presence of profileBranch would then trigger the @Shared annotation, if you 
still need it.

After thinking about it some more, I still believe it would be better to detect 
the use of profileBranch during a C2 compile task, and feed that to the 
too_many_traps logic.  I agree it is much easier to stick the annotation on in 
the IBGen; the problem is that because of a minor phase ordering problem you 
are introducing an annotation which flows from the JDK to the VM.  Here's one 
more suggestion at reducing this coupling…

Note that C-set_trap_count is called when each Parse phase processes a whole 
method.  This means that information about the contents of the nmethod 
accumulates during the parse.  Likewise, add a flag method 
C-{has,set}_injected_profile, and set the flag whenever the parser sees a 
profileBranch intrinsic (with or without a constant profile array; your call).  
Then consult that flag from too_many_traps.  It is true that code which is 
parsed upstream of the very first profileBranch will potentially issue a 
non-trapping fallback, but by definition that code would be unrelated to the 
injected profile, so I don't see a harm in that.  If this approach works, then 
you can remove the annotation altogether, which is clearly preferable.  We 
understand the annotation now, but it has the danger of becoming a maintainer's 
puzzlement.

 
 In 'updateCounters', if the counter overflows, you'll get continuous
 creation of ArithmeticExceptions.  Will that optimize or will it cause a
 permanent slowdown?  Consider a hack like this on the exception path:
counters[idx] = Integer.MAX_VALUE / 2;
 I had an impression that VM optimizes overflows in Math.exact* intrinsics, 
 but it's not the case - it always inserts an uncommon trap. I used the 
 workaround you proposed.

Good.

 
 On the Name Bikeshed:  It looks like @IgnoreProfile (ignore_profile in
 the VM) promises too much ignorance, since it suppresses branch counts
 and traps, but allows type profiles to be consulted.  Maybe something
 positive like @ManyTraps or @SharedMegamorphic?  (It's just a name,
 and this is just a suggestion.)
 What do you think about @LambdaForm.Shared?

That's fine.  Suggest changing the JVM accessor to is_lambda_form_shared, 
because the term shared is already overused in the VM.

Or, to be much more accurate, s/@Shared/@CollectiveProfile/.  Better yet, get 
rid of it, as suggested above.

(I just realized that profile pollution looks logically parallel to the 
http://en.wikipedia.org/wiki/Tragedy_of_the_commons 
http://en.wikipedia.org/wiki/Tragedy_of_the_commons .)

Also, in the comment explaining the annotation:
  s/mostly useless/probably polluted by conflicting behavior from multiple call 
sites/

I very much like the fact that profileBranch is the VM intrinsic, not 
selectAlternative.  A VM intrinsic should be nice and narrow like that.  In 
fact, you can delete selectAlternative from vmSymbols while you are at it.

(We could do profileInteger and profileClass in a similar way, if that turned 
out to be useful.)

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8069591: Customize LambdaForms which are invoked using MH.invoke/invokeExact

[John Rose]

On Jan 21, 2015, at 9:31 AM, Remi Forax fo...@univ-mlv.fr wrote:
 
 in Invokers.java, I think that checkCustomized should take an Object and not 
 a MethodHandle
 exactly like getCallSiteTarget takes an Object and not a CallSite.

The use of erased types (any ref = Object) in the MH runtime is an artifact of 
bootstrapping difficulties, early in the project.  I hope it is not necessary 
any more.  That said, I agree that the pattern should be consistent.

Vladimir, would you please file a tracking bug for this cleanup, to change MH 
library functions to use stronger types instead of Object?

 in MethodHandle.java, customizationCount is declared as a byte and there is 
 no check that
 the CUSTOMIZE_THRESHOLD is not greater than 127.

Yes.  Also, the maybeCustomize method has a race condition that could cause the 
counter to wrap.  It shouldn't use +=1 to increment; it should load the old 
counter value, test it, increment it (in a local), and then store the updated 
value.  That is also one possible place to deal with jumbo CUSTOMIZE_THRESHOLD 
values.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8063137: Never-taken branches should be pruned when GWT LambdaForms are shared

[John Rose]

On Jan 16, 2015, at 9:16 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:
 
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.00/hotspot/ 
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.00/hotspot/
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.00/jdk/ 
 http://cr.openjdk.java.net/~vlivanov/8063137/webrev.00/jdk/
 https://bugs.openjdk.java.net/browse/JDK-8063137 
 https://bugs.openjdk.java.net/browse/JDK-8063137
 ...
 PS: as a summary, my experiments show that fixes for 8063137  8068915 [2] 
 almost completely recovers peak performance after LambdaForm sharing [3]. 
 There's one more problem left (non-inlined MethodHandle invocations are more 
 expensive when LFs are shared), but it's a story for another day.

This performance bump is excellent news.  LFs are supposed to express 
emergently common behaviors, like hidden classes.  We are much closer to that 
goal now.

I'm glad to see that the library-assisted profiling turns out to be relatively 
clean.

In effect this restores the pre-LF CountingMethodHandle logic from 2011, which 
was so beneficial in JDK 7:
  
http://hg.openjdk.java.net/jdk7u/jdk7u/jdk/file/02de5cdbef21/src/share/classes/java/lang/invoke/CountingMethodHandle.java

I have some suggestions to make this version a little cleaner; see below.

Starting with the JDK changes:

In LambdaForm.java, I'm feeling flag pressure from all the little boolean 
fields and constructor parameters.

(Is it time to put in a bit-encoded field private byte LambdaForm.flags, or 
do we wait for another boolean to come along?  But see next questions, which 
are more important.)

What happens when a GWT LF gets inlined into a larger LF?  Then there might be 
two or more selectAlternative calls.
Will this confuse anything or will it Just Work?  The combined LF will get 
profiled as usual, and the selectAlternative calls will also collect profile 
(or not?).

This leads to another question:  Why have a boolean 'isGWT' at all?  Why not 
just check for one or more occurrence of selectAlternative, and declare that 
those guys override (some of) the profiling.  Something like:

  -+ if (PROFILE_GWT  lambdaForm.isGWT) 
  ++ if (PROFILE_GWT  lambdaForm.containsFunction(NF_selectAlternative))
(...where LF.containsFunction(NamedFunction) is a variation of 
LF.contains(Name).)

I suppose the answer may be that you want to inline GWTs (if ever) into 
customized code where the JVM profiling should get maximum benefit.  In that 
case case you might want to set the boolean to false to distinguish 
immature GWT combinators from customized ones.

If that's the case, perhaps the real boolean flag you want is not 'isGWT' but 
'sharedProfile' or 'immature' or some such, or (inverting) 'customized'.  (I 
like the feel of a 'customized' flag.)  Then @IgnoreProfile would get attached 
to a LF that (a ) contains selectAlternative and (b ) is marked as 
non-customized/immature/shared.  You might also want to adjust the call to 
'profileBranch' based on whether the containing LF was shared or customized.

What I'm mainly poking at here is that 'isGWT' is not informative about the 
intended use of the flag.

In 'updateCounters', if the counter overflows, you'll get continuous creation 
of ArithmeticExceptions.  Will that optimize or will it cause a permanent 
slowdown?  Consider a hack like this on the exception path:
   counters[idx] = Integer.MAX_VALUE / 2;

On the Name Bikeshed:  It looks like @IgnoreProfile (ignore_profile in the VM) 
promises too much ignorance, since it suppresses branch counts and traps, but 
allows type profiles to be consulted.  Maybe something positive like 
@ManyTraps or @SharedMegamorphic?  (It's just a name, and this is just a 
suggestion.)

Going to the JVM:

In library_call.cpp, I think you should change the assert to a guard:
  -+ assert(aobj-length() == 2, );
  ++  aobj-length() == 2) {

In Parse::dynamic_branch_prediction, the mere presence of the Opaque4 node is 
enough to trigger replacement of profiling.  I think there should *not* be a 
test of method()-ignore_profile().  That should provide better integration 
between the two sources of profile data to JVM profiling?

Also, I think the name 'Opaque4Node' is way too… opaque.  Suggest 
'ProfileBranchNode', since that's exactly what it does.

Suggest changing the log element profile_branch to observe 
source='profileBranch', to make a better hint as to the source of the info.

— John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9, 8u40] RFR (XXS): 8066746: MHs.explicitCastArguments does incorrect type checks for VarargsCollector

[John Rose]

Reviewed.  — John

 On Dec 8, 2014, at 3:47 PM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
 wrote:
 
 http://cr.openjdk.java.net/~vlivanov/8066746/webrev.00/
 https://bugs.openjdk.java.net/browse/JDK-8066746
 
 Recent changes (8057656 [1]) broke MHs.explicitCastArguments for 
 VarargsCollector case. It introduced an equivalence check between 
 MHs.explicitCastArguments and MethodHandle.asType() which doesn't work for 
 VarargsCollector case as expected.
 
 VarargsCollector has special asType() implementation, which supports 
 collecting any number of trailing positional arguments into an array 
 argument. It doesn't play well with MHs.explicitCastArguments, because the 
 latter is meant to be a pairwise argument and return type conversion.
 
 The fix is to ensure that adapted method handle has fixed arity.
 
 Testing: regression test, jck (api/java_lang/invoke), jdk/java/lang/invoke
 
 Thanks!
 
 Best regards,
 Vladimir Ivanov
 
 [1] https://bugs.openjdk.java.net/browse/JDK-8057656

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9, 8u40] RFR (M): 8057020: LambdaForm caches should support eviction

[John Rose]

Reviewed.

I sympathize with Paul's gnarly comment.

Nice bit of stream-ology (rheology) in the test code.

Regarding:

 On Dec 2, 2014, at 8:20 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
 wrote:
 
 In src/java.base/share/classes/java/lang/invoke/LambdaFormEditor.java
 
  366 lambdaForm.transformCache = c = ta;
 
 Do you need to set c? It's a local variable and by this point the method 
 should return rather than loop.
 I did it mostly as a cleanup. Now I think that it doesn't help much. Removed 
 (+ similar change in another place).

The c =  bit can be viewed as a bug-stopper.  It prevents a later expression 
(if introduced in a future change) from accidentally using 'c' and getting an 
out-of-date value.  A better bug-stopper would be a declaration that c is dead 
as of this point, and cannot be used any more, but the language does not 
support that.  I don't mind seeing the assignment deleted.

— John

 On Dec 1, 2014, at 8:58 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
 wrote:
 
 http://cr.openjdk.java.net/~vlivanov/8057020/webrev.00/
 https://bugs.openjdk.java.net/browse/JDK-8057020

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] [8u40] RFR (M): 8059877: GWT branch frequencies pollution due to LF sharing

[John Rose]

Good, I'm happy.  Reviewed.  — John

On Oct 28, 2014, at 12:04 PM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 John, thanks for the feedback!
 See my answers inline.

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (S): 8058293: Bit set computation in MHs.findFirstDupOrDrop/findFirstDrop is broken

[John Rose]

On Sep 15, 2014, at 9:48 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 https://bugs.openjdk.java.net/browse/JDK-8058293
 http://cr.openjdk.java.net/~vlivanov/8058293/webrev.00
 
 Bit set computation in MHs.findFirstDupOrDrop/findFirstDrop is incorrect due 
 to widening primitive conversions (int - long) taking place. The fix is to 
 explicitly use long where needed.
 
 Also, fixed a case when reorder represents a permutation (zeroPos == 
 newArity).
 
 Testing: jck (api/java_lang/invoke), jdk/java/lang/invoke, 
 jdk/java/util/streams w/ -ea -esa and COMPILE_THRESHOLD={0,30}

Wow, that's foul.  Thanks for patching it up.  Next time I'll start with 
BIT_LIMIT = 31.

Reviewed.

FTR, it's worth exercising the code with BIT_LIMIT = 0, just to stress the slow 
path.

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (S): 8058291: Missing some checks during parameter validation

[John Rose]

Reviewed.

Nice tuning of the asserts.

— John

On Sep 15, 2014, at 9:03 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 http://cr.openjdk.java.net/~vlivanov/8058291/webrev.00/
 https://bugs.openjdk.java.net/browse/JDK-8058291
 
 8057656  8050166 changes accidentally removed some checks during argument 
 validation in MHs.dropArguments() and MHs.explicitCastArguments(). The fix is 
 to restore them.
 
 Testing: jck (api/java_lang/invoke), jdk/java/lang/invoke, 
 jdk/java/util/streams w/ -ea -esa and COMPILE_THRESHOLD={0,30}
 
 Thanks!
 
 Best regards,
 Vladimir Ivanov

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Unloading LambdaForm bytecode

[John Rose]

On Sep 9, 2014, at 12:11 AM, Martin Traverso mtrave...@gmail.com wrote:

 Hi John,
 
 Thanks for the detailed explanation.
 
 I ran a few additional experiments after I wrote a simpler program to try to 
 reproduce the issue. I don't see the permgen leak, so something else must be 
 causing it in Presto. I do see the Loaded... messages without a 
 corresponding Unloaded..., so your hypothesis about that happening for 
 anonymous classes seems likely. Interestingly, it has another side-effect: 
 the Loaded classes counter in VisualVM and in the ClassLoadingMXBean don't 
 seem to work correctly under this scenario.
 
 It also reproduces the issue I mentioned regarding seemingly duplicate LFs. 
 The code is here if you want to take a quick look: 
 https://github.com/martint/lftest. The weird behavior goes away if I change 
 the indy callsite to return Object instead of String

OK, got it.  The path through makeReferenceIdentity is not cached in JDK 7 or 
8, for reference types other than Object.

The fix for this is pending:
  https://bugs.openjdk.java.net/browse/JDK-8050884
  http://cr.openjdk.java.net/~vlivanov/lfc/2014-09-03/12.8050884.identity

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Unloading LambdaForm bytecode

[John Rose]

Hi Martin.

A few LFs are cached in 8u20, and many more will be in 8u40; those will not be 
unloaded.

A non-cached LF can be viewed as customized, to the exact MH that caused its 
creation.

It should be the case that a LF that is customized to LFs will be unloaded 
(with its bytecodes) as soon as the last reference to it goes away.

An indy instruction bound to a LF (via a call site and MH) will typically be 
the only reference to the LF, so if the class containing the indy goes away, 
the LF should go also.

LF bytecodes are loaded with Unsafe.defineAnonymousClass, which means that the 
class is not registered in any class loader or dictionary, but should get 
unloaded as soon as everybody drops the java.lang.Class for the LF bytecodes.  
A MH with anonymous bytecodes indirectly refers to the java.lang.Class.

All that's to say that you should not have a storage leak if your classes are 
getting unloaded.  (And if you don't cache MHs, etc.)  If you have a leak, 
there's either an inadvertent reference keeping the MH stuff alive, or else 
there is a bug in the defineAnonymousClass mechanism.

(I can't remember offhand whether anonymous classes report unload events.  If 
the reporting is coupled to the system dictionary, then you won't get a report, 
since ACs don't affect the system dictionary.)

But.  The example you give below comes from MethodHandles.constant (a K 
combinator instance).  Those are cached, I think.  They also should not be 
numerous, since their types are erased to basic-types.

The string inside the dummy method is intended to be human-readable 
documentation of what the LF does.

In this example, the class Species_L is a bound method handle containing one 
reference field.  Its method argL0 returns that unique value.  (If it also had 
a second double field, the method would be Species_LD.argD1, etc.)  The 
receiver argument of this LF (a0) is a one-field Species_L which carries the 
constant K value in its argL0 field; when invoked, it returns that value.

If you have two LF classes with the same string, there might be bug.

— John

On Sep 8, 2014, at 4:54 PM, Martin Traverso mtrave...@gmail.com wrote:

 Do the generated bytecodes for LambdaForms ever get unloaded? In Presto (a 
 distributed SQL engine) we generate bytecode on the fly for each query and 
 use invokedynamic in a number of places as a way to link constants. We 
 recently ran into a permgen leak (we run 7u45). Looking at the output of 
 -verbose:class, I noticed a bunch of lines like these:
 
 [Loaded java.lang.invoke.LambdaForm$MH227/995942612 from 
 java.lang.invoke.LambdaForm]
 [Loaded java.lang.invoke.LambdaForm$MH228/489675590 from 
 java.lang.invoke.LambdaForm]
 [Loaded java.lang.invoke.LambdaForm$MH229/2095286826 from 
 java.lang.invoke.LambdaForm]
 [Loaded java.lang.invoke.LambdaForm$MH230/538456879 from 
 java.lang.invoke.LambdaForm]
 [Loaded java.lang.invoke.LambdaForm$MH231/1550961803 from 
 java.lang.invoke.LambdaForm]
 ...
 
 We load our generated bytecode in standalone classloaders and it eventually 
 gets unloaded. I never see an unload message for the LambdaForm classes, 
 though.
 
 This is what’s inside one of those classes. They all look similar:
 
  javap -c 'LambdaForm$MH499.class'
 
 Compiled from LambdaForm$MH499
 final class java.lang.invoke.LambdaForm$MH499 extends 
 java.lang.invoke.LambdaForm {
   static java.lang.Object identity(java.lang.Object);
 Code:
0: aload_0   
1: checkcast #12 // class 
 java/lang/invoke/BoundMethodHandle$Species_L
4: getfield  #16 // Field 
 java/lang/invoke/BoundMethodHandle$Species_L.argL0:Ljava/lang/Object;
7: astore_1  
8: ldc   #18 // String 
 CONSTANT_PLACEHOLDER_0 MethodHandle(Object)Object
   10: checkcast #20 // class 
 java/lang/invoke/MethodHandle
   13: aload_1   
   14: invokevirtual #23 // Method 
 java/lang/invoke/MethodHandle.invokeBasic:(Ljava/lang/Object;)Ljava/lang/Object;
   17: areturn   
 
   static void dummy();
 Code:
0: ldc   #27 // String 
 identity=Lambda(a0:L)={\nt1:L=Species_L.argL0(a0:L);\n
 t2:L=ValueConversions.identity(t1:L);t2:L}
2: pop   
3: return
 }
 ___
 mlvm-dev mailing list
 mlvm-dev@openjdk.java.net
 http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (S) 8050173: Generalize BMH.copyWith API to all method handles

[John Rose]

On Jul 16, 2014, at 1:50 AM, Paul Sandoz paul.san...@oracle.com wrote:

 Why not make the second parameter be DirectMethodHandle mh ?

Good suggestion; thanks.  Makes the restrictReceiver logic less magic.  — John

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (L): 8057042: LambdaFormEditor: derive new LFs from a base LF

[John Rose]

On Sep 3, 2014, at 10:35 AM, Mark Roos mr...@roos.com wrote:

 From Morris 
 
 All that assert laden code is nice to see. 
 
 I just finished watching a video from Doug Lea where he mentioned that having 
 asserts can 
 inhibit inlining due to the additional byte codes.  So he sadly does not use 
 them due to 
 performance issues. 
 
 Does anyone have any insights on this? 

Yep.

http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-August/028450.html
https://bugs.openjdk.java.net/browse/JDK-6316156

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (L): 8057042: LambdaFormEditor: derive new LFs from a base LF

[John Rose]

On Sep 3, 2014, at 8:46 AM, Morris Meyer morris.me...@oracle.com wrote:

 src/share/classes/java/lang/invoke/BoundMethodHandle.java
 
  Could we keep /* */ comment style consistent throughout?
 
   @Override // there is a default binder in the super class, for 'L' 
 types only
  /*non-public*/

Most comments are // style, or else javadoc.

The locution /*non-public*/ is a stand-in for a modifier keyword that 
emphasizes that the API is not to be made public.  There have been proposals to 
use 'default' or 'package' for this purpose, but they may never be adopted, and 
until that time, we need something that can do the job of a modifier, at least 
for documentation.

 src/share/classes/java/lang/invoke/LambdaForm.java
 
 ! Object transformCache;  // managed by LambdaFormEditor
 
 This is sort of odd.  Could we have a TransformCache class that has a 
 Transform[] part and a ConcurrentHashMapTransform, Transform part?

That's maybe worth a comment.  The first-class way to do this would be a CHM, 
but it would add an order of magnitude overhead to something that is (at least 
currently) a footprint problem, and also (probably) a startup bottleneck.  The 
given design is optimized for a cache arity of close to 1, at the expense of 
clear static typing (but the type weakness is localized) and possible 
over-retention.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: DMH to fields, casts and type profiling was Re: [9] RFR (M): 8037209: Improvements and cleanups to bytecode assembly for lambda forms

[John Rose]

On Aug 28, 2014, at 7:38 AM, Paul Sandoz paul.san...@oracle.com wrote:

 On Jul 8, 2014, at 9:09 PM, John Rose john.r.r...@oracle.com wrote:
 
 Regarding the extra cast in accessor logic that Paul picked up on:  That may 
 be a left-over from obsolete versions of the code, or it may cover for some 
 corner cases, or it could possibly be a re-assurance to the JIT.
 
 
 I had some enlightening discussions with Roland on this.
 
 It seems quite tricky to solve in general the removal of the null check due 
 to the aggressive nature in which the null branch is reduce to a trap, but 
 IIUC may be possible to turn Class.cast into an intrinsic to handle the 
 specific case (although that seems costly).
 
 I was labouring under the misapprehension that an explicit Class.cast was a 
 profiling point but now i realize it's only certain byte codes (like 
 checkcast/invokehandle). Nothing specific to the DHM access logic showed up 
 with regards to type profiling when analysing the MethodData output from some 
 simple examples [*]. Therefore i presume it's more likely to be the first or 
 third reason you state.
 
 So i propose to proceed with the experiment with a patch to replace the casts 
 with asserts in the accessor logic and run that through the usual tests.
 
 Paul.
 
 
 [*] Also i have so far failed to concoct a simple example for VarHandles 
 where i can trigger profile pollution and failed inlining

Here's something to try first:  Force a profile point before Class.cast, even 
without Roland's enhancements.
You should be able to type-profile x by inserting push x; checkcast 
java/lang/Object; pop.
See last line of https://wiki.openjdk.java.net/display/HotSpot/MethodData

— John
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: The Great Startup Problem

[John Rose]

On Aug 22, 2014, at 1:08 PM, Charles Oliver Nutter head...@headius.com wrote:

 Marcus coaxed me into making a post about our indy issues. Our indy
 issues mostly surround startup and warmup time, so I'm making this a
 general post about startup and warmup.

This is a vigorous and interesting discussion.  I will make some piecemeal 
replies to
specific points, but first, as a HotSpot team member, I'll comment on the 
startup
problem and then set out our position and vision for indy and dynamic languages.

Achilles had two heels, and so does Java:  Startup and warmup.  Many teams of 
many
people have worked on these issues for years, with hard-won progress.  Compared
with C programs (say, /bin/awk), the JVM takes a long time to get going.

Fundamentally this comes from the decision to package programs as bytecodes
(JARs, etc.) rather than machine instructions plus mappable data (dylibs, etc.).
As everyone on this list knows and appreciates, Java bytecodes are a portable,
stable, and compact intermediate representation for both code and data 
structure.
You can do an amazing variety of great things with them, and there is no 
credible
proposal (IMO) to replace them wholesale with some other representation.

As an inherent property, bytecodes cannot be executed directly, requiring
additional machinery to execute: an interpreter, JIT compiler, and/or AOT 
engine.
Making this machinery look small is an ongoing challenge for us JVM 
implementors.
I say ongoing (not just historic) because software complexity scales up 
with time.

Our basic move is organizing cold stuff differently from hot stuff.
The C2 JIT spends lots of effort on the hot paths in hot methods.
On the cold side, Java's class data sharing feature tries to organize large
amounts of start-up bytecode in a form that can be quickly loaded and discarded.
The most useful distinctions between hot and cold data and code usually
require some sort of online measurement, which is one reason Java is
competitive with C++, which generally lacks behavior-dependent optimizations.

Building on these basic considerations, an ideal Java implementation would
quickly get through the start-up process quickly by executing cold code and/or
loading pre-configured data structures, with the result that data which must be
configured by every JVM on startup, or by every invocation of a given 
application,
is quickly assembled.  This would be done in some cases by loading a
pre-composed bitwise data image, or perhaps more efficiently by decompressing
an operationally encoded description of the initial data image by executing
some sort of code.  In fact, that is what Java interpreters are pretty good at,
when augmented by a pre-composed class data archive that maps in
a pre-parsed version of classes from rt.jar.  (Sometimes this image is
found in a file called classes.jsa.)

But if more of the initial state of a Java application could be demand-paged 
from
an image file (like the initial data of C programs) there might be less latency,
since there would certainly be fewer order constraints between the different
loading events (at the page and cache line level), allowing prefetch machinery
to hide latency.  This is an important area of growth for Java.

Second, an ideal Java implementation would quickly discover the particular
on-line characteristics of the application under execution, and produce
tailor-made machine code for that execution, with the result that the
application's critical inner loops would execute at full performance,
almost immediately.  This would be done in some cases by capturing
profile information from early phases of the application and applying them
to the creation of customized code.  In fact, the profiling interpreter and
tiered C1 JIT are pretty good at gathering such information, and
feeding it to the C2 JIT.

What's missing?  Probably a certain amount of prediction of behavior
based on previous runs of the application, combined with a way of
loading data and executing code that benefits from that prediction,
even before the application has gotten that far.  To me that sounds
like a static compiler of some sort, though an open-world kind that
allows the compiler's model of the world to change during execution.

Maybe another missing bit is a slightly more static initialization model
for Java, under which the assembly of initial data can be predicted
off-line with greater accuracy.  For starters, we should have array
initializers that are more distinct from random bytecode execution,
plus a little more discipline about the effects of clinit methods,
separating those effects into yes we can statically model this
vs. do you really want to do this crazy thing?.

One wrong answer would be to do more JIT stuff, earlier and oftener.
But JIT optimizations are inherently throughput-oriented; they do
nothing do reduce start-up or spin-up overheads; they often worsen
those overheads.  (Example:  Iteration range splitting which makes
the middle of a loop run 

Re: Defining anonymous classes

[John Rose]

On Aug 15, 2014, at 5:03 AM, Florian Weimer fwei...@redhat.com wrote:

 On 08/14/2014 10:15 PM, Mark Roos wrote:
 Look into sun.Misc.Unsafe
 
 [and at defineAnonymousClass(Class, byte[], Object[])]
 
 Thanks.  Could we turn this into a supported API, with a suitable security 
 manager check?

Hi Florian and Mark.

Here's the story about that.

Unsafe.defineAnonymousClass is used inside the JDK for loading small classes of 
indeterminate lifetime.  Many of them contain single static methods, which 
makes it, in effect, an anonymous method loader.

Because the small bits of code often need to be injected into a pre-existing 
class, a host class can be designated, which means the anonymous class has 
access to everything the host class can access, including host class privates.

Because the small bits of dynamically generated code can depend on (meta-)data 
values that are dynamically computed and/or are anonymous, the class's constant 
pool can be patched to refer to live values that do not have symbolic names.

So it allows some low-level tinkering with the JVM internals.  Like any 
non-standard internal API, if you use it, you are tightly coupled to the JVM 
implementation, and you are on your own if something goes wrong.  We fix bugs 
in this internal feature only if it fixes a bug in a standard J2SE feature that 
somehow depends on it.

I expect that defineAC and its use cases will help inform future consideration 
of next-generation class loading APIs.  I don't expect it to become a standard 
feature on its own.  For one thing, the design is not quite right:  It should 
really be a method loader, but we don't have a separate format for methods.  
The constant pool patching part of the API, while necessary, requires knowledge 
of constant pool layout, which feels wrong too.

Specifically, in the context of a (not yet existing) classfile 2.0 effort, 
defineAC should help us think about the format of a stand-alone method, about 
the interaction of packaging and access rights, and about the division of labor 
between constant pool and bytecode.  Since these are (to my mind) open-ended 
design questions, I don't think we will promote the existing ad hoc API to a 
public one, unless there is a compelling short term reason to do so.

The most obvious security interaction is with the host-class feature; as it 
stands the API is very unsafe because you could use it to inject code into any 
system class.

Just allowing the existing API with a security manager check is too 
coarse-grained to make the feature useful, except to highly trusted code.

If the host-class token were changed to a MethodHandles.Lookup object, we could 
restrict the host-class to be one which the user already had appropriate access 
to.  Seems simple, but of course the rest of the project is complicated:   API 
design, spec completion, security analysis, positive and negative test 
creation, code development, quality assurance—all these would be expensive, and 
(again) most easily justified in the context of a larger refresh of our 
classfile format.

Do you have a use case in mind that could be expressed as a more tightly 
focused API?  It might be easier to standardize on a very specific API that 
used dAC.

Or, most or all of dAC could be simulated using regular class loading, into a 
single-use ClassLoader object.  The nominal bytecodes would have to be 
rewritten to use invokedynamic to manage the linking, at least to host-class 
names.  But given that ASM is inside the JDK, the tools are all available.  
(Remi could do most of it in an afternoon. :-) )  Given such a simulation, the 
internal dAC mechanism could be used as an optimization, when available, but 
there would be a standard (complex) semantics derived from ordinary classes and 
indy.

— John

 I'm surprised there aren't any callers of this method in Fedora. Anonymous 
 classes look very useful for run-time code generation.
 
 -- 
 Florian Weimer / Red Hat Product Security
 ___
 mlvm-dev mailing list
 mlvm-dev@openjdk.java.net
 http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8037209: Improvements and cleanups to bytecode assembly for lambda forms

[John Rose]

On Jul 9, 2014, at 3:14 AM, Paul Sandoz paul.san...@oracle.com wrote:

 
 I quickly verified the fold up does as you expect. Also, if i do the 
 following the null check goes away:
 
 ...
 
   void testLoop() {
   for (int i = 0; i  100; i++) {
   testLoopOne(a);
   testLoopOne(snull);
   }
   }

Good observation.  So rather than missing a null-case fold-up (good it's not!), 
the optimizer is speculating not-nullness (based on profile) and adding a test. 
 (Either the test is being used for a downstream optimization, or else the test 
is not being detected as useless and removed—which would be bad!.)

 I am probably obsessing too much over some micro/nano-benchmarks,

(Hi, I'm John and I'm a micro-obsess-aholic.)

 but i am wondering if this could cause some unwanted de-opt/recompilation 
 effects when all is good with no null values then suddenly a null triggers 
 de-optimization.

Besides jumping after the micro-benchmark and chewing on the optimizer until 
the code shrinks, there are two other things we can do:

1. Mentally file the issue and watch real benchmarks for evidence of the 
problem.  (This works pretty well, provided enough time and focus, and provided 
enough people have some consciousness of the optimizer's workings.)

2. Create a self-test and check it into the test base.  It could be either a 
unit test of assertion.  In this case, I don't see an easy way to do it, but 
creating clever permanent tests almost always pays off much better than 
cleverly pounding on the micro-benchmark of the moment.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8037209: Improvements and cleanups to bytecode assembly for lambda forms

[John Rose]

Regarding the extra cast in accessor logic that Paul picked up on:  That may be 
a left-over from obsolete versions of the code, or it may cover for some corner 
cases, or it could possibly be a re-assurance to the JIT.

Generally speaking, we lean heavily on MH types to guarantee a priori 
correctness of argument types.  Paul is right that the stored value type is 
already validated and (except for corner cases we may be neglecting) does not 
need re-validation via a cast.

It might be an interesting experiment to replace the cast with an assert.

Sometimes corner cases bite you.  For example, an array store check is 
necessary, if the type is an interface, because interfaces are weakly checked 
all the way up to aastore or invokeinterface.

Sometimes the JIT cannot see the type assurances implicit in a MH type, and so 
(when choosing internal MH code shapes) we must choose between handing the JIT 
code that is not locally verifiable, or adding reassurance casts to repair 
the local verifiability of the code.  If the JIT thinks it sees badly-typed 
code, it might bail out.Note that locality of verifiability is a fluid 
concept, depending sometime on vagaries of inlining decisions.  This is the 
reason for some awkward looking belt and suspenders MH internals, such as the 
free use of casts in LF bytecode rendering.

Usually, redundant type verifications (including casts and signatures of 
incoming arguments) are eliminated, but they can cause (as noted) an extra null 
check.  In theory, that should fold up also, if the null value is replaced by 
another null, as (p == null ? null : identityFunction(p)).

— John

On Jul 8, 2014, at 3:09 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 I'd like to revive this review thread.
 
 Updated version:
 http://cr.openjdk.java.net/~vlivanov/8037209/webrev.04/
 
 Paul, I'd like to address your case (if possible) separately. Right now, I 
 don't see any way to avoid null check you see with your tests.


On Apr 1, 2014, at 8:29 AM, Paul Sandoz paul.san...@oracle.com wrote:

 On Apr 1, 2014, at 4:10 PM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
 wrote:
 
 Paul,
 
 Unfortunately, additional profiling doesn't work for Accessor.checkCast 
 case. The problem is Accessor.checkCast is called from multiple places, so 
 type profile is very likely to be polluted. And it kills the benefits.
 
 
 So is there any point in doing such a cast in this case? 
 
 The type performing the cast is the field type declared as a parameter in the 
 MethodType of the MethodHandle and also held by the Accessor. 
 
 IIUC the Invokers.checkExactType should ensure no unsavoury instances of 
 the wrong type gets through? (the holder instance is only checked for null, 
 via checkBase).


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (XS): 8046903: VM anonymous class members can't be statically invocable

[John Rose]

Reviewed.  — John

On Jun 16, 2014, at 9:50 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 http://cr.openjdk.java.net/~vlivanov/8046903/webrev.00/
 https://bugs.openjdk.java.net/browse/JDK-8046903
 
 j.l.i.InvokerBytecodeGenerator::isStaticallyInvocable doesn't distinguish 
 between VM anonymous classes and ordinary classes. In some very specific 
 circumstances (VM anonymous class declared in one of predefined core 
 packages), it can consider a member of VM anonymous class as statically 
 invocable and symbolically reference it from generated bytecode. It's wrong 
 because such class can't be looked up by name and the attempt to run such 
 code ends up with NoClassDefFoundError.
 
 The fix is to disable static invocation for members of VM anonymous classes.
 
 Testing: regression test.
 
 Thanks!
 
 Best regards,
 Vladimir Ivanov

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (S): 8032400: JSR292: invokeSpecial: InternalError attempting to lookup a method

[John Rose]

Reviewed.

This is not a requirement, but I would prefer (in general) to see less test 
logic in ASM-generated bytecode and more in Java.  I am guessing that the 
invokeExact call could have been replaced by a simple weakly-typed invoke call 
in the framing code, and likewise with most of the other invokes (methodType, 
findSpecial) which are not caller-sensitive.

— John

On Jun 5, 2014, at 3:25 PM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 Thanks for review, Paul.
 
 
 Looks ok.
 
 The behaviour of MethodHandles.Lookup.findSpecial got me confused for a 
 while :-)
 
 Minor point: is it also worth exposing a T3.lookup() method and on the 
 returned Lookup calling findSpecial(T1.class, m, 
 MethodType.methodType(int.class), T3.class).invokeExact() to ensure the 
 programmatic path also works?
 Good point. Updated webrev:
 http://cr.openjdk.java.net/~vlivanov/8032400/webrev.01/
 
 Best regards,
 Vladimir Ivanov
 ___
 mlvm-dev mailing list
 mlvm-dev@openjdk.java.net
 http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: RFR (L) 8037210: Get rid of char-based descriptions 'J' of basic types

[John Rose]

On Apr 3, 2014, at 6:33 PM, Christian Thalinger 
christian.thalin...@oracle.com wrote:

 Of course they are popular because these are the type names.  There is no 
 type L; it’s an object.  I don’t understand why we have to use different 
 names just because they are used in other namespaces.  This is not a C define.

They stand for JVM signatures as well as basic types.  The letters are 
signature letters.  Can we move on from this?

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: RFR (L) 8037210: Get rid of char-based descriptions 'J' of basic types

[John Rose]

On Mar 21, 2014, at 8:49 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 Thanks for the feedback.
 
 What do you think about the following:
 http://cr.openjdk.java.net/~vlivanov/8037210/webrev.01/

That looks nice.  Strong typing; who woulda' thunk it.  :-)

The uses of .ordinal() are the extra cost relative to using just small 
integers.  They seem totally reasonable in the code.

I suggest either wrapping ordinal as something like id or else changing 
temp names like id to ord, to reinforce the use of a common name.

Don't make the enum class public.  And especially don't make the mutable arrays 
public:

+public static final BasicType[] ALL_TYPES = { L_TYPE, I_TYPE, J_TYPE, 
F_TYPE, D_TYPE, V_TYPE };
+public static final BasicType[] ARG_TYPES = { L_TYPE, I_TYPE, J_TYPE, 
F_TYPE, D_TYPE };

— John

P.S.  That would only be safe if we had (what we don't yet) a notion of frozen 
arrays like:

+public static final BasicType final[] ALL_TYPES = { L_TYPE, I_TYPE, 
J_TYPE, F_TYPE, D_TYPE, V_TYPE };
+public static final BasicType final[] ARG_TYPES = { L_TYPE, I_TYPE, 
J_TYPE, F_TYPE, D_TYPE };

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: RFR (L) 8037210: Get rid of char-based descriptions 'J' of basic types

[John Rose]

On Mar 18, 2014, at 1:36 PM, Christian Thalinger 
christian.thalin...@oracle.com wrote:

 Why are we not using an Enum instead of an untyped byte?

Byte is moderately typed, in the sense (which I rely on during development) 
that you can't assign an int or char to a byte w/o a cast.
That's why it is not just a plain int.

But those values (L_TYPE etc.) are used a lot as numbers, and specifically as 
low-level array indexes, and also comparisons (x  V_TYPE).

To turn them into enums, we'd have to add lots of calls to '.ordinal()' to turn 
them right back to numbers.  That dilutes (completely IMO) the value they have 
as enums to raise the level of the code.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (S): 8036117: MethodHandles.catchException doesn't handle VarargsCollector right (8034120 failed)

[John Rose]

Reviewed. 

– John

 On Mar 10, 2014, at 3:21 PM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
 wrote:
 
 Chris, thanks for the review.
 
 John suggested an elegant way to fix the problem - use asFixedArity.
 
 Updated fix:
 http://cr.openjdk.java.net/~vlivanov/8036117/webrev.01/
 
 Best regards,
 Vladimir Ivanov
 
 On 3/8/14 4:51 AM, Christian Thalinger wrote:
 Seems good to me.  I’d like to have another name for this method:
 
 + private static Object invokeCustom(MethodHandle target, Object... 
 args) throws Throwable {
 
 On Mar 4, 2014, at 12:00 PM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
 wrote:
 
 http://cr.openjdk.java.net/~vlivanov/8036117/webrev.00/
 https://bugs.openjdk.java.net/browse/JDK-8036117
 84 lines changed: 74 ins; 3 del; 7 mod
 
 I have to revert a cleanup I did for 8027827.
 MethodHandle.invokeWithArguments (and generic invocation) has unpleasant
 peculiarity in behavior when used with VarargsCollector. So,
 unfortunately, invokeWithArguments is not an option there.
 
 Looking at the API (excerpts from javadoc [1] [2]), the following
 condition doesn't hold in that case:
   trailing parameter type of the caller is a reference type identical
 to or assignable to the trailing parameter type of the adapter.
 
 Example:
   target.invokeWithArguments((Object[])args)
   =
   target.invoke((Object)o1,(Object)o2,(Object)o3)
   =/
   target.invokeExact((Object)o1, (Object)o2, (Object[])o3)
 
 because Object !: Object[].
 
 The fix is to skip unnecessary conversion when invoking a method handle
 and just do a pairwise type conversion.
 
 Testing: failing test case, nashorn w/ experimental features (octane)
 
 Thanks!
 
 Best regards,
 Vladimir Ivanov
 
 [1] MethodHandle.invokeWithArguments
 Performs a variable arity invocation, ..., as if via an inexact invoke
 from a call site which mentions only the type Object, and whose arity is
 the length of the argument array.
 
 [2] MethodHandle.asVarargsCollector
 When called with plain, inexact invoke, if the caller type is the same
 as the adapter, the adapter invokes the target as with invokeExact.
 (This is the normal behavior for invoke when types match.)
 
 Otherwise, if the caller and adapter arity are the same, and the
 trailing parameter type of the caller is a reference type identical to
 or assignable to the trailing parameter type of the adapter, the
 arguments and return values are converted pairwise, as if by asType on a
 fixed arity method handle.
 
 Otherwise, the arities differ, or the adapter's trailing parameter type
 is not assignable from the corresponding caller type. In this case, the
 adapter replaces all trailing arguments from the original trailing
 argument position onward, by a new array of type arrayType, whose
 elements comprise (in order) the replaced arguments.
 ___
 mlvm-dev mailing list
 mlvm-dev@openjdk.java.net
 http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
 
 ___
 mlvm-dev mailing list
 mlvm-dev@openjdk.java.net
 http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
 ___
 mlvm-dev mailing list
 mlvm-dev@openjdk.java.net
 http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Signature of MethodHandleInfo.reflectAs is not specific enough

[John Rose]

On Feb 25, 2014, at 3:13 AM, Ali Ebrahimi ali.ebrahimi1...@gmail.com wrote:

 I know, this is too late, but I want to share my suggestion:
 
 public T extends AccessibleObjectAnnotatedElement T reflectAs(Class? 
 super T expected, MethodHandles.Lookup lookup)

Isn't this the same as 

public T extends AccessibleObject T reflectAs...

?

I think we considered AccessibleObject but rejected it as not buying anything 
significant compared with Member which is an interface.

Perhaps 

public T extends Member  AnnotatedElement T reflectAs...

with both interfaces, would have been slightly better.

As the API is written (and yes it is too late to change) I don't think there 
are any use cases (at least with ground types) which require an extra cast.

Thank you for looking at it.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8027827: Improve performance of catchException combinator

[John Rose]

On Feb 26, 2014, at 3:44 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 
 Maybe use invokeWithArguments with target and catcher?  That at least is
 a one-liner, and probably more efficient.
 
 Yes, that's a good idea! At least, it considerably simplifies the code.
 
 Updated webrev:
 http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.03/

Thumbs up.

Your use of invokeWithArguments in the unspecialized code is a good design 
pattern.  The semantics are clear in the original method.  This in turn gives a 
clear basis for specializing for each combination of argument arities and 
types.  Specialization should be done using low-level, high-leverage mechanisms 
like bytecode spinning or even JIT optimizations.

Put another way, if we have reasonable bytecode-generation intrinsics, feeding 
to good JIT optimizations, we don't need top-level specializations in the 
source code.  The need for those has always been a mark of weakness in the 
HotSpot implementation of MHs.  (Fredrik's JRockit implementation did it all in 
the JIT!)  We will continue to push down specializations to lower layers.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: RFR (S): 8033666: Make sure @ForceInline is everywhere it needs to be in sun.misc and java.lang.invoke

[John Rose]

On Feb 25, 2014, at 4:16 PM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 As an interim fix, I moved castReference method into 
 java.lang.invoke.MethodHandleImpl class and added new entry point 
 ValueConversions::cast which accepts a method handle to a method which 
 should be used for casting.

P.S.  Reviewed!  Ship it.___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: [9] RFR (M): 8027827: Improve performance of catchException combinator

[John Rose]

On Feb 20, 2014, at 9:57 AM, Vladimir Ivanov vladimir.x.iva...@oracle.com 
wrote:

 Updated webrev:
 http://cr.openjdk.java.net/~vlivanov/8027827/final/webrev.01/
 
 I finally figured out how to make caching work. This webrev contains these 
 changes.
 
 I changed LF representation a bit and added 2 auxiliary method handles - 
 argument boxing and wrapping into Object[] and result unboxing. These 
 operations depend on actual type and can't be shared among arbitrary 
 combinators with the same basic type. They are used only during LF 
 interpretation and are completely ignored in compiled LFs.

This is a good step forward, thanks!

Some comments:

I prefer the bounds check expression pos+3  lambdaForm.names.length.  (One 
integer constant, limit to right.)

The predicate isGuardWithCatch must test all three subforms.  Or else there 
must be assertions to ensure that names[pos+2] is of the expected form.  The 
problem is that LF's can sometimes be edited (e.g., by binding operations) and 
there is no insurance that your pattern of three expressions will be preserved 
in all cases.

I see you are trying to do unboxing elimination here; this is not a safe or 
effective way to do it, IMO.  Put in a FIXME comment and file a bug to deal 
better with unboxing ops in LFs.  I have some WIP code toward this end which we 
can talk about.  You've probably seen of that business, about internally LF 
marking expressions as intrinsics to guide bytecode generation.

Why is the logic about cachedLambdaForm commented out?  It looks correct, but 
is there a bug?

Consider replacing GUARD_WITH_CATCH with Lazy.NF_guardWithCatch, and using the 
NF instead of MH for the intrinsic.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: problems with step debugging

[John Rose]

On Feb 4, 2014, at 4:21 AM, Jochen Theodorou blackd...@gmx.org wrote:

 If I know how it is supposed to look like, I can change our compiler to 
 emit the line number information exactly where suggested and then see if 
 it works. But for that I missing that information :(

Here is the information I know about, which is very little, from the JVMS:

 4.7.12 The LineNumberTable Attribute
 
 The LineNumberTable attribute is an optional variable-length attribute in the 
 attributes table of a Code (§4.7.3) attribute. It may be used by debuggers to 
 determine which part of the Java Virtual Machine code array corresponds to a 
 given line number in the original source file.

It may be used by debuggers... which implies on the other hand it may not.

It could be that javac emits something reasonable for line numbers and IDEs 
and debuggers do something reasonable with it, and nobody can say what's 
wrong if the handshake fails.

If this is how it works, it's OK as long as only a few people are involved in 
the handshake.  Perhaps somebody in the know needs to write a little blog entry 
or wiki page on how line numbering works.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev