Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[John Rose]

On May 5, 2017, at 5:44 AM, Vladimir Ivanov  
wrote:
> 
> In other words, LF specialization influence how many classes for compiled LFs 
> are loaded, but doesn't change what actually happen during MH invocation. (No 
> inlining on bytecode level is needed during specialization. JIT will already 
> do that during compilation. No need to help it.)

Not that hidden frames are terribly important, but it does strike me as odd, at 
this moment, that stack traces for customized MHs and "raw" MHs look the pretty 
much the same.

Idea:  Have two levels of hiding:  Hide always, and hide if inlined into some 
caller.  That way a successfully optimized MH will show fewer stack frames on 
close inspection.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[Vladimir Ivanov]

Jochen,


Can you elaborate a bit, please? What kind of stress do you observe: MH
instantiation overhead or increased footprint? Does memory increase come
from method handles alone or there are plenty of classes loaded at
runtime for compiled LFs?


My biggest problem in terms of creation performance are transformations
of the handle using asType so far. Having to create many many different
MethodHandles increases the memory footprint, but probably stabilizes.
As for plenty of classes... well.. potentially yes. I can easily create
such a program in Groovy.


example... foo(x,y) is mapped to MyInvokerFallback.handle(receiver,
"foo", x, y); with the method taking a String and an Object[]. How do I
get the name in there without insertArguments? Don't I have to create at
least one handle per name I find?


One important detail is how method handles are actually used.

Yes, you do have to create a method handle per call site, but it is
placed in a CallSite instance and bound to indy call site. In that case,
there's no need in LambdaForm specialization: JIT-compiler will inline
the whole method handle chain at indy call site which is equivalent to
bytecode specialization.


is that now true for all handles? Since the forms do no longer show up
in the traces I cannot tell. Also I am required to have MutableCallsite,
since I have to handle the dispatch based on runtime types. This
multiplies the number of handles I create. Example:


Yes, it's true for all handles. LF specialization is tightly coupled 
with JIT-compilers and is triggered only for method handles which aren't 
inlined into all callers. It never happens for indy call sites - JITs 
can always inline (and do so) through them. (Even when they are linked 
to mutable CSs. In that case, there's a dependency on compiled method 
registered to track future modifications.)


But I suspect it's not what you asked about.

FYI with -XX:+ShowHiddenFrames the JVM will include LF frames in stack 
trackes. But it's not about stack frames: there's still a single frame 
per method handle in a method handle chain in interpreter.


LambdaForm specialization is about generating a dedicated class for a 
LambdaForm instance.


So, irrespective of LF specialization, you'll observe the same number of 
stack frames, but the methods being executed will refer to either shared 
or customized LFs.


In other words, LF specialization influence how many classes for 
compiled LFs are loaded, but doesn't change what actually happen during 
MH invocation. (No inlining on bytecode level is needed during 
specialization. JIT will already do that during compilation. No need to 
help it.)



Object myMethod(Object singleArg);
Object myMethod(String singleArg);

myMethod(x)

In Java, now depending on the defined type of x we know which of the two
methods to call. Which means, if I could use a static callsite here. In
Groovy I have to first put in a handle, that directs to my method
selector, which will then install the target handle (and call it), as
well as a guard to check that the argument is as expected.


I'd like to differentiate method handles and lambda forms. If you create 
a new method handle, it doesn't imply a new lambda form is also created.


Method handles aren't compiled to bytecode themselves, only lambda forms 
are. So, when you instantiate a new method handle, from footprint 
perspective you pay a cost of a single object instance. Most likely, the 
costs of the lambda form & associated class are amortized across all 
method handles which share them.


For example, my experiments with Nashorn showed 1000x ratio between 
instantiated MHs & LFs (millions handles vs thousands LFs on Octane 
benchmarks).


Also, LF caches are SoftReference-based, so footprint measurements don't 
reflect how many LFs are actually used. It's pretty expensive to 
construct a LF, so it's benefitical to keep it alive longer that weak 
references allow.


You mentioned MH.asType() and, unfortunately, from LF sharing 
perspective it's a weak point right now. There's some sharing possible, 
but the current LF shape for asType() transformation is hard to share.


It hasn't been addressed yet mostly because we don't have a good 
understanding how much overhead does it cause. So, if you have any data 
on that, please, share.



Also, LambdaForms are aggressively shared, so you shouldn't observe
significant growth in their number at runtime (unless there are lots of
unique "erased" signatures present; that's where LF sharing can't help
now).


there is a high number of "runtime signatures"


What is important is how many unique erased signatures exist (erased to 
basic types [1]). It's still possible to trigger explosion in number of 
LFs (5^255 is still pretty large, isn't it? ;-)), but now it's a corner 
case.


Best regards,
Vladimir Ivanov

[1] 5 in total: int, long, float, double, Object
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[Jochen Theodorou]

On 04.05.2017 15:05, Vladimir Ivanov wrote:

Jochen,


I think avoiding to create many of them is actually not trivial. The
indy port of Groovy has a similar problem. And I do have to use a lot of
insertArguments, exception catching handles and other things. So the
stress is actually pretty high at times.


Can you elaborate a bit, please? What kind of stress do you observe: MH
instantiation overhead or increased footprint? Does memory increase come
from method handles alone or there are plenty of classes loaded at
runtime for compiled LFs?


My biggest problem in terms of creation performance are transformations 
of the handle using asType so far. Having to create many many different 
MethodHandles increases the memory footprint, but probably stabilizes. 
As for plenty of classes... well.. potentially yes. I can easily create 
such a program in Groovy.



example... foo(x,y) is mapped to MyInvokerFallback.handle(receiver,
"foo", x, y); with the method taking a String and an Object[]. How do I
get the name in there without insertArguments? Don't I have to create at
least one handle per name I find?


One important detail is how method handles are actually used.

Yes, you do have to create a method handle per call site, but it is
placed in a CallSite instance and bound to indy call site. In that case,
there's no need in LambdaForm specialization: JIT-compiler will inline
the whole method handle chain at indy call site which is equivalent to
bytecode specialization.


is that now true for all handles? Since the forms do no longer show up 
in the traces I cannot tell. Also I am required to have MutableCallsite, 
since I have to handle the dispatch based on runtime types. This 
multiplies the number of handles I create. Example:


Object myMethod(Object singleArg);
Object myMethod(String singleArg);

myMethod(x)

In Java, now depending on the defined type of x we know which of the two 
methods to call. Which means, if I could use a static callsite here. In 
Groovy I have to first put in a handle, that directs to my method 
selector, which will then install the target handle (and call it), as 
well as a guard to check that the argument is as expected.



Also, LambdaForms are aggressively shared, so you shouldn't observe
significant growth in their number at runtime (unless there are lots of
unique "erased" signatures present; that's where LF sharing can't help
now).


there is a high number of "runtime signatures"

bye Jochen
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[Vladimir Ivanov]

Jochen,


I think avoiding to create many of them is actually not trivial. The
indy port of Groovy has a similar problem. And I do have to use a lot of
insertArguments, exception catching handles and other things. So the
stress is actually pretty high at times.


Can you elaborate a bit, please? What kind of stress do you observe: MH 
instantiation overhead or increased footprint? Does memory increase come 
from method handles alone or there are plenty of classes loaded at 
runtime for compiled LFs?



example... foo(x,y) is mapped to MyInvokerFallback.handle(receiver,
"foo", x, y); with the method taking a String and an Object[]. How do I
get the name in there without insertArguments? Don't I have to create at
least one handle per name I find?


One important detail is how method handles are actually used.

Yes, you do have to create a method handle per call site, but it is 
placed in a CallSite instance and bound to indy call site. In that case, 
there's no need in LambdaForm specialization: JIT-compiler will inline 
the whole method handle chain at indy call site which is equivalent to 
bytecode specialization.


Also, LambdaForms are aggressively shared, so you shouldn't observe 
significant growth in their number at runtime (unless there are lots of 
unique "erased" signatures present; that's where LF sharing can't help 
now).


Hope it helps. FTR I covered some of those topics in details in my 
j.l.i-related talk at JVMLS'15 [1].


Best regards,
Vladimir Ivanov

[1] http://cr.openjdk.java.net/~vlivanov/talks/2015-JVMLS_State_of_JLI.pdf


On 04.05.2017 08:16, John Rose wrote:

On May 3, 2017, at 9:37 PM, Wenlei Xie > wrote:


Thank you Vladimir for the help ! I see the point why MH.bindTo() is
not a good fit for implementing lambda capturing.


A simple rule for using MHs is that they are designed to be another form
of code.
Creating many of them at a high rate is likely to stress JVM in ways
similar
to loading many small classes at a high rate.

So bindTo is really code customization, which is not the same thing as
data capture.
The MH before bindTo is an algorithm with a variable "hole" in it, where
the MH after
bindTo is a customized version of the algorithm, with the hole filed by
a constant.
It's a little like a C++ template instance.

I'd like high-count bindTo to be cheaper, of course, but it's not the
design center,
and it's not where we are investing optimization effort.  Maybe in the
future.

— John


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[Jochen Theodorou]

I think avoiding to create many of them is actually not trivial. The 
indy port of Groovy has a similar problem. And I do have to use a lot of 
insertArguments, exception catching handles and other things. So the 
stress is actually pretty high at times.


example... foo(x,y) is mapped to MyInvokerFallback.handle(receiver, 
"foo", x, y); with the method taking a String and an Object[]. How do I 
get the name in there without insertArguments? Don't I have to create at 
least one handle per name I find?


bye Jochen

On 04.05.2017 08:16, John Rose wrote:

On May 3, 2017, at 9:37 PM, Wenlei Xie > wrote:


Thank you Vladimir for the help ! I see the point why MH.bindTo() is
not a good fit for implementing lambda capturing.


A simple rule for using MHs is that they are designed to be another form
of code.
Creating many of them at a high rate is likely to stress JVM in ways similar
to loading many small classes at a high rate.

So bindTo is really code customization, which is not the same thing as
data capture.
The MH before bindTo is an algorithm with a variable "hole" in it, where
the MH after
bindTo is a customized version of the algorithm, with the hole filed by
a constant.
It's a little like a C++ template instance.

I'd like high-count bindTo to be cheaper, of course, but it's not the
design center,
and it's not where we are investing optimization effort.  Maybe in the
future.

— John


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[John Rose]

On May 3, 2017, at 9:37 PM, Wenlei Xie  wrote:
> 
> Thank you Vladimir for the help ! I see the point why MH.bindTo() is not a 
> good fit for implementing lambda capturing. 

A simple rule for using MHs is that they are designed to be another form of 
code.
Creating many of them at a high rate is likely to stress JVM in ways similar
to loading many small classes at a high rate.

So bindTo is really code customization, which is not the same thing as data 
capture.
The MH before bindTo is an algorithm with a variable "hole" in it, where the MH 
after
bindTo is a customized version of the algorithm, with the hole filed by a 
constant.
It's a little like a C++ template instance.

I'd like high-count bindTo to be cheaper, of course, but it's not the design 
center,
and it's not where we are investing optimization effort.  Maybe in the future.

— John___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[Wenlei Xie]

Thank you Vladimir for the help ! I see the point why MH.bindTo() is not a
good fit for implementing lambda capturing.

We cannot easily directly pass the values from table explicitly to the
generated MethodHandle, as we allow UDF/extract function used in lambda
functions.

I see Brain talked about use InvokeDynamic to implement Java lambda capture
(http://wiki.jvmlangsummit.com/images/7/7b/Goetz-jvmls-lambda.pdf) and the
LambdaMetaFactoryBenchmark class, I did some benchmark and it seems to have
better performance for capture support :). However I do see some strange
performance regression after the invocation number exceeds some threshold,
which is probably better fit in a separate email thread :)

Best,
Wenlei

On Wed, May 3, 2017 at 5:34 AM, Vladimir Ivanov <
vladimir.x.iva...@oracle.com> wrote:

> Thanks for the report and for the test case, Wenlei.
>
> What you observe is an unfortunate consequence of LambdaForm
> customization. It was introduced to speedup invocations of non-constant
> method handles (MH.invoke/invokeExact on a method handle which isn't a
> constant during JIT compilation).
>
> As an example from your use case, in order to optimize for the value of
> bound argument, the JIT compiler has to "see" it during the compilation.
> The only way to achieve it right now is by issuing "specialized" bytecode
> for the particular method handle and that's exactly what happens during
> LambdaForm customization.
>
> The generated class should go away once the method handle it was generated
> for becomes unreachable, but it seems you construct plenty of method
> handles for every query.
>
> As a workaround, you can turn it off by specifying:
>   -Djava.lang.invoke.MethodHandle.CUSTOMIZE_THRESHOLD=-1
>
> But I agree with Remi that it's a sign of a deeper problem in how you use
> method handles. MH.bindTo() always produces a new method handle and doesn't
> look like a good fit for implementing lambda capturing.
>
> Method handles are designed for fast invocation. Some non-trivial amount
> of work happens during method handle instantiation, so it should be avoided
> in hot code. From performance perspective, one-time usage of method handles
> never pays off. You should try to cache and reuse them in order to observe
> speedups.
>
> In particular, reusing the same method handle chain for all rows and
> passing the value (from the table) explicitly should lead to a better
> generated code.
>
> Best regards,
> Vladimir Ivanov
>
>
> On 5/2/17 10:29 PM, Wenlei Xie wrote:
>
>> Hi,
>>
>> We are implementing Lambda function with capture support in a SQL
>> Engine. We currently implement by compiling user-written Lambda
>> Expression into a MethodHandle. And use bindTo to captured fields. Thus
>> for each row we will have a Bound Method Handle.
>>
>> However, we found JVM will generate the byte code Bound Method Handle
>> once it's invoked more than 128 times. This cause in some cases (when
>> the table has large arrays), the Metaspace fills with generated
>> LambdaForm$BMH class.
>>
>> Here is the simple code to reproduce the
>> issue: https://github.com/wenleix/BMHTest . It looks we cannot increase
>> java.lang.invoke.MethodHandle.CUSTOMIZE_THRESHOLD beyond 128. Any
>> suggestions to implement Lambda with Capture Support on JVM?
>>
>> Thank you !!
>>
>> Best,
>> Wenlei
>>
>>
>>
>>
>>
>> ___
>> mlvm-dev mailing list
>> mlvm-dev@openjdk.java.net
>> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>>
>> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>



-- 
Best Regards,
Wenlei Xie (谢文磊)

Email: wenlei@gmail.com
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[Vladimir Ivanov]

Thanks for the report and for the test case, Wenlei.

What you observe is an unfortunate consequence of LambdaForm 
customization. It was introduced to speedup invocations of non-constant 
method handles (MH.invoke/invokeExact on a method handle which isn't a 
constant during JIT compilation).


As an example from your use case, in order to optimize for the value of 
bound argument, the JIT compiler has to "see" it during the compilation. 
The only way to achieve it right now is by issuing "specialized" 
bytecode for the particular method handle and that's exactly what 
happens during LambdaForm customization.


The generated class should go away once the method handle it was 
generated for becomes unreachable, but it seems you construct plenty of 
method handles for every query.


As a workaround, you can turn it off by specifying:
  -Djava.lang.invoke.MethodHandle.CUSTOMIZE_THRESHOLD=-1

But I agree with Remi that it's a sign of a deeper problem in how you 
use method handles. MH.bindTo() always produces a new method handle and 
doesn't look like a good fit for implementing lambda capturing.


Method handles are designed for fast invocation. Some non-trivial amount 
of work happens during method handle instantiation, so it should be 
avoided in hot code. From performance perspective, one-time usage of 
method handles never pays off. You should try to cache and reuse them in 
order to observe speedups.


In particular, reusing the same method handle chain for all rows and 
passing the value (from the table) explicitly should lead to a better 
generated code.


Best regards,
Vladimir Ivanov

On 5/2/17 10:29 PM, Wenlei Xie wrote:

Hi,

We are implementing Lambda function with capture support in a SQL
Engine. We currently implement by compiling user-written Lambda
Expression into a MethodHandle. And use bindTo to captured fields. Thus
for each row we will have a Bound Method Handle.

However, we found JVM will generate the byte code Bound Method Handle
once it's invoked more than 128 times. This cause in some cases (when
the table has large arrays), the Metaspace fills with generated
LambdaForm$BMH class.

Here is the simple code to reproduce the
issue: https://github.com/wenleix/BMHTest . It looks we cannot increase
java.lang.invoke.MethodHandle.CUSTOMIZE_THRESHOLD beyond 128. Any
suggestions to implement Lambda with Capture Support on JVM?

Thank you !!

Best,
Wenlei





___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev


___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[Wenlei Xie]

Hi Remi,

Thank you for the prompt response!!

Consider the following query:

SELECT transform(arr, x->x + value)
FROM test_table

The underlying table is
arr |   value

[1, 2] |   1
[7, 8, 9] |   2

The expected output is

[2, 3]
[9, 10, 11]


Now our implementation is first to generate a MethodHandle mh that takes
two argument and output the sum. During the query processing, for each row,
we first call

capturedMh = mh.bindTo(value)

And then use capturedMh for each value in the array.

That's why we are having a different method handle for each row.

Best,
Wenlei






On Tue, May 2, 2017 at 1:02 PM, Remi Forax  wrote:

> Hi Wenei,
> The idea of bindTo()/insertArguments is that you are requesting a partial
> evaluation, so you are asking the VM/JIT to specialize the method handle if
> the method handle is used often.
> So compiling a method handle to a bytecode snippet in that case is the
> expected behavior, not a bug.
>
> What i do not understand is why you need to have a different method handle
> for each row ?
>
> Did you try to use invokeWithArguments instead of bindTo().invokeExact() ?
>
> cheers,
> Rémi
>
> --
>
> *De: *"Wenlei Xie" 
> *À: *mlvm-dev@openjdk.java.net
> *Envoyé: *Mardi 2 Mai 2017 21:29:38
> *Objet: *Implementing Lambda with Capture support makes Metaspace fills
> LambdaForms$BMH class
>
> Hi,
> We are implementing Lambda function with capture support in a SQL Engine.
> We currently implement by compiling user-written Lambda Expression into a
> MethodHandle. And use bindTo to captured fields. Thus for each row we will
> have a Bound Method Handle.
>
> However, we found JVM will generate the byte code Bound Method Handle once
> it's invoked more than 128 times. This cause in some cases (when the table
> has large arrays), the Metaspace fills with generated LambdaForm$BMH class.
>
> Here is the simple code to reproduce the issue: https://github.com/
> wenleix/BMHTest . It looks we cannot increase
> java.lang.invoke.MethodHandle.CUSTOMIZE_THRESHOLD beyond 128. Any
> suggestions to implement Lambda with Capture Support on JVM?
>
> Thank you !!
>
> Best,
> Wenlei
>
>
>
>
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
>
> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
>
>


-- 
Best Regards,
Wenlei Xie (谢文磊)

Email: wenlei@gmail.com
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev

Re: Implementing Lambda with Capture support makes Metaspace fills LambdaForms$BMH class

[Remi Forax]

Hi Wenei, 
The idea of bindTo()/insertArguments is that you are requesting a partial 
evaluation, so you are asking the VM/JIT to specialize the method handle if the 
method handle is used often. 
So compiling a method handle to a bytecode snippet in that case is the expected 
behavior, not a bug. 

What i do not understand is why you need to have a different method handle for 
each row ? 

Did you try to use invokeWithArguments instead of bindTo().invokeExact() ? 

cheers, 
Rémi 

> De: "Wenlei Xie" 
> À: mlvm-dev@openjdk.java.net
> Envoyé: Mardi 2 Mai 2017 21:29:38
> Objet: Implementing Lambda with Capture support makes Metaspace fills
> LambdaForms$BMH class

> Hi,
> We are implementing Lambda function with capture support in a SQL Engine. We
> currently implement by compiling user-written Lambda Expression into a
> MethodHandle. And use bindTo to captured fields. Thus for each row we will 
> have
> a Bound Method Handle.

> However, we found JVM will generate the byte code Bound Method Handle once 
> it's
> invoked more than 128 times. This cause in some cases (when the table has 
> large
> arrays), the Metaspace fills with generated LambdaForm$BMH class.

> Here is the simple code to reproduce the issue:
> https://github.com/wenleix/BMHTest . It looks we cannot increase
> java.lang.invoke.MethodHandle.CUSTOMIZE_THRESHOLD beyond 128. Any suggestions
> to implement Lambda with Capture Support on JVM?

> Thank you !!

> Best,
> Wenlei

> ___
> mlvm-dev mailing list
> mlvm-dev@openjdk.java.net
> http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev
___
mlvm-dev mailing list
mlvm-dev@openjdk.java.net
http://mail.openjdk.java.net/mailman/listinfo/mlvm-dev