Re: Loading classes with many methods is very expensive

Sorry, I apologize. I'll start a new thread on core-libs only. Peter On 10/30/2014 10:47 AM, Aleksey Shipilev wrote: Guys, can you please start a new RFR thread for Peter's change? You have a bug ID now, and the discussion should be searchable by bug ID. Also, you are spamming two aliases (cor

Re: Loading classes with many methods is very expensive

Guys, can you please start a new RFR thread for Peter's change? You have a bug ID now, and the discussion should be searchable by bug ID. Also, you are spamming two aliases (corelibs + hotspot-runtime) instead of one ;) -Aleksey. On 10/30/2014 12:43 PM, Peter Levart wrote: > On 10/30/2014 10:29

Re: Loading classes with many methods is very expensive

On 10/30/2014 10:29 AM, Peter Levart wrote: I might get away with excluding seq from hashCode computation and only use it in equals(). This way Key(s) could be modified in-place (hashCode would not change, entry would stay in same bucket), it would just become equal to some other Key. Modificat

Re: Loading classes with many methods is very expensive

Hi Stanimir, On 10/30/2014 12:00 AM, Stanimir Simeonoff wrote: Hi Peter, The removal of value wrapper is a clever approach to reduce the new instances created although it feels very unnatural (at least to me). Number of objects created is practically the same. Previous approach used same Met

Re: Loading classes with many methods is very expensive

Hi Peter, The removal of value wrapper is a clever approach to reduce the new instances created although it feels very unnatural (at least to me). Small optimization; eagerly calculate the hash in the c'tor, hash = 149 method.getReturnType().hashCode() ^ 150 Syst

Re: inconsistency between Class.getMethod and Class.getMethods Was: Loading classes with many methods is very expensive

Hi Peter, On 29 Oct 2014, at 18:16, Peter Levart wrote: > On 10/29/2014 02:08 PM, Joel Borggrén-Franck wrote: >> Hi Peter, >> >> I’m not entirely convinced this is a bug. >> >> The lookup order for getMethod has for a long time been walk up superclasses >> and return what you find there first

Re: Loading classes with many methods is very expensive

On 10/29/2014 02:08 PM, Joel Borggrén-Franck wrote: Hi Peter, I’m not entirely convinced this is a bug. The lookup order for getMethod has for a long time been walk up superclasses and return what you find there first without even looking at interfaces. It might be desirable to change that bu

Re: Loading classes with many methods is very expensive

Hi Peter, I’m not entirely convinced this is a bug. The lookup order for getMethod has for a long time been walk up superclasses and return what you find there first without even looking at interfaces. It might be desirable to change that but I’m not sure. cheers /Joel On 29 okt 2014, at 12:2

Re: Loading classes with many methods is very expensive

Hi Joel, I found an inconsistency between getMethod() and getMethods() results that is present in current JDK8/9 code and in my latest webrev.02. The following program: import java.util.stream.Collectors; import java.util.stream.Stream; public class GetMethodTest { static void test(Clas

Re: Loading classes with many methods is very expensive

Jeremy has already filed a bug (8062116), and is just finishing up the patch (I jumped the gun a bit by attaching it to the bug - it has a few bugs in it). I'll probably have something in reasonable shape to review tomorrow (I've submitted it internally, and I want to let it spend a day or so havi

Re: Loading classes with many methods is very expensive

On 27.10.2014 22:01, Martin Buchholz wrote: > I'm having trouble keeping up with this thread I started, but I did > update the test/benchmark > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class.getMethods-benchmark/ >

Re: Loading classes with many methods is very expensive

I'm having trouble keeping up with this thread I started, but I did update the test/benchmark http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Class.getMethods-benchmark/ I added a test *LoadAllClassesAndMethods *that does what its name says. Although originally written as just a benchmark, it

Re: Loading classes with many methods is very expensive

Hi Peter, On 10/27/2014 03:45 PM, Peter Levart wrote: > I merged the functionality of the method-signature-key with the > linked-list node, holding just a reference to Method object and a link > to 'next' node. I think this is the most compact temporary datastructure > representation for that purp

Re: Loading classes with many methods is very expensive

Hi Joel, Thanks for peeking into the code. On 10/27/2014 02:45 PM, Joel Borggrén-Franck wrote: Hi Peter, As always, thanks for doing this! It has been on my todolist for a while but never quite bubbling up to the top. I don’t have time to look att his right now, but I expect to have some fre

Re: Loading classes with many methods is very expensive

Hi Peter, As always, thanks for doing this! It has been on my todolist for a while but never quite bubbling up to the top. I don’t have time to look att his right now, but I expect to have some free time next week, but i have two short comments First, I have been thinking about moving MethodA

Re: Loading classes with many methods is very expensive

Hi Aleksey, On 10/27/2014 12:23 PM, Aleksey Shipilev wrote: On 10/27/2014 01:53 AM, Peter Levart wrote: As you might have noticed, the number of methods obtained from patched code differed from original code. I have investigated this and found that original code treats abstract class methods th

Re: Loading classes with many methods is very expensive

On Mon, Oct 27, 2014 at 1:23 PM, Aleksey Shipilev < aleksey.shipi...@oracle.com> wrote: > On 10/27/2014 01:53 AM, Peter Levart wrote: > > As you might have noticed, the number of methods obtained from patched > > code differed from original code. I have investigated this and found > > that origina

Re: Loading classes with many methods is very expensive

On 10/27/2014 01:53 AM, Peter Levart wrote: > As you might have noticed, the number of methods obtained from patched > code differed from original code. I have investigated this and found > that original code treats abstract class methods the same as abstract > interface methods as far as multiple

Re: Loading classes with many methods is very expensive

> > 55 java/lang/reflect jtreg tests still pass. As they did before, which > means that we don't have a coverage for such cases. I'll see where I can > add such a case (EnumSet for example, which inherits from Set interface and > AbstractColection class via two different paths, so Set.size()/iterat

Re: Loading classes with many methods is very expensive

On 10/26/2014 09:25 PM, Peter Levart wrote: 19657 classes loaded in 1.987373401 seconds. 494141 methods obtained in 1.02493941 seconds. vs. 19657 classes loaded in 2.084409717 seconds. 494124 methods obtained in 0.915928578 seconds. Hi, As you might have noticed, the number of methods obtai

Re: Loading classes with many methods is very expensive

On Mon, Oct 27, 2014 at 12:36 AM, Peter Levart wrote: > > On 10/26/2014 11:21 PM, Stanimir Simeonoff wrote: > > Great effort. > > From first glance: the hashCode and equals of MethodList use > m.getParameterTypes() which is cloned. I'd rather pay the collision costs > and rely on the default has

Re: Loading classes with many methods is very expensive

On 10/26/2014 11:21 PM, Stanimir Simeonoff wrote: Great effort. From first glance: the hashCode and equals of MethodList use m.getParameterTypes() which is cloned. I'd rather pay the collision costs and rely on the default hashCode/equals of Method itself (hashCode ignores the parameters). P

Re: Loading classes with many methods is very expensive

Great effort. >From first glance: the hashCode and equals of MethodList use m.getParameterTypes() which is cloned. I'd rather pay the collision costs and rely on the default hashCode/equals of Method itself (hashCode ignores the parameters). Possibly hashCode can include the returnType but equals

Re: Loading classes with many methods is very expensive

Hi all, I revamped the Class.getMethods() implementation so that it is faster for common cases and O(n) at the same time, which makes Martin's test happy (at least in part that measures getMethods() speed - the class loading / linkage in VM is a separate issue). With the following test that

Re: Loading classes with many methods is very expensive

On 23 Oct 2014, at 17:44, Peter Levart wrote: > > Is there a test that validates correctness of getMethods() or at least a set > of interfaces and/or classes to exercise the algorithm and compare it to a > different implementation? > There is :) http://hg.openjdk.java.net/jdk9/dev/jdk/file/

Re: Loading classes with many methods is very expensive

On 10/23/2014 05:44 PM, Peter Levart wrote: interface A6 extends B6, C6 {} interface B6 extends D6 { void m(); } interface C6 extends D6 {} interface D6 { default void m() {}; } A6.class.getMethods() returns B6.m, D6.m Ah, B6.m re-abstracts the default method D6.m. I can see the rule here. N

Re: Loading classes with many methods is very expensive

On 10/23/2014 05:06 PM, Joel Borggrén-Franck wrote: Hi Martin, On 23 okt 2014, at 00:53, Martin Buchholz wrote: Here at Google we have both kinds of scalability problems - loading classes from a classpath with 10,000 jars, and loading a single class file packed with the maximal number of meth

Re: Loading classes with many methods is very expensive

On 10/23/2014 03:48 PM, Karen Kinnear wrote: Peter, Which hotspot algorithm is the one you identified as slow? Would that be for loading the class or for Class.getMethods? thanks, Karen Martin did it. And Alexey profiled it. I just noted that Java part of Class.getMethods(), should it be re

Re: Loading classes with many methods is very expensive

Hi Martin, On 23 okt 2014, at 00:53, Martin Buchholz wrote: > Here at Google we have both kinds of scalability problems - loading classes > from a classpath with 10,000 jars, and loading a single class file packed > with the maximal number of methods. This message is about the latter. > > If y

Re: Loading classes with many methods is very expensive

I was under the impression the declaring order has been abandoned in 1.7 which I considered quite a let down as allowed ordering the function calls in JMX in a non-chaotic way. On Thu, Oct 23, 2014 at 4:37 PM, Peter Levart wrote: > On 10/23/2014 01:57 AM, Stanimir Simeonoff wrote: > >> Class.j

Re: Loading classes with many methods is very expensive

Peter, Which hotspot algorithm is the one you identified as slow? Would that be for loading the class or for Class.getMethods? thanks, Karen On Oct 23, 2014, at 9:37 AM, Peter Levart wrote: > On 10/23/2014 01:57 AM, Stanimir Simeonoff wrote: >> Class.java can easily be improved by adding an aux

Re: Loading classes with many methods is very expensive

On 10/23/2014 01:57 AM, Stanimir Simeonoff wrote: Class.java can easily be improved by adding an auxiliary HasMap indexing the position in the array by name and then checking only few overloaded signatures in case of a match. The auxiliary map can be deployed only past some threshold of methods,

Re: Loading classes with many methods is very expensive

Hi Martin, On 23.10.2014 02:53, Martin Buchholz wrote: > If you have a class with ~64k methods with a superclass that also has ~64k > methods, class loading that single class will cost you ~30sec and calling > Class.getMethods another ~10sec. Both are unacceptably slow. I think both > are due to

Re: Loading classes with many methods is very expensive

Class.java can easily be improved by adding an auxiliary HasMap indexing the position in the array by name and then checking only few overloaded signatures in case of a match. The auxiliary map can be deployed only past some threshold of methods, so it should not affect the common case. Alternati

Loading classes with many methods is very expensive

Here at Google we have both kinds of scalability problems - loading classes from a classpath with 10,000 jars, and loading a single class file packed with the maximal number of methods. This message is about the latter. If you have a class with ~64k methods with a superclass that also has ~64k me