Re: WITHDRAWN Re: Proposal: Fully Concurrent ClassLoading

2013-01-31 Thread David Holmes
Hi Peter, On 31/01/2013 11:07 PM, Peter Levart wrote: Hi David, Could the parallel classloading be at least space optimized somehow in the JDK8 timeframe if there was a solution ready? If there is something that does not impact any of the existing specified semantics regarding the classloade

Re: WITHDRAWN Re: Proposal: Fully Concurrent ClassLoading

2013-01-31 Thread Peter Levart
Hi David, Could the parallel classloading be at least space optimized somehow in the JDK8 timeframe if there was a solution ready? Regards, Peter On 01/31/2013 10:26 AM, David Holmes wrote: Regretfully this proposal has to be withdrawn from Java 8. The VM facilities that this depends upon ha

WITHDRAWN Re: Proposal: Fully Concurrent ClassLoading

2013-01-31 Thread David Holmes
Regretfully this proposal has to be withdrawn from Java 8. The VM facilities that this depends upon have not been specified formally and so an update to the Java Virtual Machine Specification (JVMS) and further updates to the ClassLoader.defineClass method are required. There was simply no time

Re: Proposal: Fully Concurrent ClassLoading

2012-12-16 Thread David Holmes
On 17/12/2012 2:12 AM, Zhong Yu wrote: On Sat, Dec 15, 2012 at 3:24 AM, David Holmes wrote: Can you suggest any actual use of the result of getClassLoadingLock() that won't cause a NullPointerException if null is returned? I'm saying a more specific exception could be more helpful than NPE, f

Re: Proposal: Fully Concurrent ClassLoading

2012-12-16 Thread Zhong Yu
On Sat, Dec 15, 2012 at 3:24 AM, David Holmes wrote: > On 15/12/2012 12:58 AM, Zhong Yu wrote: >> >> On Wed, Dec 12, 2012 at 5:48 PM, David Holmes >> wrote: >>> >>> On 13/12/2012 1:51 AM, Zhong Yu wrote: If a class loader is declared fully concurrent, yet getClassLoadingLock()

Re: Proposal: Fully Concurrent ClassLoading

2012-12-15 Thread David Holmes
On 15/12/2012 12:58 AM, Zhong Yu wrote: On Wed, Dec 12, 2012 at 5:48 PM, David Holmes wrote: On 13/12/2012 1:51 AM, Zhong Yu wrote: If a class loader is declared fully concurrent, yet getClassLoadingLock() is invoked, what's the harm of returning a dedicated lock anyway, exactly like what's d

Re: Proposal: Fully Concurrent ClassLoading

2012-12-14 Thread Zhong Yu
On Wed, Dec 12, 2012 at 5:48 PM, David Holmes wrote: > On 13/12/2012 1:51 AM, Zhong Yu wrote: >> >> If a class loader is declared fully concurrent, yet >> getClassLoadingLock() is invoked, what's the harm of returning a >> dedicated lock anyway, exactly like what's done before? > > > The whole poi

Re: Proposal: Fully Concurrent ClassLoading

2012-12-12 Thread David Holmes
On 13/12/2012 1:51 AM, Zhong Yu wrote: If a class loader is declared fully concurrent, yet getClassLoadingLock() is invoked, what's the harm of returning a dedicated lock anyway, exactly like what's done before? The whole point is to get rid of the lockMap and these lock objects. I could retur

Re: Proposal: Fully Concurrent ClassLoading

2012-12-12 Thread Zhong Yu
On Wed, Dec 12, 2012 at 10:05 AM, Peter Levart wrote: > On 12/12/2012 04:51 PM, Zhong Yu wrote: >> >> If a class loader is declared fully concurrent, yet >> getClassLoadingLock() is invoked, what's the harm of returning a >> dedicated lock anyway, exactly like what's done before? > > To encourage

Re: Proposal: Fully Concurrent ClassLoading

2012-12-12 Thread Peter Levart
On 12/12/2012 04:51 PM, Zhong Yu wrote: If a class loader is declared fully concurrent, yet getClassLoadingLock() is invoked, what's the harm of returning a dedicated lock anyway, exactly like what's done before? To encourage people to not use locking in the first place ;-) No, seriously, to be

Re: Proposal: Fully Concurrent ClassLoading

2012-12-12 Thread Zhong Yu
If a class loader is declared fully concurrent, yet getClassLoadingLock() is invoked, what's the harm of returning a dedicated lock anyway, exactly like what's done before? On Tue, Dec 11, 2012 at 7:40 PM, David Holmes wrote: > On 11/12/2012 9:58 PM, Peter Levart wrote: >> >> On 12/11/2012 12:27

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread David M. Lloyd
Ah, well for our concurrent class loader we just keep simple counters for all that stuff. If we can boost that up to the ClassLoader level that'd be nifty. The stats we track that have been useful are: - Number of defined classes - Number of 'races' i.e. define when already existent - Number

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread David Holmes
On 11/12/2012 9:58 PM, Peter Levart wrote: On 12/11/2012 12:27 PM, David Holmes wrote: Peter, You are convincing me that all superclasses must be fully concurrent too. Otherwise we are just trying to second-guess a whole bunch of what-ifs. :) If you think some more, yes. The superclass might

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread David Holmes
On 12/12/2012 1:10 AM, David M. Lloyd wrote: No problem. What do you mean by "instrumentation"? Any means of "seeing" when parallel define class actually occurred: - -verbose:class logging ? - jvmstat counter ? - -XX:+TraceParallelDefine ? - ??? Thanks, David On 12/10/2012 11:18 PM, David

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread David M. Lloyd
On 12/11/2012 02:12 AM, Peter Levart wrote: On 12/11/2012 04:41 AM, David M. Lloyd wrote: On JDK 8 with your patches, we are loading around 4750 classes and there are, as expected, 0 define races (I believe, however, that we're getting a false count though whenever defineClass() returns an alrea

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread David M. Lloyd
No problem. What do you mean by "instrumentation"? On 12/10/2012 11:18 PM, David Holmes wrote: David, Many thanks for trialling this so promptly! Do you have any suggestions for what instrumentation you would like to see accompany this? David On 11/12/2012 1:41 PM, David M. Lloyd wrote: On

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread Peter Levart
On 12/11/2012 12:27 PM, David Holmes wrote: Peter, You are convincing me that all superclasses must be fully concurrent too. Otherwise we are just trying to second-guess a whole bunch of what-ifs. :) If you think some more, yes. The superclass might not use getClassLoadingLock() but rely on

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread David Holmes
Peter, You are convincing me that all superclasses must be fully concurrent too. Otherwise we are just trying to second-guess a whole bunch of what-ifs. :) Thanks, David On 11/12/2012 7:44 PM, Peter Levart wrote: On 12/11/2012 10:29 AM, David Holmes wrote: On 11/12/2012 7:20 PM, Peter Leva

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread Peter Levart
On 12/11/2012 10:29 AM, David Holmes wrote: On 11/12/2012 7:20 PM, Peter Levart wrote: On 12/11/2012 03:55 AM, David Holmes wrote: Question on the source code: registerAsFullyConcurrent has confusing comment - do the super classes all need to be parallel capable? Or do the super classes all nee

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread Peter Levart
Hi again, There might be a middle-ground variant. No registration API changes as described below. When ClassLoader X declares that it is fully-concurrent make it fully-concurrent. But if any superclass registered as only parallel-capable, provide getClassLoadingLock() locks nevertheless. Only

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread David Holmes
On 11/12/2012 7:20 PM, Peter Levart wrote: On 12/11/2012 03:55 AM, David Holmes wrote: Question on the source code: registerAsFullyConcurrent has confusing comment - do the super classes all need to be parallel capable? Or do the super classes all need to be FullyConcurrent? I assume the latter,

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread Peter Levart
On 12/11/2012 03:55 AM, David Holmes wrote: Question on the source code: registerAsFullyConcurrent has confusing comment - do the super classes all need to be parallel capable? Or do the super classes all need to be FullyConcurrent? I assume the latter, so just fix the comments. Actually it i

Re: Proposal: Fully Concurrent ClassLoading

2012-12-11 Thread Peter Levart
On 12/11/2012 04:41 AM, David M. Lloyd wrote: On JDK 8 with your patches, we are loading around 4750 classes and there are, as expected, 0 define races (I believe, however, that we're getting a false count though whenever defineClass() returns an already-defined class - it would be nice if ther

Re: Proposal: Fully Concurrent ClassLoading

2012-12-10 Thread David Holmes
David, Many thanks for trialling this so promptly! Do you have any suggestions for what instrumentation you would like to see accompany this? David On 11/12/2012 1:41 PM, David M. Lloyd wrote: On 12/10/2012 06:36 PM, David Holmes wrote: On 10/12/2012 11:53 PM, David M. Lloyd wrote: On 12/

Re: Proposal: Fully Concurrent ClassLoading

2012-12-10 Thread David Holmes
FYI I've made some small updates to the blog entry. Just to quantify the inefficiencies here is the instrumented output of a simple test that tries to load all classes in rt.jar and prints out the size of the lock maps: 18521 classes loaded. sun.misc.Launcher$AppClassLoader@f2f585 - loc

Re: Proposal: Fully Concurrent ClassLoading

2012-12-10 Thread David M. Lloyd
On 12/10/2012 06:36 PM, David Holmes wrote: On 10/12/2012 11:53 PM, David M. Lloyd wrote: On 12/09/2012 10:03 PM, David Holmes wrote: That sounds promising. Are you in a position to try out this proposal on a testbed with your app server? Sure, I'd love to. What tree should I build? Please

Re: Proposal: Fully Concurrent ClassLoading

2012-12-10 Thread David Holmes
Hi Karen, Thanks for your feedback. On 11/12/2012 12:31 PM, Karen Kinnear wrote: David, I fully support the concept of a fully concurrent class loader, which allows parallel defineClass. I would agree that we deprecate registerAsParallelCapable - starting deprecation in JDK8. Okay that se

Re: Proposal: Fully Concurrent ClassLoading

2012-12-10 Thread Karen Kinnear
David, I fully support the concept of a fully concurrent class loader, which allows parallel defineClass. I would agree that we deprecate registerAsParallelCapable - starting deprecation in JDK8. My memory is that customers asked us for the getClassLoadingLock API so that they could do their

Re: Proposal: Fully Concurrent ClassLoading

2012-12-10 Thread David Holmes
On 10/12/2012 11:53 PM, David M. Lloyd wrote: On 12/09/2012 10:03 PM, David Holmes wrote: That sounds promising. Are you in a position to try out this proposal on a testbed with your app server? Sure, I'd love to. What tree should I build? Please apply the patches from the webrevs here:

Re: Proposal: Fully Concurrent ClassLoading

2012-12-10 Thread David M. Lloyd
On 12/09/2012 10:03 PM, David Holmes wrote: Hi David, On 7/12/2012 1:06 AM, David M. Lloyd wrote: On 12/06/2012 05:35 AM, Alan Bateman wrote: On 05/12/2012 11:59, David Holmes wrote: Java 7 introduced support for parallel classloading by adding to each class loader a ConcurrentHashMap, refere

Re: Proposal: Fully Concurrent ClassLoading

2012-12-10 Thread Alan Bateman
On 10/12/2012 03:54, David Holmes wrote: : I am tempted to suggest that registerAsParallelCapable should be deprecated too. I think that may be premature for 8, perhaps for 9? I think we should just it over with and deprecate it is now to discourage usage. I don't have data but I'll bet that

Re: Proposal: Fully Concurrent ClassLoading

2012-12-09 Thread David Holmes
Thanks for the comments Mandy. On 8/12/2012 9:46 AM, Mandy Chung wrote: On 12/5/12 3:59 AM, David Holmes wrote: Java 7 introduced support for parallel classloading by adding to each class loader a ConcurrentHashMap, referenced through a new field, parallelLockMap. This contains a mapping from

Re: Proposal: Fully Concurrent ClassLoading

2012-12-09 Thread David Holmes
Hi David, On 7/12/2012 1:06 AM, David M. Lloyd wrote: On 12/06/2012 05:35 AM, Alan Bateman wrote: On 05/12/2012 11:59, David Holmes wrote: Java 7 introduced support for parallel classloading by adding to each class loader a ConcurrentHashMap, referenced through a new field, parallelLockMap. Th

Re: Proposal: Fully Concurrent ClassLoading

2012-12-09 Thread David Holmes
Hi Peter, I think your middle-ground sounds like it may have been a better choice for JDK 7. But for this proposal I want to try and leave parallel loaders behind and move to the fully concurrent model. The real crux of my proposal is how often you really do get concurrent load attempts and

Re: Proposal: Fully Concurrent ClassLoading

2012-12-09 Thread David Holmes
On 6/12/2012 9:35 PM, Alan Bateman wrote: On 05/12/2012 11:59, David Holmes wrote: Java 7 introduced support for parallel classloading by adding to each class loader a ConcurrentHashMap, referenced through a new field, parallelLockMap. This contains a mapping from class names to Objects to use a

Re: Proposal: Fully Concurrent ClassLoading

2012-12-07 Thread Mandy Chung
On 12/5/12 3:59 AM, David Holmes wrote: Java 7 introduced support for parallel classloading by adding to each class loader a ConcurrentHashMap, referenced through a new field, parallelLockMap. This contains a mapping from class names to Objects to use as a classloading lock for that class name.

Re: Proposal: Fully Concurrent ClassLoading

2012-12-06 Thread David M. Lloyd
On 12/06/2012 05:35 AM, Alan Bateman wrote: On 05/12/2012 11:59, David Holmes wrote: Java 7 introduced support for parallel classloading by adding to each class loader a ConcurrentHashMap, referenced through a new field, parallelLockMap. This contains a mapping from class names to Objects to use

Re: Proposal: Fully Concurrent ClassLoading

2012-12-06 Thread Alan Bateman
On 05/12/2012 11:59, David Holmes wrote: Java 7 introduced support for parallel classloading by adding to each class loader a ConcurrentHashMap, referenced through a new field, parallelLockMap. This contains a mapping from class names to Objects to use as a classloading lock for that class name

Re: Proposal: Fully Concurrent ClassLoading

2012-12-06 Thread Peter Levart
Hi David, What about a middle-ground mode where findLoadedClass() and delegation to parent would be called without locks and only findClass() would be guarded by a per-class-name-per-classloader lock? In this mode concurrent attempts to define the same class would still be possible, but the p

Proposal: Fully Concurrent ClassLoading

2012-12-05 Thread David Holmes
Java 7 introduced support for parallel classloading by adding to each class loader a ConcurrentHashMap, referenced through a new field, parallelLockMap. This contains a mapping from class names to Objects to use as a classloading lock for that class name. This scheme has a number of inefficienc