Re: [infinispan-dev] Invoking distributed exec and mapreduce over hotrod
On Wed, May 4, 2011 at 23:51, Manik Surtani wrote: > > So I reckon Javascript is the way to go, at least from a protocol > standpoint. Now how we expose this in remote client APIs (Java, Python, > etc) needs some thought, but at first glance it would seem as though we > won't have a direct mapping to what we do on the embedded side of things. > E.g., http://www.mongodb.org/display/DOCS/MapReduce > Actually, using javax.script, we could default to Javascript (which is included in every Java6 implementation out there) but also be able to specify alternatives via a mimetype. In this way one could use any of the JSR 223 languages (Jython, JRuby, Groovy, etc). Tristan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Invoking distributed exec and mapreduce over hotrod
Well, I feel exposing M/R over Hot Rod - from a protocol standpoint - would require a platform-independent mechanism of defining a closure (keeping in mind we need to allow this from non-Java clients too). So I reckon Javascript is the way to go, at least from a protocol standpoint. Now how we expose this in remote client APIs (Java, Python, etc) needs some thought, but at first glance it would seem as though we won't have a direct mapping to what we do on the embedded side of things. E.g., http://www.mongodb.org/display/DOCS/MapReduce - Manik On 4 May 2011, at 04:50, Vladimir Blagojevic wrote: > Galder, > > I believe the ability to invoke distributed executors and mapreduce over > hotrod would be very interesting. However, I quickly realized that > internals of both DistributedExecutorService and MapReduceTask rely > heavily on some Cache internals (RpcManager, CommandsFactory, > InterceptoChain) that are only available in non-remote caches. There is > no way to fake this by simply passing RemoteCache instead of Cache. > Either we rethink the internals of DistributedExecutorService and > MapReduceTask or we somehow bridge to these abstractions from a thin > client. > > Any thoughts how we could potentially achieve hotrodization of dist. exec? > > Regards, > Vladimir > > > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
2011/5/4 Pete Muir : > > On 4 May 2011, at 09:55, Dan Berindei wrote: > >> On Wed, May 4, 2011 at 7:18 AM, "이희승 (Trustin Lee)" >> wrote: >>> On 05/03/2011 09:33 PM, Sanne Grinovero wrote: 2011/5/3 "이희승 (Trustin Lee)" : > On 05/03/2011 05:08 AM, Sanne Grinovero wrote: >> 2011/5/2 Manik Surtani : >>> >>> On 1 May 2011, at 13:38, Pete Muir wrote: >>> >> As in, user API? That's a little intrusive... e.g., put(K, V, cl) ? > > Not for put, since you have the class, just get, and I was thinking > something more like: > > Foo foo = getUsing(key, Foo.class) This would be a pretty useful addition to the API anyway to avoid user casts. >>> >>> Maybe as an "advanced" API, so as not to pollute the basic API? A bit >>> like: >>> >>> Foo f = cache.getAdvancedCache().asClass(Foo.class).get(key); >> >> doesn't look much better than a cast, but is more cryptical :) >> >> getting back to the classloader issue, what about: >> >> Cache c = cacheManager.getCache( cacheName, classLoader ); >> >> or >> Cache c = cacheManager.getCache( cacheName >> ).usingClassLoader(classLoader ); >> >> BTW if that's an issue on the API, maybe you should propose it to >> JSR-107 as well ? > > We have a configurable Marshaller, right? Then why don't we just use > the class loader that the current Marshaller uses? +1 I like the clean approach, not sure how you configure the "current Marshaller" to use the correct CL ? Likely hard to do via configuration file :) >>> >>> By default, we could use the class loader that the current Unmarshaller >>> uses, and let user choose a class loader for a certain get() call. >>> >>> So, we need to deal with the two issues here: >>> >>> 1) Make sure that user can configure the default class loader in runtime >>> and calling get() (and consequently related unmarshaller) will use the >>> default class loader: >>> >>> cache.setDefaultClassLoader(UserClass.class.getClassLoader()) >>> >>> 2) Provide an additional API that allows a user to specify a class >>> loader for the current transaction or context: >>> >>> cache.get(K key, Class clazz) // +1 to Pete's suggestion >>> >> >> If V is Set then Set.class.getClassLoader() will >> give you the system classloader, which in most cases won't be able the >> MyObject class. It may not even be a Set of course, it could >> be just Set. >> >> We could have a "shortcut" so the users can pass in a class instead of >> a classloader to avoid writing ".getClassLoader()", but we shouldn't >> require any relation between the class passed in and the class of the >> return value. > > There are two forces at play here. Firstly the desire to introduce greater > type safety into the Cache API, by returning a type for user, rather than > just an object, and secondly the need to specify the CL. > > We could have an API like: > > V get(Object key, Class classLoaderType); > > and an overloaded form > > V get(Object key, ClassLoader cl); > > This does involve Infinispan having to do a runtime cast to V without the > class type passed in as a parameter, but this may be a necessary evil in the > interest of improving the user api. > Cache extends Map, so we already have runtime casts to V for V get(Object key) It seems natural to extend it to also return objects of the cache's V type parameter instead of returning an arbitrary type: V get(Object key, Class classLoaderType); V get(Object key, ClassLoader cl); Besides, allowing get() to return an arbitrary type will only make the user code look more type safe, without changing its behaviour at all. If the user wants type safety he can already define different caches with different V type parameters and in that case we will actually enforce that the put argument is assignable to V at compile time. > An alternative which is somewhat more verbose, somewhat safer, and somewhat > more complex to explain (but ultimately more logical): > > // Use the classloader of the expectedType, this would be the normal case > V get(Object key, Class expectedType); > > // Specify the classloader to use as a class > V get(Object key, Class expectedType, Class classLoader); > > // Specify the classloader to use as a classloader > V get(Object key, Class expectedType, ClassLoader classLoader); > > We could also include > > // Use the TCCL > V get(Object key); > > Any thoughts? > I'd allow the user to set another default classloader on the cache instead of the TCCL, like in Sanne's proposal: Cache c = cacheManager.getCache( cacheName, defaultClassLoader ); or Cache c = cacheManager.getCache( cacheName ).usingDefaultClassLoader( classLoader ); Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On 05/04/2011 09:08 AM, Pete Muir wrote: > > On 4 May 2011, at 09:55, Dan Berindei wrote: > >> On Wed, May 4, 2011 at 7:18 AM, "이희승 (Trustin >> Lee)" wrote: >>> On 05/03/2011 09:33 PM, Sanne Grinovero wrote: 2011/5/3 "이희승 (Trustin Lee)": > On 05/03/2011 05:08 AM, Sanne Grinovero wrote: >> 2011/5/2 Manik Surtani: >>> >>> On 1 May 2011, at 13:38, Pete Muir wrote: >>> >> As in, user API? That's a little intrusive... >> e.g., put(K, V, cl) ? > > Not for put, since you have the class, just get, and > I was thinking something more like: > > Foo foo = getUsing(key, Foo.class) This would be a pretty useful addition to the API anyway to avoid user casts. >>> >>> Maybe as an "advanced" API, so as not to pollute the >>> basic API? A bit like: >>> >>> Foo f = >>> cache.getAdvancedCache().asClass(Foo.class).get(key); >> >> doesn't look much better than a cast, but is more cryptical >> :) >> >> getting back to the classloader issue, what about: >> >> Cache c = cacheManager.getCache( cacheName, classLoader ); >> >> or Cache c = cacheManager.getCache( cacheName >> ).usingClassLoader(classLoader ); >> >> BTW if that's an issue on the API, maybe you should propose >> it to JSR-107 as well ? > > We have a configurable Marshaller, right? Then why don't we > just use the class loader that the current Marshaller uses? +1 I like the clean approach, not sure how you configure the "current Marshaller" to use the correct CL ? Likely hard to do via configuration file :) >>> >>> By default, we could use the class loader that the current >>> Unmarshaller uses, and let user choose a class loader for a >>> certain get() call. >>> >>> So, we need to deal with the two issues here: >>> >>> 1) Make sure that user can configure the default class loader in >>> runtime and calling get() (and consequently related unmarshaller) >>> will use the default class loader: >>> >>> cache.setDefaultClassLoader(UserClass.class.getClassLoader()) >>> >>> 2) Provide an additional API that allows a user to specify a >>> class loader for the current transaction or context: >>> >>> cache.get(K key, Class clazz) // +1 to Pete's suggestion >>> >> >> If V is Set then Set.class.getClassLoader() >> will give you the system classloader, which in most cases won't be >> able the MyObject class. It may not even be a Set of >> course, it could be just Set. >> >> We could have a "shortcut" so the users can pass in a class instead >> of a classloader to avoid writing ".getClassLoader()", but we >> shouldn't require any relation between the class passed in and the >> class of the return value. > > There are two forces at play here. Firstly the desire to introduce > greater type safety into the Cache API, by returning a type for user, > rather than just an object, and secondly the need to specify the CL. > > We could have an API like: > > V get(Object key, Class classLoaderType); See note below about the CL type parameter. > and an overloaded form > > V get(Object key, ClassLoader cl); > > This does involve Infinispan having to do a runtime cast to V without > the class type passed in as a parameter, but this may be a necessary > evil in the interest of improving the user api. There's no reason to do the runtime cast. This should be spelled: Object get(Object key, ClassLoader cl); That way if there's a CCE, the user will stand a better chance of understanding it (because hey, they did a cast). > An alternative which is somewhat more verbose, somewhat safer, and > somewhat more complex to explain (but ultimately more logical): > > // Use the classloader of the expectedType, this would be the normal > case V get(Object key, Class expectedType); > > // Specify the classloader to use as a class > V get(Object key, Class expectedType, Class classLoader); This one doesn't make a lot of sense to me. In particular there's no reason to have a type parameter CL unless you give it a bound in relationship to V, in which case you could have just used the class loader class in the first place. Maybe you meant: V get(Object key, Class expected, Class classWhoseClassLoaderShouldBeUsed); And I would name that parameter exactly thus :) > // Specify the classloader to use as a classloader > V get(Object key, Class expectedType, ClassLoader classLoader); > > We could also include > > // Use the TCCL > V get(Object key); Or a preset default CL? -- - DML ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On Wed, May 4, 2011 at 5:09 PM, Pete Muir wrote: > > On 4 May 2011, at 05:34, Dan Berindei wrote: > >> On Wed, May 4, 2011 at 10:14 AM, Galder Zamarreño wrote: >>> >>> On May 3, 2011, at 2:33 PM, Sanne Grinovero wrote: >>> 2011/5/3 "이희승 (Trustin Lee)" : > On 05/03/2011 05:08 AM, Sanne Grinovero wrote: >> 2011/5/2 Manik Surtani : >>> >>> On 1 May 2011, at 13:38, Pete Muir wrote: >>> >> As in, user API? That's a little intrusive... e.g., put(K, V, cl) ? > > Not for put, since you have the class, just get, and I was thinking > something more like: > > Foo foo = getUsing(key, Foo.class) This would be a pretty useful addition to the API anyway to avoid user casts. >>> >>> Maybe as an "advanced" API, so as not to pollute the basic API? A bit >>> like: >>> >>> Foo f = cache.getAdvancedCache().asClass(Foo.class).get(key); >> >> doesn't look much better than a cast, but is more cryptical :) >> >> getting back to the classloader issue, what about: >> >> Cache c = cacheManager.getCache( cacheName, classLoader ); >> >> or >> Cache c = cacheManager.getCache( cacheName >> ).usingClassLoader(classLoader ); >> >> BTW if that's an issue on the API, maybe you should propose it to >> JSR-107 as well ? > > We have a configurable Marshaller, right? Then why don't we just use > the class loader that the current Marshaller uses? +1 I like the clean approach, not sure how you configure the "current Marshaller" to use the correct CL ? Likely hard to do via configuration file :) >>> >>> Well, the marshaller is a global component and so it's a cache manager >>> level. You can't make any assumptions about it's classloader, particularly >>> when lazy deserialization is configured and you want to make sure that the >>> data of the cache is deserialized with the correct classloader when the >>> user reads the data from the cache. This is gonna become even more >>> important when we for example move to having a single cache for all 2LC >>> entities or all EJB3 SFSBs where we'll definitely need multiple >>> classloaders to access a single cache. >>> >> >> The current unmarshaller uses the TCCL, which is a great idea for >> non-modular environments and will still work in AS7 for application >> classes (so it's still a good default). It probably won't work if >> Hibernate wants to store its own classes in the cache, because >> Hibernate's internal classes may not be reachable from the >> application's classloader. >> >> It gets even trickier if Hibernate wants to store a >> PrivateHibernateCollection class in the cache containing instances of >> application classes inside. Then I don't think there will be any >> single classloader that could reach both the Hibernate classes and the >> application classes so it can properly unmarshal both. Perhaps that's >> just something for the Hibernate folks to worry about? Or maybe we >> should allow the users to register more than one classloader with a >> cache? > > You need to use a bridge classloader in this case. You're right of course, Hibernate must have received/guessed the application's classloader and so it is able to create a bridge classloader that "includes" both. And if the application classes include references to classes from another module(s), the application has to provide a bridge classloader to Hibernate anyway. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On 4 May 2011, at 05:34, Dan Berindei wrote: > On Wed, May 4, 2011 at 10:14 AM, Galder Zamarreño wrote: >> >> On May 3, 2011, at 2:33 PM, Sanne Grinovero wrote: >> >>> 2011/5/3 "이희승 (Trustin Lee)" : On 05/03/2011 05:08 AM, Sanne Grinovero wrote: > 2011/5/2 Manik Surtani : >> >> On 1 May 2011, at 13:38, Pete Muir wrote: >> > As in, user API? That's a little intrusive... e.g., put(K, V, cl) ? Not for put, since you have the class, just get, and I was thinking something more like: Foo foo = getUsing(key, Foo.class) >>> >>> This would be a pretty useful addition to the API anyway to avoid user >>> casts. >> >> Maybe as an "advanced" API, so as not to pollute the basic API? A bit >> like: >> >> Foo f = cache.getAdvancedCache().asClass(Foo.class).get(key); > > doesn't look much better than a cast, but is more cryptical :) > > getting back to the classloader issue, what about: > > Cache c = cacheManager.getCache( cacheName, classLoader ); > > or > Cache c = cacheManager.getCache( cacheName ).usingClassLoader(classLoader > ); > > BTW if that's an issue on the API, maybe you should propose it to > JSR-107 as well ? We have a configurable Marshaller, right? Then why don't we just use the class loader that the current Marshaller uses? >>> >>> +1 >>> I like the clean approach, not sure how you configure the "current >>> Marshaller" to use the correct CL ? >>> Likely hard to do via configuration file :) >> >> Well, the marshaller is a global component and so it's a cache manager >> level. You can't make any assumptions about it's classloader, particularly >> when lazy deserialization is configured and you want to make sure that the >> data of the cache is deserialized with the correct classloader when the user >> reads the data from the cache. This is gonna become even more important when >> we for example move to having a single cache for all 2LC entities or all >> EJB3 SFSBs where we'll definitely need multiple classloaders to access a >> single cache. >> > > The current unmarshaller uses the TCCL, which is a great idea for > non-modular environments and will still work in AS7 for application > classes (so it's still a good default). It probably won't work if > Hibernate wants to store its own classes in the cache, because > Hibernate's internal classes may not be reachable from the > application's classloader. > > It gets even trickier if Hibernate wants to store a > PrivateHibernateCollection class in the cache containing instances of > application classes inside. Then I don't think there will be any > single classloader that could reach both the Hibernate classes and the > application classes so it can properly unmarshal both. Perhaps that's > just something for the Hibernate folks to worry about? Or maybe we > should allow the users to register more than one classloader with a > cache? You need to use a bridge classloader in this case. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On 4 May 2011, at 09:55, Dan Berindei wrote: > On Wed, May 4, 2011 at 7:18 AM, "이희승 (Trustin Lee)" wrote: >> On 05/03/2011 09:33 PM, Sanne Grinovero wrote: >>> 2011/5/3 "이희승 (Trustin Lee)" : On 05/03/2011 05:08 AM, Sanne Grinovero wrote: > 2011/5/2 Manik Surtani : >> >> On 1 May 2011, at 13:38, Pete Muir wrote: >> > As in, user API? That's a little intrusive... e.g., put(K, V, cl) ? Not for put, since you have the class, just get, and I was thinking something more like: Foo foo = getUsing(key, Foo.class) >>> >>> This would be a pretty useful addition to the API anyway to avoid user >>> casts. >> >> Maybe as an "advanced" API, so as not to pollute the basic API? A bit >> like: >> >> Foo f = cache.getAdvancedCache().asClass(Foo.class).get(key); > > doesn't look much better than a cast, but is more cryptical :) > > getting back to the classloader issue, what about: > > Cache c = cacheManager.getCache( cacheName, classLoader ); > > or > Cache c = cacheManager.getCache( cacheName ).usingClassLoader(classLoader > ); > > BTW if that's an issue on the API, maybe you should propose it to > JSR-107 as well ? We have a configurable Marshaller, right? Then why don't we just use the class loader that the current Marshaller uses? >>> >>> +1 >>> I like the clean approach, not sure how you configure the "current >>> Marshaller" to use the correct CL ? >>> Likely hard to do via configuration file :) >> >> By default, we could use the class loader that the current Unmarshaller >> uses, and let user choose a class loader for a certain get() call. >> >> So, we need to deal with the two issues here: >> >> 1) Make sure that user can configure the default class loader in runtime >> and calling get() (and consequently related unmarshaller) will use the >> default class loader: >> >> cache.setDefaultClassLoader(UserClass.class.getClassLoader()) >> >> 2) Provide an additional API that allows a user to specify a class >> loader for the current transaction or context: >> >> cache.get(K key, Class clazz) // +1 to Pete's suggestion >> > > If V is Set then Set.class.getClassLoader() will > give you the system classloader, which in most cases won't be able the > MyObject class. It may not even be a Set of course, it could > be just Set. > > We could have a "shortcut" so the users can pass in a class instead of > a classloader to avoid writing ".getClassLoader()", but we shouldn't > require any relation between the class passed in and the class of the > return value. There are two forces at play here. Firstly the desire to introduce greater type safety into the Cache API, by returning a type for user, rather than just an object, and secondly the need to specify the CL. We could have an API like: V get(Object key, Class classLoaderType); and an overloaded form V get(Object key, ClassLoader cl); This does involve Infinispan having to do a runtime cast to V without the class type passed in as a parameter, but this may be a necessary evil in the interest of improving the user api. An alternative which is somewhat more verbose, somewhat safer, and somewhat more complex to explain (but ultimately more logical): // Use the classloader of the expectedType, this would be the normal case V get(Object key, Class expectedType); // Specify the classloader to use as a class V get(Object key, Class expectedType, Class classLoader); // Specify the classloader to use as a classloader V get(Object key, Class expectedType, ClassLoader classLoader); We could also include // Use the TCCL V get(Object key); Any thoughts? ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On Wed, May 4, 2011 at 7:18 AM, "이희승 (Trustin Lee)" wrote: > On 05/03/2011 09:33 PM, Sanne Grinovero wrote: >> 2011/5/3 "이희승 (Trustin Lee)" : >>> On 05/03/2011 05:08 AM, Sanne Grinovero wrote: 2011/5/2 Manik Surtani : > > On 1 May 2011, at 13:38, Pete Muir wrote: > As in, user API? That's a little intrusive... e.g., put(K, V, cl) ? >>> >>> Not for put, since you have the class, just get, and I was thinking >>> something more like: >>> >>> Foo foo = getUsing(key, Foo.class) >> >> This would be a pretty useful addition to the API anyway to avoid user >> casts. > > Maybe as an "advanced" API, so as not to pollute the basic API? A bit > like: > > Foo f = cache.getAdvancedCache().asClass(Foo.class).get(key); doesn't look much better than a cast, but is more cryptical :) getting back to the classloader issue, what about: Cache c = cacheManager.getCache( cacheName, classLoader ); or Cache c = cacheManager.getCache( cacheName ).usingClassLoader(classLoader ); BTW if that's an issue on the API, maybe you should propose it to JSR-107 as well ? >>> >>> We have a configurable Marshaller, right? Then why don't we just use >>> the class loader that the current Marshaller uses? >> >> +1 >> I like the clean approach, not sure how you configure the "current >> Marshaller" to use the correct CL ? >> Likely hard to do via configuration file :) > > By default, we could use the class loader that the current Unmarshaller > uses, and let user choose a class loader for a certain get() call. > > So, we need to deal with the two issues here: > > 1) Make sure that user can configure the default class loader in runtime > and calling get() (and consequently related unmarshaller) will use the > default class loader: > > cache.setDefaultClassLoader(UserClass.class.getClassLoader()) > > 2) Provide an additional API that allows a user to specify a class > loader for the current transaction or context: > > cache.get(K key, Class clazz) // +1 to Pete's suggestion > If V is Set then Set.class.getClassLoader() will give you the system classloader, which in most cases won't be able the MyObject class. It may not even be a Set of course, it could be just Set. We could have a "shortcut" so the users can pass in a class instead of a classloader to avoid writing ".getClassLoader()", but we shouldn't require any relation between the class passed in and the class of the return value. Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On 05/03/2011 11:18 PM, "이희승 (Trustin Lee)" wrote: > By default, we could use the class loader that the current Unmarshaller > uses, and let user choose a class loader for a certain get() call. > > So, we need to deal with the two issues here: > > 1) Make sure that user can configure the default class loader in runtime > and calling get() (and consequently related unmarshaller) will use the > default class loader: > > cache.setDefaultClassLoader(UserClass.class.getClassLoader()) > > 2) Provide an additional API that allows a user to specify a class > loader for the current transaction or context: > > cache.get(K key, Class clazz) // +1 to Pete's suggestion Agreed on all parts. If the default CL isn't set then TCCL is probably a reasonable default. But the user should be allowed to set the default class loader for resolution. To be fair though I don't know whether that makes sense at a cache level, but from a marshalling and modularity perspective this makes a lot of sense. -- - DML ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On Wed, May 4, 2011 at 10:14 AM, Galder Zamarreño wrote: > > On May 3, 2011, at 2:33 PM, Sanne Grinovero wrote: > >> 2011/5/3 "이희승 (Trustin Lee)" : >>> On 05/03/2011 05:08 AM, Sanne Grinovero wrote: 2011/5/2 Manik Surtani : > > On 1 May 2011, at 13:38, Pete Muir wrote: > As in, user API? That's a little intrusive... e.g., put(K, V, cl) ? >>> >>> Not for put, since you have the class, just get, and I was thinking >>> something more like: >>> >>> Foo foo = getUsing(key, Foo.class) >> >> This would be a pretty useful addition to the API anyway to avoid user >> casts. > > Maybe as an "advanced" API, so as not to pollute the basic API? A bit > like: > > Foo f = cache.getAdvancedCache().asClass(Foo.class).get(key); doesn't look much better than a cast, but is more cryptical :) getting back to the classloader issue, what about: Cache c = cacheManager.getCache( cacheName, classLoader ); or Cache c = cacheManager.getCache( cacheName ).usingClassLoader(classLoader ); BTW if that's an issue on the API, maybe you should propose it to JSR-107 as well ? >>> >>> We have a configurable Marshaller, right? Then why don't we just use >>> the class loader that the current Marshaller uses? >> >> +1 >> I like the clean approach, not sure how you configure the "current >> Marshaller" to use the correct CL ? >> Likely hard to do via configuration file :) > > Well, the marshaller is a global component and so it's a cache manager level. > You can't make any assumptions about it's classloader, particularly when lazy > deserialization is configured and you want to make sure that the data of the > cache is deserialized with the correct classloader when the user reads the > data from the cache. This is gonna become even more important when we for > example move to having a single cache for all 2LC entities or all EJB3 SFSBs > where we'll definitely need multiple classloaders to access a single cache. > The current unmarshaller uses the TCCL, which is a great idea for non-modular environments and will still work in AS7 for application classes (so it's still a good default). It probably won't work if Hibernate wants to store its own classes in the cache, because Hibernate's internal classes may not be reachable from the application's classloader. It gets even trickier if Hibernate wants to store a PrivateHibernateCollection class in the cache containing instances of application classes inside. Then I don't think there will be any single classloader that could reach both the Hibernate classes and the application classes so it can properly unmarshal both. Perhaps that's just something for the Hibernate folks to worry about? Or maybe we should allow the users to register more than one classloader with a cache? Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On Wed, May 4, 2011 at 10:16 AM, Galder Zamarreño wrote: > > On May 3, 2011, at 8:06 AM, Dan Berindei wrote: >> >> I'm not even sure we should allow using more than one classloader, >> otherwise a get operation might return an object loaded from the wrong >> classloader. After all, we will only use the provided classloader if >> we need to get the object from another node or if storeAsBinary is set >> to true. > > This is not a right assumption. For example, we don't foresee supporting > asymmetric clusters in the short term as discussed in London, so this will > require all 2LC entities to be stored in the same cache, and these entities > could easily belong to a different classloaders. > I thought Hibernate has some other problems with storing entities in the cache so it stores individual field values instead? If true, then the cache would only store Hibernate classes. Anyway, we already support the multiple classloaders scenario today by using the TCCL, so I agree that it would be a bad idea to require a single class loader per cache from now on. Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] Invoking distributed exec and mapreduce over hotrod
Galder, I believe the ability to invoke distributed executors and mapreduce over hotrod would be very interesting. However, I quickly realized that internals of both DistributedExecutorService and MapReduceTask rely heavily on some Cache internals (RpcManager, CommandsFactory, InterceptoChain) that are only available in non-remote caches. There is no way to fake this by simply passing RemoteCache instead of Cache. Either we rethink the internals of DistributedExecutorService and MapReduceTask or we somehow bridge to these abstractions from a thin client. Any thoughts how we could potentially achieve hotrodization of dist. exec? Regards, Vladimir ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On May 3, 2011, at 8:06 AM, Dan Berindei wrote: > I think Sanne's idea was that you shouldn't have to specify the class > or class loader on every get, as it's very likely that all operations > on that cache will use the same classloader. Most applications will > only use one classloader anyway. > > I'm not even sure we should allow using more than one classloader, > otherwise a get operation might return an object loaded from the wrong > classloader. After all, we will only use the provided classloader if > we need to get the object from another node or if storeAsBinary is set > to true. This is not a right assumption. For example, we don't foresee supporting asymmetric clusters in the short term as discussed in London, so this will require all 2LC entities to be stored in the same cache, and these entities could easily belong to a different classloaders. > > Dan > > > On Mon, May 2, 2011 at 11:24 PM, Thomas P. Fuller > wrote: >> How about just using "as" and "using" instead of "asClass" and >> "usingClassloader"? >> >> Consider something like the following which kind of looks like a dsl: >> >> Foo f = cache.getAdvancedCache().as (Foo.class).using (classLoader).get >> (key); >> >> T >> >> >> From: Sanne Grinovero >> To: infinispan -Dev List >> Sent: Mon, 2 May, 2011 21:08:47 >> Subject: Re: [infinispan-dev] [Pull Request] Modular Classloading >> Compatibility >> >> 2011/5/2 Manik Surtani : >>> >>> On 1 May 2011, at 13:38, Pete Muir wrote: >>> >> As in, user API? That's a little intrusive... e.g., put(K, V, cl) ? > > Not for put, since you have the class, just get, and I was thinking > something more like: > > Foo foo = getUsing(key, Foo.class) This would be a pretty useful addition to the API anyway to avoid user casts. >>> >>> Maybe as an "advanced" API, so as not to pollute the basic API? A bit >>> like: >>> >>> Foo f = cache.getAdvancedCache().asClass(Foo.class).get(key); >> >> doesn't look much better than a cast, but is more cryptical :) >> >> getting back to the classloader issue, what about: >> >> Cache c = cacheManager.getCache( cacheName, classLoader ); >> >> or >> Cache c = cacheManager.getCache( cacheName ).usingClassLoader(classLoader ); >> >> BTW if that's an issue on the API, maybe you should propose it to >> JSR-107 as well ? >> >> Sanne >> >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> ___ >> infinispan-dev mailing list >> infinispan-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > ___ > infinispan-dev mailing list > infinispan-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On May 3, 2011, at 2:33 PM, Sanne Grinovero wrote: > 2011/5/3 "이희승 (Trustin Lee)" : >> On 05/03/2011 05:08 AM, Sanne Grinovero wrote: >>> 2011/5/2 Manik Surtani : On 1 May 2011, at 13:38, Pete Muir wrote: >>> As in, user API? That's a little intrusive... e.g., put(K, V, cl) ? >> >> Not for put, since you have the class, just get, and I was thinking >> something more like: >> >> Foo foo = getUsing(key, Foo.class) > > This would be a pretty useful addition to the API anyway to avoid user > casts. Maybe as an "advanced" API, so as not to pollute the basic API? A bit like: Foo f = cache.getAdvancedCache().asClass(Foo.class).get(key); >>> >>> doesn't look much better than a cast, but is more cryptical :) >>> >>> getting back to the classloader issue, what about: >>> >>> Cache c = cacheManager.getCache( cacheName, classLoader ); >>> >>> or >>> Cache c = cacheManager.getCache( cacheName ).usingClassLoader(classLoader ); >>> >>> BTW if that's an issue on the API, maybe you should propose it to >>> JSR-107 as well ? >> >> We have a configurable Marshaller, right? Then why don't we just use >> the class loader that the current Marshaller uses? > > +1 > I like the clean approach, not sure how you configure the "current > Marshaller" to use the correct CL ? > Likely hard to do via configuration file :) Well, the marshaller is a global component and so it's a cache manager level. You can't make any assumptions about it's classloader, particularly when lazy deserialization is configured and you want to make sure that the data of the cache is deserialized with the correct classloader when the user reads the data from the cache. This is gonna become even more important when we for example move to having a single cache for all 2LC entities or all EJB3 SFSBs where we'll definitely need multiple classloaders to access a single cache. -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Questions on AtomicMap and improvement proposals
Well, there's a fundamental difference to how atomic map works to normal cache
operations and that's the fact that the creation of atomic maps is hidden
behind AtomicMapLookup.getAtomicMap. It's not like std cache usage where the
client creates the map that goes into the cache.
So, I don't see why AtomicMapLookup.removeAtomicMap should let that map escape
its boundaries. IOW, the underlying map is owned by the AML class, not the the
user code.
On Apr 30, 2011, at 9:14 PM, Emmanuel Bernard wrote:
> Another behavior I'm seeing that is surprising me is:
>
> tx.begin();
> Map map = AtomicMapLookup.getAtomicMap(cache, key);
> assert map.size() == 3;
> AtomicMapLookup.removeAtomicMap(cache, key);
> assert map.size() == 0;
> tx.commit();
>
> I can sort understand in a twisted way that removeAtomicMap clears the
> underlying map but that's for sure surprising and can lead to weird bugs for
> ATM users. Is that necessary?
>
> Emmanuel
>
> On 14 avr. 2011, at 10:30, Manik Surtani wrote:
>
>> Apologies for the late response.
>>
>> On 13 Apr 2011, at 10:38, Emmanuel Bernard wrote:
>>
>>> In exchange for answers, I will improve at least the JavaDoc and maybe
>>> create a wiki from this info.
>>
>> Sounds like a fair exchange. :-)
>>
>>> I've tried to search the wiki but found nothing on AtomicMap (only forum
>>> posts) and I've read the AtomicMap JavaDoc but I am still a bit unsure of
>>> what's going on.
>>>
>>> When I need to create an AtomicMap, I must use
>>> AtomicMap resultset = AtomicMapLookup.getAtomicMap(cache,
>>> "my_atomic_map_key");
>>
>> Well, AtomicMapLookup.getAtomicMap(cache, myAtomicMapName).
>>
>>> Questions
>>> Here are a few questions for you
>>>
>>> 1. How can I apply advanced flags to get or create the AtomicMap?
>>> cache.withFlag(SKIP_CACHE_STORE,FORCE_ASYNCHRONOUS).get("my_atomic_map_key");
>>
>> You can't right now; as a "dirty hack", you can just do what getAtomicMap()
>> does internally, adding the flags you need.
>>
>> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/atomic/AtomicMapLookup.java#L49
>>
>> The real solution: create a feature request on JIRA so that we can add flags
>> to getAtomicMap() as well.
>>
>>> 3. Is that legal to use other cache methods like replace on an AtomicMap
>>> value, what are the consequences?
>>
>> cache.putIfAbsent() is used internally by getAtomicMap(). replace() is
>> harder, since it may mean that a thread may be storing stuff in an AtomicMap
>> that has been replaced, so those changes get lost. Why would you need to
>> replace() an atomic map?
>>
>>> 3. When I expect a map to be present in the key, can I use
>>> cache.get("my_atomic_map_key") and still benefit from my properly proxied
>>> AtomicMap? Or am I forced to use AtomicMapLookup.getAtomicMap?
>>
>> You need to use AML.getAtomicMap() to make sure your proxy is properly
>> created/wrapped. See:
>>
>> https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/atomic/AtomicMapLookup.java#L49
>>
>>> 4. Using AtomicMapLookup.getAtomicMap, how can I differentiate an empty key
>>> where a new map has been created from a retrieval of an empty map stored in
>>> the key?
>>
>> You could use getAtomicMap(cache, mapName, createIfAbsent) so you get a null
>> if createIfAbsent == false and no such atomic map exists. Does this help?
>>
>>> 5. Must I use AtomicMapLookup.remove method to remove an atomic map?
>>
>> You can just use cache.remove(), but for the sake of clean generics, it is
>> better to go via AML.remove().
>>
>>> 6. Can I clone an AtomicMap?
>>
>> No, it's not cloneable, since a clone that is not stored in the cache (in a
>> threadsafe manner) is meaningless, and Java's clone() API doesn't allow you
>> to pass in a new name for the clone.
>>
>> However, we could provide a AtomicMap.clone(nameOfClone) method, which could
>> take care of this (internally call AML.getAtomicMap(nameOfClone) and copy
>> contents across).
>>
>>> For using methods like replace(), it seems one would need to clone the
>>> atomicmap to compare it to the initial value. When I tried briefly Iw as
>>> unsure I could do that.
>>
>> Yep.
>>
>>> Proposed improvements
>>>
>>> I'm asking all the questions because these can potentially make AtomicMap
>>> users life quite hard and abstracting Infinispan from OGM will make a damn
>>> ugly interface / contract
>>>
>>> 1. Would it be possible to let people get their proxied AtomicMap from
>>> cache.get() and other get methods?
>>> It seems that either the marshaller or the get operations (probably the get
>>> ops as local cache should work as well) should be able to intercept an
>>> AtomicMap and essentually do what AtomicMapLookup is doing (and make sure
>>> it is properly wrapped).
>>> If that's possible that would be already a big benefit for the user as it
>>> would only be limited to AtomicMap creation or cloning
>>
>> Fair point, certai
