Re: [infinispan-dev] ISPN-1049
The solution outlined on the jira looks good to me. Sent from my mobile phone On 11 May 2011, at 10:58, Mircea Markus mircea.mar...@jboss.com wrote: Hi, I've added a possible solution to https://issues.jboss.org/browse/ISPN-1049, can you please take a look and tell me what you think? Cheers, Mircea ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Describing infinispan
Hi Paolo - I'll ping you off-list with these details. On 29 Apr 2011, at 18:47, Paolo Romano wrote: Hi guys, we're preparing a paper describing the self-tuning replication mechanism (Primary backup vs the 2PC-based replication mechanism currently used by Infinispan), and we'd like to stress how relevant Infinispan is for the J2EE community, and (I was thinking) in particular for the JBoss AS. Is it correct if we say that Infinispan is the reference in-memory NoSQL data grid for the JBoss Application Server, which is the most popular open source J2EE AS ? Do you have access to any public statistics on the number of users of Infinispan/JBoss AS? Of course, feel free to suggest any other cool phrase to highlight the relevance of Infinispan in the NoSQL data grid domain! Cheers, Paolo -- Paolo Romano, PhD Coordinator of the Cloud-TM ICT FP7 Project (www.cloudtm.eu) Senior Researcher INESC-ID Rua Alves Redol, 9 1000-059, Lisbon Portugal Tel. + 351 21 3100300 Fax + 351 21 3145843 Webpage http://www.gsd.inesc-id.pt/~romanop ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] Ideas for locking improvements
Hello all, as some of us met recently at events, we've been discussing some details about the current locking implementations and possible improvements. Some interesting ideas came out and we've been writing them down on the wiki so that everyone can be involved: http://community.jboss.org/wiki/PossibleLockingImprovements as always, more feedback, comments and more suggestions are welcome. Please bear with us if something is not very clear as they are drafts of unimplemented ideas, so if you see something that you like to know more about please start a discussion or comment about it and we'll try to polish the explanation, refining the ideas. Cheers, Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Infinispan Query entity discovery (Was: Re: new Infinispan Query API - ISPN-194)
On 29 Apr 2011, at 19:36, Sanne Grinovero wrote: 2011/4/29 Israel Lacerra israe...@gmail.com: What about use D) and also give a way to user specify the default classes to all queries? Yes that's the idea; but we need to figure out how the user specifies the default classes; so far nobody liked any proposal. Are you suggesting something declarative? E.g., a list of FQCNs in your config file? Might work, but I still prefer mandating the type to query on, on a per-query basis. Cheers Manik -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] cache name in logs
Have we got a JIRA for this BTW? On 2 May 2011, at 08:16, Galder Zamarreño wrote: +1 to limiting to named cache components On Apr 29, 2011, at 7:23 PM, Manik Surtani wrote: I think any log attached to a named cache component (versus a global component) should present cache name. -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
[infinispan-dev] Eventing over Hot Rod vs JMS Was: Hot Rod protocol improvements JIRA
Thanks for this. One of the things Sanne and I discussed is eventing and whether we really need events over Hot Rod, and how urgent this is. For the JBW keynote demo we effectively mimicked the same by using JMS. I.e., listeners on the remote notes putting interesting events on a topic which remote nodes could subscribe to. Maybe this could be detailed as a pattern on the wiki, so when folks ask for eventing over Hot Rod, we could point them here, as this may solve 90% of peoples issues. Cheers Manik On 9 May 2011, at 15:14, Galder Zamarreño wrote: Guys, I've create https://issues.jboss.org/browse/ISPN-1090 to track improvements required to the Hot Rod protocol for the next version. I've added a couple that had been in my head for a while. If you think of any improvements that the protocol needs, make sure you add them there for when we next tackle new functionality in the protocol (i.e. remote querying, or even listeners,...etc) Cheers, -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On 5/12/11 4:18 AM, Dan Berindei wrote: On Wed, May 11, 2011 at 11:18 PM, David Bosschaertda...@redhat.com wrote: On 11/05/2011 17:54, Dan Berindei wrote: On Wed, May 11, 2011 at 7:08 PM, Pete Muirpm...@redhat.comwrote: Were we developing for OSGi I would certainly agree with you. However in many environments today we can reasonably expect the TCCL to be set and to be able to load the classes we need. So whilst making it part of the API is the safest option, it's also making complicated an API for the sake of the few at the cost of the many. Further this also seems kinda nasty to me. We know the class (and hence bundle/module) when we put the object into Infinispan, therefore why do we require people to respecify this again? David, can we not actually do something here akin to what we are discussing for Weld? Whereby we can serialize out the bundle id and then find the correct CL based on that when we deserialize. What if the object is a java.util.ArrayList? Each element in the list could belong to a different bundle, so you'd have to write a bundle id for every element in the list. Yes, if you know the Bundle-SymbolicName and Version (or the Bundle ID) you can find its classloader. On the other question, if you're passing in a class object then you can obtain its classloader and hence the bundle where it came from. But, and I think this is what Dan allused to above, is it always true that the class your passing in comes from the bundle that you need to have or could it also come from one of its parent class loaders? Exactly David, sorry if my message was a little cryptic. I think in order to handle every case properly you would have to go through the entire object graph being stored in the cache in order to find all the classloaders/bundle ids that you will need on get(). This approach just doesn't scale, and it would not work in all environments (there is no gaurantee you can get the original classloader if the environment doesn't allow you to look them up by some kind of unique id). That seems like a lot of overhead to me, and forcing the user to provide the classloader doesn't seem that bad in comparison. Perhaps we should use something other than a thread-local for this though Yes thread locals are brittle, and tend to leak if they aren't managed correctly. Using a context specific instance is a much better approach. so that users can do a onto the result of a cacheManager.getCache(A).usingClassLoader(A.class) and never have to provide the classloader again. That's similar to what I was proposing with the CacheSession notion. The basic notion is that you move thread/context specific state to a separate object that the caller uses (optionaly), and it mirrors the same Cache API. In fact I think this is a good idea for the invocation flags we already have, too. It would involve creating lots of overloads in CacheDelegate with a PreInvocationContext parameter and a new CacheDelegateWithContext class to invoke those methods, but the public API would remain the same. You dont need to reuse the same impl. Just create a new delegate which passes an extra invocation state parameter with ever call. The overhead of that is tiny. -- Jason T. Greene JBoss, a division of Red Hat ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] JGroupsDistSync and ISPN-83
Yes, please have a look. If we are relying on lock upgrades then that's really bad. I am aware of the inability to (safely) upgrade a RWL and I'm pretty sure we don't try, but the dist sync codebase has evolved a lot and could do with some careful analysis. Sent from my mobile phone On 12 May 2011, at 09:24, Vladimir Blagojevic vblag...@redhat.com wrote: On 11-05-11 11:23 AM, Dan Berindei wrote: If ReentrantReadWriteLock would allow upgrades then you would get a deadlock when two threads both hold the read lock and try to upgrade to a write lock at the same time. There's always a trade-off... I'm not familiar with the code, but are you sure the read lock is being held by the same thread that is trying to acquire the write lock? Not sure and it sounds counter intuitive that a thread holding a read lock from cluster invocation is doing state generation for state transfer as well. But this lock business is fishy and I plan to get to the bottom of it... ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977
Yes, collocation of all keys is a large concern of my application(s). Currently, I can handle keys I'm in control of (like database-generated keys), where I can play around with the hash code. What I would love to do is collocate that data with keys I can't control (like UUIDs) so that all cache operations can be done in the same transaction on the data owner's node. Erik -Original Message- From: infinispan-dev-boun...@lists.jboss.org [mailto:infinispan-dev-boun...@lists.jboss.org] On Behalf Of Manik Surtani Sent: Friday, May 13, 2011 5:25 AM To: infinispan -Dev List Subject: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977 On 11 May 2011, at 18:47, Erik Salter wrote: Wouldn't any rehash affect the locality of these generated keys, or am I missing something? It would. And hence ISPN-977, to address that. Or is your concern keys already generated before the rehash? The latter would certainly be a problem. Perhaps, if this was important to the application, on detecting a change in topology, re-generate keys and move data around? For other apps, move the session to the appropriate node? Cheers Manik -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977
On Fri, May 13, 2011 at 6:38 PM, Erik Salter esal...@bnivideo.com wrote: Yes, collocation of all keys is a large concern of my application(s). Currently, I can handle keys I'm in control of (like database-generated keys), where I can play around with the hash code. What I would love to do is collocate that data with keys I can't control (like UUIDs) so that all cache operations can be done in the same transaction on the data owner's node. By playing around with the hash code do you mean you set the hashcode for all the keys you want on a certain server to the same value? I imagine that would degrade performance quite a bit, because we have HashMaps everywhere and your keys will always end up in the same hash bucket. ISPN-312 seems to me like a much better fit for your use case than the KeyAffinityService. Even if you added a listener to change your keys when the topology changes, that would mean on a rehash the keys could get moved to the new server and then back to the old server, whereas with ISPN-312 they will either all stay on the old node or they will all move to the new node. Cheers Dan Erik -Original Message- From: infinispan-dev-boun...@lists.jboss.org [mailto:infinispan-dev-boun...@lists.jboss.org] On Behalf Of Manik Surtani Sent: Friday, May 13, 2011 5:25 AM To: infinispan -Dev List Subject: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977 On 11 May 2011, at 18:47, Erik Salter wrote: Wouldn't any rehash affect the locality of these generated keys, or am I missing something? It would. And hence ISPN-977, to address that. Or is your concern keys already generated before the rehash? The latter would certainly be a problem. Perhaps, if this was important to the application, on detecting a change in topology, re-generate keys and move data around? For other apps, move the session to the appropriate node? Cheers Manik -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977
Hi Dan, I don't necessarily care about which server it's on, as long as the keys for my set of caches all remain collocated. I understand they will all end up in the same bucket, but for one hash code, that's at most 200 keys. I must acquire a lock for a subset of them during a transaction -- so I make liberal use of the eagerLockSingleNode option and redirecting my calling application to execute a transaction on the local node. Acquiring cluster-wide locks is an absolute throughput killer. I took a look at the KeyAffinityService a while ago (when it came out) and quickly realized it would not be suitable for my purposes. I was wondering if ISPN-977 would allow me to use it. But you're right. What I ultimately want is ISPN-312. Erik -Original Message- From: infinispan-dev-boun...@lists.jboss.org [mailto:infinispan-dev-boun...@lists.jboss.org] On Behalf Of Dan Berindei Sent: Friday, May 13, 2011 12:58 PM To: infinispan -Dev List Subject: Re: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977 On Fri, May 13, 2011 at 6:38 PM, Erik Salter esal...@bnivideo.com wrote: Yes, collocation of all keys is a large concern of my application(s). Currently, I can handle keys I'm in control of (like database-generated keys), where I can play around with the hash code. What I would love to do is collocate that data with keys I can't control (like UUIDs) so that all cache operations can be done in the same transaction on the data owner's node. By playing around with the hash code do you mean you set the hashcode for all the keys you want on a certain server to the same value? I imagine that would degrade performance quite a bit, because we have HashMaps everywhere and your keys will always end up in the same hash bucket. ISPN-312 seems to me like a much better fit for your use case than the KeyAffinityService. Even if you added a listener to change your keys when the topology changes, that would mean on a rehash the keys could get moved to the new server and then back to the old server, whereas with ISPN-312 they will either all stay on the old node or they will all move to the new node. Cheers Dan Erik -Original Message- From: infinispan-dev-boun...@lists.jboss.org [mailto:infinispan-dev-boun...@lists.jboss.org] On Behalf Of Manik Surtani Sent: Friday, May 13, 2011 5:25 AM To: infinispan -Dev List Subject: [infinispan-dev] Generated keys affected by rehash Was: https://issues.jboss.org/browse/ISPN-977 On 11 May 2011, at 18:47, Erik Salter wrote: Wouldn't any rehash affect the locality of these generated keys, or am I missing something? It would. And hence ISPN-977, to address that. Or is your concern keys already generated before the rehash? The latter would certainly be a problem. Perhaps, if this was important to the application, on detecting a change in topology, re-generate keys and move data around? For other apps, move the session to the appropriate node? Cheers Manik -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev The information contained in this message is legally privileged and confidential, and is intended for the individual or entity to whom it is addressed (or their designee). If this message is read by anyone other than the intended recipient, please be advised that distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please notify the sender immediately and delete or destroy all copies of this message. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] [Pull Request] Modular Classloading Compatibility
On Fri, May 13, 2011 at 5:48 PM, Jason T. Greene jason.gre...@redhat.com wrote: On 5/12/11 4:18 AM, Dan Berindei wrote: On Wed, May 11, 2011 at 11:18 PM, David Bosschaertda...@redhat.com wrote: On 11/05/2011 17:54, Dan Berindei wrote: On Wed, May 11, 2011 at 7:08 PM, Pete Muirpm...@redhat.com wrote: Were we developing for OSGi I would certainly agree with you. However in many environments today we can reasonably expect the TCCL to be set and to be able to load the classes we need. So whilst making it part of the API is the safest option, it's also making complicated an API for the sake of the few at the cost of the many. Further this also seems kinda nasty to me. We know the class (and hence bundle/module) when we put the object into Infinispan, therefore why do we require people to respecify this again? David, can we not actually do something here akin to what we are discussing for Weld? Whereby we can serialize out the bundle id and then find the correct CL based on that when we deserialize. What if the object is a java.util.ArrayList? Each element in the list could belong to a different bundle, so you'd have to write a bundle id for every element in the list. Yes, if you know the Bundle-SymbolicName and Version (or the Bundle ID) you can find its classloader. On the other question, if you're passing in a class object then you can obtain its classloader and hence the bundle where it came from. But, and I think this is what Dan allused to above, is it always true that the class your passing in comes from the bundle that you need to have or could it also come from one of its parent class loaders? Exactly David, sorry if my message was a little cryptic. I think in order to handle every case properly you would have to go through the entire object graph being stored in the cache in order to find all the classloaders/bundle ids that you will need on get(). This approach just doesn't scale, and it would not work in all environments (there is no gaurantee you can get the original classloader if the environment doesn't allow you to look them up by some kind of unique id). That seems like a lot of overhead to me, and forcing the user to provide the classloader doesn't seem that bad in comparison. Perhaps we should use something other than a thread-local for this though Yes thread locals are brittle, and tend to leak if they aren't managed correctly. Using a context specific instance is a much better approach. so that users can do a onto the result of a cacheManager.getCache(A).usingClassLoader(A.class) and never have to provide the classloader again. That's similar to what I was proposing with the CacheSession notion. The basic notion is that you move thread/context specific state to a separate object that the caller uses (optionaly), and it mirrors the same Cache API. The way I understood your proposal was that the session would be something visible to the user, so he would have to call cacheManager.getCache(A).getSession() in order to access the context functionality. My version essentially the same thing, except the session would be created transparently when the user calls usingClassLoader() or withFlags(). In fact I think this is a good idea for the invocation flags we already have, too. It would involve creating lots of overloads in CacheDelegate with a PreInvocationContext parameter and a new CacheDelegateWithContext class to invoke those methods, but the public API would remain the same. You dont need to reuse the same impl. Just create a new delegate which passes an extra invocation state parameter with ever call. The overhead of that is tiny. Some of the methods in CacheDelegate do have lots of logic in them, so we have to reuse them. The way I see it the existing methods will change to receive the context parameter explicitly instead of implicitly, and the methods with the old signature will call them with a default context. usingClassLoader/withFlags will create an instance of CacheDelegateSession, and the methods in CacheDelegateSession will modify/use the context stored in that instance. I agree the overhead is tiny, and the best thing is users can optimize it away by moving the usingClassLoader/withFlags calls outside of loops. Cheers Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Infinispan Query entity discovery (Was: Re: new Infinispan Query API - ISPN-194)
2011/5/13 Manik Surtani ma...@jboss.org: On 29 Apr 2011, at 19:36, Sanne Grinovero wrote: 2011/4/29 Israel Lacerra israe...@gmail.com: What about use D) and also give a way to user specify the default classes to all queries? Yes that's the idea; but we need to figure out how the user specifies the default classes; so far nobody liked any proposal. Are you suggesting something declarative? E.g., a list of FQCNs in your config file? Might work, but I still prefer mandating the type to query on, on a per-query basis. I'm also not a big fan of listing FQCNs, but for example when doing a FROM in JPA-QL you can target a supertype and be returned subtypes; in our case you'd have to list all subtypes as well. (i.e. FROM java.lang.Object is a valid query). Or does Reflections provide some cool way to find all subtypes of a class? Even if it's a bit expensive, we already cache all new class processing as for each new type we have to restart the Search engine which is quite expensive anyway. Sanne Cheers Manik -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
