Re: [infinispan-dev] getting ISPN config as XML via JMX
On Fri, Oct 14, 2011 at 7:40 PM, Pete Muir pm...@redhat.com wrote: What is the use case for that method? I've never know anyone actually want to export a running config as XML. So I was planning to loose it. It may be useful to take the configuration out of a cache configured in AS' ISPN subsystem and run it in standalone mode to isolate bugs... Cheers Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] getting ISPN config as XML via JMX
On Fri, 2011-10-14 at 17:40 +0100, Pete Muir wrote: What is the use case for that method? I've never know anyone actually want to export a running config as XML. So I was planning to loose it. It would be far better to interrogate the cache state and behavior and check the config has actually taken effect. Full integration testing is considerably more powerful than anemic unit tests. I know EDG doesn't officially support it, but there is no chance of installing a servlet that can access the AdvancedCache and then using e.g. REST to interrogate it. How do you mean this? EDG currently doesn't support deploying applications into it. And even if I was able to do this I wouldn't get to the advanced cache because I'm accessing the cache remotely - using RemoteCache. On 13 Oct 2011, at 14:12, Mircea Markus wrote: On 13 Oct 2011, at 12:40, Martin Gencur wrote: Hi, one of our efforts around EDG testing is to test whether all XML elements/attributes being set in standalone.xml take effect, i.e. whether org.infinispan.config.Configuration, which in turn is used to create caches, is populated properly based on the xml config file. (this test goal is mentioned in our testplan - second row - https://docspace.corp.redhat.com/docs/DOC-79912) I think one possible way to test this would be to start EDG with certain config. defined in standalone.xml and look at how it was really configured via JMX. There's a JIRA I reported last week which shows that this is not working currently (https://issues.jboss.org/browse/ISPN-1443). Would there be a chance to make it work? Or has this ever been working? Looking at CacheImpl.getConfigurationAsXmlString - this *should* work based on some changes introduced in 5.0.0 final: https://issues.jboss.org/browse/ISPN-948 Perhaps Pete can keep an eye on this as part of rewriting the configuration parsing? If anyone has any thoughts how to test this in a different way, they can tell me. The problem is that we're testing client-server mode where we cannot access AdvancedCache and so on, so getting the real configuration is hard (if possible at all). ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Martin Gencur -- JBoss QE, Enterprise Data Grid Desk phone: +420 532 294 192, ext. 62192 ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Hibernate OGM URL
Wahey! On 12 Oct 2011, at 06:19, Emmanuel Bernard wrote: http://ogm.hibernate.org is now live. It's a redirect to the more complicated http://www.hibernate.org/subprojects/ogm That should make it easier to mention in your slides. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani ma...@jboss.org twitter.com/maniksurtani Lead, Infinispan http://www.infinispan.org ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] FineGrainedAtomicMap - remaining items
On 17 October 2011 10:19, Mircea Markus mircea.mar...@jboss.com wrote: On 17 Oct 2011, at 09:13, Dan Berindei wrote: [cut] 2) This one is a bit technical. What should we do if tx1 deletes entire AtomicMap while tx2 updates entries in the same Map. Should we create a new Map and apply deltas to a new fresh map *or* simply discard delta changes because entire Map has been deleted? I would say not simply discard the changes, but throw an exception and roll back tx2 as well. That would work. Also if a FGAM method is called and that FGAM has been deleted/replaced then the FGAM method should throw an exception. I agree that if it's going to discard the changes then we should have the transaction rolled back, but are we sure if it makes sense to discard them? I think that for the use case of OGM both approaches are fine, simply because this scenario will not happen. Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] getting ISPN config as XML via JMX
On 17 Oct 2011, at 09:37, Martin Gencur wrote: On Fri, 2011-10-14 at 17:40 +0100, Pete Muir wrote: What is the use case for that method? I've never know anyone actually want to export a running config as XML. So I was planning to loose it. It would be far better to interrogate the cache state and behavior and check the config has actually taken effect. Full integration testing is considerably more powerful than anemic unit tests. I know EDG doesn't officially support it, but there is no chance of installing a servlet that can access the AdvancedCache and then using e.g. REST to interrogate it. How do you mean this? EDG currently doesn't support deploying applications into it. Understood, however I wasn't sure if this was actually prohibited or not... And even if I was able to do this I wouldn't get to the advanced cache because I'm accessing the cache remotely - using RemoteCache. I was talking about creating a second connection. On 13 Oct 2011, at 14:12, Mircea Markus wrote: On 13 Oct 2011, at 12:40, Martin Gencur wrote: Hi, one of our efforts around EDG testing is to test whether all XML elements/attributes being set in standalone.xml take effect, i.e. whether org.infinispan.config.Configuration, which in turn is used to create caches, is populated properly based on the xml config file. (this test goal is mentioned in our testplan - second row - https://docspace.corp.redhat.com/docs/DOC-79912) I think one possible way to test this would be to start EDG with certain config. defined in standalone.xml and look at how it was really configured via JMX. There's a JIRA I reported last week which shows that this is not working currently (https://issues.jboss.org/browse/ISPN-1443). Would there be a chance to make it work? Or has this ever been working? Looking at CacheImpl.getConfigurationAsXmlString - this *should* work based on some changes introduced in 5.0.0 final: https://issues.jboss.org/browse/ISPN-948 Perhaps Pete can keep an eye on this as part of rewriting the configuration parsing? If anyone has any thoughts how to test this in a different way, they can tell me. The problem is that we're testing client-server mode where we cannot access AdvancedCache and so on, so getting the real configuration is hard (if possible at all). ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Martin Gencur -- JBoss QE, Enterprise Data Grid Desk phone: +420 532 294 192, ext. 62192 ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] getting ISPN config as XML via JMX
On 17 Oct 2011, at 09:06, Dan Berindei wrote: On Fri, Oct 14, 2011 at 7:40 PM, Pete Muir pm...@redhat.com wrote: What is the use case for that method? I've never know anyone actually want to export a running config as XML. So I was planning to loose it. It may be useful to take the configuration out of a cache configured in AS' ISPN subsystem and run it in standalone mode to isolate bugs... I would have thought a xslt transform would be better here, especially as it can be run without container being up. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Time measurement and expiry
On 17 October 2011 09:02, Dan Berindei dan.berin...@gmail.com wrote: On Sat, Oct 15, 2011 at 7:25 PM, Sanne Grinovero sa...@infinispan.org wrote: Hi all, I'm looking at some performance details, and noticed the current code is using System.currentTimeMillis() to estimate performance for short lived operations. For the purpose of CacheMgmtInterceptor we should use System.nanoTime() instead: http://blogs.oracle.com/dholmes/entry/inside_the_hotspot_vm_clocks Very interesting, I knew that in Windows currentTimeMillis() basically just reads a volatile because I got bit by the 15 millisecond accuracy issue before, so I thought it would always be very fast. I had no idea on Linux it would have the same performance as nanoTime(). Even in Windows it seems to be a bit more expensive than just reading a volatile; Searching for currentTimeMillis() in the code base reveals it's being used for expiry of entries as well. If someone messes with system clocks data will expire; is it expected that entries expire at calendar time and not at a set lifetime? I would say if someone's messing with system clocks they already have a much bigger problem than their cached data expiring... Absolutely, but this is just an example. I'd expect that if I'm using it as a cache, the data will stay in it for a certain amount of relative time. It doesn't have to be a user changing the clock, for example there might be automated changes like daylight adjustments (summer/winter time) or relatively small updates via NTP to keep the clocks in sync.. of course they might not affect the functionality significantly, but I'm just pointing out that we might prefer to use the nanoTime source as our API suggests a period of entry validity in milliseconds, not an expiry in terms of calendar Date. Besides, someone may intentionally change the system time to test the ISPN expiration algorithm :) I understand we need to use the absolute time to be able to transmit the value to other nodes, where the nanoTime of different nodes might not be comparable, but we could store one and use both: transmit the absolute value only to other nodes and refer to the other for accurate expiries. Other nodes receiving the absolute value will check for the remaining lifespane and store the corresponding nanoTime. ExpiryHelper itself will invoke the currentTimeMillis() operation internally, that means that it's going to be invoked at least once for each entry being accessed and might result in a lot of invocations when traversing several entries; I'm wondering if it shouldn't take a millisecond parameter to consider as current, so that this relatively expensive method can be invoked only once at the beginning of a batch of operations. + 1 for adding a currentTime parameter to the isExpiredX() methods in ExpiryHelper and also to isExpired() CacheEntry and its implementations. Agreed: ISPN-1459 Also reading this time information is a high overhead in some configurations, I'm wondering if we should make it possible to configure Infinispan to not track performance on each cache operation? Someone might prefer to estimate an average from multiple calls; I'm going to remove CacheMgmtInterceptor for my tests. CacheMgmtInterceptor doesn't seem to be enabled when JMX statistics are disabled, so I don't think you need to explicitly remove CacheMgmtInterceptor for your performance tests. Yes that's how I do it ;) What concerns me is that there is going to be a relatively strong overhead just by enabling statistics. Cheers Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Infinispan 5.1 + AS 7.1
Hey Dan, Any progress on this? I'd like to push the Infinispan/JGroups upgrade this week if possible, but this is currently blocking my progress. Thanks, Paul On Fri, 2011-10-14 at 15:36 +0300, Dan Berindei wrote: Hi Paul Yeah, I have changed replicated mode's state transfer to transfer state in parallel from all the existing nodes and I'm using a consistent hash to determine which keys should be pushed by each node. This one looks tricky, the transport reports 0 members but it should always have at least the current node as a member. I'll try to run with REPL_ASYNC to see if it changes anything, but I don't think it should. Cheers Dan On Fri, Oct 14, 2011 at 1:27 AM, Paul Ferraro paul.ferr...@redhat.com wrote: Hey all, I'm a bit stuck with the Infinispan 5.1 upgrade in AS 7.1. I've tried both with BETA1 and a SNAPSHOT build from today. When a deployment forces a cache to start, I see the following stacktrace: 14:06:07,584 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-8) MSC1: Failed to start service jboss.infinispan.web.repl: org.jboss.msc.service.StartException in service jboss.infinispan.web.repl: Failed to start service at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1780) [jboss-msc-1.0.1.GA.jar:1.0.1.GA] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [:1.6.0_23] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [:1.6.0_23] at java.lang.Thread.run(Thread.java:679) [:1.6.0_23] Caused by: org.infinispan.CacheException: Unable to invoke method private void org.infinispan.statetransfer.BaseStateTransferManagerImpl.start() throws java.lang.Exception on object at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205) at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:825) at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:624) at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:527) at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:177) at org.infinispan.CacheImpl.start(CacheImpl.java:462) at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:574) at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:453) at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:467) at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:84) at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:73) at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:73) at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1824) [jboss-msc-1.0.1.GA.jar:1.0.1.GA] at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1759) [jboss-msc-1.0.1.GA.jar:1.0.1.GA] ... 3 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [:1.6.0_23] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [:1.6.0_23] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_23] at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_23] at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203) ... 16 more Caused by: java.lang.IllegalArgumentException: Invalid cache list for consistent hash: [] at org.infinispan.distribution.ch.AbstractWheelConsistentHash.setCaches(AbstractWheelConsistentHash.java:96) at org.infinispan.distribution.ch.ConsistentHashHelper.createConsistentHash(ConsistentHashHelper.java:122) at org.infinispan.statetransfer.ReplicatedStateTransferManagerImpl.createConsistentHash(ReplicatedStateTransferManagerImpl.java:56) at org.infinispan.statetransfer.BaseStateTransferManagerImpl.start(BaseStateTransferManagerImpl.java:143) What's particularly puzzling is that this is a REPL_ASYNC cache with state transfer enabled. Why are we attempting to create a consistent hash here? Dan, is this related to your work? Thanks in advance, Paul ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev
Re: [infinispan-dev] Infinispan 5.1 + AS 7.1
I'm afraid I can't reproduce it here and I don't know of any situation where the transport could return 0 members. Could I get your code and debug it on my machine? Cheers Dan On Mon, Oct 17, 2011 at 5:25 PM, Paul Ferraro paul.ferr...@redhat.com wrote: Hey Dan, Any progress on this? I'd like to push the Infinispan/JGroups upgrade this week if possible, but this is currently blocking my progress. Thanks, Paul On Fri, 2011-10-14 at 15:36 +0300, Dan Berindei wrote: Hi Paul Yeah, I have changed replicated mode's state transfer to transfer state in parallel from all the existing nodes and I'm using a consistent hash to determine which keys should be pushed by each node. This one looks tricky, the transport reports 0 members but it should always have at least the current node as a member. I'll try to run with REPL_ASYNC to see if it changes anything, but I don't think it should. Cheers Dan On Fri, Oct 14, 2011 at 1:27 AM, Paul Ferraro paul.ferr...@redhat.com wrote: Hey all, I'm a bit stuck with the Infinispan 5.1 upgrade in AS 7.1. I've tried both with BETA1 and a SNAPSHOT build from today. When a deployment forces a cache to start, I see the following stacktrace: 14:06:07,584 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-8) MSC1: Failed to start service jboss.infinispan.web.repl: org.jboss.msc.service.StartException in service jboss.infinispan.web.repl: Failed to start service at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1780) [jboss-msc-1.0.1.GA.jar:1.0.1.GA] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [:1.6.0_23] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [:1.6.0_23] at java.lang.Thread.run(Thread.java:679) [:1.6.0_23] Caused by: org.infinispan.CacheException: Unable to invoke method private void org.infinispan.statetransfer.BaseStateTransferManagerImpl.start() throws java.lang.Exception on object at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205) at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:825) at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:624) at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:527) at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:177) at org.infinispan.CacheImpl.start(CacheImpl.java:462) at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:574) at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:453) at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:467) at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:84) at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:73) at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:73) at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1824) [jboss-msc-1.0.1.GA.jar:1.0.1.GA] at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1759) [jboss-msc-1.0.1.GA.jar:1.0.1.GA] ... 3 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [:1.6.0_23] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [:1.6.0_23] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_23] at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_23] at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203) ... 16 more Caused by: java.lang.IllegalArgumentException: Invalid cache list for consistent hash: [] at org.infinispan.distribution.ch.AbstractWheelConsistentHash.setCaches(AbstractWheelConsistentHash.java:96) at org.infinispan.distribution.ch.ConsistentHashHelper.createConsistentHash(ConsistentHashHelper.java:122) at org.infinispan.statetransfer.ReplicatedStateTransferManagerImpl.createConsistentHash(ReplicatedStateTransferManagerImpl.java:56) at org.infinispan.statetransfer.BaseStateTransferManagerImpl.start(BaseStateTransferManagerImpl.java:143) What's particularly puzzling is that this is a REPL_ASYNC cache with state transfer enabled. Why are we attempting to create a consistent hash here? Dan, is this related to your work? Thanks in
[infinispan-dev] strictly not returning expired values
I've noticed that every time we perform a get() operation (or any other retrieval) the value is NOT returned to the client if it's expired, which is going to receive a null instead. It looks like that these checks are in place to prevent a) a race condition with the eviction process b) to not return very old values from a very large wakeUpInterval between eviction activity c) to load from cacheLoaders Now for the first case, as a user of a caching library I would prefer to get the value instead. Expiry is in place only to prevent me from wasting memory, but this entry wasn't collected yet so why is Infinispan denying me access to the value I want to retrieve? We're introducing unneeded cache misses. Since even with the current code we can not provide guarantees that we won't ever return an expired entry (between the check and the actual return to user code we might be briefly suspended), I don't see why we should not make it clear that the expiry timeout is a best-effort time and relax our code checks a bit; there are many reasons to do so: 1) avoid fictitious cache misses. 2) Avoid the need to invoke a System.currentTimeMillis() for each retrieval operation - that's expensive! 3) Some operations like size() or values() now explicitly iterate the whole result set twice to make sure all expired entries are removed. This is unnecessary following the reasoning above. Regarding the other cases: b) We don't need to care imo. If people need more precision they should use a lower wakeUpInterval. c) We should keep this logic in CacheLoader, just because having an aggressive wakeUpInterval might be very expensive in this case. I'm asking as I've built a quick POC for ISPN-1459 but since now the current time needs to be passed to the entry, I end up needing to invoke the wall clock even for immortal cache entries making many other use cases slower: not happy with that, I'd rather speed up both use cases. Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] strictly not returning expired values
On Mon, Oct 17, 2011 at 7:51 PM, Sanne Grinovero sa...@infinispan.org wrote: I've noticed that every time we perform a get() operation (or any other retrieval) the value is NOT returned to the client if it's expired, which is going to receive a null instead. It looks like that these checks are in place to prevent a) a race condition with the eviction process b) to not return very old values from a very large wakeUpInterval between eviction activity c) to load from cacheLoaders Now for the first case, as a user of a caching library I would prefer to get the value instead. Expiry is in place only to prevent me from wasting memory, but this entry wasn't collected yet so why is Infinispan denying me access to the value I want to retrieve? We're introducing unneeded cache misses. Since even with the current code we can not provide guarantees that we won't ever return an expired entry (between the check and the actual return to user code we might be briefly suspended), I don't see why we should not make it clear that the expiry timeout is a best-effort time and relax our code checks a bit; there are many reasons to do so: 1) avoid fictitious cache misses. 2) Avoid the need to invoke a System.currentTimeMillis() for each retrieval operation - that's expensive! 3) Some operations like size() or values() now explicitly iterate the whole result set twice to make sure all expired entries are removed. This is unnecessary following the reasoning above. Regarding the other cases: b) We don't need to care imo. If people need more precision they should use a lower wakeUpInterval. I don't think this is an option, as a 5 seconds wakeUpInterval was deemed to short not long ago: https://source.jboss.org/viewrep/Infinispan/core/src/main/java/org/infinispan/config/Configuration.java?r1=9fe9a8cb6e49308b9e45f09b2f5324a7daefdee3r2=9cda361dc6df7bcecde95abc31fcbab11f734360 c) We should keep this logic in CacheLoader, just because having an aggressive wakeUpInterval might be very expensive in this case. I'm asking as I've built a quick POC for ISPN-1459 but since now the current time needs to be passed to the entry, I end up needing to invoke the wall clock even for immortal cache entries making many other use cases slower: not happy with that, I'd rather speed up both use cases. I guess you could pass in a Clock that lazily calls System.currentTimeMillis() and saves it for subsequent calls, but that will complicate things again. Dan Sanne ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] getting ISPN config as XML via JMX
Pete, Although https://issues.jboss.org/browse/ISPN-791 task is for 5.2 can we somehow ensure we are talking these requirements into account! Thank you, Vladimir ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] strictly not returning expired values
On 17 Oct 2011, at 18:21, Dan Berindei wrote: On Mon, Oct 17, 2011 at 7:51 PM, Sanne Grinovero sa...@infinispan.org wrote: I've noticed that every time we perform a get() operation (or any other retrieval) the value is NOT returned to the client if it's expired, which is going to receive a null instead. It looks like that these checks are in place to prevent a) a race condition with the eviction process b) to not return very old values from a very large wakeUpInterval between eviction activity c) to load from cacheLoaders Now for the first case, as a user of a caching library I would prefer to get the value instead. Expiry is in place only to prevent me from wasting memory, but this entry wasn't collected yet so why is Infinispan denying me access to the value I want to retrieve? We're introducing unneeded cache misses. Since even with the current code we can not provide guarantees that we won't ever return an expired entry (between the check and the actual return to user code we might be briefly suspended), I don't see why we should not make it clear that the expiry timeout is a best-effort time and relax our code checks a bit; there are many reasons to do so: 1) avoid fictitious cache misses. 2) Avoid the need to invoke a System.currentTimeMillis() for each retrieval operation - that's expensive! 3) Some operations like size() or values() now explicitly iterate the whole result set twice to make sure all expired entries are removed. This is unnecessary following the reasoning above. Regarding the other cases: b) We don't need to care imo. If people need more precision they should use a lower wakeUpInterval. I don't think this is an option, as a 5 seconds wakeUpInterval was deemed to short not long ago: https://source.jboss.org/viewrep/Infinispan/core/src/main/java/org/infinispan/config/Configuration.java?r1=9fe9a8cb6e49308b9e45f09b2f5324a7daefdee3r2=9cda361dc6df7bcecde95abc31fcbab11f734360 perhaps a configuration option disabled by default? c) We should keep this logic in CacheLoader, just because having an aggressive wakeUpInterval might be very expensive in this case. I'm asking as I've built a quick POC for ISPN-1459 but since now the current time needs to be passed to the entry, I end up needing to invoke the wall clock even for immortal cache entries making many other use cases slower: not happy with that, I'd rather speed up both use cases. I guess you could pass in a Clock that lazily calls System.currentTimeMillis() and saves it for subsequent calls, but that will complicate things again. Complicate by introducing an inaccuracy on the exact time? Code-wise seems reasonably simple... ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Time measurement and expiry
On 17 Oct 2011, at 14:13, Sanne Grinovero wrote: Very interesting, I knew that in Windows currentTimeMillis() basically just reads a volatile because I got bit by the 15 millisecond accuracy issue before, so I thought it would always be very fast. I had no idea on Linux it would have the same performance as nanoTime(). Indeed very nice! I miss the part where the article says nanoTime has the same performance on modern Linux as currentTimeMillis, are you sure? Even in Windows it seems to be a bit more expensive than just reading a volatile; Searching for currentTimeMillis() in the code base reveals it's being used for expiry of entries as well. If someone messes with system clocks data will expire; is it expected that entries expire at calendar time and not at a set lifetime? I would say if someone's messing with system clocks they already have a much bigger problem than their cached data expiring... Absolutely, but this is just an example. I'd expect that if I'm using it as a cache, the data will stay in it for a certain amount of relative time. It doesn't have to be a user changing the clock, for example there might be automated changes like daylight adjustments (summer/winter time) or relatively small updates via NTP to keep the clocks in sync.. I'd expect System.currentTimeMillis() to not be affected in any way by the daylight savings, as it returns: the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC. of course they might not affect the functionality significantly, +- 10/15 millis but I'm just pointing out that we might prefer to use the nanoTime source as our API suggests a period of entry validity in milliseconds, not an expiry in terms of calendar Date. the calendar bit (i.e. timesavings) isn't counted here. Besides, someone may intentionally change the system time to test the ISPN expiration algorithm :) I understand we need to use the absolute time to be able to transmit the value to other nodes, where the nanoTime of different nodes might not be comparable, but we could store one and use both: transmit the absolute value only to other nodes and refer to the other for accurate expiries. Other nodes receiving the absolute value will check for the remaining lifespane and store the corresponding nanoTime. ExpiryHelper itself will invoke the currentTimeMillis() operation internally, that means that it's going to be invoked at least once for each entry being accessed and might result in a lot of invocations when traversing several entries; I'm wondering if it shouldn't take a millisecond parameter to consider as current, so that this relatively expensive method can be invoked only once at the beginning of a batch of operations. + 1 for adding a currentTime parameter to the isExpiredX() methods in ExpiryHelper and also to isExpired() CacheEntry and its implementations. Agreed: ISPN-1459 Also reading this time information is a high overhead in some configurations, I'm wondering if we should make it possible to configure Infinispan to not track performance on each cache operation? Someone might prefer to estimate an average from multiple calls; I'm going to remove CacheMgmtInterceptor for my tests. I think it is relevant for local caches and not that relevant for clustered caches. CacheMgmtInterceptor doesn't seem to be enabled when JMX statistics are disabled, so I don't think you need to explicitly remove CacheMgmtInterceptor for your performance tests. Yes that's how I do it ;) What concerns me is that there is going to be a relatively strong overhead just by enabling statistics. Cheers Dan ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] FineGrainedAtomicMap - remaining items
On 17 Oct 2011, at 12:27, Sanne Grinovero wrote: On 17 October 2011 10:19, Mircea Markus mircea.mar...@jboss.com wrote: On 17 Oct 2011, at 09:13, Dan Berindei wrote: [cut] 2) This one is a bit technical. What should we do if tx1 deletes entire AtomicMap while tx2 updates entries in the same Map. Should we create a new Map and apply deltas to a new fresh map *or* simply discard delta changes because entire Map has been deleted? I would say not simply discard the changes, but throw an exception and roll back tx2 as well. That would work. Also if a FGAM method is called and that FGAM has been deleted/replaced then the FGAM method should throw an exception. I agree that if it's going to discard the changes then we should have the transaction rolled back, but are we sure if it makes sense to discard them? I'm not sure, but I tend to go this way because: - a AtomicMap within Cache is somehow similar to the way the Cache operates within a CacheManager: if a cache is stopped then all other transactions operating with the cache receive exceptions on calling Cache's method. - this is somehow counter-intitive and more difficult to implement than the previous approach: unless community specifically asks for this feature IMO we should stick with the simple approach. I think that for the use case of OGM both approaches are fine, simply because this scenario will not happen. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Long time no hear, or ISPN-78 continued
On 14 Oct 2011, at 21:36, Olaf Bergner wrote:
Hi Ales,
yes, that sounds reasonable.
+1. Guess that would be an overloaded version of the existing method?
In fact, I have been worrying about the user relinquishing control over how
Infinispan handles streaming large objects. One quick question, though: I
envisaged
OutputStream writeToKey(K key)
to be the standard way of streaming large objects into Infinispan. Doesn't
it fit your bill?
- Olaf
Am 14.10.11 22:29, schrieb Ales Justin:
@Olaf:
Looking at the API you proposed
public interface StreamingHandlerK {
void writeToKey(K key, InputStream largeObject);
OutputStream writeToKey(K key);
InputStream readFromKey(K key);
boolean removeKey(K key);
StreamingHandlerK withFlags(Flag... flags);
}
and what I actually need:
*
http://code.google.com/p/googleappengine/source/browse/trunk/java/src/main/com/google/appengine/api/files/FileService.java
-- see openRecordWriteChannel and openRecordReadChannel
Where the read has a way of jumping backforth in the channel,
and write can seal the written bytes with closeFinally.
*
http://code.google.com/p/googleappengine/source/browse/trunk/java/src/main/com/google/appengine/api/files/FileReadChannel.java
*
http://code.google.com/p/googleappengine/source/browse/trunk/java/src/main/com/google/appengine/api/files/FileWriteChannel.java
Hence I guess we could change it to
Closeable writeToKey(K key, InputStream largeObject); -- returning a handle
to seal off (making it immutable from then on)
InputStream readFromKey (K key, long position); -- also taking position into
consideration
Wdyt?
(unfortunately I also don't have the time to help with impl,
but I'm definitely willing to test it (since I need it anyway, right :-)))
-Ales
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
___
infinispan-dev mailing list
infinispan-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev
Re: [infinispan-dev] Separating FluentTypes into a separate file?
On 14 Oct 2011, at 17:51, Pete Muir wrote: In doing so I moved all the of the interfaces and classes to be top level. That is the stuff within FluentConfiguration? I wish Configuration would have the same fate as well: is always a PITA to browse/update subclasses in that monster. ___ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
