Le 1 déc. 2009 à 03:52, Niclas Hedhman a écrit : > On Tue, Dec 1, 2009 at 1:27 AM, philippe van dyck <[email protected]> wrote: > >> I plan to try this on EC2 later, in order to validate the usability, hoping >> that a local network will remove the current performance problem. > > Well, has all the discussion so far been around having S3 accessed > from a "local" system? I can understand that adding Infinispan (or any > other LAN based cache) would greatly improve performance in such case, > due to latencies between your LAN and S3.
When it comes to increasing speed, any bus speed is improved by caching: network bus, memory bus,... But Infinispan is not about caching exclusively. It actually uses the JCache JSR to expose services, but here, caching is really considered "a way to access stuff stored (in a Map<K,V>)". Here are the features of Infinispan 4.0 (from http://java.dzone.com/articles/infinispan-data-grid-platform ) APIs Familiar Map-like API (org.infinispan.Cache extends java.util.concurrent.ConcurrentMap) Alternative JBoss Cache-compatible tree-like API RESTful API for remote access Support for non-JVM clients via RESTful API Operational modes Clustered and non-clustered (standalone) operation modes Clustered operational modes include invalidation, replication and distribution, making use of either synchronous and asynchronous network communications Distribution with optional L1 caching (“near-caching”) reduces network latency for frequent lookups, while maintaining the scalability of distribution JTA transaction support Infinispan is an XA resource that is compatible with any JTA-compliant transaction manager Write-through caching to pluggable CacheStores Can be configured to provide “warm starts” Ships with high-performance disk, JDBC and cloud storage based CacheStore implementations Support for custom implementations Also useful as an overflow space if memory is scarce Eviction and expiration FIFO and LRU based eviction policies Expiration of data based on lifespan and idle time Management JMX statistics and monitoring JOPR-based GUI management High-performance custom marshaling layer to provide fast, low-overhead serialization and deserialization > > As for XA transaction isolation, I am a bit skeptical, since IIUIC all > instances needs to participate in all transactions, and that doesn't > scale. Instead, what I think I have concluded is that Entity > Aggregates are the way to go, i.e. no changes to any entities other > than via the Aggregate. The "problem" is "how to do it". Ffor me, the real question is "how is it done elsewhere?", since I don't plan to reinvent the wheel ;-) I tried to do it myself, queuing updates with multiple workers in order to improve performance, but the XA is the right approach (and the standard one - industrial level). It is expensive, preparing all updates in transactions, but simple to understand. And, usually, when it is simple and short to develop, it is to maintain. > > One way is to direct all requests regarding a particular Aggregate to > the same node, but theoretically that node may be available to some > clients and not others, causing fail-over to send the requests to the > wrong node. Solution here is to use distribution with consistent hashing (http://en.wikipedia.org/wiki/Consistent_hashing) you know where your info (and the replicated ones) are exactly. Infinispan implements exactly that in "distribution" mode (design document here http://www.jboss.org/community/wiki/DIST-distributedcachemode). > > "Communication channels" between nodes don't really work, since one > node may have already "be done" with an aggregate when another node > starts working on it, without those changes to be visible at the > second node. You could use a fined-grain approach of locking the modified entries until changes are applied (eager-locking / see below) ? I don't really know exactly how Infinispan does it... And I don't care as long as my transaction boundaries are respected. > > Another option would be to not allow any changes, only additions > (provided that S3 can handle atomic additions), and resolve the state > in higher level code. For some systems there could be an explosion of > storage, which could be solved by having a kind of GC chewing along in > the background. > > Another option would be to ignore it all... > > > Cheez, not sure what to make of it all... What about letting the clustered-XA manage all this ? If I am able to start a transaction before I apply changes (in the ES) and then commit it afterwards... It is solved ! Let's look at the transaction configuration options (http://infinispan.sourceforge.net/4.0/apidocs/config.html#ce_default_transaction) transaction Defines transactional (JTA) characteristics of the cache. Attribute Type Default Description syncRollbackPhase boolean false If true, the cluster-wide rollback phase in two-phase commit (2PC) transactions will be synchronous, so Infinispan will wait for responses from all nodes to which the rollback was sent. Otherwise, the rollback phase will be asynchronous. Keeping it as false improves performance of 2PC transactions. syncCommitPhase boolean false If true, the cluster-wide commit phase in two-phase commit (2PC) transactions will be synchronous, so Infinispan will wait for responses from all nodes to which the commit was sent. Otherwise, the commit phase will be asynchronous. Keeping it as false improves performance of 2PC transactions, since any remote failures are trapped during the prepare phase anyway and appropriate rollbacks are issued. useEagerLocking boolean false When eager locking is set to true, whenever a lock on key is required, cluster-wide locks will be acquired at the same time as local, in-VM locks. If false, cluster-wide locks are only acquired during the prepare phase in the two-phase commit protocol. Note that this setting is implicit and so it's indiscriminate. Alternatively, you can keep this setting as false and instead do eager locking explicitly on a per invocation basis by calling AdvancedCache.lock(Object). transactionManagerLookupClass string org.infinispan.transaction.lookup.GenericTransactionManagerLookup Fully qualified class name of a class that looks up a reference to a {...@link javax.transaction.TransactionManager}. The default provided is capable of locating the default TransactionManager in most popular Java EE systems, using a JNDI lookup. Does it fill your needs ? (surely fills mine) phil
_______________________________________________ qi4j-dev mailing list [email protected] http://lists.ops4j.org/mailman/listinfo/qi4j-dev
