Ok, I'll digest this in the next couple of weeks and have a look more closely at Clojure code and Terracotta's capabilities.
I have a non-linear thinking process so it helps to have some early goals. I'll create some prototype setup here to help identify issues and run some basic tests. Thank you, Luc On Sat, 2008-10-18 at 08:50 -0400, Rich Hickey wrote: > On Fri, Oct 17, 2008 at 8:01 PM, Luc Prefontaine > <[EMAIL PROTECTED]> wrote: > > I am not very far from tackling this issue. In our bus messaging system, we > > are using Terracotta with some Java components > > and it's a matter of weeks before we start to investigate how we can bridge > > Clojure and Terracotta. > > > > A customer asked us about some new functionality today and I see a need to > > fill the Terracotta/Clojure gap > > somehow. > > > > I'll comeback toward the end of November with some proposal. > > > > Any comments Rich about how you would see this integration and what Clojure > > semantics you would like to share through Terracotta ? > > I might enlarge the scope beyond what we need in our system even if not all > > the pieces are delivered in the very short term. > > > > There are lots of potential synergies. I think that one key benefit of > using Clojure is that the immutable data structures can be shared, yet > read outside of locks. As you know, Terracotta requires shared objects > to be accessed under a lock. However, once the object has been > propagated, it need not be acceessed under a lock iff it is immutable. > This was one of the first things I verified with Terracotta. > > So, for instance, you can do the normal Terracotta cache thing and put > Clojure's persistent data structures in a ConcurrentHashMap shared > root. Once you pull one out of the map, you can use it henceforth > without locking - a huge benefit IMO. Plus, since the data structures > share structure, updates are also efficient. A current hitch, which I > am looking to enhance anyway, is that some of the data structures do > lazy hash creation with volatile caches. In proc this is no problem, > nor out of proc since the hash value is a pure function of the > immutable structure value, but I think Terracotta may not be happy > with the volatile members. I have already on my todo list moving to > incrementally calculated hash values (had to wait for the unification > of hash logic with java.util's, now done). > > Of course, this leaves you with ConcurrentHashMap's last-one-in-wins, > no coordinated activity semantics. When that's not good enough, you'll > want to try STM refs + transactions. Here too, I think a lot is > possible. As I've said, I once had this working, but haven't tracked > whether or not all mechanisms I am using are supported by Terracotta. > Underneath the STM are all standard Java concurrency things - > reentrant locks, wait/notify, java.util.concurrent.atomic stuff etc. > To the extent these are supported, it should function correctly right > away. > > That said, there's one aspect of the STM I think could be tweaked for > a clustering situation. The only thing that is shared between all > transactions is a single CAS for the timestamps. In-proc, that works > fine until you get to very heavy micro-transactions, which are a bad > idea anyway. On a Terracotta cluster, that CAS will turn into a shared > lock, I think, which is much heavier. What you really want is a > central getAndIncrement server, since this capability can be done > completely with a server-side CAS with no client coordination. > Terracotta, being API-free, will want to maintain the illusion that > each JVM has its own CAS. If I had time to do Terracotta/Clojure work, > I'd probably investigate abstracting out the STM's timestamp generator > to allow for a timestamp server for clustered situations. > > Once you have that, you can make normal locality-of-reference based > architectural decisions for the refs, and get concurrency that is > scalable strictly on the transactions' degree of ref overlap. > > Rich > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
