I did something similar at one point with a trading system. The map
was something like
{ticker {resting-orders [] pending-orders []}. The pending and resting
orders were changing often, but we weren't changing what tickers we
were trading very often. The entire map was a ref, and the resting and
pending orders were also refs. It worked well as long as the changes
were isolated to one ticker. e.g. an exchange acknowledgment would
cause an order to go from pending to resting, for one ticker - that
worked fine due to there being very little contention for those refs.
However, at some point the system changed and acknowledgment of one
order could cause changes to the orders for any given ticker, and
everything broke down.However, what we were doing (assoc, dissoc, update-in +, etc) was so fast that it was easy to solve by moving the entire state map to an agent. If you're doing simple operations, you might want to look at doing the same. Cheers, Jay On Fri, Apr 6, 2012 at 10:22 AM, Bryce <[email protected]> wrote: > I have a system that consists of a map, of the rough form {1 [1 2 3] 2 > [2 3 4]}. An update to the system chooses a couple random keys, looks > them up in the map, does some math with them, and writes two new > values. If one or both of the keys don't exist, they need to be > created and their values initialized. > > My thought was to represent them as (ref {key1 (ref val1) k2 (ref > v2)...}), which according to my current understanding of the STM only > "locks" the map as a whole if a new key is being added, and otherwise > should only "lock" the val being accessed/updated. Atoms on the vals > aren't really an option, since the updates need to see a consistent > state between them. > > However, it looks like I'm spending between 50-80% of my time in > clojure.lang.transaction.BlockAndBail, and the single-threaded version > takes ~25% less time than the concurrent version, even with just 2 > threads, and even after giving the system time to "warm up" so it > should no longer be creating significant numbers of new keys. > > I have a feeling that I'm grabbing the main ref on every read, and > causing all transactions to proceed serially. Can someone confirm my > suspicion, and is there good way to represent this kind of system in a > non-blocking manner? Finally, will having an atom, vs. a ref, guard > the overall map cause Bad Things to happen? > > My code for get-or-set is: > > (defn- get-or-set-nested-ref > "targ is of form (ref {key (ref val)}). > Check outside the transaction for index and deref. If it's not > there, > enter a transaction and check in a consistent context. If it's > there, > deref and return it; if not, initialize it to a ref to a default > value." > [targ index default-fn] > @(get @targ index > (dosync (if-let [r (get @targ index)] r > (-> (alter targ > assoc index (ref (default-fn))) > (get index)))))) > > The last wrinkle is that in the context of the update code, get-or-set- > nested-ref is always called inside a dosync block to begin with, so > the first deref always happens inside a transaction - this is why I > think an atom as the main guard might help, but I'm not sure how that > interacts with the STM as a whole or if there would be possibly > inconsistent edge cases. > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to [email protected] > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to [email protected] Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/clojure?hl=en
