Re: Ring startup processing?
Ken Wesson kwess...@gmail.com writes:
Retries? We were discussing an atom here, not a ref. A dosync
transaction may be retried. The obvious implementation for swap! would
be (locking inner_value (set! inner_value.contents (apply fun
inner_value.contents))) (warning: pseudocode, I assume this stuff
would actually be implemented in Atom.java or someplace like that in
Java rather than in Clojure).
Sorry, I didn't explain very well. I basically said that it's not
implemented with locks, rather than how it *is* implemented. I should
have pointed at the Clojure documentation:
Internally, swap! reads the current value, applies the function to
it, and attempts to compare-and-set it in. Since another thread may
have changed the value in the intervening time, it may have to
retry, and does so in a spin loop. The net effect is that the value
will always be the result of the application of the supplied
function to a current value, atomically. However, because the
function might be called multiple times, it must be free of side
effects.
-- http://clojure.org/atoms
The actual implementation from clojure/lang/Atom.java is quite concise
too:
public Object swap(IFn f) throws Exception{
for(; ;) {
Object v = deref();
Object newv = f.invoke(v);
validate(newv);
if(state.compareAndSet(v, newv)) {
notifyWatches(v, newv);
return newv;
}
}
}
(let [owned (Object.)]
(swap! the-atom #(if (= % sentinel) owned %))
(if (= @the-atom owned)
(reset! the-atom (do-once-fn))
(loop []
(if (= (.getClass @the-atom) Object)
(do
(Thread/sleep 10)
(recur))
(@the-atom)
The interesting bit here deals with the case that the atom holds
someone else's owned object instead of an actual result. The loop
sleeps 10ms and checks repeatedly until the do-once-fn (running in
some other thread) has completed and the atom's final value been
assigned. It does assume that the result of do-once-fn will not be an
unspecialized java.lang.Object.
The locking version is probably cleaner than the above, but when no
return value is needed, the code further above is short and especially
concise; both are lock-free (given that swap! and reset! are
lock-free, which, if they perform retries, is presumably the case).
You've actually implemented a polling sleeplock in terms of an atom.
Using the language's own lock has the benefit that long sleeping threads
do not have to keep polling and using CPU time, the OS scheduler will
wake them up when the lock is released.
Locking is semantically correct for this situation. Imagine the webapp
case, you start up the webapp and want to initialize something on
the first request. What happens if 3 requests come in simultaneously
and all hit your initialize? You want two of the threads to block and
wait while one of them does the initialization. That's locking (mutual
exclusion).
Atoms do not lock. If multiple threads enter swap! simultaneously, they
*will* each execute f. One will win and the others will retry.
Eh. Why this implementation? To optimize for the common case that
there isn't any contention?
CAS is a lower-level primitive, locks and semaphores can be
implemented using a CAS as you demonstrated. ;-)
But yes, there are potential performance benefits, depending on
contention. For example, if f does some calculation and ends up often
returning the value unchanged then multiple threads can do that
simultaneously without blocking.
Also context switching (due to blocking on a lock) is quite expensive.
However, if there's really high contention and a lot of retrying,
locking may be better from a performance perspective as it allows the
blocked thread to sleep and other tasks to run.
Finally, the atom is a safer concurrency primitive, it's impossible to
deadlock (unless you implement a lock with one, of course).
Further reading about CAS:
http://en.wikipedia.org/wiki/Compare-and-swap
http://en.wikipedia.org/wiki/Non-blocking_algorithm
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
On Wed, Dec 1, 2010 at 4:50 AM, Alex Osborne a...@meshy.org wrote:
Ken Wesson kwess...@gmail.com writes:
Retries? We were discussing an atom here, not a ref. A dosync
transaction may be retried. The obvious implementation for swap! would
be (locking inner_value (set! inner_value.contents (apply fun
inner_value.contents))) (warning: pseudocode, I assume this stuff
would actually be implemented in Atom.java or someplace like that in
Java rather than in Clojure).
Sorry, I didn't explain very well. I basically said that it's not
implemented with locks, rather than how it *is* implemented. I should
have pointed at the Clojure documentation:
Internally, swap! reads the current value, applies the function to
it, and attempts to compare-and-set it in. Since another thread may
have changed the value in the intervening time, it may have to
retry, and does so in a spin loop. The net effect is that the value
will always be the result of the application of the supplied
function to a current value, atomically. However, because the
function might be called multiple times, it must be free of side
effects.
-- http://clojure.org/atoms
Interesting.
The actual implementation from clojure/lang/Atom.java is quite concise
too:
public Object swap(IFn f) throws Exception{
for(; ;) {
Object v = deref();
Object newv = f.invoke(v);
validate(newv);
if(state.compareAndSet(v, newv)) {
notifyWatches(v, newv);
return newv;
}
}
}
(let [owned (Object.)]
(swap! the-atom #(if (= % sentinel) owned %))
(if (= @the-atom owned)
(reset! the-atom (do-once-fn))
(loop []
(if (= (.getClass @the-atom) Object)
(do
(Thread/sleep 10)
(recur))
(@the-atom)
The interesting bit here deals with the case that the atom holds
someone else's owned object instead of an actual result. The loop
sleeps 10ms and checks repeatedly until the do-once-fn (running in
some other thread) has completed and the atom's final value been
assigned. It does assume that the result of do-once-fn will not be an
unspecialized java.lang.Object.
The locking version is probably cleaner than the above, but when no
return value is needed, the code further above is short and especially
concise; both are lock-free (given that swap! and reset! are
lock-free, which, if they perform retries, is presumably the case).
You've actually implemented a polling sleeplock in terms of an atom.
Using the language's own lock has the benefit that long sleeping threads
do not have to keep polling and using CPU time, the OS scheduler will
wake them up when the lock is released.
It looks an awful lot like swap! itself is implemented with a polling
sleeplock instead of using the language's own lock. :)
Locking is semantically correct for this situation. Imagine the webapp
case, you start up the webapp and want to initialize something on
the first request. What happens if 3 requests come in simultaneously
and all hit your initialize? You want two of the threads to block and
wait while one of them does the initialization. That's locking (mutual
exclusion).
Most of the various proposals here would have that effect. I expect
they may differ as to their efficiency, though, and perhaps different
ones would be more efficient under a) high contention and b) low
contention. I expect that there's been research done on such matters
by theorists, but I'm not sure to what extent that's informed the
design of either Java or Clojure...
Atoms do not lock. If multiple threads enter swap! simultaneously, they
*will* each execute f. One will win and the others will retry.
Eh. Why this implementation? To optimize for the common case that
there isn't any contention?
CAS is a lower-level primitive, locks and semaphores can be
implemented using a CAS as you demonstrated. ;-)
But yes, there are potential performance benefits, depending on
contention. For example, if f does some calculation and ends up often
returning the value unchanged then multiple threads can do that
simultaneously without blocking.
I expect the compare it does uses .equals rather than ==?
Finally, the atom is a safer concurrency primitive, it's impossible to
deadlock (unless you implement a lock with one, of course).
It seems to me that it is still possible to livelock with this sort
of thing (and with dosync as well) if the system ends up spending a
very large percentage of time in retries. On the other hand, with a
deadlock it would just plain hang and avoiding deadlocks can be very
complicated; whereas every time several transactions try to go through
at the same time at least *one* of them should succeed, so the system
will always be making progress and won't completely hang, and
adjusting things like the granularity of your refs can reduce
contention while at the same time you don't have to worry about
Re: Ring startup processing?
Ken Wesson kwess...@gmail.com writes:
It looks an awful lot like swap! itself is implemented with a polling
sleeplock instead of using the language's own lock. :)
I suppose it's similar to a spinlock simply because it keeps retrying,
like a spinlock keeps retrying to acquire. Very different semantics
though: an atom (or any subset of the swap! code) is not by itself a
drop in replacement for a general-purpose lock.
Locking is semantically correct for this situation. Imagine the webapp
case, you start up the webapp and want to initialize something on
the first request. What happens if 3 requests come in simultaneously
and all hit your initialize? You want two of the threads to block and
wait while one of them does the initialization. That's locking (mutual
exclusion).
Most of the various proposals here would have that effect. I expect
they may differ as to their efficiency, though, and perhaps different
ones would be more efficient under a) high contention and b) low
contention. I expect that there's been research done on such matters
by theorists, but I'm not sure to what extent that's informed the
design of either Java or Clojure...
Well all the ones that are semantically correct (don't have the retry
problems) enforce mutual exclusion on the function as that's part of the
definition of defonce as a problem.
It also depends on how long the init function takes, if it's something
that's going to take a few seconds to do, may as well block. If it's
quick then it's cheaper to spin. Often language/OS lock takes this into
account in some fashion, like spinning for a while and eventually
sleeping, or spinning until the thread that holds the lock is context
switched.
But yes, there are potential performance benefits, depending on
contention. For example, if f does some calculation and ends up often
returning the value unchanged then multiple threads can do that
simultaneously without blocking.
I expect the compare it does uses .equals rather than ==?
Not even that, it's a CPU instruction so it'll be identical?. As just
pointed out the other thread though stuff like this is true due to
structural sharing, which would help in practice:
(let [m {:x 1, :y 2}]
(identical? m (assoc m :x 1)))
Finally, the atom is a safer concurrency primitive, it's impossible to
deadlock (unless you implement a lock with one, of course).
It seems to me that it is still possible to livelock with this sort
of thing (and with dosync as well) if the system ends up spending a
very large percentage of time in retries. On the other hand, with a
deadlock it would just plain hang and avoiding deadlocks can be very
complicated; whereas every time several transactions try to go through
at the same time at least *one* of them should succeed, so the system
will always be making progress and won't completely hang, and
adjusting things like the granularity of your refs can reduce
contention while at the same time you don't have to worry about
locking order and deterministic symmetry-breaking and all that stuff
to try to keep deadlocks from happening.
Yep it could certainly suffer from starvation and lead to livelock.
Imagine there's two threads hitting an atom really hard. One of them
sends an update function that's really quick, the other sends an update
function that takes say, 2 seconds. The thread with the slow update
function is going to be starved and never get a chance to execute. If
the fast thread depends on some change the slow thread makes then that's
a livelock situation.
Generally I guess though if there's a lot of contention you're probably
not getting a good deal of benefit from concurrency anyway so you
probably need to try a different approach. I'd certainly want to avoid
an atom update function that took any significant amount of time.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
On Wed, Dec 1, 2010 at 8:38 AM, Alex Osborne a...@meshy.org wrote: I expect the compare it does uses .equals rather than ==? Not even that, it's a CPU instruction so it'll be identical?. The identical? predicate and the Java == operator do the same thing. Also, what's a CPU instruction? Everything we've been discussing is in Java and/or Clojure, so it'd be a JVM instruction. I think the .compareAndSet method call in the swap method in the Java code for Clojure atoms was the specific case under discussion. The snippet you posted didn't make it clear what method that actually was; it was invoked on an object held by a reference named state but the variable's declaration wasn't in the snippet so its type is unknown to me. My guess would be something in the java.util.concurrent package. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
Why not use good old runonce from lancet? (defn runonce Create a function that will only run once. All other invocations return the first calculated value. The function can have side effects. Returns a [has-run-predicate, reset-fn, once-fn] [function] (let [sentinel (Object.) result (atom sentinel) reset-fn (fn [] (reset! result sentinel) nil) has-run? #(not= @result sentinel)] [has-run? reset-fn (fn [ args] (locking sentinel (if (= @result sentinel) (reset! result (function)) @result)))])) --Robert McIntyre On Sun, Nov 28, 2010 at 11:28 PM, Stuart Campbell stu...@harto.org wrote: Ah, OK. We just use Tomcat 6 for everything here :) On 29 November 2010 12:16, lprefonta...@softaddicts.ca wrote: Yep but some app servers have bugs with this interface (GlassFish 2, jetty 6.1, tomcat 5.5, ...) and the interface is not called appropriately. There are less problems reported with the load-on-startup flag. Of course if you know your app server's behavior regarding this feature, it might not be a problem. I got caught once so I prefer a safer mechanism, we are not always using the same app server here. Might work in dev and not in prod... oups... Luc P. Stuart Campbell stu...@harto.org wrote .. On 28 November 2010 16:51, lprefonta...@softaddicts.ca wrote: We use a dedicated servlet for every web app to make sure all prerequisites are met. Since it's loaded first, we can find problems by looking at a single piece of the log files just after the container messages announcing that it's loading the app. There's also ServletContextListener ( http://download.oracle.com/javaee/5/api/javax/servlet/ServletContextListener.html), which is configured in web.xml too. Regards, Stuart -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en Luc P. The rabid Muppet -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
For my part, I need stuff done prior to anything being done including routing a request and I need to access the servlet context. I'll explain this in a couple of days, it's related to a deployment problem. I am in it right now. Robert McIntyre r...@mit.edu wrote .. Why not use good old runonce from lancet? (defn runonce Create a function that will only run once. All other invocations return the first calculated value. The function can have side effects. Returns a [has-run-predicate, reset-fn, once-fn] [function] (let [sentinel (Object.) result (atom sentinel) reset-fn (fn [] (reset! result sentinel) nil) has-run? #(not= @result sentinel)] [has-run? reset-fn (fn [ args] (locking sentinel (if (= @result sentinel) (reset! result (function)) @result)))])) --Robert McIntyre On Sun, Nov 28, 2010 at 11:28 PM, Stuart Campbell stu...@harto.org wrote: Ah, OK. We just use Tomcat 6 for everything here :) On 29 November 2010 12:16, lprefonta...@softaddicts.ca wrote: Yep but some app servers have bugs with this interface (GlassFish 2, jetty 6.1, tomcat 5.5, ...) and the interface is not called appropriately. There are less problems reported with the load-on-startup flag. Of course if you know your app server's behavior regarding this feature, it might not be a problem. I got caught once so I prefer a safer mechanism, we are not always using the same app server here. Might work in dev and not in prod... oups... Luc P. Stuart Campbell stu...@harto.org wrote .. On 28 November 2010 16:51, lprefonta...@softaddicts.ca wrote: We use a dedicated servlet for every web app to make sure all prerequisites are met. Since it's loaded first, we can find problems by looking at a single piece of the log files just after the container messages announcing that it's loading the app. There's also ServletContextListener ( http://download.oracle.com/javaee/5/api/javax/servlet/ServletContextListener.html), which is configured in web.xml too. Regards, Stuart -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en Luc P. The rabid Muppet -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en Luc P. The rabid Muppet -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
On Tue, Nov 30, 2010 at 5:27 PM, Robert McIntyre r...@mit.edu wrote: Why not use good old runonce from lancet? (defn runonce Create a function that will only run once. All other invocations return the first calculated value. The function can have side effects. Returns a [has-run-predicate, reset-fn, once-fn] [function] (let [sentinel (Object.) result (atom sentinel) reset-fn (fn [] (reset! result sentinel) nil) has-run? #(not= @result sentinel)] [has-run? reset-fn (fn [ args] (locking sentinel (if (= @result sentinel) (reset! result (function)) @result)))])) Eww. Why (locking ...)? Why not just (swap! result #(if (= % sentinel) (function) %))? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
On Tue, Nov 30, 2010 at 6:08 PM, lprefonta...@softaddicts.ca wrote: For my part, I need stuff done prior to anything being done including routing a request and I need to access the servlet context. Given any of the many singleton/runonce implementations that have been posted, you just need to make sure that everything that falls under your anything being done including routing a request calls the runonce before doing anything else, and the runonce code will be run just before the first instance of anything being done including routing a request. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
Ken Wesson kwess...@gmail.com writes: On Tue, Nov 30, 2010 at 5:27 PM, Robert McIntyre r...@mit.edu wrote: (defn runonce Create a function that will only run once. All other invocations return the first calculated value. The function can have side effects. Returns a [has-run-predicate, reset-fn, once-fn] [function] (let [sentinel (Object.) result (atom sentinel) reset-fn (fn [] (reset! result sentinel) nil) has-run? #(not= @result sentinel)] [has-run? reset-fn (fn [ args] (locking sentinel (if (= @result sentinel) (reset! result (function)) @result)))])) Eww. Why (locking ...)? Why not just (swap! result #(if (= % sentinel) (function) %))? From the doc string: The function can have side effects. Sometimes a lock is the right tool for the job. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
Then I would need to place the call within Ring... I tried to do what I need in middle ware but it's simply impossible. What I need to init is related to how routes are dispatched by Ring and needs to be done outside of Ring. I need the servlet context, I have to get the container call me with the servlet context somehow before getting into Ring. Using a load-on-startup I get the container to call my servlet directly before anything else in the web apps. Luc P. Ken Wesson kwess...@gmail.com wrote .. On Tue, Nov 30, 2010 at 6:08 PM, lprefonta...@softaddicts.ca wrote: For my part, I need stuff done prior to anything being done including routing a request and I need to access the servlet context. Given any of the many singleton/runonce implementations that have been posted, you just need to make sure that everything that falls under your anything being done including routing a request calls the runonce before doing anything else, and the runonce code will be run just before the first instance of anything being done including routing a request. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en Luc P. The rabid Muppet -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
On Tue, Nov 30, 2010 at 7:23 PM, Alex Osborne a...@meshy.org wrote: Eww. Why (locking ...)? Why not just (swap! result #(if (= % sentinel) (function) %))? From the doc string: The function can have side effects. Sometimes a lock is the right tool for the job. I put a println in a swap! function once and the world didn't end. It just seems to me that the code that was posted more or less reinvents a (weak) form of swap! using reset! and locking. The requirements for runonce seem clear enough: the thing should run only once, so, you should check if it has already been run and if not run it and the check-and-maybe-run needs to be done atomically. And the original code even uses an atom to represent the has-it-run-yet? state. And swap!s on a single atom are supposed to be atomic, any given swap! definitely happening before or after any other given swap! on the same atom. So... -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
Ken Wesson kwess...@gmail.com writes: I put a println in a swap! function once and the world didn't end. For a println retries are usually harmless, it just means you'll occasionally get doubled output. But it's not a good thing for a general-purpose runonce fn whose entire reason for existence is to ensure something is called only once. ;-) And swap!s on a single atom are supposed to be atomic, any given swap! definitely happening before or after any other given swap! on the same atom. The guarantee of atomicity is *dependent* on the function not having side-effects. If it has side-effects then there is no guarantee. The side-effects thing is not just about cleaner code or something, it is entirely *necessary* for correct operation. Atoms do not lock. If multiple threads enter swap! simultaneously, they *will* each execute f. One will win and the others will retry. Yes, this happening is rare unless there's a lot of contention on the atom, but that just makes the resulting heisenbugs much harder track down. Clojure makes concurrency safer, but it's not magic, it only works if you play by its rules. The rules are there for good reasons, make sure you understand why they are there before you ignore them. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
On Wed, Dec 1, 2010 at 12:55 AM, Alex Osborne a...@meshy.org wrote: Ken Wesson kwess...@gmail.com writes: I put a println in a swap! function once and the world didn't end. For a println retries are usually harmless Retries? We were discussing an atom here, not a ref. A dosync transaction may be retried. The obvious implementation for swap! would be (locking inner_value (set! inner_value.contents (apply fun inner_value.contents))) (warning: pseudocode, I assume this stuff would actually be implemented in Atom.java or someplace like that in Java rather than in Clojure). it just means you'll occasionally get doubled output. But it's not a good thing for a general-purpose runonce fn whose entire reason for existence is to ensure something is called only once. ;-) If swap! does sometimes run its argument twice then you'd need something slightly more sophisticated, like: (let [owned (Object.)] (swap! the-atom #(if (= % sentinel) owned %)) (when (= @the-atom owned) (do-once-fn) (reset! the-atom nil))) Here, the swap changes the atom to the unique object owned if and only if it's still holding the sentinel value. If afterward the atom's value is owned, we've claimed ownership of the atom and we call do-once-fn. If anyone else beat us to the punch the atom will hold either a different owned object or else nil. This works if we're not actually interested in the return value of the function, just in making it execute exactly once. If we need the result: (let [owned (Object.)] (swap! the-atom #(if (= % sentinel) owned %)) (if (= @the-atom owned) (reset! the-atom (do-once-fn)) (loop [] (if (= (.getClass @the-atom) Object) (do (Thread/sleep 10) (recur)) (@the-atom) The interesting bit here deals with the case that the atom holds someone else's owned object instead of an actual result. The loop sleeps 10ms and checks repeatedly until the do-once-fn (running in some other thread) has completed and the atom's final value been assigned. It does assume that the result of do-once-fn will not be an unspecialized java.lang.Object. The locking version is probably cleaner than the above, but when no return value is needed, the code further above is short and especially concise; both are lock-free (given that swap! and reset! are lock-free, which, if they perform retries, is presumably the case). The guarantee of atomicity is *dependent* on the function not having side-effects. How so? (Other than that it would probably be bad to reference the atom explicitly inside of the swap! function.) Atoms do not lock. If multiple threads enter swap! simultaneously, they *will* each execute f. One will win and the others will retry. Eh. Why this implementation? To optimize for the common case that there isn't any contention? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
On 28 November 2010 16:51, lprefonta...@softaddicts.ca wrote: We use a dedicated servlet for every web app to make sure all prerequisites are met. Since it's loaded first, we can find problems by looking at a single piece of the log files just after the container messages announcing that it's loading the app. There's also ServletContextListener ( http://download.oracle.com/javaee/5/api/javax/servlet/ServletContextListener.html), which is configured in web.xml too. Regards, Stuart -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
Yep but some app servers have bugs with this interface (GlassFish 2, jetty 6.1, tomcat 5.5, ...) and the interface is not called appropriately. There are less problems reported with the load-on-startup flag. Of course if you know your app server's behavior regarding this feature, it might not be a problem. I got caught once so I prefer a safer mechanism, we are not always using the same app server here. Might work in dev and not in prod... oups... Luc P. Stuart Campbell stu...@harto.org wrote .. On 28 November 2010 16:51, lprefonta...@softaddicts.ca wrote: We use a dedicated servlet for every web app to make sure all prerequisites are met. Since it's loaded first, we can find problems by looking at a single piece of the log files just after the container messages announcing that it's loading the app. There's also ServletContextListener ( http://download.oracle.com/javaee/5/api/javax/servlet/ServletContextListener.html), which is configured in web.xml too. Regards, Stuart -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en Luc P. The rabid Muppet -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
Ah, OK. We just use Tomcat 6 for everything here :) On 29 November 2010 12:16, lprefonta...@softaddicts.ca wrote: Yep but some app servers have bugs with this interface (GlassFish 2, jetty 6.1, tomcat 5.5, ...) and the interface is not called appropriately. There are less problems reported with the load-on-startup flag. Of course if you know your app server's behavior regarding this feature, it might not be a problem. I got caught once so I prefer a safer mechanism, we are not always using the same app server here. Might work in dev and not in prod... oups... Luc P. Stuart Campbell stu...@harto.org wrote .. On 28 November 2010 16:51, lprefonta...@softaddicts.ca wrote: We use a dedicated servlet for every web app to make sure all prerequisites are met. Since it's loaded first, we can find problems by looking at a single piece of the log files just after the container messages announcing that it's loading the app. There's also ServletContextListener ( http://download.oracle.com/javaee/5/api/javax/servlet/ServletContextListener.html ), which is configured in web.xml too. Regards, Stuart -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en Luc P. The rabid Muppet -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Ring startup processing?
My simple web app (http://blog.mired.org/2010/11/x10-controller-in-clojure.html) has some stuff that needs to happen just once (in this case, opening the serial port). It's not clear how to get this to happen using ring. If I do it inside my ring handler, then it gets run on every request, and I have to check to make sure it's not run multiple times. If I run it outside the handler, then it gets run when I do lein uberwar, which is simply wrong. When the deployment platform activates the war would seem to be the right time to run this (war load time?). So maybe this is a question that depends on the deployment platform, or war? However, a quick google search didn't turn up anything that looked interesting. Anyone got suggestions on how to set up code to be run when Jetty (or tomcat, or ...)? thanks, mike -- Mike Meyer m...@mired.org http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O ascii ribbon campaign - stop html mail - www.asciiribbon.org -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
On Sat, Nov 27, 2010 at 7:50 PM, Mike Meyer mwm-keyword-googlegroups.620...@mired.org wrote: My simple web app (http://blog.mired.org/2010/11/x10-controller-in-clojure.html) has some stuff that needs to happen just once (in this case, opening the serial port). It's not clear how to get this to happen using ring. If I do it inside my ring handler, then it gets run on every request, and I have to check to make sure it's not run multiple times. If I run it outside the handler, then it gets run when I do lein uberwar, which is simply wrong. When the deployment platform activates the war would seem to be the right time to run this (war load time?). So maybe this is a question that depends on the deployment platform, or war? However, a quick google search didn't turn up anything that looked interesting. Anyone got suggestions on how to set up code to be run when Jetty (or tomcat, or ...)? If it were an ordinary application, -main would be the obvious place (or some module-init function eventually called from there). In this case, though, I think your best bet may actually be a Java lazily-initialized singleton that you access from Clojure, if performance is an issue, or else the Clojure equivalent: (defmacro defsingleton [name init-sexp] `(def ~name (let [singleton-uninitialized# (Object.) a# (atom singleton-uninitialized#)] (fn [] (let [x# @a#] (if (= x# singleton-uninitialized#) (do (reset! a# ~init-sexp) @a#) x#)) user= (defsingleton foo (do (println foo initialized) 7)) #'user/foo user= (foo) foo initialized 7 user= (foo) 7 You can also abuse futures: user= (def foo (future (do (println foo initialized) 7))) #'user/foo user= @foo foo initialized 7 user= @foo 7 or delay/force, or even lazy-seq: user= (def foo (drop 1 (take 2 (iterate (fn [_] (do (println foo initialized) 7)) nil #'user/foo user= (first foo) foo initialized 7 user= (first foo) 7 -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
Hi, Normally this is done use an initialization servlet. You need a separate class inheriting javax.servlet.http.HttpServlet and a entry in your web.xml file to get it executed once at application startup. Look at load-on-startup here: http://www.caucho.com/resin-3.0/servlet/servlet.xtp#load-on-startup I'm two inches from implementing this in our medical record viewer which uses Compojure and Ring, I have the same issue. I will no have it done before next Tuesday however, I have some higher priority stuff to deliver first. Hope it helps you Mike Meyer mwm-keyword-googlegroups.620...@mired.org wrote .. My simple web app (http://blog.mired.org/2010/11/x10-controller-in-clojure.html) has some stuff that needs to happen just once (in this case, opening the serial port). It's not clear how to get this to happen using ring. If I do it inside my ring handler, then it gets run on every request, and I have to check to make sure it's not run multiple times. If I run it outside the handler, then it gets run when I do lein uberwar, which is simply wrong. When the deployment platform activates the war would seem to be the right time to run this (war load time?). So maybe this is a question that depends on the deployment platform, or war? However, a quick google search didn't turn up anything that looked interesting. Anyone got suggestions on how to set up code to be run when Jetty (or tomcat, or ...)? thanks, mike -- Mike Meyer m...@mired.org http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O ascii ribbon campaign - stop html mail - www.asciiribbon.org -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en Luc P. The rabid Muppet -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
On Sat, Nov 27, 2010 at 9:19 PM, Ken Wesson kwess...@gmail.com wrote: On Sat, Nov 27, 2010 at 7:50 PM, Mike Meyer mwm-keyword-googlegroups.620...@mired.org wrote: My simple web app (http://blog.mired.org/2010/11/x10-controller-in-clojure.html) has some stuff that needs to happen just once (in this case, opening the serial port). It's not clear how to get this to happen using ring. If I do it inside my ring handler, then it gets run on every request, and I have to check to make sure it's not run multiple times. If I run it outside the handler, then it gets run when I do lein uberwar, which is simply wrong. When the deployment platform activates the war would seem to be the right time to run this (war load time?). So maybe this is a question that depends on the deployment platform, or war? However, a quick google search didn't turn up anything that looked interesting. Anyone got suggestions on how to set up code to be run when Jetty (or tomcat, or ...)? If it were an ordinary application, -main would be the obvious place (or some module-init function eventually called from there). In this case, though, I think your best bet may actually be a Java lazily-initialized singleton that you access from Clojure, if performance is an issue, or else the Clojure equivalent: (defmacro defsingleton [name init-sexp] `(def ~name (let [singleton-uninitialized# (Object.) a# (atom singleton-uninitialized#)] (fn [] (let [x# @a#] (if (= x# singleton-uninitialized#) (do (reset! a# ~init-sexp) @a#) x#)) user= (defsingleton foo (do (println foo initialized) 7)) #'user/foo user= (foo) foo initialized 7 user= (foo) 7 You can also abuse futures: user= (def foo (future (do (println foo initialized) 7))) #'user/foo user= @foo foo initialized 7 user= @foo 7 or delay/force, or even lazy-seq: user= (def foo (drop 1 (take 2 (iterate (fn [_] (do (println foo initialized) 7)) nil #'user/foo user= (first foo) foo initialized 7 user= (first foo) 7 Or memoize: user= (defn foo [] (do (println foo initialized) 7)) #'user/foo user= (def foo (memoize foo)) #'user/foo user= (foo) foo initialized 7 user= (foo) 7 which actually is fairly close to what the defsingleton macro does. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
Mike Meyer mwm-keyword-googlegroups.620...@mired.org writes: My simple web app (http://blog.mired.org/2010/11/x10-controller-in-clojure.html) has some stuff that needs to happen just once (in this case, opening the serial port). It's not clear how to get this to happen using ring. If I do it inside my ring handler, then it gets run on every request, and I have to check to make sure it's not run multiple times. I would just open it lazily on the first request. That gives you a nice place to gracefully retry on error as well, instead of being dead until the entire servlet container is restarted. As a sysadmin I really hate apps that can get stuck in a error state and need restarting. ;-) For example you could be using a USB to Serial adapter, which somebody might have temporarily removed and plugged in later. Similarly if you're opening a database connection on startup and there's a network dropout, or perhaps your app got started before the database server did... If I run it outside the handler, then it gets run when I do lein uberwar, which is simply wrong. If you really want to do it at load time you can avoid executing something at compile time by wrapping it in this: (when-not *compile-files* ...) -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
The servlet interface includes the init method for this exact purpose. In java, this would be used by subclassing one of the abstract servlet classes, and filling in the init method with whatever initialization you need. Or by implementing the servlet interface directly. From the javadoc, the init method Initializes the servlet. The method is called once, automatically, by the servlet engine when it loads the servlet. It is guaranteed to finish before any service requests are accepted. I find this approach more straightforward than having an entire separate servlet for initialization. I was hoping that initialization of this kind would make it into the ring spec, but there wasn't much support. See the thread below for more details: http://groups.google.com/group/ring-clojure/browse_thread/thread/ec5a30b5bb4ec823# Rob On Nov 27, 4:50 pm, Mike Meyer mwm-keyword-googlegroups. 620...@mired.org wrote: My simple web app (http://blog.mired.org/2010/11/x10-controller-in-clojure.html) has some stuff that needs to happen just once (in this case, opening the serial port). It's not clear how to get this to happen using ring. If I do it inside my ring handler, then it gets run on every request, and I have to check to make sure it's not run multiple times. If I run it outside the handler, then it gets run when I do lein uberwar, which is simply wrong. When the deployment platform activates the war would seem to be the right time to run this (war load time?). So maybe this is a question that depends on the deployment platform, or war? However, a quick google search didn't turn up anything that looked interesting. Anyone got suggestions on how to set up code to be run when Jetty (or tomcat, or ...)? thanks, mike -- Mike Meyer m...@mired.org http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O ascii ribbon campaign - stop html mail -www.asciiribbon.org -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ring startup processing?
From http://docstore.mik.ua/orelly/java-ent/servlet/ch03_03.htm - Just like applets, servlets can define init() and destroy() methods. A servlet's init(ServletConfig) method is called by the server immediately after the server constructs the servlet's instance. Depending on the server and its configuration, this can be at any of these times: * When the server starts * When the servlet is first requested, just before the service() method is invoked * At the request of the server administrator In any case, init() is guaranteed to be called before the servlet handles its first request. -- We use a dedicated servlet for every web app to make sure all prerequisites are met. Since it's loaded first, we can find problems by looking at a single piece of the log files just after the container messages announcing that it's loading the app. We also avoid variable load times depending on the container's behavior and setup. Of course Mike does not need to do that if he has a only few inits related to a single servlet. Luc P. Rob Lachlan robertlach...@gmail.com wrote .. The servlet interface includes the init method for this exact purpose. In java, this would be used by subclassing one of the abstract servlet classes, and filling in the init method with whatever initialization you need. Or by implementing the servlet interface directly. From the javadoc, the init method Initializes the servlet. The method is called once, automatically, by the servlet engine when it loads the servlet. It is guaranteed to finish before any service requests are accepted. I find this approach more straightforward than having an entire separate servlet for initialization. I was hoping that initialization of this kind would make it into the ring spec, but there wasn't much support. See the thread below for more details: http://groups.google.com/group/ring-clojure/browse_thread/thread/ec5a30b5bb4ec823# Rob On Nov 27, 4:50 pm, Mike Meyer mwm-keyword-googlegroups. 620...@mired.org wrote: My simple web app (http://blog.mired.org/2010/11/x10-controller-in-clojure.html) has some stuff that needs to happen just once (in this case, opening the serial port). It's not clear how to get this to happen using ring. If I do it inside my ring handler, then it gets run on every request, and I have to check to make sure it's not run multiple times. If I run it outside the handler, then it gets run when I do lein uberwar, which is simply wrong. When the deployment platform activates the war would seem to be the right time to run this (war load time?). So maybe this is a question that depends on the deployment platform, or war? However, a quick google search didn't turn up anything that looked interesting. Anyone got suggestions on how to set up code to be run when Jetty (or tomcat, or ...)? thanks, mike -- Mike Meyer m...@mired.org http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O ascii ribbon campaign - stop html mail -www.asciiribbon.org -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en Luc P. The rabid Muppet -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
