Re: Ordering of messages to agents
On Wed, Jul 7, 2010 at 5:46 PM, Ryan Waters ryan.or...@gmail.com wrote: On Wed, Jul 7, 2010 at 4:32 PM, Meikel Brandmeyer m...@kotka.de wrote: Hi, Am 07.07.2010 um 23:11 schrieb Ryan Waters: (send a init-function) (send a f)) It's not guaranteed the init-function will complete before f. I doubt that. Since only one action can be active at a time and the ordering is preserved, init-function will be complete before f by definition: f is not started until init-function is complete. (And btw: no other g can happen between init-function and f, because the agent is not yet known to the outside world) Sincerely Meikel My apologies - I read the code wrong and should explain more of what I see. When the if is true, the ordering of init-function and f is guaranteed so init-function will complete before f. However, on any subsequent calls to example, there are no language guarantees that keep the if from evaluating to false, calling (send @my-ref f), but only after the previous call to (send a init-function). Stated another way, it's technically possible the agent calls won't complete before a subsequent call to example. I don't care if the agent calls complete before the next call to example. I care that the init function is run in the agent before any of the f functions are called. A more specific question is: is the enqueing of messages to agents part of committing the transaction, or not? The messages aren't enqueued unless the transaction commits, that much is clear. But consider the following scenario: Thread #1 comes in and calls example for the first time. So this fires off a transaction that initializes the reference, and sends two messages to the agent- initialize and f. Then, the very nanosecond that thread #1's transaction commits, the operating system decides to swap thread #1 out to disk, or preempt it, or whatever, and thread #2 comes and calls example. Now, since the first transaction has completed, thread #2 takes the else branch, and just sends it's f to the agent. Now, if the messages are sent as part of committing the transaction, there is no problem- the nanosecond the transaction commits, the messages are already safely enqueued on their way to the agent (even if they haven't been executed yet), and thread #2's f gets enqueued behind them. But if the messages are sent after the transaction commits, then there is a problem. In this case, it's possible that thread #2 can sneak in between the time that thread #1 commits and thread #1 enqueues it's messages for the agent, allowing thread #2's f to execute before the initialization function can execute. The fact that the window of vulnerability for this race condition may be very small doesn't help at all, I've both seen and had to track down race conditions where the window of vulnerability was mere nanoseconds- the words not fun come to mind. It's sounding like the answer is no, I can't depend upon this. Brian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ordering of messages to agents
2010/7/8 Brian Hurt bhur...@gmail.com On Wed, Jul 7, 2010 at 5:46 PM, Ryan Waters ryan.or...@gmail.com wrote: On Wed, Jul 7, 2010 at 4:32 PM, Meikel Brandmeyer m...@kotka.de wrote: Hi, Am 07.07.2010 um 23:11 schrieb Ryan Waters: (send a init-function) (send a f)) It's not guaranteed the init-function will complete before f. I doubt that. Since only one action can be active at a time and the ordering is preserved, init-function will be complete before f by definition: f is not started until init-function is complete. (And btw: no other g can happen between init-function and f, because the agent is not yet known to the outside world) Sincerely Meikel My apologies - I read the code wrong and should explain more of what I see. When the if is true, the ordering of init-function and f is guaranteed so init-function will complete before f. However, on any subsequent calls to example, there are no language guarantees that keep the if from evaluating to false, calling (send @my-ref f), but only after the previous call to (send a init-function). Stated another way, it's technically possible the agent calls won't complete before a subsequent call to example. I don't care if the agent calls complete before the next call to example. I care that the init function is run in the agent before any of the f functions are called. A more specific question is: is the enqueing of messages to agents part of committing the transaction, or not? The messages aren't enqueued unless the transaction commits, that much is clear. But consider the following scenario: Thread #1 comes in and calls example for the first time. So this fires off a transaction that initializes the reference, and sends two messages to the agent- initialize and f. Then, the very nanosecond that thread #1's transaction commits, the operating system decides to swap thread #1 out to disk, or preempt it, or whatever, and thread #2 comes and calls example. Now, since the first transaction has completed, thread #2 takes the else branch, and just sends it's f to the agent. Now, if the messages are sent as part of committing the transaction, there is no problem- the nanosecond the transaction commits, the messages are already safely enqueued on their way to the agent (even if they haven't been executed yet), and thread #2's f gets enqueued behind them. But if the messages are sent after the transaction commits, then there is a problem. In this case, it's possible that thread #2 can sneak in between the time that thread #1 commits and thread #1 enqueues it's messages for the agent, allowing thread #2's f to execute before the initialization function can execute. The fact that the window of vulnerability for this race condition may be very small doesn't help at all, I've both seen and had to track down race conditions where the window of vulnerability was mere nanoseconds- the words not fun come to mind. It's sounding like the answer is no, I can't depend upon this. Seems to me that your fear is justified: http://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LockingTransaction.java#L358 -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ordering of messages to agents
A better API for this particular use suggested itself to me: simply allow the state to be nil, and require the functions that are sent initialize the state if necessary. Code like: (defn my_f [ state ] (let [ state (or state (init-function)) ] ... In this case, it doesn't matter which f is the first f executed, it initializes the state, and everything works. Brian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ordering of messages to agents
Hi, On Jul 8, 4:53 pm, Brian Hurt bhur...@gmail.com wrote: A better API for this particular use suggested itself to me: simply allow the state to be nil, and require the functions that are sent initialize the state if necessary. Code like: (defn my_f [ state ] (let [ state (or state (init-function)) ] ... In this case, it doesn't matter which f is the first f executed, it initializes the state, and everything works. Instead of putting this in every f, you decorate the fs. (defn init-and-call [state initf f args] (let [state (if (nil? state) (initf) state)] (apply f state args))) (send a init-and-call init-function f) If your init is constant (which it is probably not), you can also use fnil: (send a (fnil f init)) Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Ordering of messages to agents
I'm wondering if the following pattern is safe or not. I'm in a transaction, and I want to create an agent and then send it an initializing message (the message function isn't transaction-safe, so I don't want to run it in the transaction). So I want to do something like: (def my-ref (ref nil)) (defn example [ f ] (dosync (if (nil? @my-ref) (let [ a (agent nil) ] (ref-set my-ref a) (send a init-function) (send a f)) (send @my-ref f (actually, what I want to do is rather more complicated than this, but this demonstrates the problem). So, my question is: is it guaranteed that init-function will be received by the agent before any f functions are received? Brian -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ordering of messages to agents
Hi, Am 07.07.2010 um 22:46 schrieb Brian Hurt: I'm wondering if the following pattern is safe or not. I'm in a transaction, and I want to create an agent and then send it an initializing message (the message function isn't transaction-safe, so I don't want to run it in the transaction). So I want to do something like: (def my-ref (ref nil)) (defn example [ f ] (dosync (if (nil? @my-ref) (let [ a (agent nil) ] (ref-set my-ref a) (send a init-function) (send a f)) (send @my-ref f (actually, what I want to do is rather more complicated than this, but this demonstrates the problem). So, my question is: is it guaranteed that init-function will be received by the agent before any f functions are received? The ordering of sends will be preserved, although they will be scheduled after the transaction commits. It might be due to the simplified example, but... Why don't you just initialise your agent when you create it? (def my-ref (agent (init-function))) (defn example [f] (send @my-ref f)) Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ordering of messages to agents
On Wed, Jul 7, 2010 at 5:04 PM, Meikel Brandmeyer m...@kotka.de wrote: Hi, Am 07.07.2010 um 22:46 schrieb Brian Hurt: I'm wondering if the following pattern is safe or not. I'm in a transaction, and I want to create an agent and then send it an initializing message (the message function isn't transaction-safe, so I don't want to run it in the transaction). So I want to do something like: (def my-ref (ref nil)) (defn example [ f ] (dosync (if (nil? @my-ref) (let [ a (agent nil) ] (ref-set my-ref a) (send a init-function) (send a f)) (send @my-ref f (actually, what I want to do is rather more complicated than this, but this demonstrates the problem). So, my question is: is it guaranteed that init-function will be received by the agent before any f functions are received? The ordering of sends will be preserved, although they will be scheduled after the transaction commits. It might be due to the simplified example, but... Why don't you just initialise your agent when you create it? This is due to the simplified example. I'm actually creating a map of keys to soft references to agents (and removing the references and keys from the map when the agents are reclaimed), which allows me to generate agents only when I need them, and have them freed when they are no longer needed. (def my-ref (agent (init-function))) (defn example [f] (send @my-ref f)) Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.comclojure%2bunsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ordering of messages to agents
On Wed, Jul 7, 2010 at 3:46 PM, Brian Hurt bhur...@gmail.com wrote: I'm wondering if the following pattern is safe or not. I'm in a transaction, and I want to create an agent and then send it an initializing message (the message function isn't transaction-safe, so I don't want to run it in the transaction). So I want to do something like: (def my-ref (ref nil)) (defn example [ f ] (dosync (if (nil? @my-ref) (let [ a (agent nil) ] (ref-set my-ref a) (send a init-function) (send a f)) (send @my-ref f (actually, what I want to do is rather more complicated than this, but this demonstrates the problem). So, my question is: is it guaranteed that init-function will be received by the agent before any f functions are received? Brian -- It's not guaranteed the init-function will complete before f. At the end of the Concurrency (Chapter 6) of Programming Clojure, Stuart Halloway outlines a rationale and a way to create functions that only run once. Related code can be found here - see 'runonce': http://github.com/stuarthalloway/lancet/blob/master/lancet.clj I hope that helps with what you're trying to do! Ryan -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ordering of messages to agents
Hi, Am 07.07.2010 um 23:11 schrieb Ryan Waters: (send a init-function) (send a f)) It's not guaranteed the init-function will complete before f. I doubt that. Since only one action can be active at a time and the ordering is preserved, init-function will be complete before f by definition: f is not started until init-function is complete. (And btw: no other g can happen between init-function and f, because the agent is not yet known to the outside world) Sincerely Meikel -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
Re: Ordering of messages to agents
On Wed, Jul 7, 2010 at 4:32 PM, Meikel Brandmeyer m...@kotka.de wrote: Hi, Am 07.07.2010 um 23:11 schrieb Ryan Waters: (send a init-function) (send a f)) It's not guaranteed the init-function will complete before f. I doubt that. Since only one action can be active at a time and the ordering is preserved, init-function will be complete before f by definition: f is not started until init-function is complete. (And btw: no other g can happen between init-function and f, because the agent is not yet known to the outside world) Sincerely Meikel My apologies - I read the code wrong and should explain more of what I see. When the if is true, the ordering of init-function and f is guaranteed so init-function will complete before f. However, on any subsequent calls to example, there are no language guarantees that keep the if from evaluating to false, calling (send @my-ref f), but only after the previous call to (send a init-function). Stated another way, it's technically possible the agent calls won't complete before a subsequent call to example. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
