The volatile! is needed for the case where a transducer is only used by one thread at a time, but the thread executing the transducer may change from one call to the next. This happens fairly often with core.async. If you used a non-atomic, non-volatile mutable field, the JVM would be free to perform several optimizations (like keeping the local in a CPU register) that would cause the value to not properly propagate to other threads in the case of a context switch. Using volatile! tells the JVM to flush all writes to this field by the time the next memory barrier rolls around. It also tells the JVM to make sure it doesn't cache the reads to this field across memory barriers.
It's a tricky subject, and one that's really hard to test, and frankly I probably got some of the specifics wrong in that last paragraph, but that's the general idea of why transducers use volatile!. Timothy On Sun, Apr 9, 2017 at 12:49 AM, Alexander Gunnarson < alexandergunnar...@gmail.com> wrote: > EDIT: Transducers are actually not safe in `fold` contexts as I thought: > > (let [f (fn [i x] (println (str "i " i " " (Thread/currentThread))) > (flush) x) > r-map-indexed #(r/folder %2 (map-indexed %1))] > (->> [6 7 8 9 10] > (r-map-indexed f) > (r/fold 1 (fn ([] (vector)) ([x] x) ([a b] (into a b))) conj))) > > Produces: > > i 0 Thread[ForkJoinPool-1-worker-2,5,main] > i 2 Thread[ForkJoinPool-1-worker-1,5,main] > i 3 Thread[ForkJoinPool-1-worker-1,5,main] > i 4 Thread[ForkJoinPool-1-worker-1,5,main] > i 1 Thread[ForkJoinPool-1-worker-3,5,main] > > So you would have to be careful to e.g. create different `map-indexed` > transducers for single-threaded (e.g. `unsynchronized-mutable` box) and > multi-threaded (e.g. `atom` box) contexts. > > On Sunday, April 9, 2017 at 2:10:06 AM UTC-4, Alexander Gunnarson wrote: >> >> I was wondering the same thing, shintotomoe. This thread >> <https://groups.google.com/forum/#!topic/clojure/CjxK7xEsOKQ> talks >> about it as well. I think it's safe to assume that since `ArrayList` uses >> unsynchronized mutability internally (a quick review of the GrepCode >> entry for `ArrayList` confirms this >> <http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/java/util/ArrayList.java>), >> then we can rest assured that a `volatile` box as opposed to a totally >> unsynchronized mutable variable is unnecessary, even in the context of >> `fold`. After all, `reduce` (and by extension, `transduce`) is only ever >> going to be single-threaded unless the data structure in question >> unexpectedly implements a multithreaded reduce, which should never happen >> (and if it does, you likely have bigger problems). To be honest, I'm not >> sure why `volatile` is used in transducers instead of e.g. an >> `unsynchronized-mutable` box. There may be a good reason, but I'm not >> seeing it immediately. I'd love to learn. >> >> On Thursday, January 1, 2015 at 10:36:13 PM UTC-5, shintotomoe wrote: >>> >>> Thank you for the superfast response. I take it implementing your own >>> transducing process is not something you would usually do unless you have a >>> unique use case (my own use case being already implemented by chan taking a >>> transducer). >>> >>> Still, I was wondering about the use of ArrayList in partition-all, and >>> the recommendation to use volatiles inside transducers, which seem at odds. >>> It seems we don't need to implement transducers in a thread-safe way. Is >>> that correct? >>> >>> On Friday, January 2, 2015 12:58:51 PM UTC+11, tbc++ wrote: >>>> >>>> Core.async already has pipeline, pipeline-blocking and pipeline-async. >>>> In addition you can use a transducer inside a channel. Use those instead. >>>> >>>> Timothy >>>> >>>> On Thu, Jan 1, 2015 at 6:55 PM, shintotomoe <tste...@atlassian.com> >>>> wrote: >>>> >>>>> I was wondering how to apply a transducer inside a go process. What >>>>> I've so far is the following >>>>> >>>>> (defn pipe-xform [in-ch out-ch xform] >>>>> (let [tr >>>>> (let [tr (xform (fn >>>>> ([result] result) >>>>> ([result input] (conj! result input))))] >>>>> (fn >>>>> ([] (locking tr (persistent! (tr (transient []))))) >>>>> ([input] (locking tr (persistent! (tr (transient []) input >>>>> ))))))] >>>>> (go-loop [] >>>>> (if-some [value (<! in-ch)] >>>>> (do (doseq [v (tr value)] >>>>> (>! out-ch v)) >>>>> (recur)) >>>>> (do (doseq [v (tr)] >>>>> (>! out-ch v)) >>>>> (close! out-ch)))))) >>>>> >>>>> Now, I could just do >>>>> >>>>> (let [xf-ch (chan 1 xform)] >>>>> (pipe in-ch xf-ch) >>>>> (pipe xf-ch out-ch) >>>>> >>>>> >>>>> Or just redesign my code so that I can create in-ch or out-ch with the >>>>> transducer directly, but I was wondering whether there are any obvious >>>>> flaws with the pipe-xform implementation. >>>>> >>>>> In particular, I was wondering about the locking. At first I was under >>>>> the impression that transducers are thread-safe due to the use of >>>>> volatiles, but looking at the partition-all transducer, which uses an >>>>> ArrayList for its state, It appears that's not the case. >>>>> >>>>> Any feedback greatly appreciated. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To post to this group, send email to clo...@googlegroups.com >>>>> Note that posts from new members are moderated - please be patient >>>>> with your first post. >>>>> To unsubscribe from this group, send email to >>>>> clojure+u...@googlegroups.com >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/clojure?hl=en >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to clojure+u...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> “One of the main causes of the fall of the Roman Empire was >>>> that–lacking zero–they had no way to indicate successful termination of >>>> their C programs.” >>>> (Robert Firth) >>>> >>> -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth) -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
