I'm super new to Clojure, so apologies if I've missed something.
I'm trying to use an agent to manage IO for code that involves reading a
bunch of JSON files, doing some processing, then writing the contents to
separate files. I'm using pmap to speed things up, but I've run into some
memory overrun issues and other hassles. So now I'm trying to use an agent
to manage the IO, but I'm not quite sure how to do it. I'm trying to use a
watch function, but I'm not sure that's right:
...
(def str-agent (agent {}))
(defn watch-write [k i o n] (let [in-file (:f n) out-str (:s n)]
(do (println in-file)
(spit (get-fixed-name in-file) out-str))))
(defn update-file [in-file] (let [lines (parse-file in-file)]
(doall (map update-both lines))))
(defn process-file [in-file] (do (println (str in-file))
(send-off str-agent assoc
:f in-file
:s (string/join "\n" (update-file
in-file)))))
(defn process-files [] (dorun (pmap process-file valid-files)))
(defn -main [] (do (println "Hello, world!")
(add-watch str-agent :writer watch-write)
(process-files)))
Is this the way to handle this kind of IO issue, or should I create an atom
for each process, or what?
Thanks guys!
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
