Re: io/writer + map/recur
The way you wrote your loop it will only ever run once which means you could take it out. 1 res = take from queue 2 if res nil? terminate 3 if res recur with result from with-open (always nil) 4 back to 2 and terminate Not sure if this is intended but the way it is written it does not make sense to loop. Also remember that .take will block if nothing is available meaning you are writing essentially an infinite loop unless you create some other kind of exit condition. Also if you just want to append a string to a file you can use spit [1] (spit f some-string :append true) Other than that, never use something lazy with IO unless you remember to doall. HTH, /thomas http://grimoire.arrdem.com/1.6.0/clojure.core/spit/ On Friday, October 31, 2014 9:08:01 PM UTC+1, Sam Raker wrote: I'm writing some stuff to interact with the Twitter API. I want to be able to write tweets (as JSON) to a file, so I can, e.g., test things without connecting to the API. I've proxied the LinkedBlockingQueue that Twitter's HBC library uses to use an agent, so ideally I want to be able to write the contents of the agent AND the LBQ. Here's what I have right now: (defmulti write-tweets (fn [q f] (class q))) (defmethod write-tweets clojure.lang.Agent [a f] (with-open [w (clojure.java.io/writer f :append true)] (.write w (apply str (interpose \n @a) (defmethod write-tweets java.util.concurrent.LinkedBlockingQueue [lbq f] (loop [res (.take lbq)] (if res (recur (with-open [w (clojure.java.io/writer f :append true)] (.write w (str (generate-string res) \n))) My first implementation of this used `(map #(.write w %) @a)` and had the `recur` within the `with-open` block. Unfortunately, at least with the agent part, I ran into an error about the file being closed when I tried to write to it. I assumed `with-open` kept the file open within the block, but maybe I'm missing something? I'm worried about the performance of either creating a potentially super-huge string in memory for the agent method (twitter returns pretty sizable JSON blobs) or repeatedly opening/closing a file for the LBQ method (I realize I could collapse this into one problem by taking everything out of the LBQ and putting it into an agent, but that's not really a solution...) Does `writer` auto-close the file after it's done? Is there some better way of handling this kind of situation? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
io/writer + map/recur
I'm writing some stuff to interact with the Twitter API. I want to be able to write tweets (as JSON) to a file, so I can, e.g., test things without connecting to the API. I've proxied the LinkedBlockingQueue that Twitter's HBC library uses to use an agent, so ideally I want to be able to write the contents of the agent AND the LBQ. Here's what I have right now: (defmulti write-tweets (fn [q f] (class q))) (defmethod write-tweets clojure.lang.Agent [a f] (with-open [w (clojure.java.io/writer f :append true)] (.write w (apply str (interpose \n @a) (defmethod write-tweets java.util.concurrent.LinkedBlockingQueue [lbq f] (loop [res (.take lbq)] (if res (recur (with-open [w (clojure.java.io/writer f :append true)] (.write w (str (generate-string res) \n))) My first implementation of this used `(map #(.write w %) @a)` and had the `recur` within the `with-open` block. Unfortunately, at least with the agent part, I ran into an error about the file being closed when I tried to write to it. I assumed `with-open` kept the file open within the block, but maybe I'm missing something? I'm worried about the performance of either creating a potentially super-huge string in memory for the agent method (twitter returns pretty sizable JSON blobs) or repeatedly opening/closing a file for the LBQ method (I realize I could collapse this into one problem by taking everything out of the LBQ and putting it into an agent, but that's not really a solution...) Does `writer` auto-close the file after it's done? Is there some better way of handling this kind of situation? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: io/writer + map/recur
Your first implementation probably failed because map is lazy, so none of the '.write' calls happened during the with-open block. Try doseq. On Friday, October 31, 2014 1:08:01 PM UTC-7, Sam Raker wrote: I'm writing some stuff to interact with the Twitter API. I want to be able to write tweets (as JSON) to a file, so I can, e.g., test things without connecting to the API. I've proxied the LinkedBlockingQueue that Twitter's HBC library uses to use an agent, so ideally I want to be able to write the contents of the agent AND the LBQ. Here's what I have right now: (defmulti write-tweets (fn [q f] (class q))) (defmethod write-tweets clojure.lang.Agent [a f] (with-open [w (clojure.java.io/writer f :append true)] (.write w (apply str (interpose \n @a) (defmethod write-tweets java.util.concurrent.LinkedBlockingQueue [lbq f] (loop [res (.take lbq)] (if res (recur (with-open [w (clojure.java.io/writer f :append true)] (.write w (str (generate-string res) \n))) My first implementation of this used `(map #(.write w %) @a)` and had the `recur` within the `with-open` block. Unfortunately, at least with the agent part, I ran into an error about the file being closed when I tried to write to it. I assumed `with-open` kept the file open within the block, but maybe I'm missing something? I'm worried about the performance of either creating a potentially super-huge string in memory for the agent method (twitter returns pretty sizable JSON blobs) or repeatedly opening/closing a file for the LBQ method (I realize I could collapse this into one problem by taking everything out of the LBQ and putting it into an agent, but that's not really a solution...) Does `writer` auto-close the file after it's done? Is there some better way of handling this kind of situation? -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
