Yea, that would work. I don't think the (fn) should be defined anonymously though, because I could see it being useful on its own. Give it a name!
Thanks, Stu On Mon, Mar 16, 2009 at 6:53 PM, Brian Doyle <brianpdo...@gmail.com> wrote: > > > On Mon, Mar 16, 2009 at 4:21 PM, Stu Hood <stuh...@gmail.com> wrote: > >> If you write your CSV -> XML processing as a function, you could pmap ( >> http://clojure.org/api#pmap) that function across the list of input >> files. pmap will transparently create the threads as needed, and it will >> probably be enough to saturate your disk. >> >> Thanks, >> Stu >> > > Stu, I just want to make sure I understand what you are saying here. > Would it look something like this: > > (pmap (fn [input-file] > ; open input for reading > ; open output for writing > ; read line > ; process line > ; write line > ) > ["input-file1", "input-file2", "input-file3", ....]) > > Thanks. > > >> >> >> On Mon, Mar 16, 2009 at 5:56 PM, Brian Doyle <brianpdo...@gmail.com>wrote: >> >>> I've been using Clojure for about 6 months now and really like it. I am >>> somewhat new to multi-threading >>> and using any of the parallel features in Clojure though. I have a >>> situation where I need to convert >>> 7 files from CSV to XML. Each one of these files is about 180MB apiece >>> in size. I have dual core machine >>> with 2GB of RAM and would like some advice on the best strategy for >>> processing these files in a way that >>> really utilizes both cores and my memory to really speed up the >>> processing. I'm sure this isn't the best >>> way, but I've only come up with starting up two threads at first, having >>> each thread open up a file, >>> call line-seq on that file, write out the XML for each line and then go >>> to the next file when it's complete. Any >>> advice would be great. Thanks. >>> >>> >>> >> >> >> > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
