Whoops duh. That was silly, far to early on the west coast. On Mar 23, 7:46 am, Per Vognsen <[email protected]> wrote: > It doesn't seem very accidental: the namespace binding for 'trees' is > retaining the head. You probably want to wrap it in a function: > > (defn trees [] > ...) > > -Per > > On Tue, Mar 23, 2010 at 9:40 PM, aria42 <[email protected]> wrote: > > Hi, > > I was experimenting with some code and I had an largish sequence I > > did a doseq over. The memory hit the ceiling, which you expect since > > even though the head isn't retained GC doesn't happen until you hit > > your memory limit (is there a way to change that). Once it hit the > > memory limit, 1.2 gigs in this case, the doseq slows to a halt. > > > On the other hand if rather than make a long lazy seq, I do it > > implicitly in the doseq itself (see FAST snippet below), the > > performance is great. Is there anyway to get good performance using a > > single lazy seq? Relevant snippets below. > > > Thanks, Aria > > > (defn lines > > "get lines from gz file" > > [#^String path] > > (-> path > > java.io.FileInputStream. > > java.util.zip.GZIPInputStream. > > java.io.InputStreamReader. > > java.io.BufferedReader. > > line-seq)) > > > ; the tree-from-str does some processing, but doesn't have state > > (def trees (for [l (lines "/usr/local/corpora//NANC/003.gz") > > :when (not (empty? l)) > > :let [[t _] (tree/tree-from-str l)]] > > t)) > > > ; SLOW: hits memory limit and becomes slow b/c of constant > > ; GC hits > > (doseq [t trees] > > (println (str t))) > > > ; FAST: low memory > > (doseq [l (lines "/usr/local/corpora//NANC/003.gz") > > :when (not (empty? l)) > > :let [[t _] (tree/tree-from-str l)]] > > (println (str t))) > > > -- > > You received this message because you are subscribed to the Google > > Groups "Clojure" group. > > To post to this group, send email to [email protected] > > Note that posts from new members are moderated - please be patient with > > your first post. > > To unsubscribe from this group, send email to > > [email protected] > > For more options, visit this group at > >http://groups.google.com/group/clojure?hl=en > > > To unsubscribe from this group, send email to > > clojure+unsubscribegooglegroups.com or reply to this email with the words > > "REMOVE ME" as the subject. > >
-- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to [email protected] Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
