Yeah, rewrite that to not hold the head of lines. In particular, get rid
of counts entirely, get rid of (nth lines ...), and use something like
(loop [lines (seq lines)
res {}]
(if lines
(recur (next lines) (update-res res (first lines)))
res))
as your loop.
On Tue, Nov 12, 2013 at 3:19 AM, Jiaqi Liu liujiaq...@gmail.com wrote:
hi , guys
I wrote the following codes to parse log files.
it's alright to parse small one.
But with big log files , i got the following error:
OutOfMemoryError GC overhead limit exceeded clojure.core/line-seq
(core.clj:2679)
(defn parse-file
[file]
(with-open [rdr (io/reader file)]
(println Statistic result : (parse-recur rdr
(defn parse-recur
[rdr]
(let [lines (line-seq rdr)
counts (count lines)]
(loop [len counts
res {}]
(if (zero? len)
res
(recur (dec len)
(update-res res (nth lines (- counts len
(defn update-res
[res line]
(let [params (string/split line #,)
id (if ( (count params) 1) (params 0) 0)]
(if (res id)
(update-in res [id] inc)
(assoc res id 1
How can fix this bug ?
and how to optimize this produce to run faster ?
Any suggestion will be appreciated~
BR
刘家齐 （Jacky Liu）
手机：15201091195邮箱：liujiaq...@gmail.com
Skype：jacky_liu_1987 QQ：406229156
--
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an
email to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
