hi , guys
I wrote the following codes to parse log files.
it's alright to parse small one.
But with big log files , i got the following error:
"OutOfMemoryError GC overhead limit exceeded clojure.core/line-seq
(core.clj:2679)"
(defn parse-file
""
[file]
(with-open [rdr (io/reader file)]
(println "Statistic result : " (parse-recur rdr))))
(defn parse-recur
""
[rdr]
(let [lines (line-seq rdr)
counts (count lines)]
(loop [len counts
res {}]
(if (zero? len)
res
(recur (dec len)
(update-res res (nth lines (- counts len))))))))
(defn update-res
""
[res line]
(let [params (string/split line #",")
id (if (> (count params) 1) (params 0) "0")]
(if (res id)
(update-in res [id] inc)
(assoc res id 1))))
How can fix this bug ?
and how to optimize this produce to run faster ?
Any suggestion will be appreciated~
BR
------------------------------------
刘家齐 （Jacky Liu）
手机：15201091195 邮箱：[email protected]
Skype：jacky_liu_1987 QQ：406229156
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
