On Fri, Dec 10, 2010 at 23:36, Alex Osborne <a...@meshy.org> wrote: > Michael Ossareh <ossa...@gmail.com> writes: > > > There are 54874 companies in the companies var. The OOM tends to take > place when there are 1000 or so companies to process. > > > > What is likely to be causing this issue? > > I replied on IRC but just recapping here. > > Yes, and you nailed the issue. Thanks so much!
> I think you've probably been bitten by the way Java strings share data. > The gotcha is described here better than I can: > > http://fishbowl.pastiche.org/2005/04/27/the_string_memory_gotcha/ > > One thing you can do to try to debug this sort of thing is to add > -XX:+HeapDumpOnOutOfMemoryError to the JVM command-line. Then when you > run into an OOM this will happen: > > java.lang.OutOfMemoryError: GC overhead limit exceeded > Dumping heap to java_pid8706.hprof ... > Heap dump file created [14429104 bytes in 0.154 secs] > > This was the first issue, the second issue is that I'm queueing up lots of data for the thread pools and as a result I'm able to completely exhaust the available memory. the clj-sys/work framework seems to be what I need to ensure there is a fixed number of threads. > You can then load that .hprof file into jvisualvm (shipped with the JDK) > to show what objects are using up the heap. It'll let you drill right > down and even inspect individual objects. > > I kinda want to make it break just so that I can use jvisualvm - it looks like fun. > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com<clojure%2bunsubscr...@googlegroups.com> > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
