On Tue, Oct 06, 2009 at 01:36:13PM -0700, Glenn Rempe wrote: > No. I have not noticed any correlation with the time. Sometimes I > have seen it run during the day and die as well. I've seen it die > lots of times... ;-) It seems like it is always dying though > somewhere between 2 and 6 million records processed out of 28 mm > (which might support the theory of memory starvation of some kind if > it is holding some of those records in memory unintentionally, even > though top reports nothing more than 4GB out of 15GB being used).
You might have a per-process memory limit of some sort: either ulimit (see "ulimit -a"); or a hard-coded limitation in your O/S which limits a single process to 4GB, for example; or conceivably the erlang VM could have a 4GB limit. [I do vaguely remember something about people saying you should build erlang in 32-bit mode even under a 64-bit OS, but I could well have that wrong] Either way, if your process memory usage is continually growing and also approaching 4GB, I would be concerned. I don't see any reason why building a view should take an increasing amount of memory. It sounds like a leak. Regards, Brian.
