On 2013 Dec 13 (Fri) at 13:24:41 +0000 (+0000), Zé Loff wrote:
:On Fri, Dec 13, 2013 at 07:16:06AM -0600, Shawn K. Quinn wrote:
:> On Fri, Dec 13, 2013, at 05:36 AM, Zé Loff wrote:
:> > Hi all
:> >
:> > First of all, sorry for the kind on newbie question.
:> > I'm running some memory-heavy statistical analyses using R, which
:> > require more memory than what's physically available. I.e. the machine
:> > (a x201, which is running -current amd64) has 4Gb of physical mem, but R
:> > needs at least 6Gb. If I understand correctly, this is what virtual mem
:> > is there for, but -- and here's the newbie part -- I'm not quite sure on
:> > how to make it work.
:> [...]
:> > 3. vmemoryuse=3G + datasize-max=infinity
:> > Admittedly not knowing what I was doing. Big time SNAFU.
:> > Everything slows to a crawl when memory usage goes past the available
:> > phys mem (about 3.6G). And by a crawl I mean unusable if using X,
:> > requiring great patience if on virtual consoles.
:> > top shows R using over 1000% (not a typo) CPU although the CPU summary
:> > lines say they're all idling. "state" is "run/3", "wait" column says
:> > "pagerse". Swap usage increases, though. R never gets back to a usable
:> > state.
:> >
:> > Clue bat required. Is there anything else that needs to be done to
:> > enable R to (properly) use some of the virtual memory?
:>
:> I think R is using virtual memory as best it can, and I seriously doubt
:> you will get anything resembling satisfactory performance without
:> upgrading the RAM (memory) to 8Gb.
:>
:> Basic computing terminology: "virtual (something X)" means "(something
:> X) that isn't really there." "Virtual memory" isn't really RAM (memory),
:> it's disk space. And you're going to get the performance of disk space,
:> which is orders of magnitude slower than RAM.
:>
:> So: 1) segment this problem such that R never needs more than about 3Gb
:> of RAM in one run if possible, 2) upgrade the RAM, or 3) give R a very
:> long time to complete the task at hand and back up your hard disk
:> regularly because it will get a workout.
:
:So it's normal for a system to get slowed down to the point of losing
:network connections and freezing X every time a process uses swap? I
:find that hard to believe...
:
:--
:
Using swap is a bug. Buy more ram.
--
Just because everything is different doesn't mean anything has
changed.
-- Irene Peter
