There was a significant impact on performance but the calculation did eventually complete, which surprised me. I did not look at the disk thrashing, there is no LED for the hard disk on my laptop.
I was using a 64-bit Ubuntu virtual machine under VMware fusion running on OSX. The setup was 64GB swap disk and 2GB RAM. I would not recommend doing calculations which end up relying on the swap disk ... best to redesign the algorithm if possible. But it is true that if you cannot do that, then it is possible to do it in the swap area in a "reasonable" amount of time. It would be interesting to try out different calculations on very large arrays which require the swap disk. It might be the case that arranging the calculation along certain axes first would result in better performance. I wonder if it is possible to arrange for a single swap page to correspond to an array slice. Perhaps J could resort to seamlessly using and managing its own on-disk "RAM" area and arrange for the pages to line up efficiently for whatever calculation is happening, rather than passing responsibility to the OS swap mechanism. Sounds like a lot of work though. On Mon, Apr 6, 2009 at 3:39 AM, Oleg Kobchenko <[email protected]> wrote: > > Did thrashing start? How did it impact performance? > > > >> From: Matthew Brand <[email protected]> >> >> The part of the thread which resolves the problem I was having is here: >> http://www.jsoftware.com/pipermail/programming/2008-December/013159.html >> >> My swap disk size was too low. Once I increased it as advised, J was >> able to operate on 64-bit VAS data. >> >> On Sun, Apr 5, 2009 at 10:27 PM, Matthew Brand wrote: >> > I think it allocates the memory from the virtual address space (VAS) >> > which is either 32 or 64 bit depending on your machine and OS. If the >> > space required by J fits in the RAM then it all happens in RAM, if it >> > is larger than the RAM then the OS automatically starts page swaps. >> > >> > You might be interested in this thread: >> > http://www.jsoftware.com/pipermail/programming/2008-December/013143.html >> > >> > In the end, if you have a 64-bit OS and 32GB RAM then I think that J >> > will automatically use all of the RAM if required. You will need to >> > set edit->configure->parameters->memory limit to >32GB though and use >> > 64-bit J. >> > >> > >> > 2009/4/5 Björn Helgason : >> >> I guess you will consider 64 bit operating system >> >> >> >> 2009/4/5 >> >> >> >>> I am going to buy a Dell T5400 to run J on. >> >>> >> >>> This machine allows one to install 32GB of memory. >> >>> >> >>> What is the command that would allow me to use a large hunk >> >>> of the 32GB for J? >> >>> >> >>> Or... >> >>> >> >>> Does J automatically adjust to use a certain portion of >> >>> memory when it starts? >> >>> >> >>> thanks >> >>> ---------------------------------------------------------------------- >> >>> For information about J forums see http://www.jsoftware.com/forums.htm >> >>> >> >> >> >> >> >> >> >> -- >> >> Björn Helgason, Verkfræðingur >> >> Fugl&Fiskur ehf, >> >> Þerneyjarsundi 23, Hraunborgum >> >> Po Box 127,801 Selfoss , >> >> t-póst: [email protected] >> >> gsm: +3546985532 >> >> Landslags og skrúðgarðagerð, gröfuþjónusta >> >> http://groups.google.com/group/J-Programming >> >> >> >> >> >> Tæknikunnátta höndlar hið flókna, sköpunargáfa er meistari einfaldleikans >> >> >> >> góður kennari getur stigið á tær án þess að glansinn fari af skónum >> >> /|_ .-----------------------------------. >> >> ,' .\ / | Með léttri lund verður | >> >> ,--' _,' | Dagurinn í dag | >> >> / / | Enn betri en gærdagurinn | >> >> ( -. | `-----------------------------------' >> >> | ) | (\_ _/) >> >> (`-. '--.) (='.'=) ♖♘♗♕♔♙ >> >> `. )----' (")_(") ☃☠ >> >> ---------------------------------------------------------------------- >> >> For information about J forums see http://www.jsoftware.com/forums.htm >> > >> > >> > >> > -- >> > http://www.ixquick.com/ >> > >> > Ixquick Protects Your Privacy! >> > The only search engine that does not record your IP address. >> > >> > http://www.vivapalestina.org/ >> > >> >> >> >> -- >> http://www.ixquick.com/ >> >> Ixquick Protects Your Privacy! >> The only search engine that does not record your IP address. >> >> http://www.vivapalestina.org/ >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm > > > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm -- http://www.ixquick.com/ Ixquick Protects Your Privacy! The only search engine that does not record your IP address. http://www.vivapalestina.org/ ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
