I've been dealing with problems with large arrays lately.
One possibility I've toyed with, though have not implemented, is to have one large array in one session and another large array in another session and pass data between the sessions using sockets. This might work for me because I have only two large arrays I care about. However, I've been dealing with arrays too large for one session by breaking them into arbitrary pieces written to file as numbered variables, e.g. var0, var1, etc., and working on a piece at a time. I use my functions found at http://www.jsoftware.com/jwiki/Scripts/File_J_Variables for this. I can apply a function to each piece using an adverb like this: NB.* getVarInfo: apply arbitrary function to each (filed) var named. getVarInfo=: 1 : 0 'dd varnm'=. y. NB. Vars dir, var names. rc=. dd unfileVar_WS_ varnm if. >{.rc do. rc=. 1;u. ".varnm [4!:55 <varnm end. rc NB.EG ({."1,.{:"1) getVarInfo &.>(<'C:\data\');&.>'var1';'var2';'var3' NB.EG dts=: (3&{) getVarInfo&.>(<VDIR);&.>MVN ) On 12/11/06, Eric Iverson <[EMAIL PROTECTED]> wrote:
Chris has already pointed out how to use 9!:20 and that its default setting is unlikely to be your problem. How large are the files? The Windows host has a limit of about 2GB of address space for an application. In practice the max file size you could map would be less than 1BG given that you had enough hard disk space allocated for virtual memory use. If you are on windows then there is an OS limit on the amount of virtual memory an application. You may want to check this in Setttngs|Control Panel|Advanced|Performance Settings (virtual memory). ----- Original Message ----- From: "Kairit Sirts" <[EMAIL PROTECTED]> To: "'Programming forum'" <[email protected]> Sent: Monday, December 11, 2006 2:48 PM Subject: [Jprogramming] how to increase memory > Hi! > > I try to process large data files and while doing that I get 'out of > memory' > error message. I understand that I have to increase the memory limit > by > setting 9!:20 or 9!:21, but I don't know how to do that. > > The attempts to execute 9!:20 y or 9!:21 y, where y is some desired > number > bigger that default 2^30 will give me rank error. I also do not find > any > examples of how to do that. > > How should I do that or where could I read about it? > > Thank you! > Kairit > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
-- Devon McCormick ^me^ at acm. org is my preferred e-mail ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
