Yeah, maybe that's the way. I hoped the OS and J would do it automatically.
Interestingly, in Java NIO, the limit on a single map size is Integer.MAX_VALUE 'c0'(8!:2) _1 + 2^31 2,147,483,647 --- [EMAIL PROTECTED] wrote: > Maybe I am too naive but isn't it possible to map smaller areas > consecutively and chunked the execution that way ? > > > Paul Gauthier > APL Software Developer - Senior > [EMAIL PROTECTED] > Phone: 312-739-3467 > Fax: 312-739-3496 > > CheckFree. The Company that Powers Payment on the WebSM. > http://www.checkfree.com/paybillsonline > > > > > Oleg Kobchenko <[EMAIL PROTECTED]> > Sent by: [EMAIL PROTECTED] > 05/10/2006 12:54 PM > Please respond to > Programming forum <[email protected]> > > > To > Programming forum <[email protected]> > cc > > Subject > Re: [Jprogramming] Multiply&sum for two big arrays > > > > > > > Example 1, I meant to illustrate that mapped > nouns do not decrease memory footprint. Indeed, > they are at best taking not less memory. > > A2=: i.20 20 > B2=: %>:i.-20 20 > 7!:2 'C2=: A2 * B2' > 4992 > > So the question remains: how using mapped files, > to decrease memory in the operation C=: A f B > where f is a simple logical or arithmetic verb. > > I understand, similar special operations > A=: A,B and A=: B i}A > do not create temps. > > Though, they could be detected at parse > time, whereas the mapped situation is not > known until execution. Is that the reason, > why it cannot be made special? > > > --- Roger Hui <[EMAIL PROTECTED]> wrote: > > > In example 1, the expression A * B takes different space than > > C1=: A1 * B1 for two reasons: > > - the first expression is shorter than the second > > - the first expression does not do an assignment > > Mapping or not mapping is not material. > > > > A f B creates a temp in memory (not mapped) even if A > > and B are mapped, even if it is then immediately assigned > > to a mapped name. > > > > What would improve the situation is if +/@:* (and in general > > f/@:g) is supported by special code. This may be done for a > > future release. It is probably too late for J601. > > > > > > > > ----- Original Message ----- > > From: "Oleg Kobchenko" <[EMAIL PROTECTED]> > > To: "Programming forum" <[email protected]> > > Sent: Sunday, May 07, 2006 1:11 AM > > Subject: Re: [Jprogramming] Multiply&sum for two big arrays > > > > Yes this is really strange: I thought operations > > with MMF nouns should not take up physical memory. > > Especially if the result is mapped, moreover > > if there are no intermediate operations. > > > > But even simple operations, like C=: A*B all mapped > > take even more memory than not mapped. > > (Using left arg "JFL map_jmf_ y" does not help.) > > > > Example 1. > > require 'jmf files dir' > > ]P=: jpath'~temp' > > c:\math\j601\temp > > > > A=: i.20 20 > > B=: %>:i.-20 20 > > > > createjmf_jmf_ (P,'/A.jmf');*/20 20 8 > > createjmf_jmf_ (P,'/B.jmf');*/20 20 8 > > createjmf_jmf_ (P,'/C.jmf');*/20 20 8 > > map_jmf_ 'A1';P,'/A.jmf' > > map_jmf_ 'B1';P,'/B.jmf' > > map_jmf_ 'C1';P,'/C.jmf' > > A1=: i.20 20 > > B1=: %>:i.-20 20 > > > > 7!:2 'A * B' > > 4800 > > 7!:2 'C1=: A1 * B1' > > 4992 > > > > > > Example 2. > > (Choose total sizes exceeding your physical + swap file, > > swap file should be fixed). > > > > createjmf_jmf_ (P,'/a.jmf'); 40e6 * 8 > > createjmf_jmf_ (P,'/b.jmf'); 40e6 * 8 > > createjmf_jmf_ (P,'/c.jmf'); 40e6 * 8 > > > > map_jmf_ 'A';P,'/a.jmf' > > map_jmf_ 'B';P,'/b.jmf' > > map_jmf_ 'C';P,'/c.jmf' > > showmap_jmf_'' > > > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > > |name |fn |sn|fh |mh |address |header |ts|msize > |refs| > > > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > > |A_base_|c:\math\j601\temp/a.jmf| |224|228|539820032 |539820032 | > |320000000|2 | > > > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > > |B_base_|c:\math\j601\temp/b.jmf| |232|236|859832320 |859832320 | > |320000000|2 | > > > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > > |C_base_|c:\math\j601\temp/c.jmf| |240|244|1179844608|1179844608| > |320000000|2 | > > > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > > > > > > A=: i.40e6 > > B=: i.40e6 > > B=: >:B > > B=: %B > > |out of memory > > | B=: %B > > > > unmapall_jmf_'' NB. OK let's restart > > 0 0 0 > > map_jmf_ 'B';P,'/b.jmf' > > B=: %B > > unmapall_jmf_'' > > 0 > > map_jmf_ 'C';P,'/c.jmf' > > C=: 40e6#0.1 NB. initialize to form of results > > unmapall_jmf_'' > > 0 0 0 > > map_jmf_ 'A';P,'/a.jmf' > > map_jmf_ 'B';P,'/b.jmf' > > map_jmf_ 'C';P,'/c.jmf' > > C=: A*B > > |out of memory > > | C=:A *B > > dir P > > 1.ijs 417 23-Apr-06 16:25:24 > > 1.ijx 4509 07-May-06 03:38:21 > > 2.ijs 1050 27-Apr-06 06:39:33 > > a.jmf 320000284 07-May-06 03:40:21 > > b.jmf 320000284 07-May-06 03:40:24 > > c.jmf 320000284 07-May-06 03:40:27 > > temp.txt 111 22-Apr-06 08:19:34 > > > > > > > > --- Anssi Seppälä <[EMAIL PROTECTED]> wrote: > > > > > I have two big 2-dimensional arrays in mapped files A and B. I need > > > to calculate +/ A * B efficiently and without "out of memory". > > > > > > In the case A was a vector the . (dot) can make it +/ . * , but this > > > does not work if A is a matrix. Is there a way to do this without a > > > program loop? > > > > > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > === message truncated === __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
