Maybe I am too naive but isn't it possible to map smaller areas consecutively and chunked the execution that way ?
Paul Gauthier APL Software Developer - Senior [EMAIL PROTECTED] Phone: 312-739-3467 Fax: 312-739-3496 CheckFree. The Company that Powers Payment on the WebSM. http://www.checkfree.com/paybillsonline Oleg Kobchenko <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 05/10/2006 12:54 PM Please respond to Programming forum <[email protected]> To Programming forum <[email protected]> cc Subject Re: [Jprogramming] Multiply&sum for two big arrays Example 1, I meant to illustrate that mapped nouns do not decrease memory footprint. Indeed, they are at best taking not less memory. A2=: i.20 20 B2=: %>:i.-20 20 7!:2 'C2=: A2 * B2' 4992 So the question remains: how using mapped files, to decrease memory in the operation C=: A f B where f is a simple logical or arithmetic verb. I understand, similar special operations A=: A,B and A=: B i}A do not create temps. Though, they could be detected at parse time, whereas the mapped situation is not known until execution. Is that the reason, why it cannot be made special? --- Roger Hui <[EMAIL PROTECTED]> wrote: > In example 1, the expression A * B takes different space than > C1=: A1 * B1 for two reasons: > - the first expression is shorter than the second > - the first expression does not do an assignment > Mapping or not mapping is not material. > > A f B creates a temp in memory (not mapped) even if A > and B are mapped, even if it is then immediately assigned > to a mapped name. > > What would improve the situation is if +/@:* (and in general > f/@:g) is supported by special code. This may be done for a > future release. It is probably too late for J601. > > > > ----- Original Message ----- > From: "Oleg Kobchenko" <[EMAIL PROTECTED]> > To: "Programming forum" <[email protected]> > Sent: Sunday, May 07, 2006 1:11 AM > Subject: Re: [Jprogramming] Multiply&sum for two big arrays > > Yes this is really strange: I thought operations > with MMF nouns should not take up physical memory. > Especially if the result is mapped, moreover > if there are no intermediate operations. > > But even simple operations, like C=: A*B all mapped > take even more memory than not mapped. > (Using left arg "JFL map_jmf_ y" does not help.) > > Example 1. > require 'jmf files dir' > ]P=: jpath'~temp' > c:\math\j601\temp > > A=: i.20 20 > B=: %>:i.-20 20 > > createjmf_jmf_ (P,'/A.jmf');*/20 20 8 > createjmf_jmf_ (P,'/B.jmf');*/20 20 8 > createjmf_jmf_ (P,'/C.jmf');*/20 20 8 > map_jmf_ 'A1';P,'/A.jmf' > map_jmf_ 'B1';P,'/B.jmf' > map_jmf_ 'C1';P,'/C.jmf' > A1=: i.20 20 > B1=: %>:i.-20 20 > > 7!:2 'A * B' > 4800 > 7!:2 'C1=: A1 * B1' > 4992 > > > Example 2. > (Choose total sizes exceeding your physical + swap file, > swap file should be fixed). > > createjmf_jmf_ (P,'/a.jmf'); 40e6 * 8 > createjmf_jmf_ (P,'/b.jmf'); 40e6 * 8 > createjmf_jmf_ (P,'/c.jmf'); 40e6 * 8 > > map_jmf_ 'A';P,'/a.jmf' > map_jmf_ 'B';P,'/b.jmf' > map_jmf_ 'C';P,'/c.jmf' > showmap_jmf_'' > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > |name |fn |sn|fh |mh |address |header |ts|msize |refs| > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > |A_base_|c:\math\j601\temp/a.jmf| |224|228|539820032 |539820032 | |320000000|2 | > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > |B_base_|c:\math\j601\temp/b.jmf| |232|236|859832320 |859832320 | |320000000|2 | > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > |C_base_|c:\math\j601\temp/c.jmf| |240|244|1179844608|1179844608| |320000000|2 | > +-------+-----------------------+--+---+---+----------+----------+--+---------+----+ > > > A=: i.40e6 > B=: i.40e6 > B=: >:B > B=: %B > |out of memory > | B=: %B > > unmapall_jmf_'' NB. OK let's restart > 0 0 0 > map_jmf_ 'B';P,'/b.jmf' > B=: %B > unmapall_jmf_'' > 0 > map_jmf_ 'C';P,'/c.jmf' > C=: 40e6#0.1 NB. initialize to form of results > unmapall_jmf_'' > 0 0 0 > map_jmf_ 'A';P,'/a.jmf' > map_jmf_ 'B';P,'/b.jmf' > map_jmf_ 'C';P,'/c.jmf' > C=: A*B > |out of memory > | C=:A *B > dir P > 1.ijs 417 23-Apr-06 16:25:24 > 1.ijx 4509 07-May-06 03:38:21 > 2.ijs 1050 27-Apr-06 06:39:33 > a.jmf 320000284 07-May-06 03:40:21 > b.jmf 320000284 07-May-06 03:40:24 > c.jmf 320000284 07-May-06 03:40:27 > temp.txt 111 22-Apr-06 08:19:34 > > > > --- Anssi Seppälä <[EMAIL PROTECTED]> wrote: > > > I have two big 2-dimensional arrays in mapped files A and B. I need > > to calculate +/ A * B efficiently and without "out of memory". > > > > In the case A was a vector the . (dot) can make it +/ . * , but this > > does not work if A is a matrix. Is there a way to do this without a > > program loop? > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
