The timings you should be comparing to are a database engine's 2k data append query on a 2MB table. Even if its done in place, there's reliance on OS's append files facilities, and even if they are faster than "erase old file, create new file , add contents (old data + new data) to new file, rename new file to old file", its not necessarily as fast as disk access on smaller files.
----- Original Message ---- From: John Randall <[EMAIL PROTECTED]> To: Programming forum <[email protected]> Sent: Wednesday, November 1, 2006 8:50:34 AM Subject: Re: [Jprogramming] Appending to mapped files Roger Hui wrote: > Have you tried f=: f,a ? It should be in-place. > I have tried appending to a jmf file. Monitoring memory allocation suggests that the append is done in place, but timings do not. I would be grateful if anyone could shed some light on this. Consider the following script. Each of the verbs testmemory and testjmf repeatedly appends i.1e4 to an array, and prints an append count and the time for the append. require 'jmf files' testmemory=:3 : 0 f=:0$0 for_i. i.y do. smoutput i, 6!:2 'f=:f,i.1e4' end. 0 0$0 ) testjmf=:3 : 0 fn=.jpath '~temp/temp.jmf' createjmf_jmf_ fn;4e8 map_jmf_ 'f';fn f=:0$0 for_i. i.y do. smoutput i, 6!:2 'f=:f,i.1e4' end. unmap_jmf_ 'f' ferase fn 0 0$0 ) The verb testmemory does this in memory. As expected, the timings remain approximately constant over trials. testmemory 10000 0 0.000269 1 0.000235 2 0.000136 3 0.000391 4 0.000127 5 0.00013 6 0.000591 7 0.000131 8 0.000127 9 0.00013 10 0.000128 ........... 990 0.000154 991 0.000146 992 0.000149 993 0.00016 994 0.000144 995 0.000148 996 0.000149 997 0.005005 998 0.000173 999 0.000149 The verb testjmf does the appends in a jmf file. The timings increase approximately linearly, giving overall quadratic performance. testjmf 1000 0 0.000416 1 0.000403 2 0.00054 3 0.000764 4 0.000931 5 0.000994 6 0.001221 7 0.001297 8 0.001759 9 0.001929 10 0.001796 ............ 990 0.166764 991 0.166541 992 0.165332 993 0.166796 994 0.167025 995 0.16505 996 0.168694 997 0.165894 998 0.166543 999 0.168175 I am new to jmf files, so I may be doing something fundamentally wrong. Any insight would be appreciated. John ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
