I am indeed using a 64-bit version of OpenSuse, that's how I can run J64.
On Mon, Mar 31, 2008 at 9:55 AM, Michael Dykman <[EMAIL PROTECTED]> wrote: > In your listing, I did not see that you are using a 64-bits version of > linux. EVen on 64 bits hardware, this will fail without a 64 bit > kernel and a 64 bit glibc. Are you sure you have those installed? I > know SUSE has such a configuration available but, generally, if the > distro does not explicitly state that it is 64-bit, it isn't. > > - michael dykman > > > > > On Mon, Mar 31, 2008 at 10:06 AM, Eric Iverson <[EMAIL PROTECTED]> wrote: > > I am interested in the problem you report but haven't had a chance to work > > on it yet. With luck I will be able to do so in a few days. > > > > Thanks for taking the initial steps of fixing the SetFilePointerR vs > 64bit > > etc problems and we'll see that this work gets included in a future J > libary > > release. > > > > The first thing I wouold do is a web search on Linux and large mapped > files > > and wonder if you have already done that. > > > > (I replied to your original message to programming and not the one to chat > > as this is quite properly a programming issue) > > > > > > > > ----- Original Message ----- > > From: "Jorge Antonio Arredondo Vargas" <[EMAIL PROTECTED]> > > To: <[email protected]> > > Sent: Thursday, March 27, 2008 11:36 PM > > Subject: [Jprogramming] Memory-map a large file on Linux > > > > > > > I'm trying to memory-map a large file (more than 6 GB) from J64 on > > > Linux, but I'm getting a ENOMEM error which I haven't been able to > > > solve. > > > I can successfully map such kind of file, (indeed bigger: 200GB) on > > > Windows XP x64 and use it as a J noun, however. > > > > > > Have any of you done this already? > > > > > > My system has the following specifications: > > > OS= OpenSuse 10.2 > > > RAM= 8GB > > > CPU= 2 Dual Core Opteron64 @2GHz > > > 9!:12'' > > > 5 > > > 9!:14'' > > > j601/2006-09-04/09:20 > > > > > > I start getting problems after testing with files bigger than 6GB, > > > which is close to the physical amount of RAM in my machine, so I'm > > > thinking the mmap system call might be trying to actually load the > > > file into memory, perhaps this problem can be solved by using a > > > different flag on the mmap call. > > > > > > > > > When I first tried to create the file I want to map, I discovered > > > jmf.ijs didn't worked with big files: > > > ======================================================================== > > > require 'jmf files' > > > FN=: '/usr/test.jmf' > > > SIZE_6GB=: 6 * 1024(<[EMAIL PROTECTED])3 > > > HS=: 568 > > > > > > createjmf_jmf_ FN;SIZE_6GB > > > assert (SIZE_6GB+HS)-:fsize FN > > > |assertion failure: assert > > > | assert (SIZE_6GB+HS)-:fsize FN > > > |[-3] > > > fsize FN > > > 64 > > > ======================================================================== > > > > > > > > > I solved this by defining c_open and c_lseek with large file support > > > (LFS) function equivalents and then running the relevant code of > > > createjmf_jmf_, here is the code: > > > ======================================================================== > > > cd=: 15!:0 > > > c_open=: 'libc.so.6 open64 i *c i i'&cd > > > c_close=: 'libc.so.6 close i i'&cd > > > c_lseek=: 'libc.so.6 lseek64 i i l i'&cd > > > c_write=: 'libc.so.6 write i i * i'&cd > > > O_RDWR=: 2 > > > O_CREAT=: 64 > > > O_TRUNC=: 512 > > > JINT=: 4 > > > SZI=: 8 > > > AFNJA=: 2 > > > SEEK_SET=: 0 > > > > > > ]fn=: FN > > > /usr/test.jmf > > > ]ms=: SIZE_6GB > > > 6442450944 > > > ]ts=: HS+ms > > > 6442451512 > > > > > > ]fh=: 0 pick c_open fn; ((23 b.)/O_RDWR,O_CREAT,O_TRUNC); 8b666 > > > 19 > > > c_lseek fh;(<:ts);SEEK_SET > > > +-----------+--+----------+-+ > > > |_2147483081|19|6442451511|0| > > > +-----------+--+----------+-+ > > > c_write fh; (,0{a.); 0+1 > > > +-+--+-+-+ > > > |1|19| |1| > > > +-+--+-+-+ > > > c_lseek fh;0 ;SEEK_SET > > > +-+--+-+-+ > > > |0|19|0|0| > > > +-+--+-+-+ > > > ]d=: HS,AFNJA,ms,JINT,0,0,1,0 NB. integer empty list > > > 568 2 6442450944 4 0 0 1 0 > > > c_write fh;d;(SZI*#d) > > > +--+--+--------------------------+--+ > > > |64|19|568 2 6442450944 4 0 0 1 0|64| > > > +--+--+--------------------------+--+ > > > c_close fh > > > +-+--+ > > > |0|19| > > > +-+--+ > > > assert (SIZE_6GB+HS)-:fsize FN > > > ======================================================================== > > > > > > > > > After creating the file, I tried using map_jmf_, but J crashed with a > > > segmentation fault: > > > ======================================================================== > > > map_jmf_ 'test';FN > > > 9183 Segmentation fault > > > ======================================================================== > > > > > > > > > So I defined c_mmap with LFS, and I could finally perform the mapping > > > by running the relevant map_jmf_ code: > > > ======================================================================== > > > cderx=: 15!:11 > > > c_mmap=: 'libc.so.6 mmap64 * * x i i i l'&cd > > > c_munmap=: 'libc.so.6 munmap i * x'&cd > > > O_RDONLY=: 0 > > > PROT_READ=: 1 > > > MAP_SHARED=: 1 > > > ENOMEM=: 12 > > > > > > ]ts=: 1!:4 <fn > > > 6442451512 > > > ]fh=: >0 { c_open fn;O_RDONLY;0 > > > 19 > > > mh=: ts > > > ]fad=: >0{ c_mmap (<0);ts;PROT_READ;MAP_SHARED;fh;0 > > > 46913860665344 > > > assert ENOMEM ~: >0{cderx'' > > > c_munmap (<fad);mh > > > +-+----------------+----------+ > > > |0|+--------------+|6442451512| > > > | ||46913860665344|| | > > > | |+--------------+| | > > > +-+----------------+----------+ > > > c_close fh > > > +-+--+ > > > |0|19| > > > +-+--+ > > > ======================================================================== > > > > > > > > > The problem is that the c_mmap call fails when trying with larger > > > files, for instance, here is the result of working with an 8GB file: > > > ======================================================================== > > > SIZE_8GB=: 8 * 1024(<[EMAIL PROTECTED])3 > > > > > > ]fn=: FN > > > /usr/test.jmf > > > ]ms=: SIZE_8GB > > > 8589934592 > > > ]ts=: HS+ms > > > 8589935160 > > > > > > NB. create the file > > > ]fh=: 0 pick c_open fn; ((23 b.)/O_RDWR,O_CREAT,O_TRUNC); 8b666 > > > 20 > > > c_lseek fh;(<:ts);SEEK_SET > > > +---+--+----------+-+ > > > |567|20|8589935159|0| > > > +---+--+----------+-+ > > > c_write fh; (,0{a.); 0+1 > > > +-+--+-+-+ > > > |1|20| |1| > > > +-+--+-+-+ > > > c_lseek fh;0 ;SEEK_SET > > > +-+--+-+-+ > > > |0|20|0|0| > > > +-+--+-+-+ > > > ]d=: HS,AFNJA,ms,JINT,0,0,1,0 NB. integer empty list > > > 568 2 8589934592 4 0 0 1 0 > > > c_write fh;d;(SZI*#d) > > > +--+--+--------------------------+--+ > > > |64|20|568 2 8589934592 4 0 0 1 0|64| > > > +--+--+--------------------------+--+ > > > c_close fh > > > +-+--+ > > > |0|20| > > > +-+--+ > > > > > > NB. no errors so far, now.try to map the file > > > ]ts=: 1!:4 <fn > > > 8589935160 > > > ]fh=: >0 { c_open fn;O_RDONLY;0 > > > 20 > > > > > > mh=: ts > > > ]fad=: >0{ c_mmap (<0);ts;PROT_READ;MAP_SHARED;fh;0 > > > _1 > > > assert ENOMEM ~: >0{cderx'' > > > |assertion failure: assert > > > | assert ENOMEM~:>0{cderx'' > > > |[-21] > > > c_close fh > > > +-+--+ > > > |0|20| > > > +-+--+ > > > ======================================================================== > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > > > > -- > - michael dykman > - [EMAIL PROTECTED] > > - All models are wrong. Some models are useful. > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
