Did you take a look at ulimit?
On Tue, Jul 15, 2014 at 1:44 PM, Raul Miller <[email protected]> wrote: > For some reason, the OS starts giving me failures when I shell out too many > times. I never did figure out why, but it might have had something to do > with memory fragmentation and the large memory footprint of my J process. > (I was shelling out to make directories -- now I use fewer of them and I > make them all before I start J.) > > That was ubuntu 12 instead of ubuntu 14, but I've not had the patience to > go back and see if this intermittent issue has gone away. > > Thanks, > > -- > Raul > > > On Tue, Jul 15, 2014 at 8:20 AM, Joe Bogner <[email protected]> wrote: > >> Thanks for sharing... Did you consider shelling out a process for it? >> >> If the slowness was due to size and not number of calls, this may suffice: >> >> md5 =: 3 : 0 >> FILE=.'/tmp/md5_input.txt' >> y fwrite FILE >> > 0{"1 ;: 2!:0 ('md5sum ', FILE) >> ) >> >> smoutput md5 ((140*1024) $ '.') >> >> It requires the md5sum utility, or some other equivalent. Your approach >> also requires a step to compile the library. I was just curious to >> understand your thought process on compiling a md5 library >> >> >> >> >> On Tue, Jul 15, 2014 at 12:40 AM, Raul Miller <[email protected]> >> wrote: >> >> > I've updated with these changes. >> > >> > Thank you, >> > >> > -- >> > Raul >> > >> > >> > >> > On Tue, Jul 15, 2014 at 12:06 AM, bill lam <[email protected]> wrote: >> > >> > > It works ok on linux32. But for linux64, it needs the following >> > > patch for gcc apart from -m64 in Makefile >> > > >> > > /* UINT4 defines a four byte word */ >> > > -typedef unsigned long int UINT4; >> > > +typedef unsigned int UINT4; >> > > >> > > 9!:29]1 >> > > - md5raw=: ((1!:43''),'/libmd5.so MD5String > i *c i')&(15!:0)@(;#) >> > > + md5raw=: ((1!:43''),'/libmd5.so MD5String > x *c i')&(15!:0)@(;#) >> > > >> > > May be the cc referenced by md5 rfc is very ancient. >> > > >> > > Пн, 14 июл 2014, Raul Miller написал(а): >> > > > I've been using the convert/misc/md5 addon, but it is rather slow. >> > > > >> > > > Using it on a 140k string on a moderately sized box was taking >> between >> > 6 >> > > > and 7 seconds. And I've got a lot of strings for it to grind through >> > > (many >> > > > gigabytes). >> > > > >> > > > So I decided I wanted to use compiled code. And, since we don't have >> a >> > J >> > > > compiler (yet), I decided to use C: >> > > > >> > > > The first thing I did was look at the source for the md5 on an ubuntu >> > > > system. That's in coreutils, and it was something of a mess. >> > > > >> > > > After backing off from trying to extract the useful parts of that as >> a >> > > > shared library, I found that RFC 1321 contains a reference >> > implementation >> > > > (which looks like it was the basis for the mess I was trying to work >> > > > with*). That was clean enough that I could rather easily extract it, >> > > build >> > > > it, and get it to work. >> > > > >> > > > The result is at https://github.com/rdm/libmd5.so >> > > > >> > > > The J test suite ( >> > > https://github.com/rdm/libmd5.so/blob/master/md5test.ijs) >> > > > was also extracted from RFC 1321, except for a couple lines which >> > define >> > > > the interface to the compiled code). >> > > > >> > > > Note that I'm using this with j602 (the 32 bit version), because >> that's >> > > the >> > > > only version where the sax xml parsing works. And, I've only tested >> it >> > on >> > > > ubuntu 14.04 lts. But if someone else needs a fast md5 >> implementation, >> > > > perhaps it won't be too much work to get this working on whatever >> > system >> > > > you work with. >> > > > >> > > > FYI, >> > > > >> > > > -- >> > > > Raul >> > > > >> > > > * P.S. I say "mess" because it has been loaded down with so many >> layers >> > > of >> > > > config toolchain and abstraction that it's really hard to find where >> > > > anything happens. That's fine for some contexts, but useless for >> > others. >> > > > >> ---------------------------------------------------------------------- >> > > > For information about J forums see >> http://www.jsoftware.com/forums.htm >> > > >> > > -- >> > > regards, >> > > ==================================================== >> > > GPG key 1024D/4434BAB3 2008-08-24 >> > > gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3 >> > > gpg --keyserver subkeys.pgp.net --armor --export 4434BAB3 >> > > ---------------------------------------------------------------------- >> > > For information about J forums see http://www.jsoftware.com/forums.htm >> > ---------------------------------------------------------------------- >> > For information about J forums see http://www.jsoftware.com/forums.htm >> > >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm >> > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
