For some reason, the OS starts giving me failures when I shell out too many times. I never did figure out why, but it might have had something to do with memory fragmentation and the large memory footprint of my J process. (I was shelling out to make directories -- now I use fewer of them and I make them all before I start J.)
That was ubuntu 12 instead of ubuntu 14, but I've not had the patience to go back and see if this intermittent issue has gone away. Thanks, -- Raul On Tue, Jul 15, 2014 at 8:20 AM, Joe Bogner <[email protected]> wrote: > Thanks for sharing... Did you consider shelling out a process for it? > > If the slowness was due to size and not number of calls, this may suffice: > > md5 =: 3 : 0 > FILE=.'/tmp/md5_input.txt' > y fwrite FILE > > 0{"1 ;: 2!:0 ('md5sum ', FILE) > ) > > smoutput md5 ((140*1024) $ '.') > > It requires the md5sum utility, or some other equivalent. Your approach > also requires a step to compile the library. I was just curious to > understand your thought process on compiling a md5 library > > > > > On Tue, Jul 15, 2014 at 12:40 AM, Raul Miller <[email protected]> > wrote: > > > I've updated with these changes. > > > > Thank you, > > > > -- > > Raul > > > > > > > > On Tue, Jul 15, 2014 at 12:06 AM, bill lam <[email protected]> wrote: > > > > > It works ok on linux32. But for linux64, it needs the following > > > patch for gcc apart from -m64 in Makefile > > > > > > /* UINT4 defines a four byte word */ > > > -typedef unsigned long int UINT4; > > > +typedef unsigned int UINT4; > > > > > > 9!:29]1 > > > - md5raw=: ((1!:43''),'/libmd5.so MD5String > i *c i')&(15!:0)@(;#) > > > + md5raw=: ((1!:43''),'/libmd5.so MD5String > x *c i')&(15!:0)@(;#) > > > > > > May be the cc referenced by md5 rfc is very ancient. > > > > > > Пн, 14 июл 2014, Raul Miller написал(а): > > > > I've been using the convert/misc/md5 addon, but it is rather slow. > > > > > > > > Using it on a 140k string on a moderately sized box was taking > between > > 6 > > > > and 7 seconds. And I've got a lot of strings for it to grind through > > > (many > > > > gigabytes). > > > > > > > > So I decided I wanted to use compiled code. And, since we don't have > a > > J > > > > compiler (yet), I decided to use C: > > > > > > > > The first thing I did was look at the source for the md5 on an ubuntu > > > > system. That's in coreutils, and it was something of a mess. > > > > > > > > After backing off from trying to extract the useful parts of that as > a > > > > shared library, I found that RFC 1321 contains a reference > > implementation > > > > (which looks like it was the basis for the mess I was trying to work > > > > with*). That was clean enough that I could rather easily extract it, > > > build > > > > it, and get it to work. > > > > > > > > The result is at https://github.com/rdm/libmd5.so > > > > > > > > The J test suite ( > > > https://github.com/rdm/libmd5.so/blob/master/md5test.ijs) > > > > was also extracted from RFC 1321, except for a couple lines which > > define > > > > the interface to the compiled code). > > > > > > > > Note that I'm using this with j602 (the 32 bit version), because > that's > > > the > > > > only version where the sax xml parsing works. And, I've only tested > it > > on > > > > ubuntu 14.04 lts. But if someone else needs a fast md5 > implementation, > > > > perhaps it won't be too much work to get this working on whatever > > system > > > > you work with. > > > > > > > > FYI, > > > > > > > > -- > > > > Raul > > > > > > > > * P.S. I say "mess" because it has been loaded down with so many > layers > > > of > > > > config toolchain and abstraction that it's really hard to find where > > > > anything happens. That's fine for some contexts, but useless for > > others. > > > > > ---------------------------------------------------------------------- > > > > For information about J forums see > http://www.jsoftware.com/forums.htm > > > > > > -- > > > regards, > > > ==================================================== > > > GPG key 1024D/4434BAB3 2008-08-24 > > > gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3 > > > gpg --keyserver subkeys.pgp.net --armor --export 4434BAB3 > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
