I had a recent experience that others might find useful. I have some
j code in a Linux #! script (jwork) that takes pairs of files in a
sendmail queue (header and body files) and puts them together into a
usable email object. Long years of habit had me starting with an
empty result and doing something like -
result =: result, grind files
inside a for. loop -- and then, after the work was done -
stdout result
By moving the stdout bit inside the for. loop and removing the
catenation in the loop, a partial result is catenated to the standard
output pipe and the result file is built by adding the results of
each iteration in the script. That is, something like
# ls q* | jwork > output
causes output to be built as jwork iterates through the files from the list.
This change caused a speedup of 4 or more times because of the simple
elimination of copies of the catenated temporary result variable.
This made things work a whole lot more reasonably and take a lot less
memory - and has the interesting side effect of generating a
perfectly usable partial result if interrupted during operation.
The point of this post is that sometimes what seems like "natural
array thinking" is counter productive - maybe others know this
instinctively, but it was an eye opener for me.
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
