I am not sure about overlapped either. Raul's idea about
special-casing sounds good. And the discussion on
spread of copy. In my test, the impact was 5-7%
or so -- a good price for streaming.
I think the bottle neck is in looping in u;.2
and the line proc itself.
I ran the UNIX wc, and it felt like x100 faster.
Then I ran jpm on wc, and the line proc takes
bulk of the time.
As for removing CR/LF, I would suggest optionizing
with default: both removed. For simplicity, handle
them as one option, because turning them on is
low-level stuff, to be handled as such in user code.
For example:
lineproc fapplylines fname NB. terminators removed
1 lineproc fapplylines fname NB. terminators preserved
I just had another idea: besides the adverb, to have
a conjunction with additional verb to insert between
line results. Then wc will become:
lwc2=: 1 , #@;:@(CRLF-.~]) , #
1 lwc2 finsertlines + fn
--- Chris Burke <[EMAIL PROTECTED]> wrote:
> Oleg Kobchenko wrote:
> > It's a great idea to include line reading
> > into a standard library. Here is a few comments.
> >
> > There are two differences from the original
> > readlines:
> > - overlapped reading (not once and only once)
> > (with asserting presence of LF in current block)
> > - automatic removal of terminators
>
> Agreed on leaving in the LF, in fapplylines. Do you agree on removing
> the CR or think this should be left in as well?
>
> I am in two minds on the buffer. It does impact performance, though not
> by much. But it means that after the block of 1e6 bytes is read in, it
> is immediately copied because it is appended to the tail of the previous
> block. So the question is whether this performance hit is worthwhile to
> permit the code to be used for stdin or sockets. I don't feel strongly
> on this and wonder if there are other opinions on it.
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
