Yeah, that quadratic slowdown from filter was my fault. This causes the
Nth parsed element to consume a byte stream that is being filtered N
times, which is the source of the slow down. Fortunately, Renzo's fix of
using `nextSkipEmpty` solves this problem.
I'm glad Patrick caught the subtle error where it wasn't passing the
parsed pipe state.
On 3/15/2014 5:15 PM, Renzo Carbonara wrote:
On Sat, Mar 15, 2014 at 8:34 PM, Michael Thompson
<[email protected]> wrote:
Sorry, seem to have sent two overlapping messages, (having, as I thought,
deleted what I was writing when Renzo's message was appearing. Anyway, my
point was that my `pipes1` diverges at the moment, but not if I get rid of
the `filter` in `parseL`. I don't see why this is yet.
Michael,
I'm not sure why this was happening yet, but in any case, I replaced
that `filter` with the newly introduced internal `nextSkipEmpty` which
should be faster anyway; it's a variant of `Pipes.next` that ignores
empty chunks I added internally.
In any case, I should work in improving the performance of `parsed`
(probably by not relying in `parseL` at all), as it's slower than the
`pipes3` example you shared:
% for i in streams pipes1 pipes2 pipes3; do (echo $i && time ./mto
$i); done
streams
12500002500000
./mto $i 1.03s user 0.05s system 99% cpu 1.081 total
pipes1
12500002500000
./mto $i 4.57s user 0.04s system 99% cpu 4.617 total
pipes2
12500002499999
./mto $i 4.72s user 0.03s system 99% cpu 4.754 total
pipes3
12500002500000
./mto $i 0.94s user 0.04s system 99% cpu 0.985 total
(By the way, in the `pipes2` example I had to change `drop 1` to `drop
2` in order to skip “1\n". This is fine.)
--
You received this message because you are subscribed to the Google Groups "Haskell
Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
