I wonder whether you really want lazy streams. -- Matthias
On Sep 9, 2013, at 10:19 AM, Lawrence Woodman wrote: > On 09/09/13 07:19, Stephen Chang wrote: >> Konrad's exactly right. Your filtered-nums blows up because you named >> the stream and then traversed the entire thing. In general, if you >> hang onto the head of the stream while traversing then the GC can't >> collect anything because since you have a pointer to the head, every >> element of the stream is still reachable. >> >> Compare to a common traversal pattern like: >> >> (let loop ([s <some stream>]) ... (loop (stream-rest s))) >> >> where the head is dropped on each loop iteration. >> >> Other comments: >> - (stream-length (gen-filtered-nums)) is fine because there's no >> pointer to the head, so the GC collects as you traverse. >> >> - in-range is fine because it's constant space and not a >> cons-cell-based stream. It's more like a generator. >> >> - the for/sum is actually collecting while traversing, just more >> slowly. I'm not exactly sure why, I may look into it. On my machine, >> it got up to 500mb or so but it finished. > > Thanks, and to Konrad too. You're mention of in-range being more > like a generator, actually makes me wonder whether a generator would > be a better choice for processing large data sets, from databases and > csv files, sequentially. What do you think? > > > Lorry > > -- > vLife Systems Ltd > Registered Office: The Meridian, 4 Copthall House, Station Square, Coventry, > CV1 2FL > Registered in England and Wales No. 06477649 > http://vlifesystems.com > > ____________________ > Racket Users list: > http://lists.racket-lang.org/users ____________________ Racket Users list: http://lists.racket-lang.org/users
