(accidentally sent this privately only, now re-sending to the list)
In the Perl 6 specification, there are plans for lazy and
memory-releasing ways to parse strings that are either too large to fit
into memory at once or that are generated lazily (like being streamed in
through the network or using "live" data sources). Sadly, none of those
features are implemented in either of our backends.
The simplest thing we have is the <cut> rule, which should instruct the
grammar engine to deallocate the parts of the input data that are before
the current cursor. Sadly, this is not going to help you much at this stage.
Another thing that will be unhelpful is that our lazy lists (such as the
ones you can generate with gather/take or what lines() will give you)
will keep all items from the very first to the last you've requested
around until the whole list becomes garbage and gets collected.
It would seem like you'll want to do a line-by-line iteration through
the data using not lines() but get() and manually parse the individual
lines; the grammar seems sufficiently simple for that to work.
Something that does surprise me is that your tests seem to imply that :p
for subparse doesn't work. I'll look into that, because I believe it
ought to be implemented already. Perhaps not properly hooked up, though.
Hope to help!