On Sun, Nov 22, 2020 at 6:15 AM Alexander Ilin wrote:
> So, the question is, is it possible to use the peg vocab to create this
> layered parsing architecture, where the first layer would consume a stream of
> file input, and the second would consume a stream of tokens from the first
> layer?
Right now this is true:
binary stream-seekable?
But none of the decoders allow stream-seeking. Maybe we should support
that for ascii and other 8-bit encodings...
On Sun, Nov 22, 2020 at 3:17 PM Jon Harper wrote:
> I didn't know about the seekable streams implementation in factor:
>
I didn't know about the seekable streams implementation in factor:
https://docs.factorcode.org/content/article-stream-protocol.html
We have:
"/tmp/toto" ascii stream-seekable?
f
"/tmp/toto" (file-reader)
t
! it works, stream-seek can rewind or fast forward and stream-read1
will get the bytes
What you are describing reminds me of the built-in nesting of parsers with
the ebnf tokenizer :
https://docs.factorcode.org/content/article-peg.ebnf.tokenizers.html
Not sure if it's really applicable to your use case though.
As for streams, aren't packrat parsers backtracking (with
I have another question to follow this up. I want to create a multilayer parser.The first layer would read a bunch of textual log files (with date-time in their names) and search for various types of events using PEG patterns per line and skipping noise. This would produce a stream of interesting
Hello!
I see that the peg vocab operates on strings. Can it operate on streams?
I want to parse some log files, and thought it would make sense not to load
them all at once.
Especially since I want to stream multiple files concatenated as one.
---=---
Александр
