Re: Can I use the R2 scanless interface with an external tokenizer?

2014-10-07 Thread Michael Roberts
And I should say at this point that the new docs are much clearer. Good work! On Monday, September 29, 2014 11:09:17 PM UTC+2, Jeffrey Kegler wrote: > > That section is being rewritten, literally "as we speak". No changes to > functionality, but I was unhappy with level of completeness and clar

Re: Can I use the R2 scanless interface with an external tokenizer?

2014-09-29 Thread Jeffrey Kegler
That section is being rewritten, literally "as we speak". No changes to functionality, but I was unhappy with level of completeness and clarity in the docs. It's not a tutorial -- I'm talking about reference doc type clarity. On Mon, Sep 29, 2014 at 12:56 PM, Michael Roberts wrote: > On Mon,

Re: Can I use the R2 scanless interface with an external tokenizer?

2014-09-29 Thread Michael Roberts
On Mon, 29 Sep 2014 21:27:52 +0200, Ruslan Shvedov wrote: External scanning ... Thank you! That was *exactly* the pointer I needed to get off the ground. Michael -- You received this message because you are subscribed to the Google Groups "marpa parser" group. To unsubscribe from this gr

Re: Can I use the R2 scanless interface with an external tokenizer?

2014-09-29 Thread Ruslan Shvedov
External scanning is usually performed by reading lexemes using ...— https://github.com/jeffreykegler/Marpa--R2/blob/master/cpan/pod/Scanless/R.pod Examples https://github.com/jeffreykegler/Marpa--R2/blob/master/cpan/t/sl_durand.t https://github.com/jeffreykegler/Marpa--R2/blob/master/cpan/t/sl_ex

Can I use the R2 scanless interface with an external tokenizer?

2014-09-29 Thread Michael Roberts
I have a natural-language tokenizer that I can call to return a stream of tokens of the form ['word', 'POS'], where POS is the part of speech. For the purposes of argument, this would be something along the lines of (['The', 'art'], ['dog', 'n'], ['barked', 'v'], ['.', 'P']). What I'd like to d