I recommend to default to using `token` rather than `rule` or `regex`.
If you need backtracking, use `regex`
If you have a lot of parts that match whitespace use `rule` (generally
used for combining other tokens.)
On Sun, Sep 23, 2018 at 7:13 PM Patrick R. Michaud <pmich...@pobox.com> wrote:
>
> I suspect the rule:
>
> rule other { . }
>
> means that in
>
> $input = '~i <<<YZ';
> my $w = Weaver.new();
> Weave.parse($input, :actions($w));
>
> the "other" rule cannot match the "Y" or the "Z" because there would need to
> be a space between them. The use of "rule" as a regex declarator implies
> that spaces in the regex source are :sigspace, which means that whitespace in
> the rule prevents two word characters from being adjacent at that point and
> any available whitespace is consumed.
>
> Change "rule" to "token" on these rules and I suspect you'll get <other> to
> match (although <other> will also end up matching the space after the "i" in
> the text string, since white spaces are no longer significant). Or try just
> changing the <other> rule to be a token and leave the others as rules.
>
> Phrased another way, the <other> rule as written now is roughly equivalent to
> writing
>
> token other { . <!ww> \s* }
>
> which will match a word character only when it's not immediately followed by
> another word character.
>
> Pm
>
>
> On Sun, Sep 23, 2018 at 08:01:31PM -0400, yary wrote:
> > Let's see.
> >
> > If you have my $input = '~i o<<<', then <other> matches.
> >
> > 'rule' turns on :sigspace. If you use 'token' instead of 'rule' then
> > <other> matches.
> >
> > I don't quite have the full picture of what's happening.
> >
> > -y
> >
> > On Sun, Sep 23, 2018 at 7:07 PM, Mark Carter <alt.mcar...@gmail.com> wrote:
> >
> > > My grammar doesn't seem to match the 'other' rule. What's wrong with it?
> > >
> > > grammar Weave {
> > > token TOP { <el> * }
> > > rule el { <lt> | <tilde> | <other> }
> > > rule lt { '<' }
> > > rule tilde { '~' \S+ }
> > > rule other { . }
> > > }
> > >
> > > class Weaver {
> > > has Str $.outstr;
> > >
> > > method TOP ($/) { make $<el> ; put("top called") ; put($<el>) }
> > > method el ($/) { put($/) }
> > > method tilde ($/) { say 'tilde called' }
> > > method lt ($/) { make '<' ; put('<'); $!outstr ~= 'X' }
> > > method other ($/) { $!outstr ~= '.'; say 'other called'; put('.');
> > > }
> > >
> > > }
> > >
> > > $input = '~i <<<YZ';
> > > my $w = Weaver.new();
> > > Weave.parse($input, :actions($w));
> > > say $w.outstr; # outputs XXX
> > >
> > > It never once says 'other called'. It seems to be matching the '<' signs
> > > OK, and I think the '~' is OK, too. It's just any other token that it's
> > > not
> > > matching.
> > >
