Hi Moritz,

Thanks that was interesting. My investigation into grammars took a while but 
here are the results thus far:

> Grammar rules and regexes are just methods…

I hadn't thought about what a grammar and rule actually was before. This 
inspired me to try:

grammar Gram{
    has $.x;
    rule TOP{
        {say $.x}
    method test{
        say $.x
my Gram $test .= new(:x("hello"));
$test.parse("ignore this");
say $test.TOP;
which outputs:
Any()                          #output of TOP in parse
hello                            #output of test.test
hello                            #outputted on direct call to rule
Gram.new(x => Any) #the return value of $test.TOP

So rules can't interpolate their grammar's attributes when being called by 
'parse' but can when called as a method. Also rules being called directly as 
methods return the parent grammar. I'm not sure whether either of these things 
are intended…


Also I tried rules with arguments and it worked from grammar->parse but not 
from calling directly as a method. 

grammar Gram{
    rule TOP{

    rule test_rule($a){

my Gram $test .= new();
$test.parse("hello") #returns true
$test.test_rule("hello") #error

The error is:

Invalid operation on null string
  in any !LITERAL at src/stage2/QRegex.nqp:653
  in method INTERPOLATE at src/gen/CORE.setting:9731
 (at the line where test_rule starts)


Ok now to try the things you mentioned:

First I tried using a parcel instead of an array as the role prototype (array 
resulted in error):
role roley [$foo]{
    token tokeny { $foo }

grammar gram {
    token TOP { <tokeny> }
my gram $gram .= new  does roley[('this','or', 'that')];
$gram.parse('this or that');  #returns true

So parcels get joined with spaces into one token


Now to try the around about way:

role roley [$foo]{
    token tokeny:sym<dynamic> { $foo }

grammar gram {
    token TOP { <tokeny>[\ <tokeny>]* }
    proto token tokeny {*}

my gram $gram .= new;
$gram does roley[$_] for <that this>;
$gram.parse('this'); #matches
$gram.parse('that'); #nope

Each iteration overwrites the previous one in terms of what 'tokeny' resolves 
to rather than adding it (symmetrically? is that what sym is short for?)


One more thing I found which seems to be a bug. I defined my nouns/pronouns 

token PN:sym<John> { <.sym> } #The dot should mean it doesn't get captured
token N:sym<ball> { <.sym> }

when my grammar parses this it ends up with a tree like this:
 sentence => q[John hit the ball]
  statement => q[John hit the ball]
   NP => q[John]
    PN => q[John]
      => q[John]
   VP => q[hit the ball]
    verb => q[hit]
      => q[hit]
    NP => q[the ball]
     D => q[the]
       => q[the]
     N => q[ball]
       => q[ball]

Notice the empty slots on the left. Rather than not capturing the <sym>  the 
<.sym> just means it doesn't capture it's name :S


So after all this I have a much better understanding of what grammars really 
are but I'm still confused about a few things:

grammars are like classes. They are special because they have a method called 
'parse' which applies a rule/token definition (regex) called TOP (or whatever 
is set by the  :rule argument to parse).
Q: Are grammars meant to be able to have attributes like classes and are they 
meant to be able to interpolate them into their rules/token?
rules and tokens are just special types of methods who's body is a regex rather 
than perl6 code.
Q: What is the meaning of the return values of tokens/rules when called as 
Q: Is it possible to write a normal method that conforms the the same interface 
as rules/tokens (whatever that is). i.e. where we can use <normal_method> in 
rules/tokens which is passed arguments and somehow matches and sets position 
Q: Are rules/tokens meant to be able to have arguments like methods and if so 
how do they fit in.
grammars don't check whether the things in their tokens/rules like <foo> are 
actually defined until it comes time to call them
Q: Is this the way it's meant to be?

I saw your post on doc.perl6.org docs. If I can get my head around all this I 
would be happy to help document grammars!



On 27/06/2012, at 12:49 AM, Moritz Lenz wrote:

> On 06/26/2012 02:04 PM, Lard Farnwell wrote:
>> Hi guys,
>> To understand and play around with perl6 grammars I was trying to do a 
>> simple NLP parts of speech parser in perl6 grammars. This is sort of what I 
>> did: 
>> ---------------------------
>> grammar Sentence{
>>        proto rule VP {*}
>>        proto rule NP {*}
>>      rule sentence {
>>              <imperative>|<statement>
>>      }
>>       rule imperative {<VP>}
>>       rule statement {<NP> <VP>}
>> }
>> grammar VerbPhrase is Sentence{
>>  rule VP:sym<hit>  {<sym>  <NP>}
>>  rule VP:sym<kill> {<sym>  <NP>}
>> }
>> grammar NounPhrase is Sentence{
>>      #define NP:sym etc
>> }
>> grammar English is NounPhrase is VerbPhrase {
>>      rule TOP {
>>              <Sentence>[\. <Sentence]*
>>        }
>> }
>> --------------------------------
>> So in case you don't get it, A sentence is made up of phrases which in turn 
>> can be made up on other phrases. And English is made up of Sentences.
>> This sort of thing works but doesn't make much sense.
>> The obvious problem is that to get the correct definitions of the proto 
>> rules in Sentence I have to say "verbPhrase is Sentence" and then "English 
>> is NounPhrase is VerbPhrase etc" .  This makes me feel like I'm doing it 
>> wrong.
> Indeed. The intended mechanism for code reuse in object oriented Perl 6
> code is role composition.
> Grammar rules and regexes are just methods, so defining them in a role
> and applying it to a class sounds like a good idea to me.
> role VerbPhrase {
>    rule VP { <verb> <NP> }
>    proto token verb  {*}
>    token verb:sym<hit>  { <sym> }
>    token verb:sym<kill> { <sym> }
> }
> Define NounPhrase in a similar way, leave out the definition of NP and
> VP from Sentence, and then write
> grammar English does NounPhrase does VerbPhrase is Sentence {
>    token TOP { ... }
> }
> Role composition has much more transparent error modes than inheritance,
> and probably works better for you.
>> How do I build a flexible dynamic grammar in a OO sort of way. For example 
>> how could I do this so:
>> 1) I define all my phrase structures (NP,VP,PP etc) in their own file while 
>> still being able to use each other. There are VPs can be made of NPs and NPs 
>> can be made up of VPs. 
> See above
>> 2) Add to these definitions dynamically. For example, here I have defined 
>> "hit and kill" VPs. What if I wanted to add "dance" VP definition at run 
>> time?
> In theory you can write
> role VerbPhrases[@verbs] {
>     token verb:sym<dynamic> { @verbs }
>     # note that 'dynamic' has no special meaning here, but since
>     # we don't use <sym> in the regex body, it doesn't matter what
>     # we write
> }
> And then instantiate your grammar as
> my $g = English.new does VerbPhrases[<dance listen juggle ...>];
> my $match = $g.parse($yourstring);
> But Rakudo doesn't yet properly handle array variables in regexes, so
> you have to write something like
> role AdditionalVerbPhrase[$verb] {
>    token verb:sym<dynamic> { $verb };
> }
> my $g = English.new;
> $g does AddtionalVerbPhrase[$_] for <dance listen juggle ...>;
> my $match = $g.parse(...);
> I haven't tested it though.
> If you experiment with it, please report your findings here, I'm curious
> about what works right now. If it doesn't work, we can surely find some
> way to make it work by going through the meta object to add methods to
> the grammar.
> Cheers,
> Moritz

Reply via email to