Re: Which Parser Generator to Use?
Hi Ovid! On Tuesday 08 Sep 2009 22:46:45 Ovid wrote: --- On Sun, 6/9/09, Shlomi Fish shlo...@iglu.org.il wrote: From: Shlomi Fish shlo...@iglu.org.il 2. Parse::RecDescent - very impressive feature set, but a little slow, and has been under-maintained (though it seemed to have improved slightly with several new releases in 2009). It also tends to be hard to debug its errors. Hi Shlomi, I didn't see it mentioned yet, but you might want to check out Damian Conway's new http://search.cpan.org/dist/Regexp-Grammars/ It requires 5.10, but it's much faster than Parse::RecDescent and has a clean syntax very close to Perl 6 rules. It's well documented and hooks directly into the Perl regex engine, hence its speed. Thanks for the recommendation! I've converted the code in the svn to use it. After some bugs I had in my grammar, due to things I didn't understand there, I got all tests to finally pass there. Now, it seems to be working pretty well. So now I can continue to enhance Qantor after a long time of inability to work on it. Regexp-Grammars seems very nifty. Regards, Shlomi Fish -- - Shlomi Fish http://www.shlomifish.org/ http://www.shlomifish.org/humour/ways_to_do_it.html Chuck Norris read the entire English Wikipedia in 24 hours. Twice.
Re: Which Parser Generator to Use?
Howdy, I haven't used it in production, but you might want to look at HOP::Parser as well: http://search.cpan.org/~ovid/HOP-Parser-0.02/ Cheers, On Sun, Sep 6, 2009 at 7:40 AM, Shlomi Fishshlo...@iglu.org.il wrote: Hi all! Which parser generator do you recommend to use for a Perl project. What I've looked at so far: 1. Berkeley Yacc for Perl - works pretty well, but is kinda limited. 2. Parse::RecDescent - very impressive feature set, but a little slow, and has been under-maintained (though it seemed to have improved slightly with several new releases in 2009). It also tends to be hard to debug its errors. 3. Parse::Yapp - http://search.cpan.org/dist/Parse-Yapp/ - I tried to use it in https://svn.berlios.de/svnroot/repos/web-cpan/Text-Qantor/ but it gives me an error for what appears to be a valid syntax, and for the life of me I cannot understand why it is. 4. There's a new version of GNU bison with support for multiple language backends. I tried writing a backend for Perl 5, but I gave up on the m4 hacking (I think that m4 must die!). 5. There's also ANTLR - http://www.antlr.org/ : http://www.antlr.org/wiki/display/ANTLR3/Code+Generation+Targets says: Perl - Early prototyping. Simple lexer is working. 6. Can I interact with the Parrot Grammar Engine (PGE)? Any input would be useful. -- I probably missed many others. Any recommendations would be appreciated. Regards, Shlomi Fish -- - Shlomi Fish http://www.shlomifish.org/ The Case for File Swapping - http://shlom.in/file-swap Chuck Norris read the entire English Wikipedia in 24 hours. Twice. -- Jonathan Leto jonat...@leto.net http://leto.net
Re: Which Parser Generator to Use?
--- On Sun, 6/9/09, Shlomi Fish shlo...@iglu.org.il wrote: From: Shlomi Fish shlo...@iglu.org.il 2. Parse::RecDescent - very impressive feature set, but a little slow, and has been under-maintained (though it seemed to have improved slightly with several new releases in 2009). It also tends to be hard to debug its errors. Hi Shlomi, I didn't see it mentioned yet, but you might want to check out Damian Conway's new http://search.cpan.org/dist/Regexp-Grammars/ It requires 5.10, but it's much faster than Parse::RecDescent and has a clean syntax very close to Perl 6 rules. It's well documented and hooks directly into the Perl regex engine, hence its speed. Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog- http://use.perl.org/~Ovid/journal/ Twitter - http://twitter.com/OvidPerl Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Re: Which Parser Generator to Use?
v6.pm has some support for grammars. here is one usage example: http://cpansearch.perl.org/src/MSILVA/Language-Tea-0.03/lib/Language/Tea/Grammar.pm - Flávio S. Glock
Re: Which Parser Generator to Use?
1. Berkeley Yacc for Perl - works pretty well, but is kinda limited. I'm not sure what (if any) practical advantage this would have over bison. I get the sense it's less well maintained. 2. Parse::RecDescent - very impressive feature set, but a little slow, and has been under-maintained (though it seemed to have improved slightly with several new releases in 2009). It also tends to be hard to debug its errors. I tried this. Works ok, two or three orders of magnitude slower than bison/C for me. Debugging tools are pretty good. Docs are pretty good. 3. Parse::Yapp - http://search.cpan.org/dist/Parse-Yapp/ - I tried to use it in https://svn.berlios.de/svnroot/repos/web-cpan/Text-Qantor/ but it gives me an error for what appears to be a valid syntax, and for the life of me I cannot understand why it is. For people who aren't experts in the field most of the grammar errors are completely inscrutable for all of these tools, even after reading the docs. Start with something simple and make it more complex until it stops working, then try phrasing it differently. Well, works for me. I haven't tried this specific tool. 4. There's a new version of GNU bison with support for multiple language backends. I tried writing a backend for Perl 5, but I gave up on the m4 hacking (I think that m4 must die!). Bison is fast and relatively simple. I'm not sure about using it directly from perl, but I wrote a C++ program to parse router configs and spit out Data::Dumper() style perl struct output. Very fast. _Relatively_ easy to add in exceptions for poorly behaved grammar. You should be able to use this directly via XS or Inline if you care about more direct integration. bison + c/c++ + valgrind works pretty well to make a well behaved parser (w/out valgrind I always leak a bunch of memory when writing in more uh.. hands on languages). I did look at yacc but bison has more features. Also bison is consistent across platforms. 5. There's also ANTLR - http://www.antlr.org/ : http://www.antlr.org/wiki/display/ANTLR3/Code+Generation+Targets says: Perl - Early prototyping. Simple lexer is working. I have used the python version of antlr. I found it to be more difficult than bison/C (steeper learning curve), and maybe one or two order of magnitude slower than bison/C. The author and the users seem very competent, but if you are like me and just want to get some parsing done this may be more effort than it's worth. Cross language support is excellent between java/python, not sure about perl. If you are trying to create an AST it will probably work fine. Beyond that... ? If you are writing your own grammar this is a really powerful tool. If you are stuck trying to parse someone else's jive it may not be as useful. Error messages for the user when something isn't parsable are also fairly inscrutable, imo. 6. Can I interact with the Parrot Grammar Engine (PGE)? Any input would be useful. If you look at the parrot stuff I would be interested to hear how well that work for you. There's also the 'roll your own' approach. That's pretty fast and less difficult than you might expect. Also depending on what you are parsing, if a grammar is poorly behaved (possibly because the !...@!@ing vendor decides to be retarded instead of consistent) it can be brutal to try to add weird exceptions in a format any of the above tools will recognize happily. Another advantage is that when you write your own parser it's easier to figure out what happens and what to do when you get something unexpected. Knowing what I know now I would tend to opt for bison/C or roll your own, depending on how speed critical your app is. If I was writing my own language maybe antlr. If you would like examples for any of the above I'd be happy to share, please email me off-list. Austin
Which Parser Generator to Use?
Hi all! Which parser generator do you recommend to use for a Perl project. What I've looked at so far: 1. Berkeley Yacc for Perl - works pretty well, but is kinda limited. 2. Parse::RecDescent - very impressive feature set, but a little slow, and has been under-maintained (though it seemed to have improved slightly with several new releases in 2009). It also tends to be hard to debug its errors. 3. Parse::Yapp - http://search.cpan.org/dist/Parse-Yapp/ - I tried to use it in https://svn.berlios.de/svnroot/repos/web-cpan/Text-Qantor/ but it gives me an error for what appears to be a valid syntax, and for the life of me I cannot understand why it is. 4. There's a new version of GNU bison with support for multiple language backends. I tried writing a backend for Perl 5, but I gave up on the m4 hacking (I think that m4 must die!). 5. There's also ANTLR - http://www.antlr.org/ : http://www.antlr.org/wiki/display/ANTLR3/Code+Generation+Targets says: Perl - Early prototyping. Simple lexer is working. 6. Can I interact with the Parrot Grammar Engine (PGE)? Any input would be useful. -- I probably missed many others. Any recommendations would be appreciated. Regards, Shlomi Fish -- - Shlomi Fish http://www.shlomifish.org/ The Case for File Swapping - http://shlom.in/file-swap Chuck Norris read the entire English Wikipedia in 24 hours. Twice.
Re: Which Parser Generator to Use?
On Sunday 06 September 2009 10:40:44 Shlomi Fish wrote:
Hi all!
Which parser generator do you recommend to use for a Perl project. What
I've looked at so far:
1. Berkeley Yacc for Perl - works pretty well, but is kinda limited.
Can't say i've used it before, but if its really Yacc then i believe that
Parse::Yapp does a better job of having a perl yacc.
2. Parse::RecDescent - very impressive feature set, but a little slow, and
has been under-maintained (though it seemed to have improved slightly with
several new releases in 2009). It also tends to be hard to debug its
errors.
I've mostly had issues with either speed or getting precedence right with
Parse::RecDescent
3. Parse::Yapp - http://search.cpan.org/dist/Parse-Yapp/ - I tried to use
it in https://svn.berlios.de/svnroot/repos/web-cpan/Text-Qantor/ but it
gives me an error for what appears to be a valid syntax, and for the life
of me I cannot understand why it is.
there's an ambiguity there in your grammar, i've run into that many times with
Math::Farnsworth
yapp -v will make a file.output that describes everything in the grammar and
can easily help with the debugging of things, the warnings about useless and
unused terminals are harmless (though could mean you've got a
typo). useless rules just means that they're in there but can't be reached,
its sort of like
if (something that is always true)
{
do something here
}
else
{
die true != true;
}
your shift reduce conflict is in the rule
plain_para_text: TEXT
| plain_para_text TEXT { my $t1 = $_[1] ; my $t2 = $_[2] ; [$t1-[0].$t2-
[0], $t1-[1]] }
;
its happening because it can recurse infinitely on itself and then see the TEXT
rule, (it'd also be expecting a second TEXT after that)
i'm not 100% sure what you were trying to do there, but the following rule
does what i THINK you were intending
plain_para_text: TEXT { my $t1 = $_[1] ; my $t2 = $_[2] ; [$t1-[0].$t2-[0],
$t1-[1]] };
as far as bison and ANTLR go i've never used either of them so i won't
comment, and i've got no idea about using PGE with perl so...
There's also Parse::Earley out there i played with that a bit but not as much
as i have with RecDescent and Yapp
P.S. sorry if this got sent twice, i found an oddity in my smtp settings and
don't think anything was ever making it out.
