Thanks Juancarlo. The tokenizer was a component that already existed in ParseKit. So when I began work on the new PEG-based static packrat generator a few weeks ago, I took a shortcut. I used the existing tokenizer for token recognition. I may unify parser/tokenizer recognition in the future for the PEG-based generator (like ANTLR). But either way, the existing tokenizer will remain as a useful stand-alone component for other applications.
TD On Sun, May 5, 2013 at 7:57 AM, Juancarlo Añez <apal...@gmail.com> wrote: > Hello Todd, > > Interesting! > > Why did you include a tokenizer in a PEG parser? > > Cheers, > > > On Thu, May 2, 2013 at 1:44 PM, Todd Ditchendorf < > todd.ditchend...@gmail.com> wrote: > >> Hello PEG Mailing List. >> >> I'd like to announce the release of an updated version of my Objective-C >> framework, ParseKit. >> >> ParseKit is an Objective-C library that converts PEG-style grammars to >> Packrat parsers in the form of Objective-C source code suitable for use on >> the iOS or Mac OS X platforms. >> >> ParseKit is heavily inspired by ANTLR, and ParseKit grammars have a very >> similar syntax. >> >> Parsekit's home page is http://parsekit.com >> >> I've written a tutorial that describes the new PEG/Packrat features: >> http://itod.github.io/ParseKitMiniMathExample/ >> >> The parsers produced by ParseKit are recursive descent, deterministic, >> packrat (memoizing), and backtracking as I believe would be expected for a >> PEG/Packrat tool. ParseKit parsers also have the Semantic Predicate feature >> copied directly from ANTLR. >> >> ParseKit also includes an *extremely weak* version of ANTLR's LL(*) >> concept: Static grammar analysis will produce simple single-token >> prediction branching for any decision which is LL(1), but use full >> backtracking speculation for any non-LL(1) decision. However, ParseKit does >> not implement anything nearly as sophisticated as ANTLR's full LL(*)/DFA >> feature. >> >> Thanks to Terence Parr and Bryan Ford for your contributions to this >> field. I have really enjoyed reading your books/papers and have learned a >> great deal from you in this process. >> >> Todd Ditchendorf >> >> _______________________________________________ >> PEG mailing list >> PEG@lists.csail.mit.edu >> https://lists.csail.mit.edu/mailman/listinfo/peg >> >> > > > -- > Juancarlo *Añez* >
_______________________________________________ PEG mailing list PEG@lists.csail.mit.edu https://lists.csail.mit.edu/mailman/listinfo/peg
