One thing to consider is whether the grammar under testing can be used to generate test inputs, as if that grammar has a bug, so will the inputs -- seems obvious, but I've not really tested it.
My experience of writing a parser for Lua was this: I took the EBNF grammar from the language spec, conferted it to SLIF manually, took the lua test suite files and tested by parsing them to AST, serializing the AST and running the lua interpreter to see if it produced the same results for oritinal file and file produced by serializing the ast. Then I wrote a round-trip parser and was able to just compare the original file and serialized AST. So, to me, testing a grammar boils down to finding a representative set of inputs and then using the reference semantics producer (lua interpreter in my case) to test serialized AST files. Hope this helps. [1] https://github.com/rns/MarpaX-Languages-Lua-AST/tree/master/t/lua5.1-tests On Fri, Aug 26, 2016 at 9:02 AM, <[email protected]> wrote: > While I realize the question is really in reference to particulars with > Marpa, I'd wondered how to "test grammars" as well. I'd initially thought > about proving equivalence, but here's a reasonable explanation about why > that's not possible: > > http://math.stackexchange.com/questions/231187/an-efficient- > way-to-determine-if-two-context-free-grammars-are-equivalent > > However, the discussion points to, > > "Comparison of Context-free Grammars Based on Parsing Generated Test Data" > http://slps.github.io/testmatch/ > > In the present paper, we leverage systematic test data generation, by >> which we mean that test data sets are generated by effective enumeration >> methods for the coverage criteria of interest. These methods do not require >> any configuration. Also, these methods imply minimality of the test data >> sets in both an intuitive and a formal sense. >> > > Has anyone any experience with this, or similar, efforts? > > > On Tuesday, August 9, 2016 at 7:21:55 PM UTC-7, [email protected] wrote: >> >> I know in advance that my target grammar is complex. So I would like to >> start at the lower, simpler levels and start testing my lexeme and grammar >> rules as I write them. >> >> * Can I change the starting rule of a (SLIF) grammar at runtime? I would >> like to test very basic rules -- the kind that I'll only see in slices far >> into a file -- (bottom-up) before defining the grammar from the top down. >> If I can specify a rule and a string (to G->parse or R->read), I can write >> easy regression tests that each rule recognizes valid strings and rejects >> invalid strings. I could modify the grammar file(s) for each test, but >> that seems like a bad idea. I'm all ears if there's a better way to do >> this. >> >> * At a higher level, can you point me to any great examples of regression >> tests for a Marpa grammar? >> >> * Even more generally, how do people develop and test a Marpa grammar? >> >> Thanks! >> >> - Ryan >> >> -- > You received this message because you are subscribed to the Google Groups > "marpa parser" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "marpa parser" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
