I was able to build a proof of concept for a Gherkin-based test runner in C#, that takes the proposed count and select features <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/gremlin-test/features/map> and runs them using C# step definitions.
It uses the Gherkin parser <https://github.com/cucumber/gherkin-dotnet> from cucumber, there isn't a release of the parser with .NET Core support so I've asked them to release one <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no limitation from their source files). If they are not able to release it in the next few days, we can implement our own as it should be pretty straight forward. On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <spmalle...@gmail.com> wrote: > Thanks for the update. I'm trying to keep the test language as simple as > possible so that we don't need an overly complicated test implementation. > Hopefully that will help make the .NET approach as easy as possible. > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra < > jorgebaygon...@gmail.com> > wrote: > > > I've been looking into Gherkin support for .NET: SpecFlow, the cucumber > > implementation for .NET <https://cucumber.io/docs# > cucumber-implementations > > >, > > does not support .NET Core platform (we use .NET Core build tools for the > > .NET GLV) and only supports .NET Framework. > > > > From what I can see <https://github.com/techtalk/SpecFlow/projects/2>, > > .NET > > Core support on SpecFlow is coming at a very slow pace and we shouldn't > > expect to land any time soon (there were some design decisions in > SpecFlow > > library that makes supporting other platforms non-trivial, like requiring > > code gen). > > > > The alternative would be to implement our own harness to support it: > from a > > xunit test, look for certain types and parse the annotations, and execute > > them using the Gherkin features. > > There is a .NET cross-platform Gherkin parser: > > https://github.com/cucumber/gherkin-dotnet > > I'll continue looking into this option and try to understand the effort > > required... > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra < > > jorgebaygon...@gmail.com> > > wrote: > > > > > Nice! Gherkin will make our lives easier with a growing number of GLVs. > > > > > > We should find a way to define the different features supported by each > > > GLV, as it's reasonable to have different maturity levels per GLV (ie: > > > lambdas support, traversal strategy, ...). I don't know if it will be > > > beneficial to do it in the Gherkin files or within each GLV > > implementation. > > > Also, we should consider the process of rolling out a new method / > class > > > in the java implementation, how that could affect each GLV. > > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette < > spmalle...@gmail.com > > > > > > wrote: > > > > > >> that's what i meant by "reflection" or as you suggest eval(). I guess > > the > > >> point is that if the language can support some way of taking the > string > > >> value and turning it automatically into a traversal in that GLVs style > > >> then > > >> we should do that. > > >> > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <m...@gremlin.guru> > > wrote: > > >> > > >> > For unparameterized queries it can probably be as easy as: > > >> > > > >> > @given("the traversal of") > > >> > def translate_traversal(step): > > >> > g = step.context.g > > >> > step.context.traversal = eval(step.text) > > >> > > > >> > > > >> > Cheers, > > >> > Daniel > > >> > > > >> > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <m...@gremlin.guru> > > >> wrote: > > >> > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for years, > > but I > > >> > > really like the BDD approach. > > >> > > > > >> > > and then you can look at the GLV Gremlin translations specifically > > >> here: > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46 > > >> > > > > >> > > > > >> > > This part is the only thing that looks weird to me. You're > basically > > >> > > writing every query twice; is there really no easier way to do > that? > > >> > > > > >> > > Cheers, > > >> > > Daniel > > >> > > > > >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette < > > >> spmalle...@gmail.com> > > >> > > wrote: > > >> > > > > >> > >> I've brought this issue up in the past and had suggested some > > >> options I > > >> > >> had > > >> > >> in mind but now I've finally put the basics of those ideas in > place > > >> so I > > >> > >> figured I'd start a fresh thread. Recall that the issue at hand > is > > >> that > > >> > we > > >> > >> don't have a test suite for GLVs as gremlin-test is bound to the > > >> JVM. We > > >> > >> have some tricks that let us test gremlin-python with it but > those > > >> > tricks > > >> > >> won't work for every language and we now have the first language > in > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which won't > support > > it > > >> > >> (yes, > > >> > >> i know that gremlin-javascript can run on the jvm but there are > > >> issues > > >> > >> with > > >> > >> getting it all to work with the test framework that make it > unduly > > >> > >> complicated). > > >> > >> > > >> > >> On other threads I offered the idea that we look to use Gherkin > to > > >> write > > >> > >> general Gremlin test specifications, which then could be read and > > >> > >> processed > > >> > >> by the wide variety of test frameworks that can read that format > - > > >> there > > >> > >> tend to be Gherkin processors in just about every language - for > > >> > example, > > >> > >> see: > > >> > >> > > >> > >> https://cucumber.io/ > > >> > >> > > >> > >> I just created this issue: > > >> > >> > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784 > > >> > >> > > >> > >> and pushed this branch: > > >> > >> > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784 > > >> > >> > > >> > >> which demonstrates how this works with gremlin-python. The basic > > >> anatomy > > >> > >> of > > >> > >> this setup involves this new directory in gremlin-test: > > >> > >> > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem > > >> > >> lin-test/features > > >> > >> > > >> > >> It contains the Gherkin .features files. These are the test > > >> > >> specifications. > > >> > >> They are written using gremlin-java as the "model" language. GLVs > > >> will > > >> > >> then > > >> > >> need to write some infrastructure to process these Gherkin files. > > The > > >> > key > > >> > >> to making this "easy" to implement will lie in our abiilty to > keep > > >> the > > >> > >> assertions we want to have relatively simple. The more simplistic > > the > > >> > >> language in the Gherkin .feature files the easier the job it will > > be > > >> for > > >> > >> GLVs to build their infrastructure. Of course, once that > > >> infrastructure > > >> > is > > >> > >> in place, the GLV developer just has to write the GLV version of > > the > > >> > >> Gremlin specified in the .feature file. So you can look at all > the > > >> > >> "infrastructure" code here in this pair of files: > > >> > >> > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem > > >> > >> lin-python/src/main/jython/radish > > >> > >> > > >> > >> and then you can look at the GLV Gremlin translations > specifically > > >> here: > > >> > >> > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem > > >> > >> lin-python/src/main/jython/radish/count_features_step.py#L34-L46 > > >> > >> > > >> > >> I think this approach works pretty well and solves our general > > >> problems > > >> > >> for > > >> > >> GLV testing. There is some pain up front in implementing the > > >> > >> "infrastructure" but after that new Gremlin tests added to > .feature > > >> > files > > >> > >> just need to translated in the GLV. I suppose we could > "automate" a > > >> good > > >> > >> portion of the translation with reflection of some sort. Anything > > >> else > > >> > >> could just be handled manually. > > >> > >> > > >> > >> Not sure if we need to use this new model to wholly replace the > old > > >> one. > > >> > >> The process test suite has its place in helping graph database > > >> providers > > >> > >> test their stuff. I also imagine that introducing this approach > in > > >> that > > >> > >> context would create a breaking change which we would then need > to > > >> push > > >> > >> off > > >> > >> to 3.4.0. I suppose that gives us time to think, but for now it > > >> might > > >> > not > > >> > >> be best to conflate the two and just treat them as separate > aspects > > >> of > > >> > the > > >> > >> test suite. > > >> > >> > > >> > >> Anyway - it's important we settle on an approach to testing as we > > >> really > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV without getting > > the > > >> > test > > >> > >> suite solid. please yell if you have any ideas or feedback on > this > > >> > >> approach. > > >> > >> > > >> > > > > >> > > > > >> > > > >> > > > > > > > > >
