Great progress! I like how you avoided using ids, even if it adds some complexity to the transformation required.
The Python step definitions are still quite simple, I think it would be mostly the same on the rest of the languages. On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <spmalle...@gmail.com> wrote: > Just another update on progress with the test suite. The language of the > .feature files is getting slightly more complex as I try to translate more > and more of our Java process suite tests into the language of the gherkin > files. You can see here where I've added the ability to include parameters: > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/ > gremlin-test/features/filter/Has.feature#L33 > > I also came up with a method of asserting edges (didn't want to rely on ids > as it makes the gherkin harder to read, plus i didn't want to assume > TinkerGraph identifiers in case these tests were every used with some other > graph database that didn't use longs): > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/ > gremlin-test/features/map/Vertex.feature#L105-L111 > > for all those additions (and others) the logic required by the GLV to > process these tests has stayed surprisingly simple: > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/ > gremlin-python/src/main/jython/radish/feature_steps.py > > There's a fair bit of regex/string manipulation involved there, but it's > processing strings from the feature file so that's the nature of it I > suppose. I think I'm of the mind that I want to port all of the tests to > feature files, so I wrote this unit test to help validate that none were > missed: > > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/ > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/ > structure/FeatureCoverageTest.java > > I don't know that we have to get to 100% porting right away, but I think > that once we have these gherkin files written they not only become the > basis for our current GLV testing work, but we might be able to simply use > them for testing the Java stuff as well - that would rid us of having the > test code duplication. It also sets us up with a portable body of tests > that can be re-used in TinkerPop 4.x. > > I'm open to suggestions if anyone has any. > > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra < > jorgebaygon...@gmail.com > > wrote: > > > I was able to build a proof of concept for a Gherkin-based test runner in > > C#, that takes the proposed count and select features > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/ > > gremlin-test/features/map> > > and runs them using C# step definitions. > > > > It uses the Gherkin parser <https://github.com/cucumber/gherkin-dotnet> > > from > > cucumber, there isn't a release of the parser with .NET Core support so > > I've > > asked them to release one > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no > > limitation from their source files). If they are not able to release it > in > > the next few days, we can implement our own as it should be pretty > straight > > forward. > > > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <spmalle...@gmail.com> > > wrote: > > > > > Thanks for the update. I'm trying to keep the test language as simple > as > > > possible so that we don't need an overly complicated test > implementation. > > > Hopefully that will help make the .NET approach as easy as possible. > > > > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra < > > > jorgebaygon...@gmail.com> > > > wrote: > > > > > > > I've been looking into Gherkin support for .NET: SpecFlow, the > cucumber > > > > implementation for .NET <https://cucumber.io/docs# > > > cucumber-implementations > > > > >, > > > > does not support .NET Core platform (we use .NET Core build tools for > > the > > > > .NET GLV) and only supports .NET Framework. > > > > > > > > From what I can see <https://github.com/techtalk/SpecFlow/projects/2 > >, > > > > .NET > > > > Core support on SpecFlow is coming at a very slow pace and we > shouldn't > > > > expect to land any time soon (there were some design decisions in > > > SpecFlow > > > > library that makes supporting other platforms non-trivial, like > > requiring > > > > code gen). > > > > > > > > The alternative would be to implement our own harness to support it: > > > from a > > > > xunit test, look for certain types and parse the annotations, and > > execute > > > > them using the Gherkin features. > > > > There is a .NET cross-platform Gherkin parser: > > > > https://github.com/cucumber/gherkin-dotnet > > > > I'll continue looking into this option and try to understand the > effort > > > > required... > > > > > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra < > > > > jorgebaygon...@gmail.com> > > > > wrote: > > > > > > > > > Nice! Gherkin will make our lives easier with a growing number of > > GLVs. > > > > > > > > > > We should find a way to define the different features supported by > > each > > > > > GLV, as it's reasonable to have different maturity levels per GLV > > (ie: > > > > > lambdas support, traversal strategy, ...). I don't know if it will > be > > > > > beneficial to do it in the Gherkin files or within each GLV > > > > implementation. > > > > > Also, we should consider the process of rolling out a new method / > > > class > > > > > in the java implementation, how that could affect each GLV. > > > > > > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette < > > > spmalle...@gmail.com > > > > > > > > > > wrote: > > > > > > > > > >> that's what i meant by "reflection" or as you suggest eval(). I > > guess > > > > the > > > > >> point is that if the language can support some way of taking the > > > string > > > > >> value and turning it automatically into a traversal in that GLVs > > style > > > > >> then > > > > >> we should do that. > > > > >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz <m...@gremlin.guru> > > > > wrote: > > > > >> > > > > >> > For unparameterized queries it can probably be as easy as: > > > > >> > > > > > >> > @given("the traversal of") > > > > >> > def translate_traversal(step): > > > > >> > g = step.context.g > > > > >> > step.context.traversal = eval(step.text) > > > > >> > > > > > >> > > > > > >> > Cheers, > > > > >> > Daniel > > > > >> > > > > > >> > > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz <m...@gremlin.guru > > > > > > >> wrote: > > > > >> > > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for > years, > > > > but I > > > > >> > > really like the BDD approach. > > > > >> > > > > > > >> > > and then you can look at the GLV Gremlin translations > > specifically > > > > >> here: > > > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem > > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py# > > L34-L46 > > > > >> > > > > > > >> > > > > > > >> > > This part is the only thing that looks weird to me. You're > > > basically > > > > >> > > writing every query twice; is there really no easier way to do > > > that? > > > > >> > > > > > > >> > > Cheers, > > > > >> > > Daniel > > > > >> > > > > > > >> > > > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette < > > > > >> spmalle...@gmail.com> > > > > >> > > wrote: > > > > >> > > > > > > >> > >> I've brought this issue up in the past and had suggested some > > > > >> options I > > > > >> > >> had > > > > >> > >> in mind but now I've finally put the basics of those ideas in > > > place > > > > >> so I > > > > >> > >> figured I'd start a fresh thread. Recall that the issue at > hand > > > is > > > > >> that > > > > >> > we > > > > >> > >> don't have a test suite for GLVs as gremlin-test is bound to > > the > > > > >> JVM. We > > > > >> > >> have some tricks that let us test gremlin-python with it but > > > those > > > > >> > tricks > > > > >> > >> won't work for every language and we now have the first > > language > > > in > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which won't > > > support > > > > it > > > > >> > >> (yes, > > > > >> > >> i know that gremlin-javascript can run on the jvm but there > are > > > > >> issues > > > > >> > >> with > > > > >> > >> getting it all to work with the test framework that make it > > > unduly > > > > >> > >> complicated). > > > > >> > >> > > > > >> > >> On other threads I offered the idea that we look to use > Gherkin > > > to > > > > >> write > > > > >> > >> general Gremlin test specifications, which then could be read > > and > > > > >> > >> processed > > > > >> > >> by the wide variety of test frameworks that can read that > > format > > > - > > > > >> there > > > > >> > >> tend to be Gherkin processors in just about every language - > > for > > > > >> > example, > > > > >> > >> see: > > > > >> > >> > > > > >> > >> https://cucumber.io/ > > > > >> > >> > > > > >> > >> I just created this issue: > > > > >> > >> > > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784 > > > > >> > >> > > > > >> > >> and pushed this branch: > > > > >> > >> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784 > > > > >> > >> > > > > >> > >> which demonstrates how this works with gremlin-python. The > > basic > > > > >> anatomy > > > > >> > >> of > > > > >> > >> this setup involves this new directory in gremlin-test: > > > > >> > >> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem > > > > >> > >> lin-test/features > > > > >> > >> > > > > >> > >> It contains the Gherkin .features files. These are the test > > > > >> > >> specifications. > > > > >> > >> They are written using gremlin-java as the "model" language. > > GLVs > > > > >> will > > > > >> > >> then > > > > >> > >> need to write some infrastructure to process these Gherkin > > files. > > > > The > > > > >> > key > > > > >> > >> to making this "easy" to implement will lie in our abiilty to > > > keep > > > > >> the > > > > >> > >> assertions we want to have relatively simple. The more > > simplistic > > > > the > > > > >> > >> language in the Gherkin .feature files the easier the job it > > will > > > > be > > > > >> for > > > > >> > >> GLVs to build their infrastructure. Of course, once that > > > > >> infrastructure > > > > >> > is > > > > >> > >> in place, the GLV developer just has to write the GLV version > > of > > > > the > > > > >> > >> Gremlin specified in the .feature file. So you can look at > all > > > the > > > > >> > >> "infrastructure" code here in this pair of files: > > > > >> > >> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/grem > > > > >> > >> lin-python/src/main/jython/radish > > > > >> > >> > > > > >> > >> and then you can look at the GLV Gremlin translations > > > specifically > > > > >> here: > > > > >> > >> > > > > >> > >> https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/grem > > > > >> > >> lin-python/src/main/jython/radish/count_features_step.py# > > L34-L46 > > > > >> > >> > > > > >> > >> I think this approach works pretty well and solves our > general > > > > >> problems > > > > >> > >> for > > > > >> > >> GLV testing. There is some pain up front in implementing the > > > > >> > >> "infrastructure" but after that new Gremlin tests added to > > > .feature > > > > >> > files > > > > >> > >> just need to translated in the GLV. I suppose we could > > > "automate" a > > > > >> good > > > > >> > >> portion of the translation with reflection of some sort. > > Anything > > > > >> else > > > > >> > >> could just be handled manually. > > > > >> > >> > > > > >> > >> Not sure if we need to use this new model to wholly replace > the > > > old > > > > >> one. > > > > >> > >> The process test suite has its place in helping graph > database > > > > >> providers > > > > >> > >> test their stuff. I also imagine that introducing this > approach > > > in > > > > >> that > > > > >> > >> context would create a breaking change which we would then > need > > > to > > > > >> push > > > > >> > >> off > > > > >> > >> to 3.4.0. I suppose that gives us time to think, but for now > > it > > > > >> might > > > > >> > not > > > > >> > >> be best to conflate the two and just treat them as separate > > > aspects > > > > >> of > > > > >> > the > > > > >> > >> test suite. > > > > >> > >> > > > > >> > >> Anyway - it's important we settle on an approach to testing > as > > we > > > > >> really > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV without > > getting > > > > the > > > > >> > test > > > > >> > >> suite solid. please yell if you have any ideas or feedback on > > > this > > > > >> > >> approach. > > > > >> > >> > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > >
