I've brought this issue up in the past and had suggested some options I had in mind but now I've finally put the basics of those ideas in place so I figured I'd start a fresh thread. Recall that the issue at hand is that we don't have a test suite for GLVs as gremlin-test is bound to the JVM. We have some tricks that let us test gremlin-python with it but those tricks won't work for every language and we now have the first language in gremlin-dotnet and upcoming gremlin-javascript which won't support it (yes, i know that gremlin-javascript can run on the jvm but there are issues with getting it all to work with the test framework that make it unduly complicated).
On other threads I offered the idea that we look to use Gherkin to write general Gremlin test specifications, which then could be read and processed by the wide variety of test frameworks that can read that format - there tend to be Gherkin processors in just about every language - for example, see: https://cucumber.io/ I just created this issue: https://issues.apache.org/jira/browse/TINKERPOP-1784 and pushed this branch: https://github.com/apache/tinkerpop/tree/TINKERPOP-1784 which demonstrates how this works with gremlin-python. The basic anatomy of this setup involves this new directory in gremlin-test: https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/gremlin-test/features It contains the Gherkin .features files. These are the test specifications. They are written using gremlin-java as the "model" language. GLVs will then need to write some infrastructure to process these Gherkin files. The key to making this "easy" to implement will lie in our abiilty to keep the assertions we want to have relatively simple. The more simplistic the language in the Gherkin .feature files the easier the job it will be for GLVs to build their infrastructure. Of course, once that infrastructure is in place, the GLV developer just has to write the GLV version of the Gremlin specified in the .feature file. So you can look at all the "infrastructure" code here in this pair of files: https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/gremlin-python/src/main/jython/radish and then you can look at the GLV Gremlin translations specifically here: https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/gremlin-python/src/main/jython/radish/count_features_step.py#L34-L46 I think this approach works pretty well and solves our general problems for GLV testing. There is some pain up front in implementing the "infrastructure" but after that new Gremlin tests added to .feature files just need to translated in the GLV. I suppose we could "automate" a good portion of the translation with reflection of some sort. Anything else could just be handled manually. Not sure if we need to use this new model to wholly replace the old one. The process test suite has its place in helping graph database providers test their stuff. I also imagine that introducing this approach in that context would create a breaking change which we would then need to push off to 3.4.0. I suppose that gives us time to think, but for now it might not be best to conflate the two and just treat them as separate aspects of the test suite. Anyway - it's important we settle on an approach to testing as we really shouldn't do a GA release of the Gremlin .NET GLV without getting the test suite solid. please yell if you have any ideas or feedback on this approach.
