I've been looking over the current test scenarios on the TINKERPOP-1784
branch, with the C# GLV test suite in mind, and oh man that's an impressive
amount of scenarios!
So far, I have a suggestion.
Instead of
Scenario: g_V_fold_countXlocalX
Given the modern graph
And the traversal of
"""
g.V().fold().count(Scope.local)
"""
We could use
Scenario: g_V_fold_countXlocalX
Given the modern graph
And parameter 1 being an enum
"""
Scope.local
"""
And the traversal of
"""
g.V().fold().count(param1)
"""
That way we can exclude enums from possible parameter values, narrowing
down the amount of types of parameters allowed, ie: const numeric, const
strings (wrapped in double quotes), traversals (starting with `__`) , ...,
making parsing the traversal a little easier.
On Thu, Oct 12, 2017 at 8:54 PM, Stephen Mallette <spmalle...@gmail.com>
wrote:
> Just a quick update on this item - it continues........
>
> Went down a bad path earlier in the week and ended up shelving a lot of
> work - ended up just coded into a corner. Anyway, I've not really changed
> much in the implementation, but I've still not reached a point where the
> addition of a new .feature files comes without meeting some new type of
> assertion that has to be dealt with. This ends up slowing down progress on
> porting over the tests. Please feel free to peruse the latest changes on
> the branch and let me know if there's any feedback.
>
>
>
> On Fri, Sep 29, 2017 at 12:05 PM, Stephen Mallette <spmalle...@gmail.com>
> wrote:
>
> > I did some pretty heavy refactoring to the python test logic (altered a
> > bit by some revision of the gherkin feature file language) and the result
> > is a much more simplified test logic file:
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-python/src/main/jython/radish/feature_steps.py
> >
> > About 120 lines of code (down from about 170). If you include the test
> > logic "setup" file:
> >
> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> > gremlin-python/src/main/jython/radish/terrain.py
> >
> > we end up with about 250 lines of test logic total (take out the
> > comment/license and we're probably well under 200 total - not too bad). I
> > think I'm closing in on the end to infrastructure building here so the
> > basic framework is getting close to final at this point I believe. I'll
> > keep scanning the tests looking for other types of assertions that I've
> not
> > yet covered, but it's getting pretty solid I think. Hopefully, there
> won't
> > need to be too many more lines of code needed to express the test logic
> as
> > I like how things are looking right now.
> >
> >
> >
> >
> > On Thu, Sep 28, 2017 at 8:02 AM, Jorge Bay Gondra <
> > jorgebaygon...@gmail.com> wrote:
> >
> >> Great progress! I like how you avoided using ids, even if it adds some
> >> complexity to the transformation required.
> >>
> >> The Python step definitions are still quite simple, I think it would be
> >> mostly the same on the rest of the languages.
> >>
> >> On Thu, Sep 28, 2017 at 1:45 PM, Stephen Mallette <spmalle...@gmail.com
> >
> >> wrote:
> >>
> >> > Just another update on progress with the test suite. The language of
> the
> >> > .feature files is getting slightly more complex as I try to translate
> >> more
> >> > and more of our Java process suite tests into the language of the
> >> gherkin
> >> > files. You can see here where I've added the ability to include
> >> parameters:
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-test/features/filter/Has.feature#L33
> >> >
> >> > I also came up with a method of asserting edges (didn't want to rely
> on
> >> ids
> >> > as it makes the gherkin harder to read, plus i didn't want to assume
> >> > TinkerGraph identifiers in case these tests were every used with some
> >> other
> >> > graph database that didn't use longs):
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-test/features/map/Vertex.feature#L105-L111
> >> >
> >> > for all those additions (and others) the logic required by the GLV to
> >> > process these tests has stayed surprisingly simple:
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-python/src/main/jython/radish/feature_steps.py
> >> >
> >> > There's a fair bit of regex/string manipulation involved there, but
> it's
> >> > processing strings from the feature file so that's the nature of it I
> >> > suppose. I think I'm of the mind that I want to port all of the tests
> to
> >> > feature files, so I wrote this unit test to help validate that none
> were
> >> > missed:
> >> >
> >> > https://github.com/apache/tinkerpop/blob/TINKERPOP-1784/
> >> > gremlin-test/src/test/java/org/apache/tinkerpop/gremlin/
> >> > structure/FeatureCoverageTest.java
> >> >
> >> > I don't know that we have to get to 100% porting right away, but I
> think
> >> > that once we have these gherkin files written they not only become the
> >> > basis for our current GLV testing work, but we might be able to simply
> >> use
> >> > them for testing the Java stuff as well - that would rid us of having
> >> the
> >> > test code duplication. It also sets us up with a portable body of
> tests
> >> > that can be re-used in TinkerPop 4.x.
> >> >
> >> > I'm open to suggestions if anyone has any.
> >> >
> >> > On Mon, Sep 25, 2017 at 11:23 AM, Jorge Bay Gondra <
> >> > jorgebaygon...@gmail.com
> >> > > wrote:
> >> >
> >> > > I was able to build a proof of concept for a Gherkin-based test
> >> runner in
> >> > > C#, that takes the proposed count and select features
> >> > > <https://github.com/apache/tinkerpop/tree/TINKERPOP-1784/
> >> > > gremlin-test/features/map>
> >> > > and runs them using C# step definitions.
> >> > >
> >> > > It uses the Gherkin parser <https://github.com/cucumber/g
> >> herkin-dotnet>
> >> > > from
> >> > > cucumber, there isn't a release of the parser with .NET Core support
> >> so
> >> > > I've
> >> > > asked them to release one
> >> > > <https://github.com/cucumber/gherkin-dotnet/issues/11> (there is no
> >> > > limitation from their source files). If they are not able to release
> >> it
> >> > in
> >> > > the next few days, we can implement our own as it should be pretty
> >> > straight
> >> > > forward.
> >> > >
> >> > > On Fri, Sep 22, 2017 at 2:23 PM, Stephen Mallette <
> >> spmalle...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Thanks for the update. I'm trying to keep the test language as
> >> simple
> >> > as
> >> > > > possible so that we don't need an overly complicated test
> >> > implementation.
> >> > > > Hopefully that will help make the .NET approach as easy as
> possible.
> >> > > >
> >> > > > On Fri, Sep 22, 2017 at 8:20 AM, Jorge Bay Gondra <
> >> > > > jorgebaygon...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > I've been looking into Gherkin support for .NET: SpecFlow, the
> >> > cucumber
> >> > > > > implementation for .NET <https://cucumber.io/docs#
> >> > > > cucumber-implementations
> >> > > > > >,
> >> > > > > does not support .NET Core platform (we use .NET Core build
> tools
> >> for
> >> > > the
> >> > > > > .NET GLV) and only supports .NET Framework.
> >> > > > >
> >> > > > > From what I can see <https://github.com/techtalk/S
> >> pecFlow/projects/2
> >> > >,
> >> > > > > .NET
> >> > > > > Core support on SpecFlow is coming at a very slow pace and we
> >> > shouldn't
> >> > > > > expect to land any time soon (there were some design decisions
> in
> >> > > > SpecFlow
> >> > > > > library that makes supporting other platforms non-trivial, like
> >> > > requiring
> >> > > > > code gen).
> >> > > > >
> >> > > > > The alternative would be to implement our own harness to support
> >> it:
> >> > > > from a
> >> > > > > xunit test, look for certain types and parse the annotations,
> and
> >> > > execute
> >> > > > > them using the Gherkin features.
> >> > > > > There is a .NET cross-platform Gherkin parser:
> >> > > > > https://github.com/cucumber/gherkin-dotnet
> >> > > > > I'll continue looking into this option and try to understand the
> >> > effort
> >> > > > > required...
> >> > > > >
> >> > > > > On Tue, Sep 19, 2017 at 6:21 PM, Jorge Bay Gondra <
> >> > > > > jorgebaygon...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Nice! Gherkin will make our lives easier with a growing number
> >> of
> >> > > GLVs.
> >> > > > > >
> >> > > > > > We should find a way to define the different features
> supported
> >> by
> >> > > each
> >> > > > > > GLV, as it's reasonable to have different maturity levels per
> >> GLV
> >> > > (ie:
> >> > > > > > lambdas support, traversal strategy, ...). I don't know if it
> >> will
> >> > be
> >> > > > > > beneficial to do it in the Gherkin files or within each GLV
> >> > > > > implementation.
> >> > > > > > Also, we should consider the process of rolling out a new
> >> method /
> >> > > > class
> >> > > > > > in the java implementation, how that could affect each GLV.
> >> > > > > >
> >> > > > > > On Thu, Sep 14, 2017 at 11:12 PM, Stephen Mallette <
> >> > > > spmalle...@gmail.com
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > >> that's what i meant by "reflection" or as you suggest
> eval(). I
> >> > > guess
> >> > > > > the
> >> > > > > >> point is that if the language can support some way of taking
> >> the
> >> > > > string
> >> > > > > >> value and turning it automatically into a traversal in that
> >> GLVs
> >> > > style
> >> > > > > >> then
> >> > > > > >> we should do that.
> >> > > > > >>
> >> > > > > >> On Thu, Sep 14, 2017 at 4:45 PM, Daniel Kuppitz
> >> <m...@gremlin.guru>
> >> > > > > wrote:
> >> > > > > >>
> >> > > > > >> > For unparameterized queries it can probably be as easy as:
> >> > > > > >> >
> >> > > > > >> > @given("the traversal of")
> >> > > > > >> > def translate_traversal(step):
> >> > > > > >> > g = step.context.g
> >> > > > > >> > step.context.traversal = eval(step.text)
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > Cheers,
> >> > > > > >> > Daniel
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> > On Thu, Sep 14, 2017 at 1:39 PM, Daniel Kuppitz
> >> <m...@gremlin.guru
> >> > >
> >> > > > > >> wrote:
> >> > > > > >> >
> >> > > > > >> > > That's great stuff. I haven't used Cucumber / Gherkin for
> >> > years,
> >> > > > > but I
> >> > > > > >> > > really like the BDD approach.
> >> > > > > >> > >
> >> > > > > >> > > and then you can look at the GLV Gremlin translations
> >> > > specifically
> >> > > > > >> here:
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-python/src/main/jython/
> radish/count_features_step.py#
> >> > > L34-L46
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > This part is the only thing that looks weird to me.
> You're
> >> > > > basically
> >> > > > > >> > > writing every query twice; is there really no easier way
> >> to do
> >> > > > that?
> >> > > > > >> > >
> >> > > > > >> > > Cheers,
> >> > > > > >> > > Daniel
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> > > On Thu, Sep 14, 2017 at 1:17 PM, Stephen Mallette <
> >> > > > > >> spmalle...@gmail.com>
> >> > > > > >> > > wrote:
> >> > > > > >> > >
> >> > > > > >> > >> I've brought this issue up in the past and had suggested
> >> some
> >> > > > > >> options I
> >> > > > > >> > >> had
> >> > > > > >> > >> in mind but now I've finally put the basics of those
> >> ideas in
> >> > > > place
> >> > > > > >> so I
> >> > > > > >> > >> figured I'd start a fresh thread. Recall that the issue
> at
> >> > hand
> >> > > > is
> >> > > > > >> that
> >> > > > > >> > we
> >> > > > > >> > >> don't have a test suite for GLVs as gremlin-test is
> bound
> >> to
> >> > > the
> >> > > > > >> JVM. We
> >> > > > > >> > >> have some tricks that let us test gremlin-python with it
> >> but
> >> > > > those
> >> > > > > >> > tricks
> >> > > > > >> > >> won't work for every language and we now have the first
> >> > > language
> >> > > > in
> >> > > > > >> > >> gremlin-dotnet and upcoming gremlin-javascript which
> won't
> >> > > > support
> >> > > > > it
> >> > > > > >> > >> (yes,
> >> > > > > >> > >> i know that gremlin-javascript can run on the jvm but
> >> there
> >> > are
> >> > > > > >> issues
> >> > > > > >> > >> with
> >> > > > > >> > >> getting it all to work with the test framework that make
> >> it
> >> > > > unduly
> >> > > > > >> > >> complicated).
> >> > > > > >> > >>
> >> > > > > >> > >> On other threads I offered the idea that we look to use
> >> > Gherkin
> >> > > > to
> >> > > > > >> write
> >> > > > > >> > >> general Gremlin test specifications, which then could be
> >> read
> >> > > and
> >> > > > > >> > >> processed
> >> > > > > >> > >> by the wide variety of test frameworks that can read
> that
> >> > > format
> >> > > > -
> >> > > > > >> there
> >> > > > > >> > >> tend to be Gherkin processors in just about every
> >> language -
> >> > > for
> >> > > > > >> > example,
> >> > > > > >> > >> see:
> >> > > > > >> > >>
> >> > > > > >> > >> https://cucumber.io/
> >> > > > > >> > >>
> >> > > > > >> > >> I just created this issue:
> >> > > > > >> > >>
> >> > > > > >> > >> https://issues.apache.org/jira/browse/TINKERPOP-1784
> >> > > > > >> > >>
> >> > > > > >> > >> and pushed this branch:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tinkerpop/tree/TINKERPOP-1784
> >> > > > > >> > >>
> >> > > > > >> > >> which demonstrates how this works with gremlin-python.
> The
> >> > > basic
> >> > > > > >> anatomy
> >> > > > > >> > >> of
> >> > > > > >> > >> this setup involves this new directory in gremlin-test:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-test/features
> >> > > > > >> > >>
> >> > > > > >> > >> It contains the Gherkin .features files. These are the
> >> test
> >> > > > > >> > >> specifications.
> >> > > > > >> > >> They are written using gremlin-java as the "model"
> >> language.
> >> > > GLVs
> >> > > > > >> will
> >> > > > > >> > >> then
> >> > > > > >> > >> need to write some infrastructure to process these
> Gherkin
> >> > > files.
> >> > > > > The
> >> > > > > >> > key
> >> > > > > >> > >> to making this "easy" to implement will lie in our
> >> abiilty to
> >> > > > keep
> >> > > > > >> the
> >> > > > > >> > >> assertions we want to have relatively simple. The more
> >> > > simplistic
> >> > > > > the
> >> > > > > >> > >> language in the Gherkin .feature files the easier the
> job
> >> it
> >> > > will
> >> > > > > be
> >> > > > > >> for
> >> > > > > >> > >> GLVs to build their infrastructure. Of course, once that
> >> > > > > >> infrastructure
> >> > > > > >> > is
> >> > > > > >> > >> in place, the GLV developer just has to write the GLV
> >> version
> >> > > of
> >> > > > > the
> >> > > > > >> > >> Gremlin specified in the .feature file. So you can look
> at
> >> > all
> >> > > > the
> >> > > > > >> > >> "infrastructure" code here in this pair of files:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/tree/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-python/src/main/jython/radish
> >> > > > > >> > >>
> >> > > > > >> > >> and then you can look at the GLV Gremlin translations
> >> > > > specifically
> >> > > > > >> here:
> >> > > > > >> > >>
> >> > > > > >> > >> https://github.com/apache/tink
> >> erpop/blob/TINKERPOP-1784/grem
> >> > > > > >> > >> lin-python/src/main/jython/
> radish/count_features_step.py#
> >> > > L34-L46
> >> > > > > >> > >>
> >> > > > > >> > >> I think this approach works pretty well and solves our
> >> > general
> >> > > > > >> problems
> >> > > > > >> > >> for
> >> > > > > >> > >> GLV testing. There is some pain up front in implementing
> >> the
> >> > > > > >> > >> "infrastructure" but after that new Gremlin tests added
> to
> >> > > > .feature
> >> > > > > >> > files
> >> > > > > >> > >> just need to translated in the GLV. I suppose we could
> >> > > > "automate" a
> >> > > > > >> good
> >> > > > > >> > >> portion of the translation with reflection of some sort.
> >> > > Anything
> >> > > > > >> else
> >> > > > > >> > >> could just be handled manually.
> >> > > > > >> > >>
> >> > > > > >> > >> Not sure if we need to use this new model to wholly
> >> replace
> >> > the
> >> > > > old
> >> > > > > >> one.
> >> > > > > >> > >> The process test suite has its place in helping graph
> >> > database
> >> > > > > >> providers
> >> > > > > >> > >> test their stuff. I also imagine that introducing this
> >> > approach
> >> > > > in
> >> > > > > >> that
> >> > > > > >> > >> context would create a breaking change which we would
> then
> >> > need
> >> > > > to
> >> > > > > >> push
> >> > > > > >> > >> off
> >> > > > > >> > >> to 3.4.0. I suppose that gives us time to think, but
> for
> >> now
> >> > > it
> >> > > > > >> might
> >> > > > > >> > not
> >> > > > > >> > >> be best to conflate the two and just treat them as
> >> separate
> >> > > > aspects
> >> > > > > >> of
> >> > > > > >> > the
> >> > > > > >> > >> test suite.
> >> > > > > >> > >>
> >> > > > > >> > >> Anyway - it's important we settle on an approach to
> >> testing
> >> > as
> >> > > we
> >> > > > > >> really
> >> > > > > >> > >> shouldn't do a GA release of the Gremlin .NET GLV
> without
> >> > > getting
> >> > > > > the
> >> > > > > >> > test
> >> > > > > >> > >> suite solid. please yell if you have any ideas or
> >> feedback on
> >> > > > this
> >> > > > > >> > >> approach.
> >> > > > > >> > >>
> >> > > > > >> > >
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
