Cool. I flip/flop back and forth about having better support for general
predicates (like text and geo) - seems like this would be a argument in
favor of adding such things to resolve this problem of having to write
server side DSL code. if done right it might save graph providers from
having to write their on extensions to GLVs .
Out of curiosity, what graph database are you using?
On Thu, Feb 15, 2018 at 11:39 AM, Moore, Branden James <bjm...@sandia.gov>
wrote:
> > Can you talk about what those custom steps do? Do you also have custom
> > TraversalStrategies which interact with them?
>
> We do not have any custom TraversalStrategies yet.
>
> We do have custom predicates, though not many... Mostly string operations
> (textContains, textRegex).
>
> Most of our custom steps can be implemented as plain Gremlin, however,
> we've found that because we know the shape of our graphs (it starts with an
> explicit tree structure, with other edges layered on top), we can write
> some steps more efficiently, or just more easily, as direct TinkerPop java
> Vertex/Edge operations.
>
> One of our custom steps, however, is a work-around to the fact that the
> NESTED_LOOP traversal requirement, still, cannot be met.
>
>
>
> On 2/15/18, 9:10 AM, "Stephen Mallette" <spmalle...@gmail.com> wrote:
>
> > Do you expect to *NOT* support server-side DSLs in the future (even
> as
> Bytecode)?
>
> I think that's it's a bit early to say that definitively, but all
> community
> discussion thus far has pointed in the direction of keeping DSLs a
> client-side construct, so I'd expect that we would not do much work to
> support them (i.e. make them easier to implement, write documentation,
> discuss them as best practices, etc) in an official capacity for
> end-users.
>
> There are a lot of reasons for that (some technical and some not), but
> I
> think that one of the biggest ones I tend to think of the most these
> days
> is that we are seeing more and more TinkerPop-enabled. graph systems
> that
> are server oriented (e.g. DSE Graph) or simply a managed service (e.g
> CosmosDB) which don't allow a lot of control of what happens on the
> server.
> They just accept bytecode/scripts, process them and send results.
> Embracing
> that model from the TinkerPop perspective vastly simplifies the
> interfaces
> we need to expose to Graph System Providers who want to be
> TinkerPop-enabled and vastly reduces the surface area of interaction
> that
> users have to face to use Gremlin. When I consider embracing that
> model in
> relation to DSLs, I tend to see DSLs as lightweight client-only
> implementations that don't introduce headaches to the Graph Systems for
> which TinkerPop doesn't really offer much answer (e.g how do i allow
> users
> to deploy code to the servers, can i allow custom steps to be deployed
> securely, what about serialization of these custom steps?). I think
> we'd
> want to take those issues off the table.
>
> > We do have a handful of custom steps
>
> That's interesting. Again, I think this puts you in a different
> category
> that goes beyond what DSLs were intended for. Going down this route
> makes
> for highly advanced usage. Custom steps on the server would mean a GLV
> would need to be extended to support those steps which ties into
> bytecode
> serialization for both client and server. Complicated stuff without
> use of
> scripts (which is why you are using them i gather).
>
> Can you talk about what those custom steps do? Do you also have custom
> TraversalStrategies which interact with them?
>
>
> On Thu, Feb 15, 2018 at 10:17 AM, Moore, Branden James <
> bjm...@sandia.gov>
> wrote:
>
> > Now that python-gremlin is mature enough to use and extend, I'm
> working to
> > migrate our environment to a fully session-less, bytecode-based
> > environment. However, we currently have significant amounts of
> "legacy"
> > groovy/gremlin hanging around. Until all of that can be migrated,
> we still
> > need to use the older model.
> >
> > We do have a handful of custom steps that are not implemented as
> > combinations of existing Gremlin steps, which does drive us to a
> > server-side DSL.
> >
> > Do you expect to *NOT* support server-side DSLs in the future (even
> as
> > Bytecode)?
> >
> >
> > On 2/15/18, 6:08 AM, "Stephen Mallette" <spmalle...@gmail.com>
> wrote:
> >
> > > Perhaps if there was a way to specify a custom "__" class to
> the
> > ImportCustomizer, this would all solve itself?
> >
> > yes - it might. i don't think we should go down that path though.
> > first of
> > all, i think the workaround i suggested seems like the way to do
> this
> > within the context of what we have right now (unless that
> doesn't work
> > for
> > some reason). second, the approach you're taking with DSLs is not
> > really
> > something we are planning on directly supporting - it's
> availability
> > is a
> > bit of a side-effect of many other older design decisions. Going
> > forward,
> > I'd say that the consensus on TinkerPop's general direction is:
> >
> > 1. prefer sessionless requests over sessions
> > 2. bytecode based traversal are the future, not scripts
> > 3. DSLs are a client side feature and are designed with item 2
> in mind
> >
> > Is there a particular use case you are trying to satisfy that
> forces
> > you
> > down the path of sessions and server-side DSLs? I think we've all
> > spent a
> > fair time considering where 1-3 breaks down and how important
> those
> > break
> > downs are for usability. To me, the main use case for
> server-side DSLs
> > would be if you had some custom steps that you'd written that you
> > wanted in
> > the DSL. But, to me, that's almost past DSL level. If you're
> doing
> > that,
> > you've drifted into the general category of Graph System
> Provider and
> > if
> > you've done that, then you have a very heavy weight
> implementation to
> > manage and as such you are in the minority of users (and thus -
> little
> > impact with that breakdown).
> >
> > Don't want to panic anyone with 1-3 if you're new to the
> list....It's
> > not
> > like TinkerPop 3.4.0 is going to drop script support or anything
> like
> > that.
> > I'm just trying to say that developers who want to be most
> inline with
> > the
> > latest thinking on the future direction of the project should
> look to
> > conform to 1-3 where possible.
> >
> >
> > On Wed, Feb 14, 2018 at 6:41 PM, Moore, Branden James <
> > bjm...@sandia.gov>
> > wrote:
> >
> > > I think I figured out why 'label' isn't getting imported...
> It is
> > being
> > > overwritten in the import map by the __.label() method static
> > import.
> > > Normally, the CoreGremlinPlugin imports 'T.label' and
> '__.label()'
> > is
> > > filtered out of the imports on line 59 of
> > ImportGroovyCustomizer.java (It
> > > was probably taken care of via line 55).
> > >
> > > Perhaps if there was a way to specify a custom "__" class to
> the
> > > ImportCustomizer, this would all solve itself?
> > >
> > >
> > > On 2/14/18, 12:32 PM, "Stephen Mallette" <spmalle...@gmail.com
> >
> > wrote:
> > >
> > > Sorry - those imports are really mysterious to me. I dug
> through
> > a
> > > fair bit
> > > of groovy code trying to figure out what the rules were
> for that
> > import
> > > stuff and i don't recall getting to the bottom of it.
> > >
> > > On Wed, Feb 14, 2018 at 2:27 PM, Moore, Branden James <
> > > bjm...@sandia.gov>
> > > wrote:
> > >
> > > > So I did a quick experiment of:
> > > >
> > > > --- a/gremlin-groovy/src/main/java/org/apache/tinkerpop/
> > > > gremlin/groovy/jsr223/GremlinGroovyScriptEngine.java
> > > > +++ b/gremlin-groovy/src/main/java/org/apache/tinkerpop/
> > > > gremlin/groovy/jsr223/GremlinGroovyScriptEngine.java
> > > > - CoreGremlinPlugin.instance().
> getCustomizers("gremlin-
> > > groovy").ifPresent(c
> > > > -> listOfCustomizers.addAll(Arrays.asList(c)));
> > > > + CoreGremlinPlugin.instance().
> getCustomizers("gremlin-
> > > groovy").ifPresent(c
> > > > -> listOfCustomizers.addAll(0, Arrays.asList(c)));
> > > >
> > > >
> > > > While this does allow my DSL's __ to be the one found,
> oddly,
> > some
> > > other
> > > > static imports no longer appear in the interpreter;
> > specifically
> > > 'label'.
> > > > (Referencing 'T.label' works fine.) It isn't that the
> symbol
> > lookup
> > > finds
> > > > the incorrect class; I get the error "groovy.lang.
> > > MissingPropertyException:
> > > > No such property: label for class: Script2", however,
> some
> > other
> > > enums (ie,
> > > > 'incr') are imported just fine.
> > > >
> > > > Any ideas why that would occur?
> > > >
> > > > On 2/14/18, 10:46 AM, "Stephen Mallette" <
> spmalle...@gmail.com
> > >
> > > wrote:
> > > >
> > > > There was a time when I looked into that in pretty
> grave
> > detail.
> > > I
> > > > don't
> > > > recall my findings exactly, but I obviously didn't
> come up
> > with
> > > a nice
> > > > solution. I'm not sure that I ever became convinced
> that
> > any of
> > > this
> > > > groovy
> > > > import stuff behaves in a completely deterministic
> way. I
> > > suppose you
> > > > could
> > > > try it out and see if that change helps or not....
> > > >
> > > > On Wed, Feb 14, 2018 at 12:42 PM, Moore, Branden
> James <
> > > > bjm...@sandia.gov>
> > > > wrote:
> > > >
> > > > > Thanks for fixing issue #1. I figured that one
> would
> > be easy
> > > to
> > > > fix.
> > > > >
> > > > > As for issue #2, I'm wondering if changing the
> > > > GremlinGroovyScriptEngine
> > > > > to add it's ImportCustomizer (line 247) to the
> > beginning of
> > > the
> > > > list of
> > > > > customizers, rather than the end, would allow DSLs
> (and
> > any
> > > other
> > > > imports)
> > > > > to override the default Gremlin imports.
> > > > >
> > > > > - Branden
> > > > >
> > > > > On 2/14/18, 8:23 AM, "Stephen Mallette" <
> > spmalle...@gmail.com>
> > > > wrote:
> > > > >
> > > > > Oops - the `getAnonymousTraversalClass()`
> should get
> > > generated
> > > > > properly. I
> > > > > created this issue:
> > > > >
> > > > > https://issues.apache.org/
> jira/browse/TINKERPOP-1890
> > > > >
> > > > > which i already pushed a fix for:
> > > > >
> > > > > https://github.com/apache/tinkerpop/commit/
> > > > > 2d7113aaa166b69a8503be27aebf36a8063b82bd
> > > > >
> > > > > your second problem....hmm - not sure what to
> do
> > with that.
> > > > Trying to
> > > > > think
> > > > > of what could be done:
> > > > >
> > > > > 1. Ugly, but you could always specify the
> package
> > > > name......super gross
> > > > > 2. You could statically import your anonymous
> steps
> > in a
> > > custom
> > > > import
> > > > > in
> > > > > Gremlin Server so instead of
> g.persons(__.xxx()) you
> > would
> > > do
> > > > > g.persons(xxx()).
> > > > > 3. It would be a bit of work, but I suppose
> the DSL
> > > processor
> > > > could be
> > > > > modified somehow to allow you to rename the DSL
> > double
> > > underscore
> > > > > class to
> > > > > something else (triple underscore ??? hehe),
> then you
> > > could just
> > > > > import it.
> > > > >
> > > > > Seems like option 2 would work best in the
> short
> > term.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Feb 13, 2018 at 6:50 PM, Moore, Branden
> > James <
> > > > > bjm...@sandia.gov>
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > We are using the @GremlinDsl annotation to
> > extend the
> > > Gremlin
> > > > > language
> > > > > > into our own DSL. I’ve run into two issues
> with
> > > anonymous
> > > > > traversals so
> > > > > > far that I’d like to bring up. One has a
> > work-around,
> > > the
> > > > other, I
> > > > > have
> > > > > > not yet found a work-around.
> > > > > >
> > > > > > First (the one with the work-around): The
> > > <DSL>TraversalSource
> > > > > class that
> > > > > > is generated does not override the
> > > > ‘getAnonymousTraversalClass()’
> > > > > method of
> > > > > > GraphTraversalSource, so the returned class
> is
> > > > > org.apache.tinkerpop.gremlin.
> > process.traversal.dsl.graph.__
> > > > > > rather than the auto-generated, DSL-specific
> > version of
> > > ‘__’.
> > > > > This can
> > > > > > be worked around by specifying your own base
> class
> > that
> > > > implementes
> > > > > > ‘getAnonymousTraversalClass()’. However,
> this
> > still
> > > > requires some
> > > > > > oddities, as the code generator interprets
> the
> > method as
> > > a
> > > > method to
> > > > > be
> > > > > > auto-generated. My solution was to create
> two
> > levels of
> > > > > inheritance:
> > > > > >
> > > > > > 1. MyDSLTraversalSourceDsl0 extends
> > > GraphTraversalSource,
> > > > and
> > > > > > implements ‘getAnonymousTraversalClass()’
> > > > > > 2. MyDSLTraversalSourceDsl1 extends
> > > > MyDSLTraversalSourceDsl0, but
> > > > > does
> > > > > > nothing else
> > > > > > 3. MyDSLTraversalDSL extends
> > > GraphTRaversal.Admin<S,E>, and
> > > > uses
> > > > > the
> > > > > > @GremlinDsl(traversalSource =
> “MyDSLTraversalDSL1”)
> > > > > >
> > > > > >
> > > > > >
> > > > > > The second issue, for which I have not yet
> found a
> > > > work-around, is
> > > > > that
> > > > > > when using the Gremlin-Groovy scriptEngine
> as a
> > Gremlin
> > > > Server, and
> > > > > sending
> > > > > > “gremlin” to the server (rather than
> bytecode),
> > anonymous
> > > > traversals
> > > > > do not
> > > > > > find my DSL’s implementation of __, but
> rather the
> > > TinkerPop
> > > > __.
> > > > > I’ve
> > > > > > added my ‘__’ to the ImportGremlinPlugin’s
> > > classImports. This
> > > > is
> > > > > > sufficient for sending bytecode and having
> my __
> > found.
> > > > However,
> > > > > when
> > > > > > sending “gremlin” to the Session Processor,
> with
> > “eval”
> > > as the
> > > > OP,
> > > > > the
> > > > > > groovy class cache finds TinkerPop’s __
> rather
> > than my
> > > __.
> > > > This is
> > > > > > appears to be because in
> > GremlinGroovyScriptEngine, the
> > > > > CoreGremlinPlugin’s
> > > > > > customizers get added last to the list of
> > > ImportCustomizers.
> > > > As it
> > > > > is
> > > > > > processed last, when building the map of
> class
> > names to
> > > > > fully-qualified
> > > > > > class names, the Gremlin ‘__’ key overwrites
> the
> > ‘__’ key
> > > > which was
> > > > > > specified to be imported in the server’s
> YAML. I
> > also
> > > came
> > > > across
> > > > > an
> > > > > > interesting comment in
> ‘ImportGroovyCustomizer’
> > which
> > > forces an
> > > > > import of
> > > > > > Tinkerpop’s ‘__’ as well.
> > > > > >
> > > > > > It's quite possible that I’m missing
> something, if
> > so,
> > > could
> > > > you
> > > > > please
> > > > > > point me to how one is supposed to enable a
> custom
> > DSL
> > > with the
> > > > > > Gremlin-Groovy script engine?
> > > > > >
> > > > > > Thanks much,
> > > > > >
> > > > > > * Branden
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
