Your summary looks good to me. I also agree that we can omit 3b from this
body of work. Going back to an earlier question that you had:

>   Can we spilt the change to support for gremlin request/script first and
bytecode/traversal separately?

I'm a bit concerned that if we don't at least have the design of the
bytecode/traversal piece thought through fully we might not get the client
side changes right to support it. Perhaps we see how things go on
implementation before making a final decision here, but as of right now, I
tend to feel like we shouldn't split this up.

To just get started, I think I'd like to get the initial development branch
setup with Gremlin Server returning some metadata. That part is pretty
easy. Once that branch is setup, anyone contributing can just submit PRs
against that or just commit directly to it. Then we can have one PR to
evaluate through the review process for this feature

I guess there is some question as to what version this body of work should
go to. I think I'd like to skip adding this to 3.2.8 and target 3.3.2 on
the tp33 branch. At this point, I think 3.2.x should really just contain
bug fixes and minor enhancements - I don't think this particular
enhancement qualifies as "minor".

Ashwini, if all that sounds good, could you please create a JIRA ticket
that contains a final summary and references this thread?





On Mon, Mar 5, 2018 at 2:47 PM, Ashwini Singh <
ashws...@microsoft.com.invalid> wrote:

>
> To summarize what we have discussed so far:
>         1.  For API using GremlinRequest/QueryScript, expose response
> attribute as part of result. Using an approach to similar to Java client
> driver (using ResultSet) . [Priority0]
>                 -- We rely on the last message for response attributes.
>         2. For GLV, add response attribute as part of Traversal. [Priority
> 0]
>                 --Rely on the last message for attributes.
>         3. Expose other server details (like server setting).  I would
> suggest to split this design discussion into two directions:
>                 a. Metadata for request execution: Only focuses on details
> related to request execution. Can be achieved through #1 and #2.
>                 b. Metadata for Gremlin Server:  Focuses on overall
> metadata for the server. Could be a separate request and fetch the data
> once for a connection. Any other suggestion ?
>
> Please add if I am missing something.
>
> Thanks a lot,
> Ashwini Singh
> Software Engineer @ Azure Cosmos DB
>
> -----Original Message-----
> From: Stephen Mallette <spmalle...@gmail.com>
> Sent: Friday, March 2, 2018 4:49 AM
> To: dev@tinkerpop.apache.org
> Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients.
>
> Perhaps another useful piece of data to return from Gremlin Server:
>
> https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%
> 2FTINKERPOP-1636&data=04%7C01%7Cashwsing%40microsoft.com%
> 7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af91ab2d7cd011
> db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=
> bVd71RV9uDiWWgsjY7SYAIYEhWtLdi7Ijgv9um%2F1JoM%3D&reserved=0
>
> On Thu, Mar 1, 2018 at 6:31 PM, Stephen Mallette <spmalle...@gmail.com>
> wrote:
>
> > I just came across this:
> >
> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue
> > s.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1494&data=04%7C01%7Cashwsing%
> > 40microsoft.com%7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af9
> > 1ab2d7cd011db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8e
> > yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata
> > =1kY8r73w2Y3euQVnLEGIuAO0fX6ZySOzl%2BVUgbhodpE%3D&reserved=0
> >
> > There's some ideas for what the open source Gremlin Server could
> > return there....for example, perhaps we send back the the host that
> > executed the script as the metadata for Gremlin Server?
> >
> > On Wed, Feb 28, 2018 at 5:37 PM, Ashwini Singh <ashws...@microsoft.com.
> > invalid> wrote:
> >
> >>
> >> Completely agree on making the change for other drivers too. The plan
> >> from our side is to make changes to other drivers as well.  My focus
> >> on Gremlin.Net was just for an example . Yes, we should make change
> >> to Gremlin Server to write the metadata for testing.
> >>
> >> I agree to adding this to ResultSet and exposing this as part of
> >> every response for all the drivers and follow the same pattern as JAVA.
> >>
> >> On the gremlin GVL part,  I agree we should support this on traversal
> >> for bytecode as well. For now, we do not have support for bytecode
> >> but we are actively working on that and need support for metadata there
> as well.
> >> Thanks for bringing this up. Just wanted to understand how we track
> >> issues at tinkerpop, Can we spilt the change to support for gremlin
> >> request/script first and bytecode/traversal separately? If we prefer
> >> to split the change in chunks, but completely fine with one change.
> >>
> >> Thanks,
> >> Ashwini Singh
> >> Software Engineer @ Azure Cosmos DB
> >>
> >> -----Original Message-----
> >> From: Stephen Mallette <spmalle...@gmail.com>
> >> Sent: Wednesday, February 28, 2018 9:58 AM
> >> To: dev@tinkerpop.apache.org
> >> Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients.
> >>
> >> This newly created issue might be related to this in some way:
> >>
> >> https://na01.safelinks.protection.outlook.com/?url=https%3A%
> >> 2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1906&
> >> data=04%7C01%7Cashwsing%40microsoft.com%7C614b32ff7f98
> >> 4c66554808d57ed4d22b%7C72f988bf86f141af91ab2d7cd011db47%7C1%
> >> 7C0%7C636554374885412678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> >> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&
> >> sdata=w7pyg6JF4KMbPdpBg6t%2F8cE%2BflbnCQQdyMUgc6HDrXw%3D&reserved=0
> >>
> >> On Wed, Feb 28, 2018 at 11:37 AM, Florian Hockmann <
> >> f...@florian-hockmann.de>
> >> wrote:
> >>
> >> > I agree that we also should make this metadata available for
> >> > traversals since we want to move users away from sending Gremlin
> >> > scripts as strings and instead use Bytecode based GLVs.
> >> >
> >> > Regarding Gremlin.Net: I think that the implementation would be
> >> > very similar to how it can be implemented in the Java driver as we
> >> > tried to stay close to the Java driver in general. The only
> >> > difference is probably that we currently don't have a ResultSet in
> >> > Gremlin.Net, but that's only because I didn't see much value in
> >> > adding that. Metadata would of course be a good argument to also
> >> > implement a ResultSet in Gremlin.Net and then the implementation
> >> > should be really basically the same as in the Java driver.
> >> >
> >> >
> >> > Am 28.02.2018 um 16:15 schrieb Stephen Mallette:
> >> > > I'm fine with using the last response message as the carrier for
> >> > > this metadata on a particular request. I can't really tell if
> >> > > there is much
> >> > work
> >> > > to do on Gremlin Server itself here. It seems like most of the
> >> > > work must occur on the various drivers (you mention the .NET api,
> >> > > but all of the drivers would need to support this feature).
> >> > > However, I would think that
> >> > we
> >> > > would want Gremlin Server itself to append in some kind metadata
> >> > > (maybe query time? something easy....) so that we could write
> >> > > tests for the drivers in TinkerPop itself. There is also the
> >> > > question of how we would expose this metadata to GLVs which don't
> >> > > see response messages at all. A traversal might need some
> >> > > metadata itself so that the user could retrieve the server
> >> > > metadata from that. The implementation between Java and the
> >> > GLVs
> >> > > might be different here as the GLV traversal class is typically
> >> > > quite lightweight and only used for generating bytecode.
> >> > >
> >> > > I'm not so sure I like the  SubmitAsynWithHeaders()  but I don't
> >> > > think
> >> > too
> >> > > much about how the .NET driver works. Is there a reason to not
> >> > > always return metadata? could it be expensive to do so? If we
> >> > > just added an
> >> > extra
> >> > > method how would remote traversals configure this option? I think
> >> > > we need another way. Generally speaking, for Java, I think I
> >> > > would like to see
> >> > the
> >> > > metadata available to the ResultSet somehow which would in turn
> >> > > make it pretty easy to get it on to a Traversal instance once
> >> > > that facility was made available.....but as to how to enable or
> >> > > disable the return of the metadata, i'm not sure how that should
> >> > > work just yet - i need to think on that some more.
> >> > >
> >> > > For committers who work on GLVs, please take a look at this
> >> > > thread and offer your thoughts on how this might work in the GLV
> >> > > driver you tend to have the most knowledge on. Let's see if we
> >> > > can come to one nice unified solution. At that point, we can
> >> > > setup a ticket in JIRA
> >> and go from there.
> >> > >
> >> > > Ashwini, thanks for offering a pull request for this by the way.
> >> > > Once we get consensus on how to do this, we'll see if tasks need
> >> > > to be divided
> >> > and
> >> > > how you might contribute.
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Feb 26, 2018 at 6:45 PM, Ashwini Singh <
> >> > > ashws...@microsoft.com.invalid> wrote:
> >> > >
> >> > >> Hi Stephen,
> >> > >>
> >> > >> Thanks for considering the change.
> >> > >>
> >> > >> We would be more inclined towards the first approach since the
> >> > >> having a ping/pong websocket message can be a bit noisy and
> >> > >> requires
> >> > sophisticated
> >> > >> handling on the client driver side.
> >> > >>
> >> > >> For handling multiple response messages. I would suggest to rely
> >> > >> on last message as these are the metadata for request execution.
> >> > >> Partial
> >> > response
> >> > >> is very internal to the client drivers (based on limited
> >> > >> understanding
> >> > of
> >> > >> tinkerpop client drivers :) , correct me if you differ) and can
> >> > >> be
> >> > exposed
> >> > >> separately (if really needed later).
> >> > >>
> >> > >> For implementation, Let us know if we can chip in there and
> >> > >> submit
> >> PR.
> >> > The
> >> > >> high level approach to achieve this is to have corresponding
> >> > >> SubmitAsynWithHeaders()  for every SubmitAsync() that returns a
> >> > >> encapsulated result with attributes and IReadOnlyCollectio<T>.
> >> > >> Let me
> >> > know
> >> > >> if you see any concerns adding a new API.
> >> > >>
> >> > >> Thanks a lot,
> >> > >> Ashwini Singh
> >> > >> Software Engineer @ Azure Cosmos DB
> >> > >>
> >> > >>
> >> > >>
> >> > >> -----Original Message-----
> >> > >> From: Stephen Mallette <spmalle...@gmail.com>
> >> > >> Sent: Friday, February 23, 2018 5:13 PM
> >> > >> To: dev@tinkerpop.apache.org
> >> > >> Subject: Re: [Discuss] Expose metadata from Gremlin Server to
> >> Clients.
> >> > >>
> >> > >> Adding those kinds of details was the reason we had the
> >> > >> ResponseStatus.attributes Map. I can really only speak for the
> >> > >> java
> >> > driver
> >> > >> as I only know that one really well (we might need other
> >> > >> TinkerPop
> >> > experts
> >> > >> to chime in for python, .net and c#).  The java driver doesn't
> >> > >> really present ways to get that information easily under usage
> >> > >> that doesn't
> >> > deal
> >> > >> directly with RequestMessage directly (which people typically
> >> > >> don't
> >> do).
> >> > >> Another thing to think about is that since a single request
> >> > >> might return multiple ResponseMessage instances you might not
> >> > >> want to return that
> >> > kind
> >> > >> of data on every response - maybe just to be returned on the
> >> > >> first (or last
> >> > >> message) and then we somehow preserve that information and make
> >> > >> it accessible on the result somehow....we sorta have some kinda
> >> > >> of
> >> > precedent
> >> > >> for that with side-effect data generated by bytecode based
> >> > >> traversals -
> >> > we
> >> > >> can probably build in something similar for this sort of thing.
> >> > >>
> >> > >> I also toyed with the idea of using ping/pong websocket messages
> >> > >> to
> >> > carry
> >> > >> general information about the server to the client. Not sure if
> >> > >> any of
> >> > the
> >> > >> metadata you want to send back would fit in there, but that
> >> > >> could be another option.
> >> > >>
> >> > >> Does any of that sound helpful?
> >> > >>
> >> > >> On Fri, Feb 23, 2018 at 3:41 PM, Ashwini Singh <
> >> > >> ashws...@microsoft.com.invalid> wrote:
> >> > >>
> >> > >>> Hi All,
> >> > >>>
> >> > >>> We are working on to expose metadata as part of gremlin to
> >> > >>> response to client. The metadata is simply a property bag to
> >> > >>> provide special message/hints to the client. But currently
> >> > >>> client libraries strip off everything and only return the data
> >> > >>> to the
> >> client.
> >> > >>>
> >> > >>> Specifically, We want to expose details like Request Charge,
> >> > >>> Rate limiting/Retry policy details etc. In the other scenarios
> >> > >>> in Cosmos DB we provide these details to the client is through
> >> > >>> response headers. We did some investigation around this and one
> >> > >>> of the options is expose these is through response attributes.
> >> > >>> Gremlin Server can add metadata as part of gremlin response
> >> > >>> attributes (For example, set the property bag on
> >> > >>> ResponseStatus.Attributes for Gremlin.Net) that can be
> >> > >>> serialized
> >> by the client drivers to the clients.
> >> > >>>
> >> > >>> We  would like to learn more if there are precedence around
> >> > >>> this and if there are any recommended ways to achieve this in
> >> > >>> Gremlin protocol and client drivers.
> >> > >>>
> >> > >>> Thanks a lot,
> >> > >>> Ashwini Singh
> >> > >>> Software Engineer @ Azure Cosmos DB
> >> > >>>
> >> > >>>
> >> >
> >> >
> >> >
> >>
> >
> >
>

Reply via email to