To summarize what we have discussed so far:
1. For API using GremlinRequest/QueryScript, expose response attribute
as part of result. Using an approach to similar to Java client driver (using
ResultSet) . [Priority0]
-- We rely on the last message for response attributes.
2. For GLV, add response attribute as part of Traversal. [Priority 0]
--Rely on the last message for attributes.
3. Expose other server details (like server setting). I would suggest
to split this design discussion into two directions:
a. Metadata for request execution: Only focuses on details
related to request execution. Can be achieved through #1 and #2.
b. Metadata for Gremlin Server: Focuses on overall metadata
for the server. Could be a separate request and fetch the data once for a
connection. Any other suggestion ?
Please add if I am missing something.
Thanks a lot,
Ashwini Singh
Software Engineer @ Azure Cosmos DB
-----Original Message-----
From: Stephen Mallette <[email protected]>
Sent: Friday, March 2, 2018 4:49 AM
To: [email protected]
Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients.
Perhaps another useful piece of data to return from Gremlin Server:
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1636&data=04%7C01%7Cashwsing%40microsoft.com%7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=bVd71RV9uDiWWgsjY7SYAIYEhWtLdi7Ijgv9um%2F1JoM%3D&reserved=0
On Thu, Mar 1, 2018 at 6:31 PM, Stephen Mallette <[email protected]>
wrote:
> I just came across this:
>
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue
> s.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1494&data=04%7C01%7Cashwsing%
> 40microsoft.com%7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af9
> 1ab2d7cd011db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8e
> yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata
> =1kY8r73w2Y3euQVnLEGIuAO0fX6ZySOzl%2BVUgbhodpE%3D&reserved=0
>
> There's some ideas for what the open source Gremlin Server could
> return there....for example, perhaps we send back the the host that
> executed the script as the metadata for Gremlin Server?
>
> On Wed, Feb 28, 2018 at 5:37 PM, Ashwini Singh <[email protected].
> invalid> wrote:
>
>>
>> Completely agree on making the change for other drivers too. The plan
>> from our side is to make changes to other drivers as well. My focus
>> on Gremlin.Net was just for an example . Yes, we should make change
>> to Gremlin Server to write the metadata for testing.
>>
>> I agree to adding this to ResultSet and exposing this as part of
>> every response for all the drivers and follow the same pattern as JAVA.
>>
>> On the gremlin GVL part, I agree we should support this on traversal
>> for bytecode as well. For now, we do not have support for bytecode
>> but we are actively working on that and need support for metadata there as
>> well.
>> Thanks for bringing this up. Just wanted to understand how we track
>> issues at tinkerpop, Can we spilt the change to support for gremlin
>> request/script first and bytecode/traversal separately? If we prefer
>> to split the change in chunks, but completely fine with one change.
>>
>> Thanks,
>> Ashwini Singh
>> Software Engineer @ Azure Cosmos DB
>>
>> -----Original Message-----
>> From: Stephen Mallette <[email protected]>
>> Sent: Wednesday, February 28, 2018 9:58 AM
>> To: [email protected]
>> Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients.
>>
>> This newly created issue might be related to this in some way:
>>
>> https://na01.safelinks.protection.outlook.com/?url=https%3A%
>> 2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1906&
>> data=04%7C01%7Cashwsing%40microsoft.com%7C614b32ff7f98
>> 4c66554808d57ed4d22b%7C72f988bf86f141af91ab2d7cd011db47%7C1%
>> 7C0%7C636554374885412678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
>> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&
>> sdata=w7pyg6JF4KMbPdpBg6t%2F8cE%2BflbnCQQdyMUgc6HDrXw%3D&reserved=0
>>
>> On Wed, Feb 28, 2018 at 11:37 AM, Florian Hockmann <
>> [email protected]>
>> wrote:
>>
>> > I agree that we also should make this metadata available for
>> > traversals since we want to move users away from sending Gremlin
>> > scripts as strings and instead use Bytecode based GLVs.
>> >
>> > Regarding Gremlin.Net: I think that the implementation would be
>> > very similar to how it can be implemented in the Java driver as we
>> > tried to stay close to the Java driver in general. The only
>> > difference is probably that we currently don't have a ResultSet in
>> > Gremlin.Net, but that's only because I didn't see much value in
>> > adding that. Metadata would of course be a good argument to also
>> > implement a ResultSet in Gremlin.Net and then the implementation
>> > should be really basically the same as in the Java driver.
>> >
>> >
>> > Am 28.02.2018 um 16:15 schrieb Stephen Mallette:
>> > > I'm fine with using the last response message as the carrier for
>> > > this metadata on a particular request. I can't really tell if
>> > > there is much
>> > work
>> > > to do on Gremlin Server itself here. It seems like most of the
>> > > work must occur on the various drivers (you mention the .NET api,
>> > > but all of the drivers would need to support this feature).
>> > > However, I would think that
>> > we
>> > > would want Gremlin Server itself to append in some kind metadata
>> > > (maybe query time? something easy....) so that we could write
>> > > tests for the drivers in TinkerPop itself. There is also the
>> > > question of how we would expose this metadata to GLVs which don't
>> > > see response messages at all. A traversal might need some
>> > > metadata itself so that the user could retrieve the server
>> > > metadata from that. The implementation between Java and the
>> > GLVs
>> > > might be different here as the GLV traversal class is typically
>> > > quite lightweight and only used for generating bytecode.
>> > >
>> > > I'm not so sure I like the SubmitAsynWithHeaders() but I don't
>> > > think
>> > too
>> > > much about how the .NET driver works. Is there a reason to not
>> > > always return metadata? could it be expensive to do so? If we
>> > > just added an
>> > extra
>> > > method how would remote traversals configure this option? I think
>> > > we need another way. Generally speaking, for Java, I think I
>> > > would like to see
>> > the
>> > > metadata available to the ResultSet somehow which would in turn
>> > > make it pretty easy to get it on to a Traversal instance once
>> > > that facility was made available.....but as to how to enable or
>> > > disable the return of the metadata, i'm not sure how that should
>> > > work just yet - i need to think on that some more.
>> > >
>> > > For committers who work on GLVs, please take a look at this
>> > > thread and offer your thoughts on how this might work in the GLV
>> > > driver you tend to have the most knowledge on. Let's see if we
>> > > can come to one nice unified solution. At that point, we can
>> > > setup a ticket in JIRA
>> and go from there.
>> > >
>> > > Ashwini, thanks for offering a pull request for this by the way.
>> > > Once we get consensus on how to do this, we'll see if tasks need
>> > > to be divided
>> > and
>> > > how you might contribute.
>> > >
>> > >
>> > >
>> > > On Mon, Feb 26, 2018 at 6:45 PM, Ashwini Singh <
>> > > [email protected]> wrote:
>> > >
>> > >> Hi Stephen,
>> > >>
>> > >> Thanks for considering the change.
>> > >>
>> > >> We would be more inclined towards the first approach since the
>> > >> having a ping/pong websocket message can be a bit noisy and
>> > >> requires
>> > sophisticated
>> > >> handling on the client driver side.
>> > >>
>> > >> For handling multiple response messages. I would suggest to rely
>> > >> on last message as these are the metadata for request execution.
>> > >> Partial
>> > response
>> > >> is very internal to the client drivers (based on limited
>> > >> understanding
>> > of
>> > >> tinkerpop client drivers :) , correct me if you differ) and can
>> > >> be
>> > exposed
>> > >> separately (if really needed later).
>> > >>
>> > >> For implementation, Let us know if we can chip in there and
>> > >> submit
>> PR.
>> > The
>> > >> high level approach to achieve this is to have corresponding
>> > >> SubmitAsynWithHeaders() for every SubmitAsync() that returns a
>> > >> encapsulated result with attributes and IReadOnlyCollectio<T>.
>> > >> Let me
>> > know
>> > >> if you see any concerns adding a new API.
>> > >>
>> > >> Thanks a lot,
>> > >> Ashwini Singh
>> > >> Software Engineer @ Azure Cosmos DB
>> > >>
>> > >>
>> > >>
>> > >> -----Original Message-----
>> > >> From: Stephen Mallette <[email protected]>
>> > >> Sent: Friday, February 23, 2018 5:13 PM
>> > >> To: [email protected]
>> > >> Subject: Re: [Discuss] Expose metadata from Gremlin Server to
>> Clients.
>> > >>
>> > >> Adding those kinds of details was the reason we had the
>> > >> ResponseStatus.attributes Map. I can really only speak for the
>> > >> java
>> > driver
>> > >> as I only know that one really well (we might need other
>> > >> TinkerPop
>> > experts
>> > >> to chime in for python, .net and c#). The java driver doesn't
>> > >> really present ways to get that information easily under usage
>> > >> that doesn't
>> > deal
>> > >> directly with RequestMessage directly (which people typically
>> > >> don't
>> do).
>> > >> Another thing to think about is that since a single request
>> > >> might return multiple ResponseMessage instances you might not
>> > >> want to return that
>> > kind
>> > >> of data on every response - maybe just to be returned on the
>> > >> first (or last
>> > >> message) and then we somehow preserve that information and make
>> > >> it accessible on the result somehow....we sorta have some kinda
>> > >> of
>> > precedent
>> > >> for that with side-effect data generated by bytecode based
>> > >> traversals -
>> > we
>> > >> can probably build in something similar for this sort of thing.
>> > >>
>> > >> I also toyed with the idea of using ping/pong websocket messages
>> > >> to
>> > carry
>> > >> general information about the server to the client. Not sure if
>> > >> any of
>> > the
>> > >> metadata you want to send back would fit in there, but that
>> > >> could be another option.
>> > >>
>> > >> Does any of that sound helpful?
>> > >>
>> > >> On Fri, Feb 23, 2018 at 3:41 PM, Ashwini Singh <
>> > >> [email protected]> wrote:
>> > >>
>> > >>> Hi All,
>> > >>>
>> > >>> We are working on to expose metadata as part of gremlin to
>> > >>> response to client. The metadata is simply a property bag to
>> > >>> provide special message/hints to the client. But currently
>> > >>> client libraries strip off everything and only return the data
>> > >>> to the
>> client.
>> > >>>
>> > >>> Specifically, We want to expose details like Request Charge,
>> > >>> Rate limiting/Retry policy details etc. In the other scenarios
>> > >>> in Cosmos DB we provide these details to the client is through
>> > >>> response headers. We did some investigation around this and one
>> > >>> of the options is expose these is through response attributes.
>> > >>> Gremlin Server can add metadata as part of gremlin response
>> > >>> attributes (For example, set the property bag on
>> > >>> ResponseStatus.Attributes for Gremlin.Net) that can be
>> > >>> serialized
>> by the client drivers to the clients.
>> > >>>
>> > >>> We would like to learn more if there are precedence around
>> > >>> this and if there are any recommended ways to achieve this in
>> > >>> Gremlin protocol and client drivers.
>> > >>>
>> > >>> Thanks a lot,
>> > >>> Ashwini Singh
>> > >>> Software Engineer @ Azure Cosmos DB
>> > >>>
>> > >>>
>> >
>> >
>> >
>>
>
>
