Completely agree on making the change for other drivers too. The plan from our side is to make changes to other drivers as well. My focus on Gremlin.Net was just for an example . Yes, we should make change to Gremlin Server to write the metadata for testing.
I agree to adding this to ResultSet and exposing this as part of every response for all the drivers and follow the same pattern as JAVA. On the gremlin GVL part, I agree we should support this on traversal for bytecode as well. For now, we do not have support for bytecode but we are actively working on that and need support for metadata there as well. Thanks for bringing this up. Just wanted to understand how we track issues at tinkerpop, Can we spilt the change to support for gremlin request/script first and bytecode/traversal separately? If we prefer to split the change in chunks, but completely fine with one change. Thanks, Ashwini Singh Software Engineer @ Azure Cosmos DB -----Original Message----- From: Stephen Mallette <[email protected]> Sent: Wednesday, February 28, 2018 9:58 AM To: [email protected] Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. This newly created issue might be related to this in some way: https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1906&data=04%7C01%7Cashwsing%40microsoft.com%7C614b32ff7f984c66554808d57ed4d22b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636554374885412678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=w7pyg6JF4KMbPdpBg6t%2F8cE%2BflbnCQQdyMUgc6HDrXw%3D&reserved=0 On Wed, Feb 28, 2018 at 11:37 AM, Florian Hockmann <[email protected]> wrote: > I agree that we also should make this metadata available for > traversals since we want to move users away from sending Gremlin > scripts as strings and instead use Bytecode based GLVs. > > Regarding Gremlin.Net: I think that the implementation would be very > similar to how it can be implemented in the Java driver as we tried to > stay close to the Java driver in general. The only difference is > probably that we currently don't have a ResultSet in Gremlin.Net, but > that's only because I didn't see much value in adding that. Metadata > would of course be a good argument to also implement a ResultSet in > Gremlin.Net and then the implementation should be really basically the > same as in the Java driver. > > > Am 28.02.2018 um 16:15 schrieb Stephen Mallette: > > I'm fine with using the last response message as the carrier for > > this metadata on a particular request. I can't really tell if there > > is much > work > > to do on Gremlin Server itself here. It seems like most of the work > > must occur on the various drivers (you mention the .NET api, but all > > of the drivers would need to support this feature). However, I would > > think that > we > > would want Gremlin Server itself to append in some kind metadata > > (maybe query time? something easy....) so that we could write tests > > for the drivers in TinkerPop itself. There is also the question of > > how we would expose this metadata to GLVs which don't see response > > messages at all. A traversal might need some metadata itself so that > > the user could retrieve the server metadata from that. The > > implementation between Java and the > GLVs > > might be different here as the GLV traversal class is typically > > quite lightweight and only used for generating bytecode. > > > > I'm not so sure I like the SubmitAsynWithHeaders() but I don't > > think > too > > much about how the .NET driver works. Is there a reason to not > > always return metadata? could it be expensive to do so? If we just > > added an > extra > > method how would remote traversals configure this option? I think we > > need another way. Generally speaking, for Java, I think I would like > > to see > the > > metadata available to the ResultSet somehow which would in turn make > > it pretty easy to get it on to a Traversal instance once that > > facility was made available.....but as to how to enable or disable > > the return of the metadata, i'm not sure how that should work just > > yet - i need to think on that some more. > > > > For committers who work on GLVs, please take a look at this thread > > and offer your thoughts on how this might work in the GLV driver you > > tend to have the most knowledge on. Let's see if we can come to one > > nice unified solution. At that point, we can setup a ticket in JIRA and go > > from there. > > > > Ashwini, thanks for offering a pull request for this by the way. > > Once we get consensus on how to do this, we'll see if tasks need to > > be divided > and > > how you might contribute. > > > > > > > > On Mon, Feb 26, 2018 at 6:45 PM, Ashwini Singh < > > [email protected]> wrote: > > > >> Hi Stephen, > >> > >> Thanks for considering the change. > >> > >> We would be more inclined towards the first approach since the > >> having a ping/pong websocket message can be a bit noisy and > >> requires > sophisticated > >> handling on the client driver side. > >> > >> For handling multiple response messages. I would suggest to rely on > >> last message as these are the metadata for request execution. > >> Partial > response > >> is very internal to the client drivers (based on limited > >> understanding > of > >> tinkerpop client drivers :) , correct me if you differ) and can be > exposed > >> separately (if really needed later). > >> > >> For implementation, Let us know if we can chip in there and submit PR. > The > >> high level approach to achieve this is to have corresponding > >> SubmitAsynWithHeaders() for every SubmitAsync() that returns a > >> encapsulated result with attributes and IReadOnlyCollectio<T>. Let > >> me > know > >> if you see any concerns adding a new API. > >> > >> Thanks a lot, > >> Ashwini Singh > >> Software Engineer @ Azure Cosmos DB > >> > >> > >> > >> -----Original Message----- > >> From: Stephen Mallette <[email protected]> > >> Sent: Friday, February 23, 2018 5:13 PM > >> To: [email protected] > >> Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. > >> > >> Adding those kinds of details was the reason we had the > >> ResponseStatus.attributes Map. I can really only speak for the java > driver > >> as I only know that one really well (we might need other TinkerPop > experts > >> to chime in for python, .net and c#). The java driver doesn't > >> really present ways to get that information easily under usage that > >> doesn't > deal > >> directly with RequestMessage directly (which people typically don't do). > >> Another thing to think about is that since a single request might > >> return multiple ResponseMessage instances you might not want to > >> return that > kind > >> of data on every response - maybe just to be returned on the first > >> (or last > >> message) and then we somehow preserve that information and make it > >> accessible on the result somehow....we sorta have some kinda of > precedent > >> for that with side-effect data generated by bytecode based > >> traversals - > we > >> can probably build in something similar for this sort of thing. > >> > >> I also toyed with the idea of using ping/pong websocket messages to > carry > >> general information about the server to the client. Not sure if any > >> of > the > >> metadata you want to send back would fit in there, but that could > >> be another option. > >> > >> Does any of that sound helpful? > >> > >> On Fri, Feb 23, 2018 at 3:41 PM, Ashwini Singh < > >> [email protected]> wrote: > >> > >>> Hi All, > >>> > >>> We are working on to expose metadata as part of gremlin to > >>> response to client. The metadata is simply a property bag to > >>> provide special message/hints to the client. But currently client > >>> libraries strip off everything and only return the data to the client. > >>> > >>> Specifically, We want to expose details like Request Charge, Rate > >>> limiting/Retry policy details etc. In the other scenarios in > >>> Cosmos DB we provide these details to the client is through > >>> response headers. We did some investigation around this and one of > >>> the options is expose these is through response attributes. > >>> Gremlin Server can add metadata as part of gremlin response > >>> attributes (For example, set the property bag on > >>> ResponseStatus.Attributes for Gremlin.Net) that can be serialized by the > >>> client drivers to the clients. > >>> > >>> We would like to learn more if there are precedence around this > >>> and if there are any recommended ways to achieve this in Gremlin > >>> protocol and client drivers. > >>> > >>> Thanks a lot, > >>> Ashwini Singh > >>> Software Engineer @ Azure Cosmos DB > >>> > >>> > > >
