Re: [Discuss] Expose metadata from Gremlin Server to Clients.
thanks for creating the issue in jira. i assigned myself to it to help manage it through the process. > For #3, Do we need to take this in this work item as well ? I believe that we should handle 3a as part of this - 3b is something else and could be removed from the description for now. As for the branchI just pushed the TINKERPOP-1919 branch: https://github.com/apache/tinkerpop/tree/TINKERPOP-1913 This was a really fast and easy first stab at this to just prove out the ideas that included: 1. a change to the server to put the "host" in the status attributes (we'll see what we ultimately want to return here as part of 3a in JIRA, but i figured that would work for now). the server changes introduced some deprecation, but it should be backward compatible 2. a change to the java driver that makes the status attributes available to the caller on the ResultSet object 3. an initial attempt at returning status attributes over a remote Traversal - i made it accessible through side-effects. not sure if this should stay that way. we've talked about having "metadata" on a traversal before. perhaps it would be best exposed that way more generally as a first class citizen, but that would entail bigger changes that might better be reserved for 3.4.0 - not sure just what we do with "remote traversals" (i.e. item 2 in JIRA) yet. Definitely happy we're not doing this on 3.2.x. I'm not so sure I even like it on 3.3.x given what I've done so far. Depending on how things develop, we might want to push this forward to 3.4.x - i think it could be more cleanly implemented there without having to worry so hard about deprecation and other things that are binding our hands a bit. There's no target date for 3.4.x yet though I would guess some time in the summer. On Tue, Mar 6, 2018 at 5:11 PM, Ashwini Singh < ashws...@microsoft.com.invalid> wrote: > Create a incident: https://issues.apache.org/jira/browse/TINKERPOP-1913 > > For this, I am fine with that. Lets take this as part of above work item. > > For #3, Do we need to take this in this work item as well ? > > Can someone setup the branch? And we can take it from there. > > Thanks a lot guys, > Ashwini Singh > > Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > From: Stephen Mallette<mailto:spmalle...@gmail.com> > Sent: Tuesday, March 6, 2018 3:53 AM > To: dev@tinkerpop.apache.org<mailto:dev@tinkerpop.apache.org> > Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. > > Your summary looks good to me. I also agree that we can omit 3b from this > body of work. Going back to an earlier question that you had: > > > Can we spilt the change to support for gremlin request/script first and > bytecode/traversal separately? > > I'm a bit concerned that if we don't at least have the design of the > bytecode/traversal piece thought through fully we might not get the client > side changes right to support it. Perhaps we see how things go on > implementation before making a final decision here, but as of right now, I > tend to feel like we shouldn't split this up. > > To just get started, I think I'd like to get the initial development branch > setup with Gremlin Server returning some metadata. That part is pretty > easy. Once that branch is setup, anyone contributing can just submit PRs > against that or just commit directly to it. Then we can have one PR to > evaluate through the review process for this feature > > I guess there is some question as to what version this body of work should > go to. I think I'd like to skip adding this to 3.2.8 and target 3.3.2 on > the tp33 branch. At this point, I think 3.2.x should really just contain > bug fixes and minor enhancements - I don't think this particular > enhancement qualifies as "minor". > > Ashwini, if all that sounds good, could you please create a JIRA ticket > that contains a final summary and references this thread? > > > > > > On Mon, Mar 5, 2018 at 2:47 PM, Ashwini Singh < > ashws...@microsoft.com.invalid> wrote: > > > > > To summarize what we have discussed so far: > > 1. For API using GremlinRequest/QueryScript, expose response > > attribute as part of result. Using an approach to similar to Java client > > driver (using ResultSet) . [Priority0] > > -- We rely on the last message for response attributes. > > 2. For GLV, add response attribute as part of Traversal. > [Priority > > 0] > > --Rely on the last message for attributes. > > 3. Expose other server details (like server setting). I would > > suggest to split this design discussion into two directions: > > a. M
RE: [Discuss] Expose metadata from Gremlin Server to Clients.
Create a incident: https://issues.apache.org/jira/browse/TINKERPOP-1913 For this, I am fine with that. Lets take this as part of above work item. For #3, Do we need to take this in this work item as well ? Can someone setup the branch? And we can take it from there. Thanks a lot guys, Ashwini Singh Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 From: Stephen Mallette<mailto:spmalle...@gmail.com> Sent: Tuesday, March 6, 2018 3:53 AM To: dev@tinkerpop.apache.org<mailto:dev@tinkerpop.apache.org> Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. Your summary looks good to me. I also agree that we can omit 3b from this body of work. Going back to an earlier question that you had: > Can we spilt the change to support for gremlin request/script first and bytecode/traversal separately? I'm a bit concerned that if we don't at least have the design of the bytecode/traversal piece thought through fully we might not get the client side changes right to support it. Perhaps we see how things go on implementation before making a final decision here, but as of right now, I tend to feel like we shouldn't split this up. To just get started, I think I'd like to get the initial development branch setup with Gremlin Server returning some metadata. That part is pretty easy. Once that branch is setup, anyone contributing can just submit PRs against that or just commit directly to it. Then we can have one PR to evaluate through the review process for this feature I guess there is some question as to what version this body of work should go to. I think I'd like to skip adding this to 3.2.8 and target 3.3.2 on the tp33 branch. At this point, I think 3.2.x should really just contain bug fixes and minor enhancements - I don't think this particular enhancement qualifies as "minor". Ashwini, if all that sounds good, could you please create a JIRA ticket that contains a final summary and references this thread? On Mon, Mar 5, 2018 at 2:47 PM, Ashwini Singh < ashws...@microsoft.com.invalid> wrote: > > To summarize what we have discussed so far: > 1. For API using GremlinRequest/QueryScript, expose response > attribute as part of result. Using an approach to similar to Java client > driver (using ResultSet) . [Priority0] > -- We rely on the last message for response attributes. > 2. For GLV, add response attribute as part of Traversal. [Priority > 0] > --Rely on the last message for attributes. > 3. Expose other server details (like server setting). I would > suggest to split this design discussion into two directions: > a. Metadata for request execution: Only focuses on details > related to request execution. Can be achieved through #1 and #2. > b. Metadata for Gremlin Server: Focuses on overall > metadata for the server. Could be a separate request and fetch the data > once for a connection. Any other suggestion ? > > Please add if I am missing something. > > Thanks a lot, > Ashwini Singh > Software Engineer @ Azure Cosmos DB > > -Original Message- > From: Stephen Mallette <spmalle...@gmail.com> > Sent: Friday, March 2, 2018 4:49 AM > To: dev@tinkerpop.apache.org > Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. > > Perhaps another useful piece of data to return from Gremlin Server: > > https://na01.safelinks.protection.outlook.com/?url= > https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse% > 2FTINKERPOP-1636=04%7C01%7Cashwsing%40microsoft.com% > 7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af91ab2d7cd011 > db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1= > bVd71RV9uDiWWgsjY7SYAIYEhWtLdi7Ijgv9um%2F1JoM%3D=0 > > On Thu, Mar 1, 2018 at 6:31 PM, Stephen Mallette <spmalle...@gmail.com> > wrote: > > > I just came across this: > > > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue > > s.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1494=04%7C01%7Cashwsing% > > 40microsoft.com%7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af9 > > 1ab2d7cd011db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8e > > yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1 > > =1kY8r73w2Y3euQVnLEGIuAO0fX6ZySOzl%2BVUgbhodpE%3D=0 > > > > There's some ideas for what the open source Gremlin Server could > > return therefor example, perhaps we send back the the host that > > executed the script as the metadata for Gremlin Server? > > > > On Wed, Feb 28, 2018 at 5:37 PM, Ashwini Singh <ashws...@microsoft.com. > > invalid> wrote: > > > >> > >> Completely agree on making
Re: [Discuss] Expose metadata from Gremlin Server to Clients.
Your summary looks good to me. I also agree that we can omit 3b from this body of work. Going back to an earlier question that you had: > Can we spilt the change to support for gremlin request/script first and bytecode/traversal separately? I'm a bit concerned that if we don't at least have the design of the bytecode/traversal piece thought through fully we might not get the client side changes right to support it. Perhaps we see how things go on implementation before making a final decision here, but as of right now, I tend to feel like we shouldn't split this up. To just get started, I think I'd like to get the initial development branch setup with Gremlin Server returning some metadata. That part is pretty easy. Once that branch is setup, anyone contributing can just submit PRs against that or just commit directly to it. Then we can have one PR to evaluate through the review process for this feature I guess there is some question as to what version this body of work should go to. I think I'd like to skip adding this to 3.2.8 and target 3.3.2 on the tp33 branch. At this point, I think 3.2.x should really just contain bug fixes and minor enhancements - I don't think this particular enhancement qualifies as "minor". Ashwini, if all that sounds good, could you please create a JIRA ticket that contains a final summary and references this thread? On Mon, Mar 5, 2018 at 2:47 PM, Ashwini Singh < ashws...@microsoft.com.invalid> wrote: > > To summarize what we have discussed so far: > 1. For API using GremlinRequest/QueryScript, expose response > attribute as part of result. Using an approach to similar to Java client > driver (using ResultSet) . [Priority0] > -- We rely on the last message for response attributes. > 2. For GLV, add response attribute as part of Traversal. [Priority > 0] > --Rely on the last message for attributes. > 3. Expose other server details (like server setting). I would > suggest to split this design discussion into two directions: > a. Metadata for request execution: Only focuses on details > related to request execution. Can be achieved through #1 and #2. > b. Metadata for Gremlin Server: Focuses on overall > metadata for the server. Could be a separate request and fetch the data > once for a connection. Any other suggestion ? > > Please add if I am missing something. > > Thanks a lot, > Ashwini Singh > Software Engineer @ Azure Cosmos DB > > -Original Message- > From: Stephen Mallette <spmalle...@gmail.com> > Sent: Friday, March 2, 2018 4:49 AM > To: dev@tinkerpop.apache.org > Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. > > Perhaps another useful piece of data to return from Gremlin Server: > > https://na01.safelinks.protection.outlook.com/?url= > https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse% > 2FTINKERPOP-1636=04%7C01%7Cashwsing%40microsoft.com% > 7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af91ab2d7cd011 > db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1= > bVd71RV9uDiWWgsjY7SYAIYEhWtLdi7Ijgv9um%2F1JoM%3D=0 > > On Thu, Mar 1, 2018 at 6:31 PM, Stephen Mallette <spmalle...@gmail.com> > wrote: > > > I just came across this: > > > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue > > s.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1494=04%7C01%7Cashwsing% > > 40microsoft.com%7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af9 > > 1ab2d7cd011db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8e > > yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1 > > =1kY8r73w2Y3euQVnLEGIuAO0fX6ZySOzl%2BVUgbhodpE%3D=0 > > > > There's some ideas for what the open source Gremlin Server could > > return therefor example, perhaps we send back the the host that > > executed the script as the metadata for Gremlin Server? > > > > On Wed, Feb 28, 2018 at 5:37 PM, Ashwini Singh <ashws...@microsoft.com. > > invalid> wrote: > > > >> > >> Completely agree on making the change for other drivers too. The plan > >> from our side is to make changes to other drivers as well. My focus > >> on Gremlin.Net was just for an example . Yes, we should make change > >> to Gremlin Server to write the metadata for testing. > >> > >> I agree to adding this to ResultSet and exposing this as part of > >> every response for all the drivers and follow the same pattern as JAVA. > >> > >> On the gremlin GVL part, I agree we should support this on traversal > >> for bytecode as well. For now, we do not have suppo
RE: [Discuss] Expose metadata from Gremlin Server to Clients.
To summarize what we have discussed so far: 1. For API using GremlinRequest/QueryScript, expose response attribute as part of result. Using an approach to similar to Java client driver (using ResultSet) . [Priority0] -- We rely on the last message for response attributes. 2. For GLV, add response attribute as part of Traversal. [Priority 0] --Rely on the last message for attributes. 3. Expose other server details (like server setting). I would suggest to split this design discussion into two directions: a. Metadata for request execution: Only focuses on details related to request execution. Can be achieved through #1 and #2. b. Metadata for Gremlin Server: Focuses on overall metadata for the server. Could be a separate request and fetch the data once for a connection. Any other suggestion ? Please add if I am missing something. Thanks a lot, Ashwini Singh Software Engineer @ Azure Cosmos DB -Original Message- From: Stephen Mallette <spmalle...@gmail.com> Sent: Friday, March 2, 2018 4:49 AM To: dev@tinkerpop.apache.org Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. Perhaps another useful piece of data to return from Gremlin Server: https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1636=04%7C01%7Cashwsing%40microsoft.com%7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1=bVd71RV9uDiWWgsjY7SYAIYEhWtLdi7Ijgv9um%2F1JoM%3D=0 On Thu, Mar 1, 2018 at 6:31 PM, Stephen Mallette <spmalle...@gmail.com> wrote: > I just came across this: > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue > s.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1494=04%7C01%7Cashwsing% > 40microsoft.com%7C621c0a838bd24f13eb4e08d5803bed5d%7C72f988bf86f141af9 > 1ab2d7cd011db47%7C1%7C0%7C636555917242398527%7CUnknown%7CTWFpbGZsb3d8e > yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1 > =1kY8r73w2Y3euQVnLEGIuAO0fX6ZySOzl%2BVUgbhodpE%3D=0 > > There's some ideas for what the open source Gremlin Server could > return therefor example, perhaps we send back the the host that > executed the script as the metadata for Gremlin Server? > > On Wed, Feb 28, 2018 at 5:37 PM, Ashwini Singh <ashws...@microsoft.com. > invalid> wrote: > >> >> Completely agree on making the change for other drivers too. The plan >> from our side is to make changes to other drivers as well. My focus >> on Gremlin.Net was just for an example . Yes, we should make change >> to Gremlin Server to write the metadata for testing. >> >> I agree to adding this to ResultSet and exposing this as part of >> every response for all the drivers and follow the same pattern as JAVA. >> >> On the gremlin GVL part, I agree we should support this on traversal >> for bytecode as well. For now, we do not have support for bytecode >> but we are actively working on that and need support for metadata there as >> well. >> Thanks for bringing this up. Just wanted to understand how we track >> issues at tinkerpop, Can we spilt the change to support for gremlin >> request/script first and bytecode/traversal separately? If we prefer >> to split the change in chunks, but completely fine with one change. >> >> Thanks, >> Ashwini Singh >> Software Engineer @ Azure Cosmos DB >> >> -Original Message- >> From: Stephen Mallette <spmalle...@gmail.com> >> Sent: Wednesday, February 28, 2018 9:58 AM >> To: dev@tinkerpop.apache.org >> Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. >> >> This newly created issue might be related to this in some way: >> >> https://na01.safelinks.protection.outlook.com/?url=https%3A% >> 2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1906& >> data=04%7C01%7Cashwsing%40microsoft.com%7C614b32ff7f98 >> 4c66554808d57ed4d22b%7C72f988bf86f141af91ab2d7cd011db47%7C1% >> 7C0%7C636554374885412678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4 >> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1& >> sdata=w7pyg6JF4KMbPdpBg6t%2F8cE%2BflbnCQQdyMUgc6HDrXw%3D=0 >> >> On Wed, Feb 28, 2018 at 11:37 AM, Florian Hockmann < >> f...@florian-hockmann.de> >> wrote: >> >> > I agree that we also should make this metadata available for >> > traversals since we want to move users away from sending Gremlin >> > scripts as strings and instead use Bytecode based GLVs. >> > >> > Regarding Gremlin.Net: I think that the implement
Re: [Discuss] Expose metadata from Gremlin Server to Clients.
I just came across this: https://issues.apache.org/jira/browse/TINKERPOP-1494 There's some ideas for what the open source Gremlin Server could return therefor example, perhaps we send back the the host that executed the script as the metadata for Gremlin Server? On Wed, Feb 28, 2018 at 5:37 PM, Ashwini Singh < ashws...@microsoft.com.invalid> wrote: > > Completely agree on making the change for other drivers too. The plan from > our side is to make changes to other drivers as well. My focus on > Gremlin.Net was just for an example . Yes, we should make change to > Gremlin Server to write the metadata for testing. > > I agree to adding this to ResultSet and exposing this as part of every > response for all the drivers and follow the same pattern as JAVA. > > On the gremlin GVL part, I agree we should support this on traversal for > bytecode as well. For now, we do not have support for bytecode but we are > actively working on that and need support for metadata there as well. > Thanks for bringing this up. Just wanted to understand how we track issues > at tinkerpop, Can we spilt the change to support for gremlin request/script > first and bytecode/traversal separately? If we prefer to split the change > in chunks, but completely fine with one change. > > Thanks, > Ashwini Singh > Software Engineer @ Azure Cosmos DB > > -Original Message- > From: Stephen Mallette <spmalle...@gmail.com> > Sent: Wednesday, February 28, 2018 9:58 AM > To: dev@tinkerpop.apache.org > Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. > > This newly created issue might be related to this in some way: > > https://na01.safelinks.protection.outlook.com/?url= > https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse% > 2FTINKERPOP-1906=04%7C01%7Cashwsing%40microsoft.com% > 7C614b32ff7f984c66554808d57ed4d22b%7C72f988bf86f141af91ab2d7cd011 > db47%7C1%7C0%7C636554374885412678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw > MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1= > w7pyg6JF4KMbPdpBg6t%2F8cE%2BflbnCQQdyMUgc6HDrXw%3D=0 > > On Wed, Feb 28, 2018 at 11:37 AM, Florian Hockmann <f...@florian-hockmann.de > > > wrote: > > > I agree that we also should make this metadata available for > > traversals since we want to move users away from sending Gremlin > > scripts as strings and instead use Bytecode based GLVs. > > > > Regarding Gremlin.Net: I think that the implementation would be very > > similar to how it can be implemented in the Java driver as we tried to > > stay close to the Java driver in general. The only difference is > > probably that we currently don't have a ResultSet in Gremlin.Net, but > > that's only because I didn't see much value in adding that. Metadata > > would of course be a good argument to also implement a ResultSet in > > Gremlin.Net and then the implementation should be really basically the > > same as in the Java driver. > > > > > > Am 28.02.2018 um 16:15 schrieb Stephen Mallette: > > > I'm fine with using the last response message as the carrier for > > > this metadata on a particular request. I can't really tell if there > > > is much > > work > > > to do on Gremlin Server itself here. It seems like most of the work > > > must occur on the various drivers (you mention the .NET api, but all > > > of the drivers would need to support this feature). However, I would > > > think that > > we > > > would want Gremlin Server itself to append in some kind metadata > > > (maybe query time? something easy) so that we could write tests > > > for the drivers in TinkerPop itself. There is also the question of > > > how we would expose this metadata to GLVs which don't see response > > > messages at all. A traversal might need some metadata itself so that > > > the user could retrieve the server metadata from that. The > > > implementation between Java and the > > GLVs > > > might be different here as the GLV traversal class is typically > > > quite lightweight and only used for generating bytecode. > > > > > > I'm not so sure I like the SubmitAsynWithHeaders() but I don't > > > think > > too > > > much about how the .NET driver works. Is there a reason to not > > > always return metadata? could it be expensive to do so? If we just > > > added an > > extra > > > method how would remote traversals configure this option? I think we > > > need another way. Generally speaking, for Java, I think I would like > > > to see > > the > > > metadata available to the ResultSet somehow which would in turn mak
RE: [Discuss] Expose metadata from Gremlin Server to Clients.
Completely agree on making the change for other drivers too. The plan from our side is to make changes to other drivers as well. My focus on Gremlin.Net was just for an example . Yes, we should make change to Gremlin Server to write the metadata for testing. I agree to adding this to ResultSet and exposing this as part of every response for all the drivers and follow the same pattern as JAVA. On the gremlin GVL part, I agree we should support this on traversal for bytecode as well. For now, we do not have support for bytecode but we are actively working on that and need support for metadata there as well. Thanks for bringing this up. Just wanted to understand how we track issues at tinkerpop, Can we spilt the change to support for gremlin request/script first and bytecode/traversal separately? If we prefer to split the change in chunks, but completely fine with one change. Thanks, Ashwini Singh Software Engineer @ Azure Cosmos DB -Original Message- From: Stephen Mallette <spmalle...@gmail.com> Sent: Wednesday, February 28, 2018 9:58 AM To: dev@tinkerpop.apache.org Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. This newly created issue might be related to this in some way: https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTINKERPOP-1906=04%7C01%7Cashwsing%40microsoft.com%7C614b32ff7f984c66554808d57ed4d22b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636554374885412678%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1=w7pyg6JF4KMbPdpBg6t%2F8cE%2BflbnCQQdyMUgc6HDrXw%3D=0 On Wed, Feb 28, 2018 at 11:37 AM, Florian Hockmann <f...@florian-hockmann.de> wrote: > I agree that we also should make this metadata available for > traversals since we want to move users away from sending Gremlin > scripts as strings and instead use Bytecode based GLVs. > > Regarding Gremlin.Net: I think that the implementation would be very > similar to how it can be implemented in the Java driver as we tried to > stay close to the Java driver in general. The only difference is > probably that we currently don't have a ResultSet in Gremlin.Net, but > that's only because I didn't see much value in adding that. Metadata > would of course be a good argument to also implement a ResultSet in > Gremlin.Net and then the implementation should be really basically the > same as in the Java driver. > > > Am 28.02.2018 um 16:15 schrieb Stephen Mallette: > > I'm fine with using the last response message as the carrier for > > this metadata on a particular request. I can't really tell if there > > is much > work > > to do on Gremlin Server itself here. It seems like most of the work > > must occur on the various drivers (you mention the .NET api, but all > > of the drivers would need to support this feature). However, I would > > think that > we > > would want Gremlin Server itself to append in some kind metadata > > (maybe query time? something easy) so that we could write tests > > for the drivers in TinkerPop itself. There is also the question of > > how we would expose this metadata to GLVs which don't see response > > messages at all. A traversal might need some metadata itself so that > > the user could retrieve the server metadata from that. The > > implementation between Java and the > GLVs > > might be different here as the GLV traversal class is typically > > quite lightweight and only used for generating bytecode. > > > > I'm not so sure I like the SubmitAsynWithHeaders() but I don't > > think > too > > much about how the .NET driver works. Is there a reason to not > > always return metadata? could it be expensive to do so? If we just > > added an > extra > > method how would remote traversals configure this option? I think we > > need another way. Generally speaking, for Java, I think I would like > > to see > the > > metadata available to the ResultSet somehow which would in turn make > > it pretty easy to get it on to a Traversal instance once that > > facility was made available.but as to how to enable or disable > > the return of the metadata, i'm not sure how that should work just > > yet - i need to think on that some more. > > > > For committers who work on GLVs, please take a look at this thread > > and offer your thoughts on how this might work in the GLV driver you > > tend to have the most knowledge on. Let's see if we can come to one > > nice unified solution. At that point, we can setup a ticket in JIRA and go > > from there. > > > > Ashwini, thanks for offering a pull request for this by the way. > > Once we get consensus on how
Re: [Discuss] Expose metadata from Gremlin Server to Clients.
This newly created issue might be related to this in some way: https://issues.apache.org/jira/browse/TINKERPOP-1906 On Wed, Feb 28, 2018 at 11:37 AM, Florian Hockmann <f...@florian-hockmann.de> wrote: > I agree that we also should make this metadata available for traversals > since we want to move users away from sending Gremlin scripts as strings > and instead use Bytecode based GLVs. > > Regarding Gremlin.Net: I think that the implementation would be very > similar to how it can be implemented in the Java driver as we tried to > stay close to the Java driver in general. The only difference is > probably that we currently don't have a ResultSet in Gremlin.Net, but > that's only because I didn't see much value in adding that. Metadata > would of course be a good argument to also implement a ResultSet in > Gremlin.Net and then the implementation should be really basically the > same as in the Java driver. > > > Am 28.02.2018 um 16:15 schrieb Stephen Mallette: > > I'm fine with using the last response message as the carrier for this > > metadata on a particular request. I can't really tell if there is much > work > > to do on Gremlin Server itself here. It seems like most of the work must > > occur on the various drivers (you mention the .NET api, but all of the > > drivers would need to support this feature). However, I would think that > we > > would want Gremlin Server itself to append in some kind metadata (maybe > > query time? something easy) so that we could write tests for the > > drivers in TinkerPop itself. There is also the question of how we would > > expose this metadata to GLVs which don't see response messages at all. A > > traversal might need some metadata itself so that the user could retrieve > > the server metadata from that. The implementation between Java and the > GLVs > > might be different here as the GLV traversal class is typically quite > > lightweight and only used for generating bytecode. > > > > I'm not so sure I like the SubmitAsynWithHeaders() but I don't think > too > > much about how the .NET driver works. Is there a reason to not always > > return metadata? could it be expensive to do so? If we just added an > extra > > method how would remote traversals configure this option? I think we need > > another way. Generally speaking, for Java, I think I would like to see > the > > metadata available to the ResultSet somehow which would in turn make it > > pretty easy to get it on to a Traversal instance once that facility was > > made available.but as to how to enable or disable the return of the > > metadata, i'm not sure how that should work just yet - i need to think on > > that some more. > > > > For committers who work on GLVs, please take a look at this thread and > > offer your thoughts on how this might work in the GLV driver you tend to > > have the most knowledge on. Let's see if we can come to one nice unified > > solution. At that point, we can setup a ticket in JIRA and go from there. > > > > Ashwini, thanks for offering a pull request for this by the way. Once we > > get consensus on how to do this, we'll see if tasks need to be divided > and > > how you might contribute. > > > > > > > > On Mon, Feb 26, 2018 at 6:45 PM, Ashwini Singh < > > ashws...@microsoft.com.invalid> wrote: > > > >> Hi Stephen, > >> > >> Thanks for considering the change. > >> > >> We would be more inclined towards the first approach since the having a > >> ping/pong websocket message can be a bit noisy and requires > sophisticated > >> handling on the client driver side. > >> > >> For handling multiple response messages. I would suggest to rely on last > >> message as these are the metadata for request execution. Partial > response > >> is very internal to the client drivers (based on limited understanding > of > >> tinkerpop client drivers :) , correct me if you differ) and can be > exposed > >> separately (if really needed later). > >> > >> For implementation, Let us know if we can chip in there and submit PR. > The > >> high level approach to achieve this is to have corresponding > >> SubmitAsynWithHeaders() for every SubmitAsync() that returns a > >> encapsulated result with attributes and IReadOnlyCollectio. Let me > know > >> if you see any concerns adding a new API. > >> > >> Thanks a lot, > >> Ashwini Singh > >> Software Engineer @ Azure Cosmos DB > >> > >> > >> > >> -Original Message- > >&g
Re: [Discuss] Expose metadata from Gremlin Server to Clients.
I agree that we also should make this metadata available for traversals since we want to move users away from sending Gremlin scripts as strings and instead use Bytecode based GLVs. Regarding Gremlin.Net: I think that the implementation would be very similar to how it can be implemented in the Java driver as we tried to stay close to the Java driver in general. The only difference is probably that we currently don't have a ResultSet in Gremlin.Net, but that's only because I didn't see much value in adding that. Metadata would of course be a good argument to also implement a ResultSet in Gremlin.Net and then the implementation should be really basically the same as in the Java driver. Am 28.02.2018 um 16:15 schrieb Stephen Mallette: > I'm fine with using the last response message as the carrier for this > metadata on a particular request. I can't really tell if there is much work > to do on Gremlin Server itself here. It seems like most of the work must > occur on the various drivers (you mention the .NET api, but all of the > drivers would need to support this feature). However, I would think that we > would want Gremlin Server itself to append in some kind metadata (maybe > query time? something easy) so that we could write tests for the > drivers in TinkerPop itself. There is also the question of how we would > expose this metadata to GLVs which don't see response messages at all. A > traversal might need some metadata itself so that the user could retrieve > the server metadata from that. The implementation between Java and the GLVs > might be different here as the GLV traversal class is typically quite > lightweight and only used for generating bytecode. > > I'm not so sure I like the SubmitAsynWithHeaders() but I don't think too > much about how the .NET driver works. Is there a reason to not always > return metadata? could it be expensive to do so? If we just added an extra > method how would remote traversals configure this option? I think we need > another way. Generally speaking, for Java, I think I would like to see the > metadata available to the ResultSet somehow which would in turn make it > pretty easy to get it on to a Traversal instance once that facility was > made available.but as to how to enable or disable the return of the > metadata, i'm not sure how that should work just yet - i need to think on > that some more. > > For committers who work on GLVs, please take a look at this thread and > offer your thoughts on how this might work in the GLV driver you tend to > have the most knowledge on. Let's see if we can come to one nice unified > solution. At that point, we can setup a ticket in JIRA and go from there. > > Ashwini, thanks for offering a pull request for this by the way. Once we > get consensus on how to do this, we'll see if tasks need to be divided and > how you might contribute. > > > > On Mon, Feb 26, 2018 at 6:45 PM, Ashwini Singh < > ashws...@microsoft.com.invalid> wrote: > >> Hi Stephen, >> >> Thanks for considering the change. >> >> We would be more inclined towards the first approach since the having a >> ping/pong websocket message can be a bit noisy and requires sophisticated >> handling on the client driver side. >> >> For handling multiple response messages. I would suggest to rely on last >> message as these are the metadata for request execution. Partial response >> is very internal to the client drivers (based on limited understanding of >> tinkerpop client drivers :) , correct me if you differ) and can be exposed >> separately (if really needed later). >> >> For implementation, Let us know if we can chip in there and submit PR. The >> high level approach to achieve this is to have corresponding >> SubmitAsynWithHeaders() for every SubmitAsync() that returns a >> encapsulated result with attributes and IReadOnlyCollectio. Let me know >> if you see any concerns adding a new API. >> >> Thanks a lot, >> Ashwini Singh >> Software Engineer @ Azure Cosmos DB >> >> >> >> -Original Message- >> From: Stephen Mallette <spmalle...@gmail.com> >> Sent: Friday, February 23, 2018 5:13 PM >> To: dev@tinkerpop.apache.org >> Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. >> >> Adding those kinds of details was the reason we had the >> ResponseStatus.attributes Map. I can really only speak for the java driver >> as I only know that one really well (we might need other TinkerPop experts >> to chime in for python, .net and c#). The java driver doesn't really >> present ways to get that information easily under usage that doesn't deal >> directly with RequestMessage directly (which peop
Re: [Discuss] Expose metadata from Gremlin Server to Clients.
I'm fine with using the last response message as the carrier for this metadata on a particular request. I can't really tell if there is much work to do on Gremlin Server itself here. It seems like most of the work must occur on the various drivers (you mention the .NET api, but all of the drivers would need to support this feature). However, I would think that we would want Gremlin Server itself to append in some kind metadata (maybe query time? something easy) so that we could write tests for the drivers in TinkerPop itself. There is also the question of how we would expose this metadata to GLVs which don't see response messages at all. A traversal might need some metadata itself so that the user could retrieve the server metadata from that. The implementation between Java and the GLVs might be different here as the GLV traversal class is typically quite lightweight and only used for generating bytecode. I'm not so sure I like the SubmitAsynWithHeaders() but I don't think too much about how the .NET driver works. Is there a reason to not always return metadata? could it be expensive to do so? If we just added an extra method how would remote traversals configure this option? I think we need another way. Generally speaking, for Java, I think I would like to see the metadata available to the ResultSet somehow which would in turn make it pretty easy to get it on to a Traversal instance once that facility was made available.but as to how to enable or disable the return of the metadata, i'm not sure how that should work just yet - i need to think on that some more. For committers who work on GLVs, please take a look at this thread and offer your thoughts on how this might work in the GLV driver you tend to have the most knowledge on. Let's see if we can come to one nice unified solution. At that point, we can setup a ticket in JIRA and go from there. Ashwini, thanks for offering a pull request for this by the way. Once we get consensus on how to do this, we'll see if tasks need to be divided and how you might contribute. On Mon, Feb 26, 2018 at 6:45 PM, Ashwini Singh < ashws...@microsoft.com.invalid> wrote: > Hi Stephen, > > Thanks for considering the change. > > We would be more inclined towards the first approach since the having a > ping/pong websocket message can be a bit noisy and requires sophisticated > handling on the client driver side. > > For handling multiple response messages. I would suggest to rely on last > message as these are the metadata for request execution. Partial response > is very internal to the client drivers (based on limited understanding of > tinkerpop client drivers :) , correct me if you differ) and can be exposed > separately (if really needed later). > > For implementation, Let us know if we can chip in there and submit PR. The > high level approach to achieve this is to have corresponding > SubmitAsynWithHeaders() for every SubmitAsync() that returns a > encapsulated result with attributes and IReadOnlyCollectio. Let me know > if you see any concerns adding a new API. > > Thanks a lot, > Ashwini Singh > Software Engineer @ Azure Cosmos DB > > > > -Original Message- > From: Stephen Mallette <spmalle...@gmail.com> > Sent: Friday, February 23, 2018 5:13 PM > To: dev@tinkerpop.apache.org > Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. > > Adding those kinds of details was the reason we had the > ResponseStatus.attributes Map. I can really only speak for the java driver > as I only know that one really well (we might need other TinkerPop experts > to chime in for python, .net and c#). The java driver doesn't really > present ways to get that information easily under usage that doesn't deal > directly with RequestMessage directly (which people typically don't do). > Another thing to think about is that since a single request might return > multiple ResponseMessage instances you might not want to return that kind > of data on every response - maybe just to be returned on the first (or last > message) and then we somehow preserve that information and make it > accessible on the result somehowwe sorta have some kinda of precedent > for that with side-effect data generated by bytecode based traversals - we > can probably build in something similar for this sort of thing. > > I also toyed with the idea of using ping/pong websocket messages to carry > general information about the server to the client. Not sure if any of the > metadata you want to send back would fit in there, but that could be > another option. > > Does any of that sound helpful? > > On Fri, Feb 23, 2018 at 3:41 PM, Ashwini Singh < > ashws...@microsoft.com.invalid> wrote: > > > Hi All, > > > > We are working on to expose metadata as part of gremlin to resp
RE: [Discuss] Expose metadata from Gremlin Server to Clients.
Hi Stephen, Thanks for considering the change. We would be more inclined towards the first approach since the having a ping/pong websocket message can be a bit noisy and requires sophisticated handling on the client driver side. For handling multiple response messages. I would suggest to rely on last message as these are the metadata for request execution. Partial response is very internal to the client drivers (based on limited understanding of tinkerpop client drivers :) , correct me if you differ) and can be exposed separately (if really needed later). For implementation, Let us know if we can chip in there and submit PR. The high level approach to achieve this is to have corresponding SubmitAsynWithHeaders() for every SubmitAsync() that returns a encapsulated result with attributes and IReadOnlyCollectio. Let me know if you see any concerns adding a new API. Thanks a lot, Ashwini Singh Software Engineer @ Azure Cosmos DB -Original Message- From: Stephen Mallette <spmalle...@gmail.com> Sent: Friday, February 23, 2018 5:13 PM To: dev@tinkerpop.apache.org Subject: Re: [Discuss] Expose metadata from Gremlin Server to Clients. Adding those kinds of details was the reason we had the ResponseStatus.attributes Map. I can really only speak for the java driver as I only know that one really well (we might need other TinkerPop experts to chime in for python, .net and c#). The java driver doesn't really present ways to get that information easily under usage that doesn't deal directly with RequestMessage directly (which people typically don't do). Another thing to think about is that since a single request might return multiple ResponseMessage instances you might not want to return that kind of data on every response - maybe just to be returned on the first (or last message) and then we somehow preserve that information and make it accessible on the result somehowwe sorta have some kinda of precedent for that with side-effect data generated by bytecode based traversals - we can probably build in something similar for this sort of thing. I also toyed with the idea of using ping/pong websocket messages to carry general information about the server to the client. Not sure if any of the metadata you want to send back would fit in there, but that could be another option. Does any of that sound helpful? On Fri, Feb 23, 2018 at 3:41 PM, Ashwini Singh < ashws...@microsoft.com.invalid> wrote: > Hi All, > > We are working on to expose metadata as part of gremlin to response to > client. The metadata is simply a property bag to provide special > message/hints to the client. But currently client libraries strip off > everything and only return the data to the client. > > Specifically, We want to expose details like Request Charge, Rate > limiting/Retry policy details etc. In the other scenarios in Cosmos DB > we provide these details to the client is through response headers. We > did some investigation around this and one of the options is expose > these is through response attributes. Gremlin Server can add metadata > as part of gremlin response attributes (For example, set the property > bag on ResponseStatus.Attributes for Gremlin.Net) that can be > serialized by the client drivers to the clients. > > We would like to learn more if there are precedence around this and > if there are any recommended ways to achieve this in Gremlin protocol > and client drivers. > > Thanks a lot, > Ashwini Singh > Software Engineer @ Azure Cosmos DB > >
Re: [Discuss] Expose metadata from Gremlin Server to Clients.
Adding those kinds of details was the reason we had the ResponseStatus.attributes Map. I can really only speak for the java driver as I only know that one really well (we might need other TinkerPop experts to chime in for python, .net and c#). The java driver doesn't really present ways to get that information easily under usage that doesn't deal directly with RequestMessage directly (which people typically don't do). Another thing to think about is that since a single request might return multiple ResponseMessage instances you might not want to return that kind of data on every response - maybe just to be returned on the first (or last message) and then we somehow preserve that information and make it accessible on the result somehowwe sorta have some kinda of precedent for that with side-effect data generated by bytecode based traversals - we can probably build in something similar for this sort of thing. I also toyed with the idea of using ping/pong websocket messages to carry general information about the server to the client. Not sure if any of the metadata you want to send back would fit in there, but that could be another option. Does any of that sound helpful? On Fri, Feb 23, 2018 at 3:41 PM, Ashwini Singh < ashws...@microsoft.com.invalid> wrote: > Hi All, > > We are working on to expose metadata as part of gremlin to response to > client. The metadata is simply a property bag to provide special > message/hints to the client. But currently client libraries strip off > everything and only return the data to the client. > > Specifically, We want to expose details like Request Charge, Rate > limiting/Retry policy details etc. In the other scenarios in Cosmos DB we > provide these details to the client is through response headers. We did > some investigation around this and one of the options is expose these is > through response attributes. Gremlin Server can add metadata as part of > gremlin response attributes (For example, set the property bag on > ResponseStatus.Attributes for Gremlin.Net) that can be serialized by the > client drivers to the clients. > > We would like to learn more if there are precedence around this and if > there are any recommended ways to achieve this in Gremlin protocol and > client drivers. > > Thanks a lot, > Ashwini Singh > Software Engineer @ Azure Cosmos DB > >
