[jira] [Commented] (TINKERPOP-2445) Speed up client initialization
[ https://issues.apache.org/jira/browse/TINKERPOP-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237802#comment-17237802 ] ASF GitHub Bot commented on TINKERPOP-2445: --- divijvaidya merged pull request #1355: URL: https://github.com/apache/tinkerpop/pull/1355 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Speed up client initialization > -- > > Key: TINKERPOP-2445 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2445 > Project: TinkerPop > Issue Type: Improvement > Components: driver >Affects Versions: 3.5.0, 3.4.8 >Reporter: Divij Vaidya >Priority: Minor > Labels: breaking > Attachments: screenshot-1.png > > > The current Java client has a lot of initialization overhead. Some of the > things we could do to trim the fat are: > 1. Parallelize the connection creation inside a connection pool, i.e. make > [this for > loop|https://github.com/apache/tinkerpop/blob/3.4-dev/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/ConnectionPool.java] > parallel. > 2. Do not create a bootstrap [for every > connection|https://github.com/apache/tinkerpop/blob/3.4-dev/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Connection.java#L111]. > A single bootstrap could be reused. > 3. Remove SASL Handler from the pipeline after negotiation is complete for a > connection. > 4. Do not initialize SASL Handler if not required. > As part of this task, we should profile the start-up time and identify other > places where we could optimize the start-up time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2480) User agent for Gremlin drivers
[ https://issues.apache.org/jira/browse/TINKERPOP-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237616#comment-17237616 ] Stephen Mallette commented on TINKERPOP-2480: - We already have one defined: https://github.com/apache/tinkerpop/blob/b6234e23751e0e0548f97f0d6e8fceea8f3f5de6/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/Tokens.java#L88 the values could be better defined though as part of this issue. it currently is meant to identify the "client application" but it could be shifted to allow for the application+driver or the like. > User agent for Gremlin drivers > -- > > Key: TINKERPOP-2480 > URL: https://issues.apache.org/jira/browse/TINKERPOP-2480 > Project: TinkerPop > Issue Type: Improvement > Components: driver, server >Affects Versions: 3.4.8 >Reporter: Divij Vaidya >Priority: Minor > > Currently, a server does not distinguish amongst the different types of > clients connecting to it. This issue is to add a new feature to add user > agent field in the HTTP and WebSocket request header which could be used to > identify the specific client from which the request was made. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (TINKERPOP-2480) User agent for Gremlin drivers
Divij Vaidya created TINKERPOP-2480: --- Summary: User agent for Gremlin drivers Key: TINKERPOP-2480 URL: https://issues.apache.org/jira/browse/TINKERPOP-2480 Project: TinkerPop Issue Type: Improvement Components: driver, server Affects Versions: 3.4.8 Reporter: Divij Vaidya Currently, a server does not distinguish amongst the different types of clients connecting to it. This issue is to add a new feature to add user agent field in the HTTP and WebSocket request header which could be used to identify the specific client from which the request was made. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2468) Stabilize shouldProcessSessionRequestsInOrder() test
[
https://issues.apache.org/jira/browse/TINKERPOP-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237584#comment-17237584
]
ASF GitHub Bot commented on TINKERPOP-2468:
---
divijvaidya commented on a change in pull request #1365:
URL: https://github.com/apache/tinkerpop/pull/1365#discussion_r528910198
##
File path:
gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
##
@@ -612,40 +612,42 @@ public void shouldProcessRequestsOutOfOrder() throws
Exception {
}
}
+/**
+ * This test validates that the session requests are processed in-order on
the server. The order of results
+ * returned to the client might be different though since each result is
handled by a different executor thread.
+ */
Review comment:
Yes. Client side threads. More specifically, in
[this](https://github.com/apache/tinkerpop/pull/1347/files#diff-1a4243209a3c0c168ac7e9817b3bca1a2791db382c294130ea85c6f0bcec120cR109)
PR we made a change to remove the possibility of blocking the executor loop on
client by the application code by passing completion of readComplete future to
client's executor threads. For two different requests they might get assigned
to different threads in executor pool here and return results at different time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Stabilize shouldProcessSessionRequestsInOrder() test
>
>
> Key: TINKERPOP-2468
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2468
> Project: TinkerPop
> Issue Type: Improvement
> Components: server
>Affects Versions: 3.4.8
>Reporter: Stephen Mallette
>Assignee: Divij Vaidya
>Priority: Minor
>
> This test has been failing somewhat randomly on Travis lately:
> {code}
> [ERROR]
> shouldProcessSessionRequestsInOrder(org.apache.tinkerpop.gremlin.server.GremlinDriverIntegrateTest)
> Time elapsed: 9.593 s <<< FAILURE!
> java.lang.AssertionError:
> Expected: iterable containing ["five", "zero"]
> but: item 0: was "zero"
> at
> org.apache.tinkerpop.gremlin.server.GremlinDriverIntegrateTest.shouldProcessSessionRequestsInOrder(GremlinDriverIntegrateTest.java:621)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2473) Allow TraversalStrategy instances to be merged
[
https://issues.apache.org/jira/browse/TINKERPOP-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237377#comment-17237377
]
Marc de Lignie commented on TINKERPOP-2473:
---
OK, great, I already have some documentation written for the Traversal
Strategies section (with the current situation) which I expect to push this
Friday. If you use that as a basis, I can rebase my PR later on. B.t.w., my PR
does not really depend on TINKERPOP-2473, but it would certainly be better if
we set this straight right now and end up with consistent behaviour that
authorization can rely on.
> Allow TraversalStrategy instances to be merged
> --
>
> Key: TINKERPOP-2473
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2473
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.4.8
>Reporter: Stephen Mallette
>Priority: Major
>
> Not sure if this is a great idea but it came up as part of TINKERPOP-2389
> where there might be a need to assign both a client-side and server-side
> {{SubgraphStrategy}}. Currently, while not explicit, you can't assign more
> than one strategy of a particular type using {{withStrategies()}}. This task
> would make that explicit and provide a direct way for mergeable strategies to
> be pushed together into one. If we did that it would also simplify
> {{OptionsStrategy}} usage in {{with()}} step which currently finds an
> existing one first if present and then adds to it (otherwise creates it new).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2468) Stabilize shouldProcessSessionRequestsInOrder() test
[
https://issues.apache.org/jira/browse/TINKERPOP-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237290#comment-17237290
]
ASF GitHub Bot commented on TINKERPOP-2468:
---
spmallette commented on a change in pull request #1365:
URL: https://github.com/apache/tinkerpop/pull/1365#discussion_r528637725
##
File path:
gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinDriverIntegrateTest.java
##
@@ -612,40 +612,42 @@ public void shouldProcessRequestsOutOfOrder() throws
Exception {
}
}
+/**
+ * This test validates that the session requests are processed in-order on
the server. The order of results
+ * returned to the client might be different though since each result is
handled by a different executor thread.
+ */
Review comment:
just to clarify , by "different executor thread" are you referring to
threads on the client being responsible for the un-ordering?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Stabilize shouldProcessSessionRequestsInOrder() test
>
>
> Key: TINKERPOP-2468
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2468
> Project: TinkerPop
> Issue Type: Improvement
> Components: server
>Affects Versions: 3.4.8
>Reporter: Stephen Mallette
>Assignee: Divij Vaidya
>Priority: Minor
>
> This test has been failing somewhat randomly on Travis lately:
> {code}
> [ERROR]
> shouldProcessSessionRequestsInOrder(org.apache.tinkerpop.gremlin.server.GremlinDriverIntegrateTest)
> Time elapsed: 9.593 s <<< FAILURE!
> java.lang.AssertionError:
> Expected: iterable containing ["five", "zero"]
> but: item 0: was "zero"
> at
> org.apache.tinkerpop.gremlin.server.GremlinDriverIntegrateTest.shouldProcessSessionRequestsInOrder(GremlinDriverIntegrateTest.java:621)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2473) Allow TraversalStrategy instances to be merged
[
https://issues.apache.org/jira/browse/TINKERPOP-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237287#comment-17237287
]
Stephen Mallette commented on TINKERPOP-2473:
-
I agree with simply not allowing multiple strategies to be assigned. I suppose
it is a breaking change but I'd have to wonder if anyone is relying on the
currently inconsistent semantics in their code. We could probably make this
change in 3.4.x without raising any eyebrows.
> Allow TraversalStrategy instances to be merged
> --
>
> Key: TINKERPOP-2473
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2473
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
>Affects Versions: 3.4.8
>Reporter: Stephen Mallette
>Priority: Major
>
> Not sure if this is a great idea but it came up as part of TINKERPOP-2389
> where there might be a need to assign both a client-side and server-side
> {{SubgraphStrategy}}. Currently, while not explicit, you can't assign more
> than one strategy of a particular type using {{withStrategies()}}. This task
> would make that explicit and provide a direct way for mergeable strategies to
> be pushed together into one. If we did that it would also simplify
> {{OptionsStrategy}} usage in {{with()}} step which currently finds an
> existing one first if present and then adds to it (otherwise creates it new).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2390) Connections not released when closed abruptly in the server side
[
https://issues.apache.org/jira/browse/TINKERPOP-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237258#comment-17237258
]
Carlos commented on TINKERPOP-2390:
---
Hi Florian,
> Your REST layer is not sitting between Gremlin.NET and the server, but using
>Gremlin.NET to provide access to the server for clients via REST, right?
yes, that's it. The REST API uses Gremlin.NET to pose the queries to
Janusgraph.
{quote}and check the nestat of the client machine
{quote}
>Here you mean the client of your REST service? And you mean {{netstat}},
>right? Could you provide the exact command you used here?
Sorry for the ambiguity, here client would be the machine running the server
providing the REST API (external client => server with the REST API * =>
Janusgraph). In the *-machine, it was netstat the command used (typo), but I
cannot recall exacltly which netstat options we used (I assume that netstat -t
tcp to check the status of all the tcp connections and check why it was
stalled).
Finally, yes, it should not be so difficult to reproduce. Exactly as you wrote,
posing some queries that will run into a timeout will do the trick.
Another possible detail to bear in mind would be that we had the
TinkerpopServer configured not to provide any concurrent service (i.e., all the
queries were processed sequentially). I forgot to mention this as with the Java
driver it didn't seem to make a difference (but just in case it could be
relevant for reproducibility purposes).
Best,
> Connections not released when closed abruptly in the server side
>
>
> Key: TINKERPOP-2390
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2390
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.4.7
> Environment: Tinkerpop 3.4.7 + Janusgraph 0.5.1 (optional opencypher
> 1.0.0)
>Reporter: Carlos
>Priority: Major
>
> We have developed a WService to query a gremlin-server (JanusGraph 0.5.1)
> using the .net driver. Using the opencypher plugin has allowed us to see a
> behaviour where the server gets completely blocked after a timeout on the
> server side. We thought this might be related to issue
> https://issues.apache.org/jira/browse/TINKERPOP-2288, so we have moved our
> driver version to the master one (3.4-dev, which includes the PR solving this
> issue). However, when facing a timeout (server side always, it is the one
> launching the exception), quite a lot of connections get stalled at
> CLOSE_WAIT status, and the server becomes unusable.
> I've been digging around other bugs and issues, and from what I've read, some
> similar behaviour happened to CosmoDB (although it might be caused in that
> situation due to the some connection leaks, in this case is the timeout). We
> have traced down the problem to the driver itself after isolating all the
> components involved (optimizing the cypher query results in a non-timeout
> situation where everything is ok; forcing the timeout from pure gremlin
> replicates the behaviour).
> We have set up the connection pool params to 16 / 4096 (we are expecting
> quite a high concurrency load).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (TINKERPOP-2390) Connections not released when closed abruptly in the server side
[
https://issues.apache.org/jira/browse/TINKERPOP-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237253#comment-17237253
]
Florian Hockmann commented on TINKERPOP-2390:
-
Thanks for getting back on this. I'm trying to understand your setup and have a
few questions on that:
Your REST layer is not sitting between Gremlin.NET and the server, but using
Gremlin.NET to provide access to the server for clients via REST, right? So,
the REST facade you mention is only necessary to have the setup completely
running, including your REST service and its client. Or is the REST facade
actually between Gremlin.NET and the server somehow? (Which would mean that it
needs to support Websockets though.)
Now about this part:
{quote}and check the nestat of the client machine
{quote}
Here you mean the client of your REST service? And you mean {{netstat}}, right?
Could you provide the exact command you used here?
If I understood your description correctly, then it should be possible to
reproduce this without the REST component by just sending some queries that
will run into a timeout.
> Connections not released when closed abruptly in the server side
>
>
> Key: TINKERPOP-2390
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2390
> Project: TinkerPop
> Issue Type: Bug
> Components: dotnet
>Affects Versions: 3.4.7
> Environment: Tinkerpop 3.4.7 + Janusgraph 0.5.1 (optional opencypher
> 1.0.0)
>Reporter: Carlos
>Priority: Major
>
> We have developed a WService to query a gremlin-server (JanusGraph 0.5.1)
> using the .net driver. Using the opencypher plugin has allowed us to see a
> behaviour where the server gets completely blocked after a timeout on the
> server side. We thought this might be related to issue
> https://issues.apache.org/jira/browse/TINKERPOP-2288, so we have moved our
> driver version to the master one (3.4-dev, which includes the PR solving this
> issue). However, when facing a timeout (server side always, it is the one
> launching the exception), quite a lot of connections get stalled at
> CLOSE_WAIT status, and the server becomes unusable.
> I've been digging around other bugs and issues, and from what I've read, some
> similar behaviour happened to CosmoDB (although it might be caused in that
> situation due to the some connection leaks, in this case is the timeout). We
> have traced down the problem to the driver itself after isolating all the
> components involved (optimizing the cypher query results in a non-timeout
> situation where everything is ok; forcing the timeout from pure gremlin
> replicates the behaviour).
> We have set up the connection pool params to 16 / 4096 (we are expecting
> quite a high concurrency load).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
