[jira] [Created] (TINKERPOP-1701) HaltedTraverserStrategy should recurse into collections for detachment.

[Marko A. Rodriguez (JIRA)]

Marko A. Rodriguez created TINKERPOP-1701:
-

 Summary: HaltedTraverserStrategy should recurse into collections 
for detachment.
 Key: TINKERPOP-1701
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1701
 Project: TinkerPop
  Issue Type: Improvement
  Components: process
Affects Versions: 3.2.5
Reporter: Marko A. Rodriguez
Assignee: Marko A. Rodriguez


We need to make sure that {{HaltedTraverserStrategy}} recurses into 
collections. This will be elegantly handled by updated {{DetachedFactory}} and 
{{ReferenceFactory}} accordingly.

For instance:

{code}
public static  D detach(final Object object) {
if (object instanceof Element) {
return (D) ReferenceFactory.detach((Element) object);
} else if (object instanceof Property) {
return (D) ReferenceFactory.detach((Property) object);
} else if (object instanceof Path) {
return (D) ReferenceFactory.detach((Path) object);
} else if (object instanceof List) {
final List list = new ArrayList();
for (final Object item : (List) object) {
list.add(ReferenceFactory.detach(item));
}
return (D) list;
} else if (object instanceof Set) {
final Set set = object instanceof LinkedHashSet ? new 
LinkedHashSet() : new HashSet();
for (final Object item : (Set) object) {
set.add(ReferenceFactory.detach(item));
}
return (D) set;
} else if (object instanceof Map) {
final Map map = object instanceof LinkedHashMap ? new 
LinkedHashMap() : new HashMap();
for (final Map.Entry entry : ((Map) 
object).entrySet()) {
map.put(ReferenceFactory.detach(entry.getKey()), 
ReferenceFactory.detach(entry.getValue()));
}
return (D) map;
} else {
return (D) object;
}
}
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TINKERPOP-741) Remove Options For Transaction Retry

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065358#comment-16065358
 ] 

ASF GitHub Bot commented on TINKERPOP-741:
--

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/633
  
VOTE: +1


> Remove Options For Transaction Retry
> 
>
> Key: TINKERPOP-741
> URL: https://issues.apache.org/jira/browse/TINKERPOP-741
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: structure
>Affects Versions: 3.0.2-incubating
>Reporter: stephen mallette
>
> Submitting a transactional workload for automatic retry currently looks like 
> this:
> {code}
> public  Workload submit(final Function work);
> {code}
> but that doesn't take into account relatively recent changes that allow 
> mutation operations to execute over a {{Traversal}}.  It also doesn't account 
> for workload constructed over multiple requests in a session to Gremlin 
> Server too well.
> Maybe we do something like:
> {code}
> public  Workload submit(final Function... work);
> public  Workload submit(final Function... work);
> public  Workload submit(final Traversal... work);
> {code}
> I suppose it might need to return a {{List}} or something like that so 
> that each piece of the transaction could maintain its own result.
> With respect to Gremlin Server and in-session requests, a driver could 
> automate transaction retry by inserting scripts to a list variable on the 
> server side until client-side {{commit()}} at which point that list could be 
> {{submit}} for retry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TINKERPOP-741) Remove Options For Transaction Retry

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065356#comment-16065356
 ] 

ASF GitHub Bot commented on TINKERPOP-741:
--

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/634
  
VOTE: +1


> Remove Options For Transaction Retry
> 
>
> Key: TINKERPOP-741
> URL: https://issues.apache.org/jira/browse/TINKERPOP-741
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: structure
>Affects Versions: 3.0.2-incubating
>Reporter: stephen mallette
>
> Submitting a transactional workload for automatic retry currently looks like 
> this:
> {code}
> public  Workload submit(final Function work);
> {code}
> but that doesn't take into account relatively recent changes that allow 
> mutation operations to execute over a {{Traversal}}.  It also doesn't account 
> for workload constructed over multiple requests in a session to Gremlin 
> Server too well.
> Maybe we do something like:
> {code}
> public  Workload submit(final Function... work);
> public  Workload submit(final Function... work);
> public  Workload submit(final Traversal... work);
> {code}
> I suppose it might need to return a {{List}} or something like that so 
> that each piece of the transaction could maintain its own result.
> With respect to Gremlin Server and in-session requests, a driver could 
> automate transaction retry by inserting scripts to a list variable on the 
> server side until client-side {{commit()}} at which point that list could be 
> {{submit}} for retry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS] Gryo 3.0

[Stephen Mallette]

Marko digging into TINKERPOP-1592 (his comments on the "[DISCUSS] Remaining
Items for 3.3.0") has made me basically realize there isn't much value to
doing Gryo 3.0 at this point. More reasoning over on that thread.

On Tue, Jun 27, 2017 at 9:25 AM, Stephen Mallette 
wrote:

> We currently have a wide range of serialization options in Gryo around
> different types of "detached" Vertex/Edge instances which typically just
> used the default Kryo FieldSerializer. As mentioned earlier in this thread,
> I've wanted to change that pattern a bit for Gryo 3.0. I think that we can
> have one serializer that will efficiently serialize each type of "detached"
> Vertex/Edge that we have. When done in conjunction with TINKERPOP-1592 I
> think we will have a good solid design in place for 3.3.0.
>
> You can take a look at what's in store for the Gryo 3.0 Element
> serializers so far here:
>
> https://github.com/apache/tinkerpop/blob/TINKERPOP-1698/
> gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/
> GryoSerializersV3d0.java
>
> Hopefully this branch can form into a PR within the coming days.
>
>
>
> On Mon, Jun 26, 2017 at 6:22 PM, Stephen Mallette 
> wrote:
>
>> Now that we have gremlin-io-test for 3.3.0 which tests different versions
>> of gryo/graphson over different versions of tinkerpop (meaning the current
>> version you build with ensure that it can still read/write the old formats)
>> do we need to do this:
>>
>> https://issues.apache.org/jira/browse/TINKERPOP-1364
>>
>> I tend to think the answer is "no" at this point. I think that for the
>> distributions we just need to have Gryo and GraphSON 3.0 and that's it (and
>> obviously the existing GraphML files). If we do that we should get rid of a
>> dozen files or so. Anyway, unless there are objections in the next 72 hours
>> I will assume lazy consensus and move forward in that direction.
>>
>>
>>
>> On Mon, Jun 26, 2017 at 10:24 AM, Stephen Mallette 
>> wrote:
>>
>>> Just wanted to note that Gryo 3.0 work is happening here in case anyone
>>> wants to follow along:
>>>
>>> https://issues.apache.org/jira/browse/TINKERPOP-1698
>>>
>>> I have a branch setup. I think I'm going to clean up Gremlin Server
>>> configuration files for 3.3.0 and remove a bunch of the old serializers.
>>> The config is starting to look really confusing. I think it would be best
>>> to just have Gryo and GraphSON 3.0 configured in the packaged sample files.
>>>
>>>
>>> On Tue, Jun 20, 2017 at 8:25 AM, Stephen Mallette 
>>> wrote:
>>>
 I'd really like to get our IO situation really solid in 3.3.0 and while
 we talk about GraphSON a lot I think Gryo still has an important place in
 our serialization story. Gryo covers an extended set of classes that goes
 beyond what we cover in GraphSON. I don't know if all those registered
 classes matter though - one would think that we just need parity with
 GraphSON defined classes.

 https://github.com/apache/tinkerpop/blob/9f13e69853c03a2fde7
 7efd40c6526b91a4b346a/gremlin-core/src/main/java/org/apache/
 tinkerpop/gremlin/structure/io/gryo/GryoVersion.java#L192

 I think that the reason this area of code has grown so much is from the
 basis of another issue with gryo which is that we don't always have a nice
 separation between implementations and serialized classes. In other words,
 if we have some complex class like DefaultTraversalMetrics, we just
 register all the classes that DefaultTraversalMetrics has as fields (and
 all the classes those classes use) until the serialization works. While
 this is really easy and, in a sense, convenient, it creates some problems:

 1. We get this big long registration list which includes types we don't
 support in GraphSON (which typically aren't tested)
 2. We have now bound the serialization of the complex class to its
 structure. If there is a problem with DefaultTraversalMetrics in the future
 and we have to change a field we essentially break all compatibility within
 gryo. (i'm facing this issue now actually).

 I would really like Gryo 3.0 to do a better job in this area, allowing
 for better forward and backward compatibility with a more limited class
 registration list that is more in line with the types defined for  GraphSON
 support. I think we can get better forward/backward compatibility if (1)
 there is better separation between the object being serialized and the
 object used in application logic (think of the "detached" classes) and (2)
 gryo serializers for 3.0 have greater intelligence around internal
 compatibility handling.

 Anyway, I was mostly writing this as notes to myself in preparation for
 3.3.0, but I'd be happy to hear any thoughts anyone might have on this.

>>>
>>>
>>
>

[jira] [Commented] (TINKERPOP-915) Gremlin Server supports REST and Websockets simultanteously

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065264#comment-16065264
 ] 

ASF GitHub Bot commented on TINKERPOP-915:
--

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/618
  
interesting - thanks for that information.


> Gremlin Server supports REST and Websockets simultanteously
> ---
>
> Key: TINKERPOP-915
> URL: https://issues.apache.org/jira/browse/TINKERPOP-915
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: server
>Affects Versions: 3.0.2-incubating
>Reporter: stephen mallette
>
> Develop a {{Channelizer}} that allows REST and Websockets to be configured at 
> the same time.  I've personally tried to do this on a couple of attempts 
> while following a Netty sample, but I've never been able to get it to work.  
> Perhaps folks like [~pluradj] or [~dmill] would like to give it a go some 
> day? :)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] tinkerpop issue #618: TINKERPOP-915 Add combined handler for Http and Websoc...

[spmallette]

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/618
  
interesting - thanks for that information.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (TINKERPOP-1519) TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette updated TINKERPOP-1519:

Fix Version/s: (was: 3.2.6)
   (was: 3.3.0)

> TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration
> -
>
> Key: TINKERPOP-1519
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1519
> Project: TinkerPop
>  Issue Type: Bug
>  Components: tinkergraph
>Affects Versions: 3.1.1-incubating
> Environment: Mac OSX
>Reporter: Felix Chapman
>Assignee: Marko A. Rodriguez
>Priority: Minor
> Fix For: 3.1.8
>
>
> When executing a VertexProgram that sends messages on multiple MessageScopes 
> in a single iteration, then the messages behave as if they were sent on all 
> scopes within that iteration.
> e.g. if you send message {{A}} on {{out}} edges, and message {{B}} on {{in}} 
> edges, then {{A}} and {{B}} will instead be sent over both {{in}} and {{out}} 
> edges.
> The problem can be resolved by using only a single MessageScope per 
> iteration, but this involves increasing the number of iterations.
> An example of this behaviour is below:
> {code:java}
> public class TinkerTest {
> public static void main(String[] args) throws ExecutionException, 
> InterruptedException {
> TinkerGraph graph = TinkerGraph.open();
> Vertex a = graph.addVertex("a");
> Vertex b = graph.addVertex("b");
> Vertex c = graph.addVertex("c");
> a.addEdge("edge", b);
> b.addEdge("edge", c);
> // Simple graph:
> // a -> b -> c
> // Execute a traversal program that sends an incoming message of "2" 
> and an outgoing message of "1" from "b"
> // then each vertex sums any received messages
> ComputerResult result = graph.compute().program(new 
> MyVertexProgram()).submit().get();
> // We expect the results to be {a=2, b=0, c=1}. Instead it is {a=3, 
> b=0, c=3}
> 
> System.out.println(result.graph().traversal().V().group().by(Element::label).by("count").next());
> }
> }
> class MyVertexProgram implements VertexProgram {
> private final MessageScope.Local countMessageScopeIn = 
> MessageScope.Local.of(__::inE);
> private final MessageScope.Local countMessageScopeOut = 
> MessageScope.Local.of(__::outE);
> private static final String MEMORY_KEY = "count";
> private static final Set COMPUTE_KEYS = 
> Collections.singleton(MEMORY_KEY);
> @Override
> public void setup(final Memory memory) {}
> @Override
> public GraphComputer.Persist getPreferredPersist() {
> return GraphComputer.Persist.VERTEX_PROPERTIES;
> }
> @Override
> public Set getElementComputeKeys() {
> return COMPUTE_KEYS;
> }
> @Override
> public Set getMessageScopes(final Memory memory) {
> return Sets.newHashSet(countMessageScopeIn, countMessageScopeOut);
> }
> @Override
> public void execute(Vertex vertex, Messenger messenger, Memory 
> memory) {
> switch (memory.getIteration()) {
> case 0:
> if (vertex.label().equals("b")) {
> messenger.sendMessage(this.countMessageScopeIn, 2L);
> messenger.sendMessage(this.countMessageScopeOut, 1L);
> }
> break;
> case 1:
> long edgeCount = 
> IteratorUtils.reduce(messenger.receiveMessages(), 0L, (a, b) -> a + b);
> vertex.property(MEMORY_KEY, edgeCount);
> break;
> }
> }
> @Override
> public boolean terminate(final Memory memory) {
> return memory.getIteration() == 1;
> }
> @Override
> public GraphComputer.ResultGraph getPreferredResultGraph() {
> return GraphComputer.ResultGraph.NEW;
> }
> @Override
> public MyVertexProgram clone() {
> try {
> return (MyVertexProgram) super.clone();
> } catch (final CloneNotSupportedException e) {
> throw new RuntimeException(e);
> }
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Reopened] (TINKERPOP-1519) TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette reopened TINKERPOP-1519:
-

> TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration
> -
>
> Key: TINKERPOP-1519
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1519
> Project: TinkerPop
>  Issue Type: Bug
>  Components: tinkergraph
>Affects Versions: 3.1.1-incubating
> Environment: Mac OSX
>Reporter: Felix Chapman
>Assignee: Marko A. Rodriguez
>Priority: Minor
> Fix For: 3.1.8
>
>
> When executing a VertexProgram that sends messages on multiple MessageScopes 
> in a single iteration, then the messages behave as if they were sent on all 
> scopes within that iteration.
> e.g. if you send message {{A}} on {{out}} edges, and message {{B}} on {{in}} 
> edges, then {{A}} and {{B}} will instead be sent over both {{in}} and {{out}} 
> edges.
> The problem can be resolved by using only a single MessageScope per 
> iteration, but this involves increasing the number of iterations.
> An example of this behaviour is below:
> {code:java}
> public class TinkerTest {
> public static void main(String[] args) throws ExecutionException, 
> InterruptedException {
> TinkerGraph graph = TinkerGraph.open();
> Vertex a = graph.addVertex("a");
> Vertex b = graph.addVertex("b");
> Vertex c = graph.addVertex("c");
> a.addEdge("edge", b);
> b.addEdge("edge", c);
> // Simple graph:
> // a -> b -> c
> // Execute a traversal program that sends an incoming message of "2" 
> and an outgoing message of "1" from "b"
> // then each vertex sums any received messages
> ComputerResult result = graph.compute().program(new 
> MyVertexProgram()).submit().get();
> // We expect the results to be {a=2, b=0, c=1}. Instead it is {a=3, 
> b=0, c=3}
> 
> System.out.println(result.graph().traversal().V().group().by(Element::label).by("count").next());
> }
> }
> class MyVertexProgram implements VertexProgram {
> private final MessageScope.Local countMessageScopeIn = 
> MessageScope.Local.of(__::inE);
> private final MessageScope.Local countMessageScopeOut = 
> MessageScope.Local.of(__::outE);
> private static final String MEMORY_KEY = "count";
> private static final Set COMPUTE_KEYS = 
> Collections.singleton(MEMORY_KEY);
> @Override
> public void setup(final Memory memory) {}
> @Override
> public GraphComputer.Persist getPreferredPersist() {
> return GraphComputer.Persist.VERTEX_PROPERTIES;
> }
> @Override
> public Set getElementComputeKeys() {
> return COMPUTE_KEYS;
> }
> @Override
> public Set getMessageScopes(final Memory memory) {
> return Sets.newHashSet(countMessageScopeIn, countMessageScopeOut);
> }
> @Override
> public void execute(Vertex vertex, Messenger messenger, Memory 
> memory) {
> switch (memory.getIteration()) {
> case 0:
> if (vertex.label().equals("b")) {
> messenger.sendMessage(this.countMessageScopeIn, 2L);
> messenger.sendMessage(this.countMessageScopeOut, 1L);
> }
> break;
> case 1:
> long edgeCount = 
> IteratorUtils.reduce(messenger.receiveMessages(), 0L, (a, b) -> a + b);
> vertex.property(MEMORY_KEY, edgeCount);
> break;
> }
> }
> @Override
> public boolean terminate(final Memory memory) {
> return memory.getIteration() == 1;
> }
> @Override
> public GraphComputer.ResultGraph getPreferredResultGraph() {
> return GraphComputer.ResultGraph.NEW;
> }
> @Override
> public MyVertexProgram clone() {
> try {
> return (MyVertexProgram) super.clone();
> } catch (final CloneNotSupportedException e) {
> throw new RuntimeException(e);
> }
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (TINKERPOP-1519) TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration

[stephen mallette (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette closed TINKERPOP-1519.
---
Resolution: Fixed

> TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration
> -
>
> Key: TINKERPOP-1519
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1519
> Project: TinkerPop
>  Issue Type: Bug
>  Components: tinkergraph
>Affects Versions: 3.1.1-incubating
> Environment: Mac OSX
>Reporter: Felix Chapman
>Assignee: Marko A. Rodriguez
>Priority: Minor
> Fix For: 3.1.8
>
>
> When executing a VertexProgram that sends messages on multiple MessageScopes 
> in a single iteration, then the messages behave as if they were sent on all 
> scopes within that iteration.
> e.g. if you send message {{A}} on {{out}} edges, and message {{B}} on {{in}} 
> edges, then {{A}} and {{B}} will instead be sent over both {{in}} and {{out}} 
> edges.
> The problem can be resolved by using only a single MessageScope per 
> iteration, but this involves increasing the number of iterations.
> An example of this behaviour is below:
> {code:java}
> public class TinkerTest {
> public static void main(String[] args) throws ExecutionException, 
> InterruptedException {
> TinkerGraph graph = TinkerGraph.open();
> Vertex a = graph.addVertex("a");
> Vertex b = graph.addVertex("b");
> Vertex c = graph.addVertex("c");
> a.addEdge("edge", b);
> b.addEdge("edge", c);
> // Simple graph:
> // a -> b -> c
> // Execute a traversal program that sends an incoming message of "2" 
> and an outgoing message of "1" from "b"
> // then each vertex sums any received messages
> ComputerResult result = graph.compute().program(new 
> MyVertexProgram()).submit().get();
> // We expect the results to be {a=2, b=0, c=1}. Instead it is {a=3, 
> b=0, c=3}
> 
> System.out.println(result.graph().traversal().V().group().by(Element::label).by("count").next());
> }
> }
> class MyVertexProgram implements VertexProgram {
> private final MessageScope.Local countMessageScopeIn = 
> MessageScope.Local.of(__::inE);
> private final MessageScope.Local countMessageScopeOut = 
> MessageScope.Local.of(__::outE);
> private static final String MEMORY_KEY = "count";
> private static final Set COMPUTE_KEYS = 
> Collections.singleton(MEMORY_KEY);
> @Override
> public void setup(final Memory memory) {}
> @Override
> public GraphComputer.Persist getPreferredPersist() {
> return GraphComputer.Persist.VERTEX_PROPERTIES;
> }
> @Override
> public Set getElementComputeKeys() {
> return COMPUTE_KEYS;
> }
> @Override
> public Set getMessageScopes(final Memory memory) {
> return Sets.newHashSet(countMessageScopeIn, countMessageScopeOut);
> }
> @Override
> public void execute(Vertex vertex, Messenger messenger, Memory 
> memory) {
> switch (memory.getIteration()) {
> case 0:
> if (vertex.label().equals("b")) {
> messenger.sendMessage(this.countMessageScopeIn, 2L);
> messenger.sendMessage(this.countMessageScopeOut, 1L);
> }
> break;
> case 1:
> long edgeCount = 
> IteratorUtils.reduce(messenger.receiveMessages(), 0L, (a, b) -> a + b);
> vertex.property(MEMORY_KEY, edgeCount);
> break;
> }
> }
> @Override
> public boolean terminate(final Memory memory) {
> return memory.getIteration() == 1;
> }
> @Override
> public GraphComputer.ResultGraph getPreferredResultGraph() {
> return GraphComputer.ResultGraph.NEW;
> }
> @Override
> public MyVertexProgram clone() {
> try {
> return (MyVertexProgram) super.clone();
> } catch (final CloneNotSupportedException e) {
> throw new RuntimeException(e);
> }
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TINKERPOP-915) Gremlin Server supports REST and Websockets simultanteously

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065220#comment-16065220
 ] 

ASF GitHub Bot commented on TINKERPOP-915:
--

Github user krlohnes commented on the issue:

https://github.com/apache/tinkerpop/pull/618
  
@spmallette I did some testing between the http channelizer and the 
combined one.

With Basic Auth, the mean latency with 1 requests for the combined 
channelizer was 148.3 ms and the standalone channelizer was 148.9 ms.

Without Basic Auth the mean latency with 1 request for the combined 
channelizer was 4.8ms and the standalone channelizer was 4.9 ms

Given the results for http and that the ChannelPipeline is only manipulated 
for http, I didn't run any performance tests for WebSockets.

I'll write up the docs once I'm done refactoring the Channelizer tests.


> Gremlin Server supports REST and Websockets simultanteously
> ---
>
> Key: TINKERPOP-915
> URL: https://issues.apache.org/jira/browse/TINKERPOP-915
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: server
>Affects Versions: 3.0.2-incubating
>Reporter: stephen mallette
>
> Develop a {{Channelizer}} that allows REST and Websockets to be configured at 
> the same time.  I've personally tried to do this on a couple of attempts 
> while following a Netty sample, but I've never been able to get it to work.  
> Perhaps folks like [~pluradj] or [~dmill] would like to give it a go some 
> day? :)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] tinkerpop issue #618: TINKERPOP-915 Add combined handler for Http and Websoc...

[krlohnes]

Github user krlohnes commented on the issue:

https://github.com/apache/tinkerpop/pull/618
  
@spmallette I did some testing between the http channelizer and the 
combined one.

With Basic Auth, the mean latency with 1 requests for the combined 
channelizer was 148.3 ms and the standalone channelizer was 148.9 ms.

Without Basic Auth the mean latency with 1 request for the combined 
channelizer was 4.8ms and the standalone channelizer was 4.9 ms

Given the results for http and that the ChannelPipeline is only manipulated 
for http, I didn't run any performance tests for WebSockets.

I'll write up the docs once I'm done refactoring the Channelizer tests.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-479) Consider Providing "getOrCreate" Functionality

[Robert Dale (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065117#comment-16065117
 ] 

Robert Dale commented on TINKERPOP-479:
---

[~dkuppitz] After some real world tests, the {{fold()}} technique and the 
original {{..coalesce(__.V().has()...}} perform very closely when working with 
an embedded graph.  However, when using {{withRemote()}},  the original 
performs 2-3x better!  And oddly enough, if the data was populated, {{fold()}} 
performance stayed the same or did worse than loading data to a clean db.  The 
original gremlin performance improved when data was populated. Looks like I'm 
staying with the original gremlin. :D

> Consider Providing "getOrCreate" Functionality
> --
>
> Key: TINKERPOP-479
> URL: https://issues.apache.org/jira/browse/TINKERPOP-479
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: structure
>Affects Versions: 3.0.2-incubating
>Reporter: stephen mallette
>
> One of the most commonly written functions used is good ol' "getOrCreate" 
> where you want to get a {{Vertex}} if it exists or create it with supplied 
> properties if it does not.  We currently have a "helper" function for this on 
> {{ElementHelper}} 
> https://github.com/tinkerpop/tinkerpop3/blob/6d0f00865f673cb0739f6f310e1868425f732924/gremlin-core/src/main/java/com/tinkerpop/gremlin/structure/util/ElementHelper.java#L62
> but perhaps it is time to treat this issue as a first class citizen as part 
> of the Graph API.  I think that some vendors might actually be able to 
> optimize this function as well.  
> Another aspect of "getOrCreate" is "upsert" as well as options to ensure 
> uniqueness.  All of these things we've at some point or another built 
> variations of outside of TinkerPop for applications, data loading, etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (TINKERPOP-1519) TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration

[Marko A. Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marko A. Rodriguez closed TINKERPOP-1519.
-
   Resolution: Fixed
 Assignee: Marko A. Rodriguez
Fix Version/s: 3.2.6
   3.1.8
   3.3.0

> TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration
> -
>
> Key: TINKERPOP-1519
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1519
> Project: TinkerPop
>  Issue Type: Bug
>  Components: tinkergraph
>Affects Versions: 3.1.1-incubating
> Environment: Mac OSX
>Reporter: Felix Chapman
>Assignee: Marko A. Rodriguez
>Priority: Minor
> Fix For: 3.3.0, 3.1.8, 3.2.6
>
>
> When executing a VertexProgram that sends messages on multiple MessageScopes 
> in a single iteration, then the messages behave as if they were sent on all 
> scopes within that iteration.
> e.g. if you send message {{A}} on {{out}} edges, and message {{B}} on {{in}} 
> edges, then {{A}} and {{B}} will instead be sent over both {{in}} and {{out}} 
> edges.
> The problem can be resolved by using only a single MessageScope per 
> iteration, but this involves increasing the number of iterations.
> An example of this behaviour is below:
> {code:java}
> public class TinkerTest {
> public static void main(String[] args) throws ExecutionException, 
> InterruptedException {
> TinkerGraph graph = TinkerGraph.open();
> Vertex a = graph.addVertex("a");
> Vertex b = graph.addVertex("b");
> Vertex c = graph.addVertex("c");
> a.addEdge("edge", b);
> b.addEdge("edge", c);
> // Simple graph:
> // a -> b -> c
> // Execute a traversal program that sends an incoming message of "2" 
> and an outgoing message of "1" from "b"
> // then each vertex sums any received messages
> ComputerResult result = graph.compute().program(new 
> MyVertexProgram()).submit().get();
> // We expect the results to be {a=2, b=0, c=1}. Instead it is {a=3, 
> b=0, c=3}
> 
> System.out.println(result.graph().traversal().V().group().by(Element::label).by("count").next());
> }
> }
> class MyVertexProgram implements VertexProgram {
> private final MessageScope.Local countMessageScopeIn = 
> MessageScope.Local.of(__::inE);
> private final MessageScope.Local countMessageScopeOut = 
> MessageScope.Local.of(__::outE);
> private static final String MEMORY_KEY = "count";
> private static final Set COMPUTE_KEYS = 
> Collections.singleton(MEMORY_KEY);
> @Override
> public void setup(final Memory memory) {}
> @Override
> public GraphComputer.Persist getPreferredPersist() {
> return GraphComputer.Persist.VERTEX_PROPERTIES;
> }
> @Override
> public Set getElementComputeKeys() {
> return COMPUTE_KEYS;
> }
> @Override
> public Set getMessageScopes(final Memory memory) {
> return Sets.newHashSet(countMessageScopeIn, countMessageScopeOut);
> }
> @Override
> public void execute(Vertex vertex, Messenger messenger, Memory 
> memory) {
> switch (memory.getIteration()) {
> case 0:
> if (vertex.label().equals("b")) {
> messenger.sendMessage(this.countMessageScopeIn, 2L);
> messenger.sendMessage(this.countMessageScopeOut, 1L);
> }
> break;
> case 1:
> long edgeCount = 
> IteratorUtils.reduce(messenger.receiveMessages(), 0L, (a, b) -> a + b);
> vertex.property(MEMORY_KEY, edgeCount);
> break;
> }
> }
> @Override
> public boolean terminate(final Memory memory) {
> return memory.getIteration() == 1;
> }
> @Override
> public GraphComputer.ResultGraph getPreferredResultGraph() {
> return GraphComputer.ResultGraph.NEW;
> }
> @Override
> public MyVertexProgram clone() {
> try {
> return (MyVertexProgram) super.clone();
> } catch (final CloneNotSupportedException e) {
> throw new RuntimeException(e);
> }
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS] Remaining Items for 3.3.0

[Marko Rodriguez]

Hi,

In this email I will argue why TINKERPOP-1592 is a bad idea.

GremlinServer does “too much.” In the future, the concept for GremlinServer for 
TinkerPop4 should be “network I/O to the Gremlin virtual machine.” That is it. 
So what is GremlinServer doing now that is bad? Currently GremlinServer is 
“detaching” elements from the graph and populating those elements with more 
data than the user requested. What do I mean?:

g.V(1)

Currently that returns a DetachedVertex which is vertex 1’s id, label, and all 
of its properties. Now here is the crazy part. What if there are 1M properties 
(e.g. timestamped multi-properties on sensor network vertices). Doh! However, 
what did the user ask for? They asked for v[1] and that is all they should get. 
This is known as ReferenceVertex. If they wanted some subset of the timestamped 
sensor network data, they should have done:

g.V(1).properties(‘sensor’).has(‘timestamp’,gt(2017))

Thus, we should only return the data people explicitly ask for in the 
traversal. The TINKERPOP-1592 ticket is a hack for DetachedVertex by allowing 
users to specify withDetachment(properties:[“not_sensor”]), but then it is not 
expressive enough. Ultimately, for generality, users will want to specify full 
traversals in their withDetachment() specification. Now you are talking 
SubgraphStrategy. Dar! — and guess what, GremlinServer doesn’t respect 
SubgraphStrategy. This is the problem with everything NOT being traversal — 
once you start using the “Blueprints API” you start getting inconsistent 
behavior/functionality. Thus, GremlinServer does too much — just execute the 
traversal and return the result.

Next, DetachedXXX starts to get I-N-S-A-N-E when you start talking GLVs. Now we 
have the Blueprints API implements in C#, Python, etc. N! GLV’s should 
only implement ReferenceXXX which is the bare minimum specification of a graph 
object such that it can be re-attached (referenced back) to the source graph. 
Thats it. Don’t start populating it with properties — “what about edges?” — 
“can it get the neighboring vertices properties too?” — “what about ...?” — if 
you want that data, you traverse to it yourself!

So, what is the solution to the problem at hand — ReferenceXXX. These element 
classes are the minimal amount of data required to re-attach to the source 
graph. Thus,  if you do g.V(1), you get back id/label. However, if you want to 
then get the sensor data, you do g.V(v1).properties(…). Moreover, there is a 
little hidden gem called HaltedTraverserStrategy that allows the user to 
specify their desired element class — 
https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/strategy/decoration/HaltedTraverserStrategy.java
 
.

If GremlinServer is yielding too much data, simply do this: 

g = g.withStrategy(HaltedTraverserStrategy.reference())
g.V(1) // ahh… fresh and clean.

The trick to software is not to write it. If you are a software developer, you 
are not as good as the guy who runs the deli down the street cause guess what, 
he lives just fine and doesn’t write a lick of code. 

Marko.

http://markorodriguez.com



> On Jun 26, 2017, at 2:21 PM, Stephen Mallette  wrote:
> 
> Looking back at this thread, I think that since there were no objections to
> doing "pre-releases" of GLVs I think we can postpone the test suite
> changes. So, i'm fine with that being off the table.
> 
> Looking at my list, I'm also surprised that I didn't include:
> 
> https://issues.apache.org/jira/browse/TINKERPOP-1592
> 
> I think that will be important for providing more flexibility to users to
> shape results returned from traversals. That, of course, is important for
> GLV remoting so that users can return only data that matters to them.
> TINKERPOP-1592 funnels into the GraphSON/Gryo 3.0 stuff mentioned
> previously as we seek to make improvements there in terms of
> efficiency/performance/usability. Marko will be taking a look at the 1592
> ticket.
> 
> I think there is a good body of nice-to-have tickets (after going through
> them all in the last couple of weeks to do some housekeeping) so we'll see
> what we can get in there and what we can't after those more crucial bits
> are done. I believe that we could start thinking about release of 3.3.0 in
> the next 4 weeks or so.
> 
> If there are any other thoughts for what's going on with 3.3.0 please let
> them be known.
> 
> 
> 
> 
> 
> 
> On Thu, Jun 1, 2017 at 2:08 PM, Stephen Mallette 
> wrote:
> 
>> I was just thinking about what needs to be done for 3.3.0 to get it ready
>> for release. I thought I'd send this email to collect any ideas:
>> 
>> + Dynamically load the MetricManager in Gremlin Server (TINKERPOP-1550)
>> + Clean up IO - both 

[jira] [Commented] (TINKERPOP-1519) TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064923#comment-16064923
 ] 

ASF GitHub Bot commented on TINKERPOP-1519:
---

Github user asfgit closed the pull request at:

https://github.com/apache/tinkerpop/pull/632


> TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration
> -
>
> Key: TINKERPOP-1519
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1519
> Project: TinkerPop
>  Issue Type: Bug
>  Components: tinkergraph
>Affects Versions: 3.1.1-incubating
> Environment: Mac OSX
>Reporter: Felix Chapman
>Priority: Minor
>
> When executing a VertexProgram that sends messages on multiple MessageScopes 
> in a single iteration, then the messages behave as if they were sent on all 
> scopes within that iteration.
> e.g. if you send message {{A}} on {{out}} edges, and message {{B}} on {{in}} 
> edges, then {{A}} and {{B}} will instead be sent over both {{in}} and {{out}} 
> edges.
> The problem can be resolved by using only a single MessageScope per 
> iteration, but this involves increasing the number of iterations.
> An example of this behaviour is below:
> {code:java}
> public class TinkerTest {
> public static void main(String[] args) throws ExecutionException, 
> InterruptedException {
> TinkerGraph graph = TinkerGraph.open();
> Vertex a = graph.addVertex("a");
> Vertex b = graph.addVertex("b");
> Vertex c = graph.addVertex("c");
> a.addEdge("edge", b);
> b.addEdge("edge", c);
> // Simple graph:
> // a -> b -> c
> // Execute a traversal program that sends an incoming message of "2" 
> and an outgoing message of "1" from "b"
> // then each vertex sums any received messages
> ComputerResult result = graph.compute().program(new 
> MyVertexProgram()).submit().get();
> // We expect the results to be {a=2, b=0, c=1}. Instead it is {a=3, 
> b=0, c=3}
> 
> System.out.println(result.graph().traversal().V().group().by(Element::label).by("count").next());
> }
> }
> class MyVertexProgram implements VertexProgram {
> private final MessageScope.Local countMessageScopeIn = 
> MessageScope.Local.of(__::inE);
> private final MessageScope.Local countMessageScopeOut = 
> MessageScope.Local.of(__::outE);
> private static final String MEMORY_KEY = "count";
> private static final Set COMPUTE_KEYS = 
> Collections.singleton(MEMORY_KEY);
> @Override
> public void setup(final Memory memory) {}
> @Override
> public GraphComputer.Persist getPreferredPersist() {
> return GraphComputer.Persist.VERTEX_PROPERTIES;
> }
> @Override
> public Set getElementComputeKeys() {
> return COMPUTE_KEYS;
> }
> @Override
> public Set getMessageScopes(final Memory memory) {
> return Sets.newHashSet(countMessageScopeIn, countMessageScopeOut);
> }
> @Override
> public void execute(Vertex vertex, Messenger messenger, Memory 
> memory) {
> switch (memory.getIteration()) {
> case 0:
> if (vertex.label().equals("b")) {
> messenger.sendMessage(this.countMessageScopeIn, 2L);
> messenger.sendMessage(this.countMessageScopeOut, 1L);
> }
> break;
> case 1:
> long edgeCount = 
> IteratorUtils.reduce(messenger.receiveMessages(), 0L, (a, b) -> a + b);
> vertex.property(MEMORY_KEY, edgeCount);
> break;
> }
> }
> @Override
> public boolean terminate(final Memory memory) {
> return memory.getIteration() == 1;
> }
> @Override
> public GraphComputer.ResultGraph getPreferredResultGraph() {
> return GraphComputer.ResultGraph.NEW;
> }
> @Override
> public MyVertexProgram clone() {
> try {
> return (MyVertexProgram) super.clone();
> } catch (final CloneNotSupportedException e) {
> throw new RuntimeException(e);
> }
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TINKERPOP-1519) TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064919#comment-16064919
 ] 

ASF GitHub Bot commented on TINKERPOP-1519:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/632
  
VOTE: +1


> TinkerGraphComputer doesn't handle multiple MessageScopes in single iteration
> -
>
> Key: TINKERPOP-1519
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1519
> Project: TinkerPop
>  Issue Type: Bug
>  Components: tinkergraph
>Affects Versions: 3.1.1-incubating
> Environment: Mac OSX
>Reporter: Felix Chapman
>Priority: Minor
>
> When executing a VertexProgram that sends messages on multiple MessageScopes 
> in a single iteration, then the messages behave as if they were sent on all 
> scopes within that iteration.
> e.g. if you send message {{A}} on {{out}} edges, and message {{B}} on {{in}} 
> edges, then {{A}} and {{B}} will instead be sent over both {{in}} and {{out}} 
> edges.
> The problem can be resolved by using only a single MessageScope per 
> iteration, but this involves increasing the number of iterations.
> An example of this behaviour is below:
> {code:java}
> public class TinkerTest {
> public static void main(String[] args) throws ExecutionException, 
> InterruptedException {
> TinkerGraph graph = TinkerGraph.open();
> Vertex a = graph.addVertex("a");
> Vertex b = graph.addVertex("b");
> Vertex c = graph.addVertex("c");
> a.addEdge("edge", b);
> b.addEdge("edge", c);
> // Simple graph:
> // a -> b -> c
> // Execute a traversal program that sends an incoming message of "2" 
> and an outgoing message of "1" from "b"
> // then each vertex sums any received messages
> ComputerResult result = graph.compute().program(new 
> MyVertexProgram()).submit().get();
> // We expect the results to be {a=2, b=0, c=1}. Instead it is {a=3, 
> b=0, c=3}
> 
> System.out.println(result.graph().traversal().V().group().by(Element::label).by("count").next());
> }
> }
> class MyVertexProgram implements VertexProgram {
> private final MessageScope.Local countMessageScopeIn = 
> MessageScope.Local.of(__::inE);
> private final MessageScope.Local countMessageScopeOut = 
> MessageScope.Local.of(__::outE);
> private static final String MEMORY_KEY = "count";
> private static final Set COMPUTE_KEYS = 
> Collections.singleton(MEMORY_KEY);
> @Override
> public void setup(final Memory memory) {}
> @Override
> public GraphComputer.Persist getPreferredPersist() {
> return GraphComputer.Persist.VERTEX_PROPERTIES;
> }
> @Override
> public Set getElementComputeKeys() {
> return COMPUTE_KEYS;
> }
> @Override
> public Set getMessageScopes(final Memory memory) {
> return Sets.newHashSet(countMessageScopeIn, countMessageScopeOut);
> }
> @Override
> public void execute(Vertex vertex, Messenger messenger, Memory 
> memory) {
> switch (memory.getIteration()) {
> case 0:
> if (vertex.label().equals("b")) {
> messenger.sendMessage(this.countMessageScopeIn, 2L);
> messenger.sendMessage(this.countMessageScopeOut, 1L);
> }
> break;
> case 1:
> long edgeCount = 
> IteratorUtils.reduce(messenger.receiveMessages(), 0L, (a, b) -> a + b);
> vertex.property(MEMORY_KEY, edgeCount);
> break;
> }
> }
> @Override
> public boolean terminate(final Memory memory) {
> return memory.getIteration() == 1;
> }
> @Override
> public GraphComputer.ResultGraph getPreferredResultGraph() {
> return GraphComputer.ResultGraph.NEW;
> }
> @Override
> public MyVertexProgram clone() {
> try {
> return (MyVertexProgram) super.clone();
> } catch (final CloneNotSupportedException e) {
> throw new RuntimeException(e);
> }
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (TINKERPOP-1700) Remove deprecated embedTypes option

[stephen mallette (JIRA)]

stephen mallette created TINKERPOP-1700:
---

 Summary: Remove deprecated embedTypes option
 Key: TINKERPOP-1700
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1700
 Project: TinkerPop
  Issue Type: Improvement
  Components: io
Affects Versions: 3.2.5
Reporter: stephen mallette
Assignee: stephen mallette


{{GraphSONMapper$Builder#embedTypes}} was long ago replaced in GraphSON 2.0 and 
in GraphSON 3.0 the option to go "typeless" will be gone as well so this 
setting is becoming increasingly useless.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (TINKERPOP-915) Gremlin Server supports REST and Websockets simultanteously

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/TINKERPOP-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064807#comment-16064807
 ] 

ASF GitHub Bot commented on TINKERPOP-915:
--

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/618
  
@krlohnes have you done any performance testing to determine if the 
combined channelizer shows any difference to the independent channelizers? i 
think that would be worthwhile to know in evaluating this PR. As an added note 
which I didn't mention before I don't think - this PR will need reference docs, 
upgrade docs and changelog entries. 


> Gremlin Server supports REST and Websockets simultanteously
> ---
>
> Key: TINKERPOP-915
> URL: https://issues.apache.org/jira/browse/TINKERPOP-915
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: server
>Affects Versions: 3.0.2-incubating
>Reporter: stephen mallette
>
> Develop a {{Channelizer}} that allows REST and Websockets to be configured at 
> the same time.  I've personally tried to do this on a couple of attempts 
> while following a Netty sample, but I've never been able to get it to work.  
> Perhaps folks like [~pluradj] or [~dmill] would like to give it a go some 
> day? :)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] tinkerpop issue #632: TINKERPOP-1519: tinker graph computer does not handle ...

[dkuppitz]

Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/632
  
VOTE: +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---