[jira] [Created] (TINKERPOP-1567) GraphSON deserialization fails with within('a')

2016-11-23 Thread Branden Moore (JIRA) 
Branden Moore created TINKERPOP-1567:


 Summary: GraphSON deserialization fails with within('a')
 Key: TINKERPOP-1567
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1567
 Project: TinkerPop
  Issue Type: Bug
  Components: language-variant, server
Affects Versions: 3.2.3
 Environment: Demonstrated on Ubuntu 14.04 and Debian Testing
Reporter: Branden Moore
Priority: Minor


Using the gremlin-python interface, when I attempt to run an example from the 
Documentation of {{where}}:
{code}
g.withSideEffect('a','josh','peter']).V(1).out('created').in_('created').values('name').where(P.within('a')).toList()
{code}
The server returns a deserialization error:

{noformat}
{{[WARN] OpExecutorHandler - Could not deserialize the JSON value as required. 
Nested exception: 
org.apache.tinkerpop.shaded.jackson.databind.JsonMappingExce[18/147]
ould not deserialize the JSON value as required. Nested exception: 
java.lang.IllegalStateException: 
org.apache.tinkerpop.gremlin.process.traversal.P.within(java.la
ng.Object)  
  
 at [Source: 
{"@type":"g:Bytecode","@value":{"source":[["withSideEffect","a",["josh","peter"]]],"step":[["V",{"@type":"g:Int32","@value":1}],["out","created"],["in
","created"],["values","name"],["where",{"@type":"g:P","@value":{"predicate":"within","value":"a"}}]]}};
 line: 1, column: 248] (through reference chain: java.util.
LinkedHashMap["step"]->java.util.ArrayList[4]->java.util.ArrayList[1])  
   
 at [Source: 
{"@type":"g:Bytecode","@value":{"source":[["withSideEffect","a",["josh","peter"]]],"step":[["V",{"@type":"g:Int32","@value":1}],["out","created"],["in
","created"],["values","name"],["where",{"@type":"g:P","@value":{"predicate":"within","value":"a"}}]]}};
 line: 1, column: 248]
org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not 
deserialize the JSON value as required. Nested exception: 
org.apache.tinkerpop.shaded.
jackson.databind.JsonMappingException: Could not deserialize the JSON value as 
required. Nested exception: java.lang.IllegalStateException: 
org.apache.tinkerpop.gr
emlin.process.traversal.P.within(java.lang.Object)  

at [Source: 
{"@type":"g:Bytecode","@value":{"source":[["withSideEffect","a",["josh","peter"]]],"step":[["V",{"@type":"g:Int32","@value":1}],["out","created"],["in
","created"],["values","name"],["where",{"@type":"g:P","@value":{"predicate":"within","value":"a"}}]]}};
 line: 1, column: 248] (through reference chain: 
java.util.LinkedHashMap["step"]->java.util.ArrayList[4]->java.util.ArrayList[1])
 at [Source: 
{"@type":"g:Bytecode","@value":{"source":[["withSideEffect","a",["josh","peter"]]],"step":[["V",{"@type":"g:Int32","@value":1}],["out","created"],["in","created"],["values","name"],["where",{"@type":"g:P","@value":{"predicate":"within","value":"a"}}]]}};
 line: 1, column: 248] 
at 
org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.from(JsonMappingException.java:216)
   at 
org.apache.tinkerpop.shaded.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:883)
   at 
org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypeDeserializer.deserialize(GraphSONTypeDeserializer.java:195)
at 
org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypeDeserializer.deserializeTypedFromAny(GraphSONTypeDeserializer.java:102)
at 
org.apache.tinkerpop.shaded.jackson.databind.deser.std.StdDeserializer.deserializeWithType(StdDeserializer.java:120)
at 
org.apache.tinkerpop.shaded.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:42)
at 
org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3789)
at 
org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2779)
at 
org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.iterateBytecodeTraversal(TraversalOpProcessor.java:339)
at 
org.apache.tinkerpop.gremlin.server.handler.OpExecutorHandler.channelRead0(OpExecutorHandler.java:68)
at 
org.apache.tinkerpop.gremlin.server.handler.OpExecutorHandler.channelRead0(OpExecutorHandler.java:43)
at 
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352)
at

[jira] [Commented] (TINKERPOP-1565) GraphSON 2.0 updates -- supporting attachment, maintaining consistency, and reducing verbosity

2016-11-23 Thread Kevin Gallardo (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691560#comment-15691560
 ] 

Kevin Gallardo commented on TINKERPOP-1565:
---

h5. Reference and Detached Elements:
I believe that this can be nicely handled by having a GraphSONModule added to 
the GraphSONMapper, that replaces all the current serializers and 
deserializers, with lighter ones, that serializes *all* the Elements as 
Reference Elements, and deserializes them accordingly to Reference Elements. 
The GraphSON2Mapper is dynamic enough to allow to override the serializers and 
deserializers for already existing types like the Graph structure types, so 
you'd have all the "g:Vertex"/"g:VertexProperty" types mapped to 
Vertex/VertexProperty but the implem would be Detached Elements instead of the 
current Detached more heavy ones. But it sounds weird to me if we had in a same 
response, different formats for the same "g:" type.

h5. Consistency with Properties
It sounds reasonable that all the classes that derive from Element, have their 
properties serialized in the same format, for consistency. Since we cannot go 
around having Vertices' VertexProperties being serialized with their types, 
then the other Elements (VertexProperty's Properties and Edge Properties) 
should follow the same path.

h5. Attaching deserialized elements
I understand the need of the parent info to reattach. However, the comment I 
made was on the way it was done that could lead to problems. Assuming the code 
in the PR was *creating a new* Vertex object when deserializing a 
VertexProperty, based on the Vertex ID info there was in the JSON. So when a 
VertexProperty is serialized on its own, that's fine. But when a VertexProperty 
is serialized as part of its parent Vertex (so the JSON is "v + vp1 + vp2 + 
"), then creating a new Vertex object when deserializing the VertexProperty 
means that the actual parent object of the VP, and what the VP thinks its 
parent is, are not the same (Java)objects 
({{v.properties('name').next().element() != v}}). That's why I introduced that 
mechanism in Jackson to indicate to the VP deserializer for example, if the VP 
to deserialize is on its own or if it's been serialized with its parent. Hope 
that's clearer

> GraphSON 2.0 updates -- supporting attachment, maintaining consistency, and 
> reducing verbosity
> --
>
> Key: TINKERPOP-1565
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1565
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 3.2.3
>Reporter: Marko A. Rodriguez
>  Labels: breaking
> Fix For: 3.2.3
>
>
> GraphSON 2.0 has some issues that should be recified for its "official 
> release" in 3.3.0.
> *Supporting Attachment*
> We need to make sure that every element and property can be attached back to 
> the main graph.
> * For {{Vertex}}, this means that we need to have the ID encoded (CHECK).
> * For {{Edge}}, this means we need to have the out/in vertex ids encoded 
> (CHECK).
> * For {{VertexProperty}}, this means we need to have the vertex ID encoded 
> (ERROR).
> * For {{Property}}, this means we need to have the the element ID (vertex 
> property or edge) encoded and then the subsequent vertex ID (ERROR).
> *Maintaining Consistency*
> Currently, property encoding in {{Edge}} is different than property encoding 
> in {{VertexProperty}}. 
> Edge -->
> {code}
> properties : {
>   key : "aKey",
>   value : { @type: "gProperty", @value : {"key","aKey","value",10}}
> }
> {code}
> VertexProperty-->
> {code}
> properties : {
>   key : "aKey",
>   value : 10
> }
> {code}
> This should be consistent.
> *Reducing Verbosity*
> We have the problem of representing both {{DetachedElements}} and 
> {{ReferenceElements}}. {{DetachedElements}} contain lots of information and 
> is typicaly used to ensure a data rich result set. {{ReferenceElements}} 
> contain only the necessary information for attachment. I think we can support 
> both representations by making certain fields "optional."
> Vertex-->
> {code}
> return new 
> Vertex(json.get("id"),json.getOrDefault("label",Vertex.DEFAULT_LABEL),json.getOrDefault("properties",Collections.emptyMap()))
> {code}
> That is, lots of {{getOrDefault()}} use will make sure that when desesired, 
> we only send/recv information that is needed for the particular 
> computatation. E.g. as dictated by {{HaltedTraverserStrategy}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

FOSDEM 2017 HPC, Bigdata and Data Science DevRoom CFP is closing soon

2016-11-23 Thread Roman Shaposhnik 
Hi!

apologies for the extra wide distribution (this exhausts my once
a year ASF mail-to-all-bigdata-projects quota ;-)) but I wanted
to suggest that all of you should consider submitting talks
to FOSDEM 2017 HPC, Bigdata and Data Science DevRoom:
https://hpc-bigdata-fosdem17.github.io/

It was a great success this year and we hope to make it an even
bigger success in 2017.

Besides -- FOSDEM is the biggest gathering of open source
developers on the face of the earth -- don't miss it!

Thanks,
Roman.

P.S. If you have any questions -- please email me directly and
see you all in Brussels!

[jira] [Commented] (TINKERPOP-1566) Kerberos authentication for gremlin-server

2016-11-23 Thread Marc de Lignie (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691288#comment-15691288
 ] 

Marc de Lignie commented on TINKERPOP-1566:
---

This issue is a follow-up on a short discussion on the dev list:
https://lists.apache.org/thread.html/dde14d6c7fd660f94de326761d72c482fa4f08b40ebfed1772d8ea14@%3Cdev.tinkerpop.apache.org%3E

I will add a link to my git repo later on, so that the work on this issue can 
be followed.

> Kerberos authentication for gremlin-server
> --
>
> Key: TINKERPOP-1566
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1566
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: server
>Affects Versions: 3.2.3
>Reporter: Marc de Lignie
>Priority: Minor
>  Labels: security
>
> Gremlin server would benefit from an explicit Kerberos authentication plugin, 
> because preparing and maintaining such a plugin is nontrivial. Also, many 
> other Apache project provide kerberized services.
> In gremlin-console the standard Krb5LoginModule can be configured. 
> Gremlin-server already includes the pluggable Sasl framework that can host 
> the proposed Kerberos authentication plugin. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (TINKERPOP-1566) Kerberos authentication for gremlin-server

2016-11-23 Thread stephen mallette (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stephen mallette updated TINKERPOP-1566:

Affects Version/s: 3.2.3

> Kerberos authentication for gremlin-server
> --
>
> Key: TINKERPOP-1566
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1566
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: server
>Affects Versions: 3.2.3
>Reporter: Marc de Lignie
>Priority: Minor
>  Labels: security
>
> Gremlin server would benefit from an explicit Kerberos authentication plugin, 
> because preparing and maintaining such a plugin is nontrivial. Also, many 
> other Apache project provide kerberized services.
> In gremlin-console the standard Krb5LoginModule can be configured. 
> Gremlin-server already includes the pluggable Sasl framework that can host 
> the proposed Kerberos authentication plugin. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (TINKERPOP-1566) Kerberos authentication for gremlin-server

2016-11-23 Thread Marc de Lignie (JIRA) 
Marc de Lignie created TINKERPOP-1566:
-

 Summary: Kerberos authentication for gremlin-server
 Key: TINKERPOP-1566
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1566
 Project: TinkerPop
  Issue Type: Improvement
  Components: server
Reporter: Marc de Lignie
Priority: Minor


Gremlin server would benefit from an explicit Kerberos authentication plugin, 
because preparing and maintaining such a plugin is nontrivial. Also, many other 
Apache project provide kerberized services.
In gremlin-console the standard Krb5LoginModule can be configured. 
Gremlin-server already includes the pluggable Sasl framework that can host the 
proposed Kerberos authentication plugin. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1520) Difference between 'has' step generated graphson2.0 in java and python glv implementation

2016-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690709#comment-15690709
 ] 

ASF GitHub Bot commented on TINKERPOP-1520:
---

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/499#discussion_r89358284
  
--- Diff: 
gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalTest.java
 ---
@@ -71,4 +80,89 @@ public void 
graphTraversalShouldHaveMethodsOfAnonymousGraphTraversal() {
 }
 }
 }
+
+@Test
+public void graphTraversalMethodsShouldGenerateCorrespondingBytecode() 
throws Exception {
+final Random random = new Random();
--- End diff --

I can amend the branch


> Difference between 'has' step generated graphson2.0 in java and python glv 
> implementation
> -
>
> Key: TINKERPOP-1520
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1520
> Project: TinkerPop
>  Issue Type: Bug
>  Components: language-variant
>Affects Versions: 3.2.3
>Reporter: Andy Tolbert
>
> Noticed that between the java and python implementations, the graphson2.0 
> payload generated for a {{has}} step is different.  i.e. for the given 
> traversal:
> {{g.E().has("weight", 0.2)}}
> The java implementation produces the following graphson:
> {code:javascript}
> {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",{"@type":"g:P","@value":{"predicate":"eq","value":{"@type":"g:Double","@value":0.2}}}]]}}
> {code}
> where the python implementation produces the following:
> {code:javascript}
>  {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",0.2]]}}
> {code}
> In the java case, a {{g\:P}} typed (predicate) value is provided, where in 
> the python case that isn't the case.
> I'm assuming the java one is correct (primarily since the graph backend seems 
> to like it and return the expected result).   Should GLV implementations 
> behave this way?  I noticed that {{GraphTraversal#has(String propertyKey, 
> Object value)}} in the java TinkerPop api wraps the value in a predicate 
> ({{P.eq}}) under the covers 
> ([link|https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java#L922])
>  so maybe implementors will need to do the same ([python 
> link|https://github.com/apache/tinkerpop/blob/master/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py#L193])?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #499: TINKERPOP-1520: Difference between 'has' step g...

2016-11-23 Thread spmallette 
Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/499#discussion_r89358284
  
--- Diff: 
gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalTest.java
 ---
@@ -71,4 +80,89 @@ public void 
graphTraversalShouldHaveMethodsOfAnonymousGraphTraversal() {
 }
 }
 }
+
+@Test
+public void graphTraversalMethodsShouldGenerateCorrespondingBytecode() 
throws Exception {
+final Random random = new Random();
--- End diff --

I can amend the branch


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1520) Difference between 'has' step generated graphson2.0 in java and python glv implementation

2016-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690634#comment-15690634
 ] 

ASF GitHub Bot commented on TINKERPOP-1520:
---

Github user okram commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/499#discussion_r89352742
  
--- Diff: 
gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalTest.java
 ---
@@ -71,4 +80,89 @@ public void 
graphTraversalShouldHaveMethodsOfAnonymousGraphTraversal() {
 }
 }
 }
+
+@Test
+public void graphTraversalMethodsShouldGenerateCorrespondingBytecode() 
throws Exception {
+final Random random = new Random();
--- End diff --

Ah yea. Thats smart. Do you want to push that to this branch or tell me how 
to do it with an example?


> Difference between 'has' step generated graphson2.0 in java and python glv 
> implementation
> -
>
> Key: TINKERPOP-1520
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1520
> Project: TinkerPop
>  Issue Type: Bug
>  Components: language-variant
>Affects Versions: 3.2.3
>Reporter: Andy Tolbert
>
> Noticed that between the java and python implementations, the graphson2.0 
> payload generated for a {{has}} step is different.  i.e. for the given 
> traversal:
> {{g.E().has("weight", 0.2)}}
> The java implementation produces the following graphson:
> {code:javascript}
> {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",{"@type":"g:P","@value":{"predicate":"eq","value":{"@type":"g:Double","@value":0.2}}}]]}}
> {code}
> where the python implementation produces the following:
> {code:javascript}
>  {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",0.2]]}}
> {code}
> In the java case, a {{g\:P}} typed (predicate) value is provided, where in 
> the python case that isn't the case.
> I'm assuming the java one is correct (primarily since the graph backend seems 
> to like it and return the expected result).   Should GLV implementations 
> behave this way?  I noticed that {{GraphTraversal#has(String propertyKey, 
> Object value)}} in the java TinkerPop api wraps the value in a predicate 
> ({{P.eq}}) under the covers 
> ([link|https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java#L922])
>  so maybe implementors will need to do the same ([python 
> link|https://github.com/apache/tinkerpop/blob/master/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py#L193])?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1520) Difference between 'has' step generated graphson2.0 in java and python glv implementation

2016-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690576#comment-15690576
 ] 

ASF GitHub Bot commented on TINKERPOP-1520:
---

Github user spmallette commented on a diff in the pull request:

https://github.com/apache/tinkerpop/pull/499#discussion_r89348850
  
--- Diff: 
gremlin-core/src/test/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversalTest.java
 ---
@@ -71,4 +80,89 @@ public void 
graphTraversalShouldHaveMethodsOfAnonymousGraphTraversal() {
 }
 }
 }
+
+@Test
+public void graphTraversalMethodsShouldGenerateCorrespondingBytecode() 
throws Exception {
+final Random random = new Random();
--- End diff --

did you see my comment above @okram ? can we go with option 1 for this PR?


> Difference between 'has' step generated graphson2.0 in java and python glv 
> implementation
> -
>
> Key: TINKERPOP-1520
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1520
> Project: TinkerPop
>  Issue Type: Bug
>  Components: language-variant
>Affects Versions: 3.2.3
>Reporter: Andy Tolbert
>
> Noticed that between the java and python implementations, the graphson2.0 
> payload generated for a {{has}} step is different.  i.e. for the given 
> traversal:
> {{g.E().has("weight", 0.2)}}
> The java implementation produces the following graphson:
> {code:javascript}
> {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",{"@type":"g:P","@value":{"predicate":"eq","value":{"@type":"g:Double","@value":0.2}}}]]}}
> {code}
> where the python implementation produces the following:
> {code:javascript}
>  {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",0.2]]}}
> {code}
> In the java case, a {{g\:P}} typed (predicate) value is provided, where in 
> the python case that isn't the case.
> I'm assuming the java one is correct (primarily since the graph backend seems 
> to like it and return the expected result).   Should GLV implementations 
> behave this way?  I noticed that {{GraphTraversal#has(String propertyKey, 
> Object value)}} in the java TinkerPop api wraps the value in a predicate 
> ({{P.eq}}) under the covers 
> ([link|https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java#L922])
>  so maybe implementors will need to do the same ([python 
> link|https://github.com/apache/tinkerpop/blob/master/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py#L193])?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1564) Distributed OLTP Traversals and the Introduction of Partition Concept

2016-11-23 Thread pieter martin (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690416#comment-15690416
 ] 

pieter martin commented on TINKERPOP-1564:
--

Will this only work for OLTP if the traversals are executed via GremlinServer?

> Distributed OLTP Traversals and the Introduction of Partition Concept
> -
>
> Key: TINKERPOP-1564
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1564
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: driver, process, server
>Affects Versions: 3.2.3
>Reporter: Marko A. Rodriguez
>
> This proposal unifies OLTP and OLAP into a single framework that removes the 
> need for OLAP {{GraphComputer}} by introducing distributed, data local 
> processing to OLTP. In essence, this is a proposal for a step-by-step query 
> routing framework within {{Traversal}}. This proposal can work across 
> machines in a cluster, threads on a machine, or in a hierarchical fashion 
> machines The example presented will discuss distribution across 
> machines in a cluster as its the most complicated scenario.
> Currently, an OLTP traversal executes at a particular machine (or thread) and 
> pulls vertex/edge/etc. data to it accordingly in order to solve the 
> traversal. In OLAP, the traversal is cloned and distributed to all machines 
> in the cluster and traversals communicate with one another by sending 
> {{Traversers}} (i.e. messages) between themselves ensuring data local 
> processing. Given recent advancements in GremlinServer and 
> {{RemoteTraversal}}, it is possible to add traverser routing to OLTP and 
> thus, effect the computational paradigm of Gremlin OLAP in Gremlin OLTP with 
> some added benefits not possible in Gremlin OLAP.
> Assume a 4 machine cluster and the following traversal:
> {code}
> g.V(1).out(‘knows’).has(‘age’,gt(20)).out(‘likes’).values(‘name’)
> {code}
> Every time there is a "walk" (adjacency), it is possible that the 
> {{Traverser}} is no longer accessing data local to the current machine. In 
> order to do data local query routing, every adjacency would feed into a 
> {{PartitionStep}}. The traversal above would be cloned (via {{Bytecode}} 
> distribution) across the cluster where "sibling" {{PartitionSteps}} would 
> have network access to one another using the same protocol of 
> {{RemoteConnection}} though called {{PartitionConnection}}. Thus, given the 4 
> node cluster example, the above traversal would be overlaid as below. Note 
> that {{partition()}} would not be a new step in the language, but simply 
> provided here to show where {{PartitionStrategy}} would insert 
> {{PartitionSteps}} into the traversal.
> {code}
> g.V(1).out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>^
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>|
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>|
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
> {code}
> The top traversal is called the "master traversal" and the other three 
> "worker traversals." Note that this is identical to current Gremlin OLAP. 
> Now, the master traversal would be the traversal that is {{.next()}}'d for 
> results. So, when the "master traversal" is {{next()}}'d, {{g.V(1)}} will 
> fetch {{v[1]}} and then its outgoing knows-adjacencies. These adjacent 
> "reference vertices" would be fed into the first {{remote()}} and a "routing 
> algorithm" would determine where in the cluster the particular vertex's data 
> is. Thus, {{partition()}} ({{PartitionStep}}) serves as a router, pushing 
> {{Traversers}} local to the data. Finally, note that the final 
> {{PartitionSteps}} can only feed back to the "master traversal" for ultimate 
> aggregation and return to the user. 
> TinkerPop currently has all the structures in place to make this possible:
>   1. Encapsulation of computational metadata via {{Traverser}}.
>   2. The ability to detach {{Traversers}} and migrate/serialize them via 
> {{Traverser.detach()}} and {{Traverser.attach()}}.
>   3. The concept of {{ReferenceElement}} so the traverser only carries 
> with it enough information to re-attach at the remote site.
>   4. {{Bytecode}} and the ability to send {{Traversals}} across the 
> cluster.
>   5. GremlinServer and {{Client}}/{{Cluster}} messaging protocol.
> What does

[GitHub] tinkerpop issue #446: TINKERPOP-1483: valueMap should always return string k...

2016-11-23 Thread robertdale 
Github user robertdale commented on the issue:

https://github.com/apache/tinkerpop/pull/446
  
Can this PR be updated with Map or should it be closed and 
open a new one?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1483) PropertyMapStep returns Map<String,E> but puts non String keys in it!

2016-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690371#comment-15690371
 ] 

ASF GitHub Bot commented on TINKERPOP-1483:
---

Github user robertdale commented on the issue:

https://github.com/apache/tinkerpop/pull/446
  
Can this PR be updated with Map or should it be closed and 
open a new one?


> PropertyMapStep returns Map but puts non String keys in it!
> -
>
> Key: TINKERPOP-1483
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1483
> Project: TinkerPop
>  Issue Type: Bug
>  Components: process
>Affects Versions: 3.2.2
>Reporter: JP Moresmau
>
> PropertyMapStep.map has return type Map, but if includeTokens is 
> true:
> {code}
> if (element instanceof VertexProperty) {
> map.put(T.id, element.id());
> map.put(T.key, ((VertexProperty) element).key());
> map.put(T.value, ((VertexProperty) element).value());
> } else {
> map.put(T.id, element.id());
> map.put(T.label, element.label());
> }
> {code}
> T.id, T.key and T.value are NOT strings, so code looping through the keys in 
> Java fails. toString() are missing... But do we rely on having these keys in 
> other operations?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1493) Groovy project doesn't build on Windows

2016-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690326#comment-15690326
 ] 

ASF GitHub Bot commented on TINKERPOP-1493:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/457
  
good to know they both work. i sorta figured `merge` would be ok in this 
case as github would just figure out the diff and only include his one commit. 
when we merged that to our stuff, it wouldn't mess up history as it came from a 
fork. though i wonder if merge would bring an ugly "merge" commit with it. if 
it did, then i was wrong and `git rebase` would have been the way to go. 

@pauljackson since @robertdale  confirmed that `git rebase` is good, 
perhaps you should go that route to keep our history neat.


> Groovy project doesn't build on Windows
> ---
>
> Key: TINKERPOP-1493
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1493
> Project: TinkerPop
>  Issue Type: Bug
>  Components: groovy
>Affects Versions: 3.2.2
>Reporter: Paul Jackson
>Assignee: Jason Plurad
>Priority: Minor
>
> Builds on Windows fail for two reasons. First the line to create extTestDir 
> is creating a path consisting of two full paths concatenated together. The 
> second drive letter is seen as an illegal character:
> {code}private static final File extTestDir = new 
> File(System.getProperty("user.dir"), 
> TestHelper.makeTestDataDirectory(DependencyGrabberTest.class));{code}
> Second, when it comes time to delete the directory it is locked. This is 
> because some instances of JarFile are created on it but not closed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #457: TINKERPOP-1493 Groovy project doesn't build on Windows

2016-11-23 Thread spmallette 
Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/457
  
good to know they both work. i sorta figured `merge` would be ok in this 
case as github would just figure out the diff and only include his one commit. 
when we merged that to our stuff, it wouldn't mess up history as it came from a 
fork. though i wonder if merge would bring an ugly "merge" commit with it. if 
it did, then i was wrong and `git rebase` would have been the way to go. 

@pauljackson since @robertdale  confirmed that `git rebase` is good, 
perhaps you should go that route to keep our history neat.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1564) Distributed OLTP Traversals and the Introduction of Partition Concept

2016-11-23 Thread Marko A. Rodriguez (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690228#comment-15690228
 ] 

Marko A. Rodriguez commented on TINKERPOP-1564:
---

{code}
g = graph.traversal().withPartitioner() 
  // uses graph.partitioner() to get partitions
  // this would have a single worker at each partitioner/machine
g = graph.traversal().withPartitioner(GlobalPartitioner.instance()) 
  // what we current have (basically, default) -- there is only one partition 
and its the entire graph (thus, one traversal only).
g = graph.traversal().withPartitioner(HashPartioner.of(graph.partitioner(),5)) 
  // creates 5 subpartitions of every graph.partitioner().partitions()
  // this would be a way to create 5 workers on every partition/machine. (i.e. 
partition threads)
g = 
graph.traversal().withPartitioner(GroupPartitioner.of(graph.partitioner(),2))
  // creates 2 superpartitioners aggregating multiple partitioners underneath
  // this would create 2 workers across the cluster where each worker is 
responsible for 1/2 the cluster.
{code}

This adds {{PartitionerStrategy}} to the traverasl source's strategies which 
will then spawn {{PartitionTraversals}} accordingly across the partitions.

> Distributed OLTP Traversals and the Introduction of Partition Concept
> -
>
> Key: TINKERPOP-1564
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1564
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: driver, process, server
>Affects Versions: 3.2.3
>Reporter: Marko A. Rodriguez
>
> This proposal unifies OLTP and OLAP into a single framework that removes the 
> need for OLAP {{GraphComputer}} by introducing distributed, data local 
> processing to OLTP. In essence, this is a proposal for a step-by-step query 
> routing framework within {{Traversal}}. This proposal can work across 
> machines in a cluster, threads on a machine, or in a hierarchical fashion 
> machines The example presented will discuss distribution across 
> machines in a cluster as its the most complicated scenario.
> Currently, an OLTP traversal executes at a particular machine (or thread) and 
> pulls vertex/edge/etc. data to it accordingly in order to solve the 
> traversal. In OLAP, the traversal is cloned and distributed to all machines 
> in the cluster and traversals communicate with one another by sending 
> {{Traversers}} (i.e. messages) between themselves ensuring data local 
> processing. Given recent advancements in GremlinServer and 
> {{RemoteTraversal}}, it is possible to add traverser routing to OLTP and 
> thus, effect the computational paradigm of Gremlin OLAP in Gremlin OLTP with 
> some added benefits not possible in Gremlin OLAP.
> Assume a 4 machine cluster and the following traversal:
> {code}
> g.V(1).out(‘knows’).has(‘age’,gt(20)).out(‘likes’).values(‘name’)
> {code}
> Every time there is a "walk" (adjacency), it is possible that the 
> {{Traverser}} is no longer accessing data local to the current machine. In 
> order to do data local query routing, every adjacency would feed into a 
> {{PartitionStep}}. The traversal above would be cloned (via {{Bytecode}} 
> distribution) across the cluster where "sibling" {{PartitionSteps}} would 
> have network access to one another using the same protocol of 
> {{RemoteConnection}} though called {{PartitionConnection}}. Thus, given the 4 
> node cluster example, the above traversal would be overlaid as below. Note 
> that {{partition()}} would not be a new step in the language, but simply 
> provided here to show where {{PartitionStrategy}} would insert 
> {{PartitionSteps}} into the traversal.
> {code}
> g.V(1).out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>^
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>|
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>|
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
> {code}
> The top traversal is called the "master traversal" and the other three 
> "worker traversals." Note that this is identical to current Gremlin OLAP. 
> Now, the master traversal would be the traversal that is {{.next()}}'d for 
> results. So, when the "master traversal" is {{next()}}'d, {{g.V(1)}} will 
> fetch {{v[1]}} and then its outgoing knows-adjacencies. These adjacent 
> "reference vertices" would be fed into

[jira] [Commented] (TINKERPOP-1564) Distributed OLTP Traversals and the Introduction of Partition Concept

2016-11-23 Thread stephen mallette (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690202#comment-15690202
 ] 

stephen mallette commented on TINKERPOP-1564:
-

We could probably change the restriction on intermediate side-effects. That was 
more done for simplicity than for anything else. Of course I'm not sure how 
side-effects would work in any case - maybe i'm thinking of it wrong - but is 
the master traversal responsible for holding the side-effects in this proposal?

> Distributed OLTP Traversals and the Introduction of Partition Concept
> -
>
> Key: TINKERPOP-1564
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1564
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: driver, process, server
>Affects Versions: 3.2.3
>Reporter: Marko A. Rodriguez
>
> This proposal unifies OLTP and OLAP into a single framework that removes the 
> need for OLAP {{GraphComputer}} by introducing distributed, data local 
> processing to OLTP. In essence, this is a proposal for a step-by-step query 
> routing framework within {{Traversal}}. This proposal can work across 
> machines in a cluster, threads on a machine, or in a hierarchical fashion 
> machines The example presented will discuss distribution across 
> machines in a cluster as its the most complicated scenario.
> Currently, an OLTP traversal executes at a particular machine (or thread) and 
> pulls vertex/edge/etc. data to it accordingly in order to solve the 
> traversal. In OLAP, the traversal is cloned and distributed to all machines 
> in the cluster and traversals communicate with one another by sending 
> {{Traversers}} (i.e. messages) between themselves ensuring data local 
> processing. Given recent advancements in GremlinServer and 
> {{RemoteTraversal}}, it is possible to add traverser routing to OLTP and 
> thus, effect the computational paradigm of Gremlin OLAP in Gremlin OLTP with 
> some added benefits not possible in Gremlin OLAP.
> Assume a 4 machine cluster and the following traversal:
> {code}
> g.V(1).out(‘knows’).has(‘age’,gt(20)).out(‘likes’).values(‘name’)
> {code}
> Every time there is a "walk" (adjacency), it is possible that the 
> {{Traverser}} is no longer accessing data local to the current machine. In 
> order to do data local query routing, every adjacency would feed into a 
> {{PartitionStep}}. The traversal above would be cloned (via {{Bytecode}} 
> distribution) across the cluster where "sibling" {{PartitionSteps}} would 
> have network access to one another using the same protocol of 
> {{RemoteConnection}} though called {{PartitionConnection}}. Thus, given the 4 
> node cluster example, the above traversal would be overlaid as below. Note 
> that {{partition()}} would not be a new step in the language, but simply 
> provided here to show where {{PartitionStrategy}} would insert 
> {{PartitionSteps}} into the traversal.
> {code}
> g.V(1).out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>^
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>|
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>|
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
> {code}
> The top traversal is called the "master traversal" and the other three 
> "worker traversals." Note that this is identical to current Gremlin OLAP. 
> Now, the master traversal would be the traversal that is {{.next()}}'d for 
> results. So, when the "master traversal" is {{next()}}'d, {{g.V(1)}} will 
> fetch {{v[1]}} and then its outgoing knows-adjacencies. These adjacent 
> "reference vertices" would be fed into the first {{remote()}} and a "routing 
> algorithm" would determine where in the cluster the particular vertex's data 
> is. Thus, {{partition()}} ({{PartitionStep}}) serves as a router, pushing 
> {{Traversers}} local to the data. Finally, note that the final 
> {{PartitionSteps}} can only feed back to the "master traversal" for ultimate 
> aggregation and return to the user. 
> TinkerPop currently has all the structures in place to make this possible:
>   1. Encapsulation of computational metadata via {{Traverser}}.
>   2. The ability to detach {{Traversers}} and migrate/serialize them via 
> {{Traverser.detach()}} and {{Traverser.attach()}}.
>   3. The concept of {{ReferenceElement}} so the traverser

[jira] [Commented] (TINKERPOP-1520) Difference between 'has' step generated graphson2.0 in java and python glv implementation

2016-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15690129#comment-15690129
 ] 

ASF GitHub Bot commented on TINKERPOP-1520:
---

Github user okram commented on the issue:

https://github.com/apache/tinkerpop/pull/499
  
You are a bad person @dkuppitz. 


> Difference between 'has' step generated graphson2.0 in java and python glv 
> implementation
> -
>
> Key: TINKERPOP-1520
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1520
> Project: TinkerPop
>  Issue Type: Bug
>  Components: language-variant
>Affects Versions: 3.2.3
>Reporter: Andy Tolbert
>
> Noticed that between the java and python implementations, the graphson2.0 
> payload generated for a {{has}} step is different.  i.e. for the given 
> traversal:
> {{g.E().has("weight", 0.2)}}
> The java implementation produces the following graphson:
> {code:javascript}
> {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",{"@type":"g:P","@value":{"predicate":"eq","value":{"@type":"g:Double","@value":0.2}}}]]}}
> {code}
> where the python implementation produces the following:
> {code:javascript}
>  {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",0.2]]}}
> {code}
> In the java case, a {{g\:P}} typed (predicate) value is provided, where in 
> the python case that isn't the case.
> I'm assuming the java one is correct (primarily since the graph backend seems 
> to like it and return the expected result).   Should GLV implementations 
> behave this way?  I noticed that {{GraphTraversal#has(String propertyKey, 
> Object value)}} in the java TinkerPop api wraps the value in a predicate 
> ({{P.eq}}) under the covers 
> ([link|https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java#L922])
>  so maybe implementors will need to do the same ([python 
> link|https://github.com/apache/tinkerpop/blob/master/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py#L193])?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #499: TINKERPOP-1520: Difference between 'has' step generate...

2016-11-23 Thread okram 
Github user okram commented on the issue:

https://github.com/apache/tinkerpop/pull/499
  
You are a bad person @dkuppitz. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: More higher order constructs?

2016-11-23 Thread Florian Hockmann 
Hi,

there are some use cases / patterns that are needed by many users and I 
think it is a good idea to support them directly in TinkerPop with such 
higher-order patterns. For me personally, I would really appreciate 
getOrCreate steps for vertices and edges and also some upsert step. Both 
are already described in an old ticket 
 and they are also 
frequently requested on this mailing list and on StackOverflow.

Another pattern that could be interesting for Gremlin is a getSimilar step. 
Elasticsearch has for example a More Like This Query 
.
 
For a graph, this could be solved by traversing the neighborhood of the 
start vertex and trying to find vertices with the same label that have 
short paths to the start vertex and ideally those paths have only few 
branches (e.g., a path including a supernode doesn't mean that those 
vertices are similar). But this use case might already be too special / 
complicated. So I would start with easier patterns like the bidirectional 
search.

Regards,
Florian

Am Mittwoch, 23. November 2016 13:46:51 UTC+1 schrieb Marko A. Rodriguez:
>
> Hi,
>
> g.V(1).as(‘a’).
>   V(2).as(‘b’).
>   shortestPath().from(‘a’).to(‘b’).
> by(outE(‘knows’).has(’since’,lt(2010)).inV()).
>   path()
>
>
> Is equivalent to:
>
> g.V(1).as(‘a’).
>   V(2).as(‘b’).
>   repeat(outE(‘knows’).has(’since’,lt(2010))
> .inV()).until(where(‘a’)).limit(1).
> path()
>
>
> However, for the Gremlin-Java VM a BiDirectionalSearchStep could then be 
> added in the compilation as a more optimal way of doing this.
>
> Basically, should we be adding more high-order constructs for common 
> patterns….? If so, what other common patterns do you know of that we should 
> make steps?
>
> Marko.
>
> http://markorodriguez.com
>
>
>
>

[GitHub] tinkerpop issue #499: TINKERPOP-1520: Difference between 'has' step generate...

2016-11-23 Thread dkuppitz 
Github user dkuppitz commented on the issue:

https://github.com/apache/tinkerpop/pull/499
  
`docker/build.sh -t -i -n` succeeded.

VOTE: +1

(don't blame me for voting on half-done work; @okram assured me that we 
won't have a release before the work, started in this PR, is actually done)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Comment Edited] (TINKERPOP-1564) Distributed OLTP Traversals and the Introduction of Partition Concept

2016-11-23 Thread Marko A. Rodriguez (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15687218#comment-15687218
 ] 

Marko A. Rodriguez edited comment on TINKERPOP-1564 at 11/23/16 12:51 PM:
--

A {{Graph}} is composed of {{Partitions}}. A {{Partition}} is responsible for 
maintaining a subgraph of the global graph. For backwards compatibility, a 
{{SinglePartitions}} is simply a wrapper around {{Graph}}.

{code}
public interface Graph {
  public default Partitions getPartitions() {
return new SingletonPartitions(this);
  }
}
{code}

A {{Partitions}} is the set of all partitions associated with the {{Graph}}. 
{{Partitions}} maintains the "hash algorithm" which determines which 
{{Partition}} an {{Element}} is in.

{code}
public interface Partitions {
  public List getPartitions();
  public Partition getPartition(final Element element);
}
{code}


A {{Partition}} is a subgraph of the entire {{Graph}}. In adjacency list graph 
systems, a {{Partition}} will maintain a set of vertices and their incident 
edges and properties. It should be possible to stream out all the 
vertices/edges associated with the {{Partition}} where the the set of all 
iterators across all partitions is equivalent to 
{{Graph.vertices()}}/{{Graph.edges()}}.

{code}
public interface Partition {
  public boolean contains(final Element element);
  public Iterator vertices(final Object... ids);
  public Iterator edges(final Object... ids);

  public static interface PhysicalPartition extends Partition {}
  public static interface VirtualPartition extends Partition {}
}
{code}

A {{PartitionTraversal}} should represent a {{Traversal}} at a particular 
{{Partition}}. We currently do this, but that partition is simply a single 
machine in the cluster. We should generalize to support {{PartitionTraversals}} 
that mirror each other across the cluster.

{code}
public interface PartitionTraversal extends Traversal.Admin {
  
  public default void insert(final Traverser.Admin traverser); // do 
something with Client/Cluster here (implementation specific to GremlinServer)

  public default void accept(final Traverser.Admin traverser) {
final Step step = this.getStepById(traverser.getStepId());
step.addStart(traverser.attach());
  }
}
{code}

Given the concept of {{Partitions}}, {{PartitionStep}} will be able to route 
traversers to different {{PartitionTraversals}}.

{code}
public class PartitionStep extends AbstractStep {
  
  private final List partitionTraversals;
  private final Partitions partitions;
  private final Partition localPartition;

  public PartitionStep(final Traversal.Admin traversal, final Partition 
localPartition, final List partitionTraversals) {
super(traversal);
this.partitionTraversals = partitionTraversals;
this.localPartition = localPartition;
  }

  public Traverser.Admin processNextStart() {
final Traverser.Admin traverser = this.starts.next();
if(traverser.get() instanceof Element && 
!this.localPartition.contains(traverser.get())) {
  final PartitionTraversal partitionTraversal = 
this.partitionTraversals.stream().
 
filter(p -> p.getPartition().contains(traverser.get())).
 
findFirst().get(); // we won't use stream, but this is the basic idea
  partitionTraversal.insert(traverser.detach());
} else
  return traverser;
  }
}
{code}

--

The architecture is thus: 

{code}
 
/--PartitionConnection->PartitionTraversal
/  [Client]
client --RemoteConnection--> 
RemoteTraversalPartitionConnection->PartitionTraversal
\  [Client]
 
\--PartitionConnection->PartitionTraversal
{code} [Client]


was (Author: okram):
A {{Graph}} is composed of {{Partitions}}. A {{Partition}} is responsible for 
maintaining a subgraph of the global graph. For backwards compatibility, a 
{{SinglePartitions}} is simply a wrapper around {{Graph}}.

{code}
public interface Graph {
  public default Partitions getPartitions() {
return new SingletonPartitions(this);
  }
}
{code}

A {{Partitions}} is the set of all partitions associated with the {{Graph}}. 
{{Partitions}} maintains the "hash algorithm" which determines which 
{{Partition}} an {{Element}} is in.

{code}
public interface Partitions {
  public List getPartitions();
  public Partition getPartition(final Element element);
}
{code}


A {{Partition}} is a subgraph of the entire {{Graph}}. In adjacency list graph 
systems, a {{Partition}} will maintain a set of vertices and their incident 
edges and properties. It should be possible to

[jira] [Updated] (TINKERPOP-1564) Distributed OLTP Traversals and the Introduction of Partition Concept

2016-11-23 Thread Marko A. Rodriguez (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marko A. Rodriguez updated TINKERPOP-1564:
--
Summary: Distributed OLTP Traversals and the Introduction of Partition 
Concept  (was: Distributed OLTP Traversals via RemoteStep)

> Distributed OLTP Traversals and the Introduction of Partition Concept
> -
>
> Key: TINKERPOP-1564
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1564
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: driver, process, server
>Affects Versions: 3.2.3
>Reporter: Marko A. Rodriguez
>
> This proposal unifies OLTP and OLAP into a single framework that removes the 
> need for OLAP {{GraphComputer}} by introducing distributed, data local 
> processing to OLTP. In essence, this is a proposal for a step-by-step query 
> routing framework within {{Traversal}}. This proposal can work across 
> machines in a cluster, threads on a machine, or in a hierarchical fashion 
> machines The example presented will discuss distribution across 
> machines in a cluster as its the most complicated scenario.
> Currently, an OLTP traversal executes at a particular machine (or thread) and 
> pulls vertex/edge/etc. data to it accordingly in order to solve the 
> traversal. In OLAP, the traversal is cloned and distributed to all machines 
> in the cluster and traversals communicate with one another by sending 
> {{Traversers}} (i.e. messages) between themselves ensuring data local 
> processing. Given recent advancements in GremlinServer and 
> {{RemoteTraversal}}, it is possible to add traverser routing to OLTP and 
> thus, effect the computational paradigm of Gremlin OLAP in Gremlin OLTP with 
> some added benefits not possible in Gremlin OLAP.
> Assume a 4 machine cluster and the following traversal:
> {code}
> g.V(1).out(‘knows’).has(‘age’,gt(20)).out(‘likes’).values(‘name’)
> {code}
> Every time there is a "walk" (adjacency), it is possible that the 
> {{Traverser}} is no longer accessing data local to the current machine. In 
> order to do data local query routing, every adjacency would feed into a 
> {{PartitionStep}}. The traversal above would be cloned (via {{Bytecode}} 
> distribution) across the cluster where "sibling" {{PartitionSteps}} would 
> have network access to one another using the same protocol of 
> {{RemoteConnection}} though called {{PartitionConnection}}. Thus, given the 4 
> node cluster example, the above traversal would be overlaid as below. Note 
> that {{partition()}} would not be a new step in the language, but simply 
> provided here to show where {{PartitionStrategy}} would insert 
> {{PartitionSteps}} into the traversal.
> {code}
> g.V(1).out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>^
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>|
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
>|   |  
>|
> 
> __.out(‘knows’).partition().has(‘age’,gt(20)).out(‘likes’).partition().values(‘name’).partition()
> {code}
> The top traversal is called the "master traversal" and the other three 
> "worker traversals." Note that this is identical to current Gremlin OLAP. 
> Now, the master traversal would be the traversal that is {{.next()}}'d for 
> results. So, when the "master traversal" is {{next()}}'d, {{g.V(1)}} will 
> fetch {{v[1]}} and then its outgoing knows-adjacencies. These adjacent 
> "reference vertices" would be fed into the first {{remote()}} and a "routing 
> algorithm" would determine where in the cluster the particular vertex's data 
> is. Thus, {{partition()}} ({{PartitionStep}}) serves as a router, pushing 
> {{Traversers}} local to the data. Finally, note that the final 
> {{PartitionSteps}} can only feed back to the "master traversal" for ultimate 
> aggregation and return to the user. 
> TinkerPop currently has all the structures in place to make this possible:
>   1. Encapsulation of computational metadata via {{Traverser}}.
>   2. The ability to detach {{Traversers}} and migrate/serialize them via 
> {{Traverser.detach()}} and {{Traverser.attach()}}.
>   3. The concept of {{ReferenceElement}} so the traverser only carries 
> with it enough information to re-attach at the remote site.
>   4. {{Bytecode}} and the ability to send {{Traversals}} across the 
> cluster.
>   5. GremlinServer and {{Client}}/{{Cluster}}

More higher order constructs?

2016-11-23 Thread Marko Rodriguez 
Hi,

g.V(1).as(‘a’).
  V(2).as(‘b’).
  shortestPath().from(‘a’).to(‘b’).
by(outE(‘knows’).has(’since’,lt(2010)).inV()).
  path()

Is equivalent to:

g.V(1).as(‘a’).
  V(2).as(‘b’).
  repeat(outE(‘knows’).has(’since’,lt(2010)).inV()).until(where(‘a’)).limit(1).
path()

However, for the Gremlin-Java VM a BiDirectionalSearchStep could then be added 
in the compilation as a more optimal way of doing this.

Basically, should we be adding more high-order constructs for common 
patterns….? If so, what other common patterns do you know of that we should 
make steps?

Marko.

http://markorodriguez.com

[jira] [Commented] (TINKERPOP-1493) Groovy project doesn't build on Windows

2016-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689973#comment-15689973
 ] 

ASF GitHub Bot commented on TINKERPOP-1493:
---

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/457
  
well - your `tp31` is now out of date because a number of changes 
(including the one that i need to test this branch with) have been added to 
that branch that are not in yours. If you were working in our repository I 
would say that to get those changes you should rebase which will effectively 
bring in the new changes and more your commits to the head of all those changes.

But it occurs to me now that you are working from a fork and perhaps you 
just need to merge `upstream/tp31` (our repo - i assume you called the remote 
"upstream") into your fork. I don't think you need to rebase your fork.


> Groovy project doesn't build on Windows
> ---
>
> Key: TINKERPOP-1493
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1493
> Project: TinkerPop
>  Issue Type: Bug
>  Components: groovy
>Affects Versions: 3.2.2
>Reporter: Paul Jackson
>Assignee: Jason Plurad
>Priority: Minor
>
> Builds on Windows fail for two reasons. First the line to create extTestDir 
> is creating a path consisting of two full paths concatenated together. The 
> second drive letter is seen as an illegal character:
> {code}private static final File extTestDir = new 
> File(System.getProperty("user.dir"), 
> TestHelper.makeTestDataDirectory(DependencyGrabberTest.class));{code}
> Second, when it comes time to delete the directory it is locked. This is 
> because some instances of JarFile are created on it but not closed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #457: TINKERPOP-1493 Groovy project doesn't build on Windows

2016-11-23 Thread spmallette 
Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/457
  
well - your `tp31` is now out of date because a number of changes 
(including the one that i need to test this branch with) have been added to 
that branch that are not in yours. If you were working in our repository I 
would say that to get those changes you should rebase which will effectively 
bring in the new changes and more your commits to the head of all those changes.

But it occurs to me now that you are working from a fork and perhaps you 
just need to merge `upstream/tp31` (our repo - i assume you called the remote 
"upstream") into your fork. I don't think you need to rebase your fork.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1493) Groovy project doesn't build on Windows

2016-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689942#comment-15689942
 ] 

ASF GitHub Bot commented on TINKERPOP-1493:
---

Github user pauljackson commented on the issue:

https://github.com/apache/tinkerpop/pull/457
  
Sorry, I’m fluent in git (shame). From reading, I gleaned that rebase is a 
type of merge from one branch to another. Since I created the pull request on 
tp31 I don’t understand what it means to rebase it on tp31. I’m happy to follow 
directions if you could give me more explicit steps.

Thanks,
-Paul


From: stephen mallette [mailto:notificati...@github.com]
Sent: Saturday, November 19, 2016 7:52 AM
To: apache/tinkerpop
Cc: Paul A. Jackson; Mention
Subject: Re: [apache/tinkerpop] TINKERPOP-1493 Groovy project doesn't build 
on Windows (#457)


Looks like #498 has merged. 
@pauljackson if you could kindly rebase this on 
tp31 for me I can give it a final test.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on 
GitHub, or 
mute the 
thread.






> Groovy project doesn't build on Windows
> ---
>
> Key: TINKERPOP-1493
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1493
> Project: TinkerPop
>  Issue Type: Bug
>  Components: groovy
>Affects Versions: 3.2.2
>Reporter: Paul Jackson
>Assignee: Jason Plurad
>Priority: Minor
>
> Builds on Windows fail for two reasons. First the line to create extTestDir 
> is creating a path consisting of two full paths concatenated together. The 
> second drive letter is seen as an illegal character:
> {code}private static final File extTestDir = new 
> File(System.getProperty("user.dir"), 
> TestHelper.makeTestDataDirectory(DependencyGrabberTest.class));{code}
> Second, when it comes time to delete the directory it is locked. This is 
> because some instances of JarFile are created on it but not closed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (TINKERPOP-1559) OLAP path query results are losing edges

2016-11-23 Thread Marko A. Rodriguez (JIRA) 

 [ 
https://issues.apache.org/jira/browse/TINKERPOP-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marko A. Rodriguez closed TINKERPOP-1559.
-
   Resolution: Fixed
 Assignee: Marko A. Rodriguez
Fix Version/s: 3.2.3

This was a bug that was introduced in a development branch and thus, never made 
it to an official release. A test case was added to {{PathTest}} that shows 
{{tp31/}} is fine, but {{tp32/}} and {{master/}} were broken. Fixed in 
{{tp32/}} and upmerged to {{master/}} (simple fix).

> OLAP path query results are losing edges
> 
>
> Key: TINKERPOP-1559
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1559
> Project: TinkerPop
>  Issue Type: Bug
>  Components: hadoop, process
>Affects Versions: 3.2.3
>Reporter: Daniel Kuppitz
>Assignee: Marko A. Rodriguez
>Priority: Critical
> Fix For: 3.2.3
>
>
> {noformat}
> gremlin> g = TinkerFactory.createModern().traversal()
> ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
> gremlin> a = g.withComputer()
> ==>graphtraversalsource[tinkergraph[vertices:6 edges:6], graphcomputer]
> gremlin> g.V().outE().inV().path()
> ==>[v[1],e[9][1-created->3],v[3]]
> ==>[v[1],e[7][1-knows->2],v[2]]
> ==>[v[1],e[8][1-knows->4],v[4]]
> ==>[v[4],e[10][4-created->5],v[5]]
> ==>[v[4],e[11][4-created->3],v[3]]
> ==>[v[6],e[12][6-created->3],v[3]]
> gremlin> a.V().outE().inV().path()
> ==>[v[1],v[2]]
> ==>[v[1],v[3]]
> ==>[v[4],v[3]]
> ==>[v[1],v[4]]
> ==>[v[6],v[3]]
> ==>[v[4],v[5]]
> {noformat}
> {noformat}
> gremlin> graph = GraphFactory.open("conf/hadoop/hadoop-gryo.properties")
> ==>hadoopgraph[gryoinputformat->gryooutputformat]
> gremlin> g = graph.traversal().withComputer(SparkGraphComputer)
> ==>graphtraversalsource[hadoopgraph[gryoinputformat->gryooutputformat], 
> sparkgraphcomputer]
> gremlin> g.V().outE().inV().path()
> ...
> ==>[v[1],v[4]]
> ==>[v[1],v[3]]
> ==>[v[4],v[3]]
> ==>[v[6],v[3]]
> ==>[v[4],v[5]]
> ==>[v[1],v[2]]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #457: TINKERPOP-1493 Groovy project doesn't build on Windows

2016-11-23 Thread pauljackson 
Github user pauljackson commented on the issue:

https://github.com/apache/tinkerpop/pull/457
  
Sorry, I’m fluent in git (shame). From reading, I gleaned that rebase is 
a type of merge from one branch to another. Since I created the pull request on 
tp31 I don’t understand what it means to rebase it on tp31. I’m happy to 
follow directions if you could give me more explicit steps.

Thanks,
-Paul


From: stephen mallette [mailto:notificati...@github.com]
Sent: Saturday, November 19, 2016 7:52 AM
To: apache/tinkerpop
Cc: Paul A. Jackson; Mention
Subject: Re: [apache/tinkerpop] TINKERPOP-1493 Groovy project doesn't build 
on Windows (#457)


Looks like #498 has merged. 
@pauljackson if you could kindly rebase this on 
tp31 for me I can give it a final test.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on 
GitHub, or 
mute the 
thread.






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1560) Cache in GroovyClassLoader may continue to grow

2016-11-23 Thread stephen mallette (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689827#comment-15689827
 ] 

stephen mallette commented on TINKERPOP-1560:
-

cool - i didn't want to be the one to have to fix that one :)

> Cache in GroovyClassLoader may continue to grow
> ---
>
> Key: TINKERPOP-1560
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1560
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: groovy
>Affects Versions: 3.2.3
>Reporter: stephen mallette
>Assignee: stephen mallette
>
> The {{GroovyClassLoader}} has a {{classCache}} that seems to continue to grow 
> for each non-parameterized script that is passed through the 
> {{GremlinGroovyScriptEngine}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1560) Cache in GroovyClassLoader may continue to grow

2016-11-23 Thread Robert Dale (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689809#comment-15689809
 ] 

Robert Dale commented on TINKERPOP-1560:


Looks like it's fixed in the next version of groovy - 2.4.8

https://issues.apache.org/jira/browse/GROOVY-7683

> Cache in GroovyClassLoader may continue to grow
> ---
>
> Key: TINKERPOP-1560
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1560
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: groovy
>Affects Versions: 3.2.3
>Reporter: stephen mallette
>Assignee: stephen mallette
>
> The {{GroovyClassLoader}} has a {{classCache}} that seems to continue to grow 
> for each non-parameterized script that is passed through the 
> {{GremlinGroovyScriptEngine}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #499: TINKERPOP-1520: Difference between 'has' step generate...

2016-11-23 Thread okram 
Github user okram commented on the issue:

https://github.com/apache/tinkerpop/pull/499
  
Integration tests ran against `master/` (the new target branch).

```
[INFO] 

[INFO] BUILD SUCCESS
[INFO] 

[INFO] Total time: 04:14 h
[INFO] Finished at: 2016-11-22T23:02:41-07:00
[INFO] Final Memory: 179M/1656M
[INFO] 

```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop issue #503: Fixed and simplified 'language not supported' formatti...

2016-11-23 Thread spmallette 
Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/503
  
We maintain three separate release branches - the 3.1.x line of code 
(currently 3.1.6) that is on tp31. tp32 is 3.2.x (currently 3.2.4) and master 
is 3.3.x (currently looking at its first release for 3.3.0). We commit changes 
to the appropriate release branches then merge tp31 -> tp32 -> master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (TINKERPOP-1520) Difference between 'has' step generated graphson2.0 in java and python glv implementation

2016-11-23 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689762#comment-15689762
 ] 

ASF GitHub Bot commented on TINKERPOP-1520:
---

Github user okram commented on the issue:

https://github.com/apache/tinkerpop/pull/499
  
Integration tests ran against `master/` (the new target branch).

```
[INFO] 

[INFO] BUILD SUCCESS
[INFO] 

[INFO] Total time: 04:14 h
[INFO] Finished at: 2016-11-22T23:02:41-07:00
[INFO] Final Memory: 179M/1656M
[INFO] 

```


> Difference between 'has' step generated graphson2.0 in java and python glv 
> implementation
> -
>
> Key: TINKERPOP-1520
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1520
> Project: TinkerPop
>  Issue Type: Bug
>  Components: language-variant
>Affects Versions: 3.2.3
>Reporter: Andy Tolbert
>
> Noticed that between the java and python implementations, the graphson2.0 
> payload generated for a {{has}} step is different.  i.e. for the given 
> traversal:
> {{g.E().has("weight", 0.2)}}
> The java implementation produces the following graphson:
> {code:javascript}
> {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",{"@type":"g:P","@value":{"predicate":"eq","value":{"@type":"g:Double","@value":0.2}}}]]}}
> {code}
> where the python implementation produces the following:
> {code:javascript}
>  {"@type":"g:Bytecode","@value":{"step":[["E"],["has","weight",0.2]]}}
> {code}
> In the java case, a {{g\:P}} typed (predicate) value is provided, where in 
> the python case that isn't the case.
> I'm assuming the java one is correct (primarily since the graph backend seems 
> to like it and return the expected result).   Should GLV implementations 
> behave this way?  I noticed that {{GraphTraversal#has(String propertyKey, 
> Object value)}} in the java TinkerPop api wraps the value in a predicate 
> ({{P.eq}}) under the covers 
> ([link|https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/dsl/graph/GraphTraversal.java#L922])
>  so maybe implementors will need to do the same ([python 
> link|https://github.com/apache/tinkerpop/blob/master/gremlin-python/src/main/jython/gremlin_python/process/graph_traversal.py#L193])?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop issue #501: Added TP2-to-TP3 GraphML XSLT to resources

2016-11-23 Thread spmallette 
Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/501
  
Anyone who has an account can create issues. In fact, I just checked and I 
think you should even have right such that you can assign them to yourself (i 
see your account there as "joshsh"). Can you please check again?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---