[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431385#comment-15431385 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user asfgit closed the pull request at: https://github.com/apache/tinkerpop/pull/386 > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431171#comment-15431171
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/386#discussion_r75714486
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/util/DefaultTraversalMetrics.java
---
@@ -54,6 +54,14 @@
public DefaultTraversalMetrics() {
}
+// This is only a convenient constructor needed for GraphSON
deserialization.
+// TODO: see if that's ok to add that.
+public DefaultTraversalMetrics(long totalStepDurationNs,
List metricsMap) {
--- End diff --
I noticed that - I'll be making some tweaks once we get it over to master.
I'll take a look at those odds and ends.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.2
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431110#comment-15431110
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/386#discussion_r75709109
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/util/MutableMetrics.java
---
@@ -42,6 +42,18 @@ public MutableMetrics(final String id, final String
name) {
this.name = name;
}
+// create a MutableMetrics from an Immutable one.
+// needed that for tests, don't know if it is worth keeping it public.
+// TODO: see if it's ok to add this
+public MutableMetrics(Metrics other) {
--- End diff --
Same here I added a convenience constructor that takes any kind of Metrics
and returns a `MutableMetrics`. It was convenient for tests, as I wanted to
serialize Metrics that I am sure have nested Metrics, and make sure the ser/de
works for those.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.2
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431108#comment-15431108
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/386#discussion_r75708714
--- Diff:
gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/util/DefaultTraversalMetrics.java
---
@@ -54,6 +54,14 @@
public DefaultTraversalMetrics() {
}
+// This is only a convenient constructor needed for GraphSON
deserialization.
+// TODO: see if that's ok to add that.
+public DefaultTraversalMetrics(long totalStepDurationNs,
List metricsMap) {
--- End diff --
Not sure who I should ask about that, I decided to add a public constructor
here for convenience during the deserialization, it takes the total duration,
and a list of all the Metrics its composed of.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.2
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431067#comment-15431067 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user PommeVerte commented on the issue: https://github.com/apache/tinkerpop/pull/386 Ok cool if we're all ok with the new features then I'm satisfied. Code looked good as well VOTE: +1 > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431041#comment-15431041 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user okram commented on the issue: https://github.com/apache/tinkerpop/pull/386 @newkek I understand. I like the `@value` concept -- its the JSON sub-object for creating the object (defined by `@type`). VOTE +1. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431027#comment-15431027 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user PommeVerte commented on the issue: https://github.com/apache/tinkerpop/pull/386 I'm waiting to hear back from marko on the points he brought up to cast my vote :) > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431020#comment-15431020 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/386 VOTE +1 > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429414#comment-15429414 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/386 All tests pass with `docker/build.sh -t -n -i` - still going through code though. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429368#comment-15429368 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/386 I just pushed corrected docs, and renamed everything "domain" to "namespace". Waiting for more consensus to change the default gremlin namespace. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429356#comment-15429356 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user PommeVerte commented on the issue: https://github.com/apache/tinkerpop/pull/386 Still reviewing I would like to finish going through the code :) will try and do this later in the day. But looking good so far > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429323#comment-15429323 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/386 is that an official +1 from you @PommeVerte or are you still reviewing ? > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429264#comment-15429264 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user PommeVerte commented on the issue: https://github.com/apache/tinkerpop/pull/386 I'm liking how this has turned out. I personally don't have much of an opinion in regards to assuming gremlin is the default namespace and getting rid of `g:` though it does have the merit of being self documenting. I don't know how much of a size difference this would represent. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429055#comment-15429055 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/386 I think I agree with @newkek to push that off until we can get his massive PR reviewed/merged. Let's see if we can solve that separately. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428974#comment-15428974 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/386 A remark: @okram requested `TraversalExplanation` to be included in the list of Graph objects. I've noticed that the current `TraversalExplanation` Graphson serializer serializes a `Traversal`. In order to follow the requirement of this GraphSON2.0 type system, we would have to write a deserializer for the `TraversalExplanation` (there isn't currently). However, the current 1.0 serializer serializes the Traversal as the `toString()` of the Traversal's step list, which means that it is not possible to deserialize to a Traversal with only such info. So I believe it would be more appropriate to wait for TP1278 before implementing the `TraversalExplanation` ser/de and leave it out of the scope of this PR. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428875#comment-15428875 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/386 *NB: the conflicts for merge are caused by the CHANGELOG* > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428820#comment-15428820
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on the issue:
https://github.com/apache/tinkerpop/pull/386
Hey @okram ,
> 1. The term is "namespace" not "domain." In case that terminology is used
in the documentation and not just in the PR notes.
Ok, it's some places here and there in the code, I'll correct it.
> 2. Why do we have `@class` and `@type`.
`@class` shouldn't be anywhere near GraphSON 2.0 anymore, everything is
`@type`, if you've seen it somewhere, that should be corrected. While writing
that I realise I might have forgotten to update the "the-graph.asciidoc" doc...
so I'll correct it. But in the actual code it shouldn't be there anymore.
> 3. I read your notes, but its still not clear to me. Why do we have
`@value`? I think we [...]
So, taking a step back and I'm going to use the exampleof ByteCode. As you
can see in the `GraphSONSerializersV2d0`, serializers don't have to care
anymore about types, serializer are put into the context of "You're in a place
where you can write whatever you want, type is handled somewhere else for you".
For the ByteCode serializer it means it would open a Map, put a field
"language" and its value, put a field "argument" and its value, put a field
"value" (or call it "bytecode") and its value, close the map. When
deserializing, read the map, get the fields values, create the ByteCode object.
Let's admit for whatever reason the ByteCode serializer in the future needs
to change and instead of writing a Map, it needs to start by an Array where the
first element of the array is 'something', the rest is the Map described
before, and close the array. Doing this is currently possible with the current
format `{"@type":"typename", "@value":value}`, because the serializer is in a
place where it can write whatever it wants to.
Now let's consider another format: `{"@type":"typename", value}`. There,
the serializer is put in the context "You are currently in a **Map**, whatever
you write, it must be in the form of a key/value pair". The "potential future"
evolution of the ByteCode serializer (writing an array) *wouldn't* be possible.
For the sake of consistency, I think it is important to keep the format as is
> 4. I think we can save some bytes and still be readable if we make our
namespace g
Totally fine with `g`, again for the sake of consistency, I think it'd be
valuable to have a 'name space' all the time.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.2
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428652#comment-15428652 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user okram commented on the issue: https://github.com/apache/tinkerpop/pull/386 First off. Thank you for doing this. Here are some notes. 1. The term is "namespace" not "domain." In case that terminology is used in the documentation and not just in the PR notes. 2. Why do we have `@class` and `@type`. Seems we should just pick one. And if the answer is cause `@class` is for Java object mapping infrastructure, that isn't good and we will also use GraphSON to create `Vertex` objects in Python and it might use UUID for its ID as well. Thus, `@class=UUID` is perhaps `@type=UUID`. That is, if an object is "typed" its `@type` and the resolver (for that language) will have to map the correct class. 3. I read your notes, but its still not clear to me. Why do we have `@value`? I think we only need `@type` (or `@class`) and then the rest of the JSON object data is up to the type serializer/deserializer. For instance, in `Bytecode` (TINKEROP-1278) we have `@type=Lambda` and then `value`, `language`, `arguments`. That is, there are 3 "values" we need to make the object and thus `@value` shouldn't be special with a `@`. Plus, we save a char. 4. I think we can save some bytes and still be readable if we make our namespace `g` instead of `gremlin`. Moreover, we should be consistent in our naming convention and either use some standard JSON naming or default to Java naming -- e.g. `g:vertexProperty`. Or better yet, if its NOT namespaced, its assumed to be "Gremlin". That is, TinkerPop owns `vertex`, `vertexProperty`, `edge`. You want to do something else -- its `myapp:vertex`. This also means we own `uuid`, `int64`, etc. etc. Thanks again for the extensive work. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428466#comment-15428466 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek closed the pull request at: https://github.com/apache/tinkerpop/pull/351 > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428465#comment-15428465 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/351 Closing this in favor of #386. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.2 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428462#comment-15428462
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
GitHub user newkek opened a pull request:
https://github.com/apache/tinkerpop/pull/386
TINKERPOP-1274: GraphSON 2.0 [revised]
For context, a precise description is provided in the PR for the first
version of the fix: #351. Please see this first for initial context.
This PR provides initial set of features defined in #351, plus the
following:
- Types for Graph objects.
- Types for **all** numeric values.
- New type IDs format.
- Avoid serializing empty properties field.
As a reminder the format for types is the following:
- A value not typed : `value`
- A value typed : `{"@type":"typeName", "@value":value}`
## New type IDs format
A type ID is now composed of 2 parts, the "domain" and the type name. A
"domain" can be used by any implementor to implement their own data type,
avoiding collisions with the existing TinkerPop type IDs. The default domain
for Graph object is "gremlin".
## Types for all numeric values
A type is now present for every numeric value, the types have been renamed
to be more understandable with regards to their memory sizes or kinds. As a
reference, here is a description of all currently existing types and their
corresponding Java implementation:
- Java `Integer`: "gremlin:int32"
- Java `Long`: "gremlin:int64"
- Java `Short`: "gremlin:int16"
- Java `Float`: "gremlin:float"
- Java `Double`: "gremlin:double"
## Types for all Graph objects.
New typeIDs introduced in #351 (time types, UUIDs, etc..) now follow the
type format defined here: "domain:typename".
Types have now been included for Graph-specific objects, here is an
exhaustive list of the existing types handled so far and their corresponding
IDs:
- `Vertex` -> "gremlin:vertex"
- `Edge` -> "gremlin:edge"
- `VertexPropery` -> "gremlin:vertexproperty"
- `Property` -> "gremlin:property"
- `Path` -> "gremlin:path"
- `Tree` -> "gremlin:tree"
- `Graph` -> "gremlin:graph"
- `Metrics` -> "gremlin:metrics"
- `TraversalMetrics` -> `gremlin:traversalmetrics"
This improvement defines a requirement to the serialization format which is
that every type *must* have a Jackson serializer and deserializer defined on
the source `ObjectMapper`. Previous not defined serializers and deserializers
have been added in this PR.
Code-wise, it's pretty much the same than for #351, the big intake in code
here is the addition of the deserializers for all Graph objects, a big
simplification to the serializers (`GraphSONSerializersV2d0`), the addition of
the "domain" to the type system, and making that new typeID format configurable
to users through the `TinkerPopJacksonModule`.
`mvn clean install` test suite passes, and it's rebased on top of current
`master`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/newkek/incubator-tinkerpop TINKERPOP-1274-rev
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tinkerpop/pull/386.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #386
commit 635be59cf6505a47a1ca1d27519d07e918fdc1ef
Author: Stephen Mallette
Date: 2016-05-18T12:41:26Z
TINKERPOP-1274: GraphSON 2.0.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.2
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372842#comment-15372842 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/351 While we have 3 +1s, I don't want to make a GraphSON 2.0 that immediately needs to be revised to a 3.0. I don't see clear consensus on what we do with types/schema and I don't think we should hold up release any further, so I think GraphSON 2.0 should be delayed for a future version when we can get everything clear. I think the core of the work done here so far however is good. I've merged this PR to here: https://github.com/apache/tinkerpop/tree/graphson-2.0 for more collaboration and discussion. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372807#comment-15372807 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user okram commented on the issue: https://github.com/apache/tinkerpop/pull/351 Adding my email on dev@ to here. I’m not following this PR too closely so what I might be saying is a already known/argued against/etc. 1. I think we should go with @robertdale proposal of int32, int64, Vertex, uuid, etc. instead of Java class names. 2. In Java we then have a `Map` for typecasting accordingly. 3. This would make GraphSON 2.0 perfect for Bytecode serialization in TINKERPOP-1278. 4. I think that if a Vertex, Edge, etc. doesn’t have properties, outV, etc. then don’t even have those fields in the representation. 5. Most of the serialization back and forth will be `ReferenceXXX` elements and thus, don’t create more Maps/lists for no reason. — less chars. For me, my interests with this work is all about a language agnostic way of sending Gremlin traversal bytecode between different languages. This work is exactly what I am looking for. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372071#comment-15372071 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user PommeVerte commented on the issue: https://github.com/apache/tinkerpop/pull/351 ok I caught up and checked code. I think what @robertdale suggests on the typing end would be really nice (`int64`, `int32`, etc). But we can keep that for the thread stephen linked to. I VOTE +1 on this. @spmallette if you could just add a mention in the documentation that the basic types are based off of the JVM types that would be a nice bonus to have. Nice work @newkek, some nice effort went into this. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369151#comment-15369151 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/351 Note that I've started a new thread on the dev mailing list related to a new IO format with GLVs and Gremlin Server i mind: https://lists.apache.org/thread.html/3d7bee51fcbf63051e687a3ab82075763154902e0baf4b1c75cb672e@%3Cdev.tinkerpop.apache.org%3E If any of you have thoughts on the matter, please feel free to join the discussion there. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369143#comment-15369143
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on the issue:
https://github.com/apache/tinkerpop/pull/351
I think we should try to merge this PR at this point (though we need one
more +1 - @PommeVerte do you have a moment to do a final review?). There are a
few little tweaks here and there, but I can quickly do those after we merge.
I configured Gremlin Server to use 2.0 and it works nicely with REST:
```text
$ curl "http://localhost:8182?gremlin=g.V(1)"
{"requestId":"7b99b15e-5d07-4cb3-b9bd-cc62d03b99ca","status":{"message":"","code":200,"attributes":{}},"result":{"data":[{"id":{"@type":"Long","@value":1},"label":"person","type":"vertex","properties":{"name":[{"id":{"@type":"Long","@value":0},"value":"marko"}],"age":[{"id":{"@type":"Long","@value":2},"value":29}]}}],"meta":{}}}
```
I expect to do more tests around the server this week.
All tests pass with `docker/build.sh -t -n -i`
VOTE +1
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15367627#comment-15367627
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on the issue:
https://github.com/apache/tinkerpop/pull/351
@robertdale the format you suggest would lead to the same inconsistencies
as in GraphSON 1.0. Since the type is at the same level than the data itself,
whether the container is an Array or an Object, the type format would not be
the same. I just pushed a change in the format that is the one @PommeVerte
suggested, which gives a consistent format, without the concern of unordered
Lists (for reference the new format is `{"@type" : "typeName", "@value" :
value}`.
> please correct me if I'm wrong, but it doesn't look like the code does
any dynamic serializing.
The `TypeIdResolver` which is the object that the serializers will call to
get a TypeID from a java `Object` is dynamic in a way, in the sense that it
returns `o.getClass().getSimpleName()`. So there is no `object` -> typeID index
reference. However for the Deser, as explained in the description, Java by
default doesn't offer a way to get a Class by its simple name, so the
`TypeIdResolver` needs to keep a reference index of typeID(which is a class's
simple name) -> Java `Class`. Don't know if that answers your question..
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366498#comment-15366498
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user robertdale commented on the issue:
https://github.com/apache/tinkerpop/pull/351
So I've caught up on the discussion and I'll offer some more food for
thought since I haven't seen any other ideas. Embedding metadata is neither
easy nor fun (not for me anyway). For any serious integration type work it's
always best to have a well-defined schema up-front.
On types:
> @spmallette
> In fact we don't always know the types ahead of time (like Titan's
GeoPoint), so using the java class name is pretty convenient
Convenience is not the same as using Java types. By "not using java types",
we mean:
- not using java package names
- not using types specific to Java
- using primitives and other common types that are concise and portable
- should include domain-specific types. e.g. Vertex, Edge, etc.
- may include other standards. e.g. GeoJSON
Defining primitives, common types:
- http://swagger.io/specification/#dataTypeFormat
- http://bsonspec.org/spec.html
- http://geojson.org/geojson-spec.html
- http://ubjson.org/type-reference/
So if your Java implementation conveniently shares the same name as the
type, then that's wonderful. But if you are to be truly language-agnostic, then
at some point the types must be known ahead of time in order to be consumed.
For instance, how can my X parser know how to handle a Titan GeoPoint if it's
all dynamic? It can't. It must be able to handle this type ahead of time. And
I can't imagine someone would want to manually read a graphson file to discover
all the types that must be handled. Maybe I'm getting out of scope as this goes
beyond language and steps into being database agnostic. @newkek, please correct
me if I'm wrong, but it doesn't look like the code does any dynamic
serializing. It looks like all types are registered anyway. So I'll argue again
if you know your types ahead of time, then you may as well have a schema.
But let's continue with embedded metadata...
In JSON, the only unambiguous types are
- array (unless you want to disambiguate from list which may be very valid)
- string
- boolean (true, false)
- null
To avoid confusion on all other types, including numbers, they should be
typed. Thus they are objects (and not lists of things). The metadata can be at
the same level as the object and alleviates these concerns: @newkek " a List in
which the first element is a Map in which the first entry's key" and
@PommeVerte "can be a pain in systems that do not necessarily order lists".
Metadata can be differentiated from member fields by a prefix (e.g. '@').
Primitive types (or objects) having only a single value would have a "value"
key which maps to the actual value.
```json
[
{
"@type":"Vertex",
"id":{
"@type":"int64",
"value":12345
},
"label":"person",
"properties":{
"@type":"VertexProperty",
"skill":{
"id":{ "@type":"int64",
"value":8723
},
"@type":"int32",
"value":5
},
"secrets":[
{ "id":{
"@type":"int64",
"value":8723
},
"@type":"uuid",
"value":"1de7bedf-f9ba-4e94-bde9-f1be28bef239"
},
{ "id":{
"@type":"int64",
"value":8724
},
"@type":"uuid",
"value":"34523adf-f9ba-4e94-bde9-f2345bcd3f45"
}
]
},
"inE":[
{ "@type":"Edge",
"label":"knows",
"id":{
"@type":"int64",
"value":987234
},
"properties":{ },
"outV":[ { } ]
}
]
}
]
```
I wouldn't concern myself with the additional payload size for metadata. I
wouldn't sacrifice conciseness for size. One could always compress the file if
size is a concern. Also, the reader/writer could be easily enhanced to support
zip. I would take the pragmatic approach and address it when it's no longer
working for people.
Anyway, maybe this is all GraphSON 3.0 stuffs. HTH.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
>
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362664#comment-15362664 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/351 This change is largely to help the consumption of GraphSON by non-JVM languages. Since I'm largely familiar with the behaviors of java based parsers and such, i'm not a good judge of that so i have to rely on @PommeVerte and others in this area. If the switch to Map is helpful then I think we should go that route. @newkek I would not be hasty in the change though. Perhaps we give it a day or so to think about to be sure that everyone is happy with that approach before you make the change and commit to it. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362655#comment-15362655
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on the issue:
https://github.com/apache/tinkerpop/pull/351
Well, I agree we cannot prevent some parsers and drivers to read the whole
JSON content and check the created object only afterward. I would not recommend
it though, because it is not great for memory consumption, but it is maybe
something we have the opportunity to avoid only thanks to Jackson and some
libraries may not offer that capability. So I guess I'll switch to the
`{"@type":"...", "value": ...}` format instead of the current one. Does that
sound good @spmallette @PommeVerte ?
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362585#comment-15362585 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user PommeVerte commented on the issue: https://github.com/apache/tinkerpop/pull/351 Ok I thought The main point here was robustness in typing for non-java languages. Hence why I suggested even typing things like Int and using java classes. Honestly depending on how you compiled your PHP your basic JSON int will be converted to `Long`. And that's the case I was highlighting. It makes sense to ignore this if you're trying to reduce de payload but we'll still be lacking on the non-lossiness end. >As I see it for how I implemented the TypeDeserializer, it acts as a meta deserializer that will read the raw text JSON sequentially, so there's no chance there can be a mixup in the order You can't guaranty that the order would be maintained in some languages hence my previous comment. These languages will parse the JSON into `List` then check the list and Map along the lines of what I said in my previous post. It would be much more efficient to simply have JSON cast to `Map` and run a simple check on keys. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15362336#comment-15362336 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/351 Just pushed the change for `@class` to `@type`. All tests pass. 'twas a 1 line fix, isn't that wonderful. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15361303#comment-15361303 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user robertdale commented on the issue: https://github.com/apache/tinkerpop/pull/351 @newkek @spmallette Sorry, my context was only this thread. I agree with you on all accounts. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15361197#comment-15361197 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/351 Just for reference: https://groups.google.com/d/msg/gremlin-users/R7qQbJXWcH8/wK9ZuWADAwAJ > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15360521#comment-15360521 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/351 Yes the `@class` bothered me a little as well, since now it doesn't really describe a "Class". But I did not want to change, for consistency for GraphSON v1.0 and I reused the `GraphSONTokens.CLASS`. So if it is ok, I'd be +1 for `@type` actually. Concerning BSON (or MsgPack) it probably is indeed a more efficient serialization solution, however the goal of this PR is to optimize and improve JSON. Implementing Graph-BSON is probably another topic adjacent to this one, on which btw I'd be happy to collaborate as well. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15360211#comment-15360211 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/351 > 1. A type is not a class. Call types 'type' or '@type' (not clear the @ is necessary since it's in the metadata payload anyway); Not @class. (Otherwise be consistent and rename all the Type* classes to be Class*. e.g. ClassDeserializer) Changing "@class" to "@type" is ok by me if others like it. I'm not tied to one or the other - "@type" does seem a little better to me the more i think on it, but I'll let @newkek (or others) weigh in on it. > 2. Don't use Java types. Use BSON as a reference. It has a nice type system and solved most if not all of the type concerns. http://bsonspec.org/spec.html Not sure about BSON typing as a solution. Ultimately we want to know if something is a `Vertex`, `Edge`, `Duration`, `GeoPoint`, etc. In fact we don't always know the types ahead of time (like Titan's `GeoPoint`), so using the java class name is pretty convenient. 3. If space and processing efficiency is a priority, then consider actually using BSON. imo, we're pretty deep into this approach having discussed it over multiple weeks in the community. making a big switch like that is probably something to reserve for the future especially since @newkek put a fair bit of effort into this work at this point and it delivers what took a while to get agreement on. If however BSON (or some other format) could be proven a more efficient network serialization format that is truly programming language agnostic, with wide support and consistently performant parsers in the major languages we support (which is what had doomed MsgPack some time back), then I think we could consider that as an additional IO format. @robertdale if you have ideas there, it would be nice to hear them. Please consider sending a message on the dev mailing list if you do. 4. Alternatively, use an external schema to define types. It could even be appended to and a part of the output. That would be an interesting option, however would mostly be good for network serialization and not so much for file storage. So far we haven't written a network only IO package, though we have written a file storage only one with GraphML. I think that we could consider a network serialization one only since dependence on Gremlin Server for non-JVM languages is going to be something we need to support in the face of GLVs. Thanks for your thoughts @robertdale > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15360153#comment-15360153 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user robertdale commented on the issue: https://github.com/apache/tinkerpop/pull/351 Some suggestions: 1. A type is not a class. Call types 'type' or '@type' (not clear the @ is necessary since it's in the metadata payload anyway); Not @class. (Otherwise be consistent and rename all the Type* classes to be Class*. e.g. ClassDeserializer) 2. Don't use Java types. Use BSON as a reference. It has a nice type system and solved most if not all of the type concerns. http://bsonspec.org/spec.html 3. If space and processing efficiency is a priority, then consider actually using BSON. 4. Alternatively, use an external schema to define types. It could even be appended to and a part of the output. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15360094#comment-15360094 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user dkuppitz commented on the issue: https://github.com/apache/tinkerpop/pull/351 I hope you guys figured it all out yesterday.I didn't follow all the comments, but started a `docker/build.sh -t -i -n` over-night job. It succeeded, thus VOTE: +1 > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359621#comment-15359621 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/351 imo, I think that if we document GraphSON this way, there should not be too much confusion as to how the type system works. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359614#comment-15359614
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on the issue:
https://github.com/apache/tinkerpop/pull/351
@spmallette yes, maybe some code will help explain.
```
ObjectMapper mapper =
GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create().createMapper();
Map map = new HashMap<>();
map.put("helo", 2);
String s = mapper.writeValueAsString(map);
// prints 's = {"helo":2}'
System.out.println("s = " + s);
Map read = mapper.readValue(s, Map.class);
// prints 'read.get("helo") = class java.lang.Integer'
System.out.println("read.get(\"helo\") = " +
read.get("helo").getClass());
Map map3 = new HashMap<>();
map3.put("helo", 2L);
String s2 = mapper.writeValueAsString(map3);
// prints 's2 = {"helo":[{"@class":"Long"},2]}'
System.out.println("s2 = " + s2);
Map read2 = mapper.readValue(s2, Map.class);
// prints 'read2.get("helo") = class java.lang.Long'
System.out.println("read2.get(\"helo\") = " +
read2.get("helo").getClass());
```
> number with decimal and no type = Java Float
Almost, by default decimals are Double, not Floats.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359580#comment-15359580 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/351 If i understand right: Serializing integer without a type tag would result in integer. You would have to specify "Long" as the type for it to be interpreted as a Long in java. This is generally just a problem for numerics so the conversion is: * number with no decimal and no type = Java integer * number with decimal and no type = Java Float * all other numerics will have a java type present (simple name) @newkek is that an appropriate description of what's happening in conversion? > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359364#comment-15359364
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on the issue:
https://github.com/apache/tinkerpop/pull/351
> I thought that java integer would be implied.
Yes with the current PR serializing an Integer will result in no type
added. Serializing a Long or a Short will result in an explicit type added. To
be very explicit, in JSON everything is Doubles and Longs, it does that because
it's the bigger container format that contains the others less precise ones.
Jackson however, considers that when an Integer is to be serialized, there is
no need for an explicit type because the precision will be kept since for JSON
it's in a Long. When it comes to serializing a Long, still no precision loss
but the format explicitly indicates to the Deserializer that what has been
serialized initially was a Long. So everything is as defined as if the Integer
was typed, because by default the reader assumes everything's an Integer, and
if not there will be a type to specify what it is. However we have save a large
payload size by not typing the integer. Same concept for Float VS Double, for
JSON everything is a Double, if a value is a Float, it will be typed, if not,
by default it's a Float.
So as I said in the description I think the outcome of the mail
conversation was to not type Simple values. Mostly because we're not sure if
this is going to be useful or not. However, adding those types in the future,
for Simple values, can be very easily done and I've left detailed comments in
the `TypeSerializer` on how to add them when we deem it necessary. Also, adding
them would not break existing code since the format would be the same, it's
just that _every_ value would follow the format for typed values. Since there's
no possibility to mix-up for the Numeric values as explained above with the
Shorts and Longs and etc... I still think we should wait somebody explicitly
requires it.
> Perhaps {"@class":"java.lang.Integer", "value": 1} is a better option.
As I see it for how I implemented the TypeDeserializer, it acts as a meta
deserializer that will read the raw text JSON sequentially, so there's no
chance there can be a mixup in the order, it does not deserialize the whole
structure and creates a `List>` object. Same for the TypeSerializer, it
does not create a Java `List` object to write the type, but it writes directly
in JSON "write a start_array token, write a start_map token, write a property
name, write a text field (the type name), write a end_array token, etc" so same
thing, there is no chance there can be a mix up in the order.
I don't know what it could look like for parsers of other languages, but it
would seem like doing something else than that would be quite inefficient in
terms of performance because it would that for every typed value you would
instantiate a `new List>` just to read a simple value. Does that makes
sense ?
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359292#comment-15359292
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on the issue:
https://github.com/apache/tinkerpop/pull/351
I thought that java integer would be implied. I believe that if we wrote a
java short for example it would type to a "Short" then you would know what kind
of number it is.
We purposely went away from the fully qualified class name which really
doesn't mean much to non-JVM languages in favor of the more brief and less
scary simple name - I thought we had decided that a smaller byte size for the
network outweighed the downside of being less specific about the type. no?
> I would also like to point out that this format
[{"@class":"java.lang.Integer"}, 1] can be a pain in systems that do not
necessarily order lists.
I seem to remember that we discussed that before but don't remember the
outcome - @newkek do you remember what was said?
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359265#comment-15359265
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69327810
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
So to sum up on that samples issue :
- the grateful-dead V1 samples have changed because for some reason, some
of the `inE` of some vertices were not written in the same order. I'm almost
sure the fix here has nothing to do with that order change, so I pushed the
changed ones. It also doesn't concern the `normalize` option of the GraphSON
mapper, since the `inE` are generally not ordered. So, quite a mystery but I
definitely don't that's something introduced by this branch.
- The `data/` folder now has the new `-v2d0` and `-v2d0-typed` graphs, but
not the `normalized` ones.
- The `gremlin-test/src/main/ressources/etc...` has all the new v2 graphs
and the normalized ones.
- I don't know what's up with the `sample.kryo` but there's near to
0.1% chances it's related to that branch. But I pushed the change though.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359255#comment-15359255
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user PommeVerte commented on the issue:
https://github.com/apache/tinkerpop/pull/351
Hey guys,
I've been super busy lately but I definitely plan on diving deep into this
PR over the weekend. One quick remark though.
1. Even the JSON supported types are not prone to lossiness in multi
language settings. They should also be typed.
2. In a multi language setting, having type names without their java
classes is not helpful.
I can illustrate both of these points with the following JSON : `{"id":1}`
The client assumes `id` is `int` but what exactly is an int? is it `16bit`,
`32bit`, or `64bit`? Languages will have their own definition here. Actually
some languages will even have different values of `int` depending on how they
were compiled.
Changing it to `{"id":[{"@class":"Integer"}, 1]}` is not helpful in this
case either. However the following is explicit and is something you can work
with: `{"id":[{"@class":"java.lang.Integer"}, 1]}`. It's immediately
identifiable and well documented. The client knows this is a 32bit Int and can
work accordingly. Without this you would have to go through documentation or
code to figure out what you were dealing with.
In conclusion:
1. Thinking about it some more it's possible that Integer is the only
special case that would need typing in the json supported types. I'll give it
some more though. We could possibly have a "verbose" option for those who
require typing of everything.
2. Type names should refer to the java class. This also seems to make sense
when dealing with custom objects.
PS: I would also like to point out that this format
`[{"@class":"java.lang.Integer"}, 1]` can be a pain in systems that do not
necessarily order lists. With these systems you need to check that your list
has two elements, that one is a map, and that the map contains a `@class` key.
Costly operation.
Perhaps `{"@class":"java.lang.Integer", "value": 1}` is a better option.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359121#comment-15359121
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69315539
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
If you re-run the tests after you generate and they all pass, i think we
should probably be safe.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359120#comment-15359120
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69314893
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
I'll try to find the differences for the grateful-dead in json but I'm not
sure I'd be able to tell concerning the diff for the `sample.kryo` as I'm not
fluent in bianry (yet) :)
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359056#comment-15359056
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69308786
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
yes - you should push the new ones. not sure what changed on the existing
ones. can you tell what the changes are? if it is just the order of keys
changing or the order of vertices or something like that then i wouldn't bother
to push it, but if it's something else i would wonder if there is something
wrong somewhere because we should be 100% backward compatible with GraphSON
1.0..nothing should have changed in that sense.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358854#comment-15358854
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69288856
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
So, since now the build creates the new v2d0 graphs, and it seems like the
other ones are pushed in the repo, should I push the new ones too? Also, it
seems that the already pushed ones have modifications. Should I push all of as
well ? Here's the diff :
https://gist.github.com/newkek/4e0488268f9bb78e0d3597821ae6b357
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358847#comment-15358847
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69288319
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
Apparently the TestHelper behaviour has changed in commit da2eb7e, but the
pom wasn't changed with regards to that change.
Ok I'll change the pom as explained earlier
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358840#comment-15358840
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69287612
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
hmm - strange that never was a problem before. something must have changed
(we haven't run with -Dio in a long long time) feel free to fix it in this
branch. thanks
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358790#comment-15358790
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69283289
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
This is also broken on master
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358784#comment-15358784
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69283048
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
Ah, looking at the pom, I'm getting this warning during the build :
```
[INFO] --- maven-resources-plugin:2.6:copy-resources
(copy-graphson-from-tmp-to-resources) @ tinkergraph-gremlin ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/Users/kevingallardo/Documents/workspace/newkek-incubator-tinkerpop/tinkergraph-gremlin/target/tinkerpop-io
```
Seems like the pom is going to search for the ressources in
`${project.build.directory}/tinkerpop-io` and
`${project.build.directory}` is defined as
`${basedir}/target`.
But the `tempPath` in IoDataGenerationTest` is `tempPath =
TestHelper.makeTestDataPath(TinkerGraphTest.class, "tinkerpop-io").getPath() +
File.separator;`
If I change the pom to search in the right directory :
`${project.build.directory}/test-case-data/TinkerGraphTest/tinkerpop-io`,
it works.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358755#comment-15358755
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69279953
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
When you run with `-Dio` it's suppose to copy the files to all the right
places. See the tinkergraph-gremlin pom.xmlnot sure why that's not
happening.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358731#comment-15358731
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69276332
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
Oh I think I see what you mean, I don't see them in the root's `data/` dir,
I don't know why
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358728#comment-15358728
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69276176
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
https://gist.github.com/newkek/643c907e54e93ab36594295281c3e4e6
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358725#comment-15358725
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69275891
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
Where are they supposed to be located ? I see them in the
`tinkergraph-gremlin/target/test-case-data/TinkerGraphTest/tinkerpop-io/`.
Should they be somewhere else?
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15358697#comment-15358697
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r69272477
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
did you run
```text
cd tinkergraph-gremlin
mvn clean install -Dio
```
I don't see the newly generated data files in the various data directories.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357513#comment-15357513 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/351 yeah - we tend to bump heads on master a little bit on the CHANGELOG - not a big deal - i can resolve that conflict when the time comes. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357479#comment-15357479 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/351 Re-conflicts with Changelog against master.. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356912#comment-15356912 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/351 @spmallette - I've written all the docs, please don't hesitate to correct them if they're not written well. - Updated the tests, parameterized them as much as possible, and focused the 2.0 tests on the 2.0 specific functionalities. - Rebased on current `master`. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356821#comment-15356821 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/351 Yes that was introduced by the ce19704 (reference docs and Changelog) > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355374#comment-15355374
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r68961204
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
yes - that's correct - thanks. you wrote it more clearly than i did :)
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355331#comment-15355331
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r68956721
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
So, write new tests in IoDataGenerationTest class that will generate V2.0
versions of the classic/modern/etc.. graphs in json serialized format (with the
files named `[graph-name]-v2d0.json`).
And put the Ser/deser test of the grateful-dead graph in another class.
Correct ?
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355300#comment-15355300
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r68951691
--- Diff:
gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/GraphSONMessageSerializerV2d0Test.java
---
@@ -0,0 +1,474 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.driver.ser;
+
+import org.apache.tinkerpop.gremlin.driver.message.RequestMessage;
+import org.apache.tinkerpop.gremlin.driver.message.ResponseMessage;
+import org.apache.tinkerpop.gremlin.driver.message.ResponseStatusCode;
+import
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+import org.apache.tinkerpop.gremlin.process.traversal.step.util.Tree;
+import org.apache.tinkerpop.gremlin.structure.*;
+import org.apache.tinkerpop.gremlin.structure.io.AbstractIoRegistry;
+import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONIo;
+import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTokens;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerFactory;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph;
+import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
+import org.apache.tinkerpop.shaded.jackson.core.JsonGenerationException;
+import org.apache.tinkerpop.shaded.jackson.core.JsonGenerator;
+import org.apache.tinkerpop.shaded.jackson.databind.JsonNode;
+import org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper;
+import org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider;
+import org.apache.tinkerpop.shaded.jackson.databind.module.SimpleModule;
+import org.apache.tinkerpop.shaded.jackson.databind.node.NullNode;
+import org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer;
+import org.apache.tinkerpop.shaded.jackson.databind.util.StdDateFormat;
+import org.junit.Test;
+
+import java.awt.*;
+import java.io.IOException;
+import java.util.*;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.core.Is.is;
+import static org.junit.Assert.*;
+
+/**
+ * These tests focus on message serialization and not "result"
serialization as test specific to results (e.g.
+ * vertices, edges, annotated values, etc.) are handled in the IO packages.
+ *
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public class GraphSONMessageSerializerV2d0Test {
--- End diff --
ok - fair enough. if they aren't the same then no worries - we just have
more tests to maintain. thanks
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355296#comment-15355296
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r68951292
--- Diff:
gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/GraphSONMessageSerializerV2d0Test.java
---
@@ -0,0 +1,474 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.driver.ser;
+
+import org.apache.tinkerpop.gremlin.driver.message.RequestMessage;
+import org.apache.tinkerpop.gremlin.driver.message.ResponseMessage;
+import org.apache.tinkerpop.gremlin.driver.message.ResponseStatusCode;
+import
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+import org.apache.tinkerpop.gremlin.process.traversal.step.util.Tree;
+import org.apache.tinkerpop.gremlin.structure.*;
+import org.apache.tinkerpop.gremlin.structure.io.AbstractIoRegistry;
+import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONIo;
+import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTokens;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerFactory;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph;
+import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
+import org.apache.tinkerpop.shaded.jackson.core.JsonGenerationException;
+import org.apache.tinkerpop.shaded.jackson.core.JsonGenerator;
+import org.apache.tinkerpop.shaded.jackson.databind.JsonNode;
+import org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper;
+import org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider;
+import org.apache.tinkerpop.shaded.jackson.databind.module.SimpleModule;
+import org.apache.tinkerpop.shaded.jackson.databind.node.NullNode;
+import org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer;
+import org.apache.tinkerpop.shaded.jackson.databind.util.StdDateFormat;
+import org.junit.Test;
+
+import java.awt.*;
+import java.io.IOException;
+import java.util.*;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.core.Is.is;
+import static org.junit.Assert.*;
+
+/**
+ * These tests focus on message serialization and not "result"
serialization as test specific to results (e.g.
+ * vertices, edges, annotated values, etc.) are handled in the IO packages.
+ *
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public class GraphSONMessageSerializerV2d0Test {
--- End diff --
I could change some of the tests to take into account the version otherwise
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355295#comment-15355295
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user newkek commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r68951122
--- Diff:
gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/GraphSONMessageSerializerV2d0Test.java
---
@@ -0,0 +1,474 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.driver.ser;
+
+import org.apache.tinkerpop.gremlin.driver.message.RequestMessage;
+import org.apache.tinkerpop.gremlin.driver.message.ResponseMessage;
+import org.apache.tinkerpop.gremlin.driver.message.ResponseStatusCode;
+import
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+import org.apache.tinkerpop.gremlin.process.traversal.step.util.Tree;
+import org.apache.tinkerpop.gremlin.structure.*;
+import org.apache.tinkerpop.gremlin.structure.io.AbstractIoRegistry;
+import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONIo;
+import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTokens;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerFactory;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph;
+import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
+import org.apache.tinkerpop.shaded.jackson.core.JsonGenerationException;
+import org.apache.tinkerpop.shaded.jackson.core.JsonGenerator;
+import org.apache.tinkerpop.shaded.jackson.databind.JsonNode;
+import org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper;
+import org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider;
+import org.apache.tinkerpop.shaded.jackson.databind.module.SimpleModule;
+import org.apache.tinkerpop.shaded.jackson.databind.node.NullNode;
+import org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer;
+import org.apache.tinkerpop.shaded.jackson.databind.util.StdDateFormat;
+import org.junit.Test;
+
+import java.awt.*;
+import java.io.IOException;
+import java.util.*;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.core.Is.is;
+import static org.junit.Assert.*;
+
+/**
+ * These tests focus on message serialization and not "result"
serialization as test specific to results (e.g.
+ * vertices, edges, annotated values, etc.) are handled in the IO packages.
+ *
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public class GraphSONMessageSerializerV2d0Test {
--- End diff --
They are not all exactly the same, because some of these inspect the
generated JSON payload manually (`shouldSerializeEdge()`) and since the format
is different, and that types are introduced now, they don't get the expected
property at the right place. [see
here](https://github.com/apache/tinkerpop/pull/351/files#diff-c199205c0fbd8f1e71e0681fdd786555R230)
(l.230) since the payload contains the type, we have to go in the Array's
second element to get the value.
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355252#comment-15355252 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user newkek commented on the issue: https://github.com/apache/tinkerpop/pull/351 > tests are failing Ah I changed something quickly this morning and did not run again those tests. Will correct that. Ok for the docs, will do. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355240#comment-15355240 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on the issue: https://github.com/apache/tinkerpop/pull/351 Note that travis isn't happy - tests are failing: ```text Failed tests: GraphSONMessageSerializerV2d0Test.shouldSerializeToJsonIteratorNullElement:147 expected:<[x]> but was:<[]> GraphSONMessageSerializerV2d0Test.shouldSerializeToJsonMap:171 expected:<[x]> but was:<[]> GraphSONMessageSerializerV2d0Test.shouldSerializeToJsonIterable:107 expected:<[x]> but was:<[]> GraphSONMessageSerializerV2d0Test.shouldSerializeToJsonIterator:126 expected:<[x]> but was:<[]> ``` Also - some documentation odds and ends: * Please update CHANGELOG with an entry for "Introduced GraphSON 2.0". * We probably need o include some reference documentation [here](https://github.com/apache/tinkerpop/blob/5395aaeb057e4c163233aa8ace2a2975ea827fe5/docs/src/reference/the-graph.asciidoc#graphson-readerwriter) - not sure what that should look like at the moment. We default to 1.0 still for now, so we probably don't need to change too much. Perhaps we need a section that shows what 2.0 format looks like and how to generate it? Maybe that is enough? * We need upgrade docs for users so they are aware of this new feature - [here](https://github.com/apache/tinkerpop/blob/5395aaeb057e4c163233aa8ace2a2975ea827fe5/docs/src/upgrade/release-3.2.x-incubating.asciidoc#upgrading-for-users) * We need upgrade docs for driver devs (need to add a section - no changes in 3.2.1 have affected drivers yet) so they are aware of how they might benefit from this change - [here](https://github.com/apache/tinkerpop/blob/5395aaeb057e4c163233aa8ace2a2975ea827fe5/docs/src/upgrade/release-3.2.x-incubating.asciidoc#upgrading-for-providers) > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355180#comment-15355180
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r68940083
--- Diff:
tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/structure/IoDataGenerationTest.java
---
@@ -275,4 +290,78 @@ else if (e.label().equals("writtenBy"))
GraphSONWriter.build().mapper(GraphSONMapper.build().embedTypes(true).create()).create().writeGraph(os4,
g);
os4.close();
}
+
+@Test
+public void shouldWriteGratefulDeadGraphSONV2d0() throws IOException {
+final TinkerGraph g = TinkerGraph.open();
+final TinkerGraph readG = TinkerGraph.open();
+
+final GraphReader reader = GryoReader.build().create();
+try (final InputStream stream =
AbstractGremlinTest.class.getResourceAsStream("/org/apache/tinkerpop/gremlin/structure/io/gryo/grateful-dead.kryo"))
{
+reader.readGraph(stream, g);
+}
+final OutputStream os2 = new FileOutputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONWriter.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().writeGraph(os2,
g);
+os2.close();
+
+final InputStream is = new FileInputStream(tempPath +
"grateful-dead-V2d0-typed.json");
+
GraphSONReader.build().mapper(GraphSONMapper.build().version(GraphSONVersion.V2_0).typeInfo(GraphSONMapper.TypeInfo.PARTIAL_TYPES).create()).create().readGraph(is,
readG);
+is.close();
+
+assertEquals(approximateGraphsCheck(g, readG), true);
+}
+
+/**
+ * Checks sequentially vertices and egdes of both graphs. Will check
sequentially Vertex IDs, Vertex Properties IDs
+ * and values and classes. Then same for edges. To use when
serializing a Graph and deserializing the supposedly
+ * same Graph.
+ */
+private boolean approximateGraphsCheck(Graph g1, Graph g2) {
--- End diff --
These are good tests, but they don't belong in `IoDataGenerationTest`. I
think you could move them somewhere else. The point of this test is to generate
the sample data that we ship with our distributions and use in tests. You
should include "tests" that generate 2.0 versions of grateful-dead, classic,
modern and crew graphs. Then, to generate the data, just run:
```text
cd tinkergraph-gremlin
mvn clean install -Dio
```
You should see the new files in the appropriate places. I guess for now
they should be named with a suffix like:
```text
grateful-dead-v2d0.json
```
Let's leave the 1.0 naming as-is for now so as not to break anything
existing that relies on the file names being what they are. I think we had also
said that we would keep 1.0 as the default for now and then look to 3.3.x to
make 2.0 the default - that would produce the least amount of breaking change,
so i guess that approach would be in line with that thinking. Does that make
sense?
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355134#comment-15355134
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
Github user spmallette commented on a diff in the pull request:
https://github.com/apache/tinkerpop/pull/351#discussion_r68936696
--- Diff:
gremlin-driver/src/test/java/org/apache/tinkerpop/gremlin/driver/ser/GraphSONMessageSerializerV2d0Test.java
---
@@ -0,0 +1,474 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.gremlin.driver.ser;
+
+import org.apache.tinkerpop.gremlin.driver.message.RequestMessage;
+import org.apache.tinkerpop.gremlin.driver.message.ResponseMessage;
+import org.apache.tinkerpop.gremlin.driver.message.ResponseStatusCode;
+import
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+import org.apache.tinkerpop.gremlin.process.traversal.step.util.Tree;
+import org.apache.tinkerpop.gremlin.structure.*;
+import org.apache.tinkerpop.gremlin.structure.io.AbstractIoRegistry;
+import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONIo;
+import org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTokens;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerFactory;
+import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph;
+import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
+import org.apache.tinkerpop.shaded.jackson.core.JsonGenerationException;
+import org.apache.tinkerpop.shaded.jackson.core.JsonGenerator;
+import org.apache.tinkerpop.shaded.jackson.databind.JsonNode;
+import org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper;
+import org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider;
+import org.apache.tinkerpop.shaded.jackson.databind.module.SimpleModule;
+import org.apache.tinkerpop.shaded.jackson.databind.node.NullNode;
+import org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer;
+import org.apache.tinkerpop.shaded.jackson.databind.util.StdDateFormat;
+import org.junit.Test;
+
+import java.awt.*;
+import java.io.IOException;
+import java.util.*;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.core.Is.is;
+import static org.junit.Assert.*;
+
+/**
+ * These tests focus on message serialization and not "result"
serialization as test specific to results (e.g.
+ * vertices, edges, annotated values, etc.) are handled in the IO packages.
+ *
+ * @author Stephen Mallette (http://stephen.genoprime.com)
+ */
+public class GraphSONMessageSerializerV2d0Test {
--- End diff --
Are the tests like this one exact copies of the v1 versions? Is it possible
to use a:
```java
@RunWith(Parameterized.class)
```
here and parameterize on the serializer version?
> GraphSON Version 2.0
>
>
> Key: TINKERPOP-1274
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1274
> Project: TinkerPop
> Issue Type: Improvement
> Components: io
>Affects Versions: 3.1.2-incubating
>Reporter: stephen mallette
>Priority: Minor
> Fix For: 3.2.1
>
>
> Develop a revised version of GraphSON that provides better support for
> non-JVM languages that consume it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355112#comment-15355112 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on a diff in the pull request: https://github.com/apache/tinkerpop/pull/351#discussion_r68934451 --- Diff: gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONMapper.java --- @@ -60,47 +61,78 @@ private final boolean normalize; private final boolean embedTypes; private final GraphSONVersion version; +private final TypeInfo typeInfo; private GraphSONMapper(final List customModules, final boolean loadCustomSerializers, --- End diff -- Could you please make the constructor match this approach: https://github.com/apache/tinkerpop/blob/5395aaeb057e4c163233aa8ace2a2975ea827fe5/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoMapper.java#L165-L172 rather than pass the individual parameters we just pass the `Bulider` object. Not sure why this wasn't changed already for `GraphSONMapper`. > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[ https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355071#comment-15355071 ] ASF GitHub Bot commented on TINKERPOP-1274: --- Github user spmallette commented on a diff in the pull request: https://github.com/apache/tinkerpop/pull/351#discussion_r68931920 --- Diff: gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/graphson/GraphSONIo.java --- @@ -22,11 +22,7 @@ import org.apache.tinkerpop.gremlin.structure.io.Io; import org.apache.tinkerpop.gremlin.structure.io.IoRegistry; -import java.io.FileInputStream; -import java.io.FileOutputStream; -import java.io.IOException; -import java.io.InputStream; -import java.io.OutputStream; +import java.io.*; --- End diff -- sorry @newkek but your IDE introduced wildcards to our imports which isn't our code style. could you please fix those? > GraphSON Version 2.0 > > > Key: TINKERPOP-1274 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1274 > Project: TinkerPop > Issue Type: Improvement > Components: io >Affects Versions: 3.1.2-incubating >Reporter: stephen mallette >Priority: Minor > Fix For: 3.2.1 > > > Develop a revised version of GraphSON that provides better support for > non-JVM languages that consume it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TINKERPOP-1274) GraphSON Version 2.0
[
https://issues.apache.org/jira/browse/TINKERPOP-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355048#comment-15355048
]
ASF GitHub Bot commented on TINKERPOP-1274:
---
GitHub user newkek opened a pull request:
https://github.com/apache/tinkerpop/pull/351
TINKERPOP-1274: GraphSON 2.0.
https://issues.apache.org/jira/browse/TINKERPOP-1274
# Summary of the changes :
Implementation of a format for value types serialization which is uniform
and not Java centric. As a reminder the new format is as follows :
- A value not typed : `value`
- A value typed : `[{"@class":"typeName"}, value]`
The content of `value` can be either a simple value, or a more complex
structure. The type prefix allows to call the right _Deserializer_ to
deserialize the `value` whatever its content is.
The default's GraphSON's 2.0 format will include types for every `value`
that is not a JSON native type
(`String/int/double/null/boolean/Map/Collection`) - called `PARTIAL_TYPES`.
This allows to significantly reduce the size of the JSON payload where those
aren't needed by the Jackson library.
GraphSON serialization _without_ types is not affected. To enable it, need
to use `NO_TYPES`.
## Quick walkthrough for the review
There are new components involved in the ser/de process that are extended
from the Jackson library :
- _TypeIdResolver_ : performs the conversion of a object's `Class` ->
`typeID`, and from a `typeID` -> `Class`.
- _TypeResolverBuilder_ : creates a _TypeIdResolver_ and instantiates a
_TypeDeserializer_ or _TypeSerializer_ with the _TypeIdResolver_ as a param.
- _TypeSerializer_ : writes the prefix and suffix for a typeID. The
_TypeSerializer_ is provided to the _Serializers_ and handles the type
serialization that respects the format put in place.
- _TypeDeserializer_ : is called before the deserializers are called. Its
role is to detect a type, and call the right _Deserializer_ to deserialize the
value.
Most of the serializers already existing for Graph had to be modified,
because most of them were *manually hardcoding* types without calling the
_TypeSerializer_ and hence, those were not respecting the format. Now all
Serializers respect the format because they call the _TypeSerializer_ given in
parameter. I followed the frame put in place by @spmallette to implement those
new serializers (prefixed with `V2d0`) without breaking existing clients code.
In _TypeDeserializer_, `baseType` represents the class given in parameter
by the user for the ser/de. If the user calls
`mapper.readValueAsString(jsonString, UUID.class)` the `baseType` in
_TypeDeserializer_ will be a _JavaType_ (which is the Jackson's custom class
for Java classes) that represents a _UUID_ class. The *wildcard* in our
deserialization mechanism is `Object.class`.
The deserialization path is as follow :
- _TypeDeserializer_ is called only for non simple JSON values (non
`String/int/double/null/boolean`).
- When called, if a type is detected, the _TypeDeserializer_ will read the
typeID, convert it to a JavaType (thanks to the _TypeIdResolver_), check that
the `baseType` is the same than what was read in the payload (only if
`baseType` is not the *wildcard*), and call the `deserialize()` method of the
_JsonDeserializer_ registered for that type.
- If a type is not detected, detect Maps or Arrays and call appropriate
Deserializer.
## Some results
GraphSON 2.0 shows a significant reduction of the payload's size for typed
serialization. And the consequence in performance is that since there's less to
process, the ser/de is faster. Results show a reduction of at least 50% in the
payload's size and evolving linearly (the bigger the payload the bigger the
difference) :
```
➜ ls -lh
tinkergraph-gremlin/target/test-case-data/TinkerGraphTest/tinkerpop-io/
-rw-r--r-- 1 kevingallardo staff 890K 28 Jun 21:50
grateful-dead-V2d0-typed.json
-rw-r--r-- 1 kevingallardo staff 1.9M 28 Jun 21:50
grateful-dead-typed.json
-rw-r--r-- 1 kevingallardo staff 851K 28 Jun 21:50 grateful-dead.json
-rw-r--r-- 1 kevingallardo staff 1.5K 28 Jun 21:50
tinkerpop-classic-V2d0-typed.json
-rw-r--r-- 1 kevingallardo staff 3.6K 28 Jun 21:50
tinkerpop-classic-typed.json
-rw-r--r-- 1 kevingallardo staff 1.3K 28 Jun 21:50
tinkerpop-classic.json
```
## Tests
Tests cover the same functionalities covered by GraphSON 1.0 typed, plus
additional features brought by GraphSON 2.0.
## Tradeoffs
- Some tricks were implemented in order to provide the types without
packages names. Since Java does not provide a way to search a class by its
simple name, the `TypeIdResolver` has to have an index that it
