[
https://issues.apache.org/jira/browse/TINKERPOP-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795847#comment-15795847
]
ASF GitHub Bot commented on TINKERPOP-1130:
-------------------------------------------
Github user spmallette commented on the issue:
https://github.com/apache/tinkerpop/pull/520
> Does this verify that we don't accidentally create a breaking IO format
change in the same IO version?
Yes. We generate GraphSON and Gryo files that are statically checked into
source control on each release. Then we make sure that those files can still be
read with the current version. For example, we currently are verifying that
3.3.0-SNAPSHOT can read Gryo 1.0 generated in 3.3.0, 3.2.4 and 3.2.3.
In doing this, I've already encountered inconsistencies which are
documented in the `Model` class which defines all the incompatibilities. So, if
we do have a break (or something untestable), we can at least document it
there. For example, there were a number of bugs in Gryo 1.0 that generated
"bad" static files to test against. I made the fixes for 3.2.4, so we can
really only test against versions 3.2.4 and later:
https://github.com/apache/tinkerpop/blob/TINKERPOP-1130/gremlin-tools/gremlin-io-test/src/main/java/org/apache/tinkerpop/gremlin/structure/io/Model.java#L100
Note that the incompatibilities in the line referenced above also include
"untyped" GraphSON there as untyped GraphSON won't deserialize to specific
types either (typically just all coerces to `Map` and `List` without a way to
deserialize back to the original type).
Actually, we still have some todos on this PR because of the changes we
need to revert in the GraphSON 2.0 work you had done a while back on properties
(those changes will be for GraphSON 3.0). That was a breaking change and the
tests are failing so I think it's doing its job.
> Does this verify that GraphSON and Gryo (and anything else down the line)
have all the same serializers? (that is, are capable of handling all the same
data -- e.g. we won't miss that Pick.none was not registered in 3.2.1 for
GraphSON but it was for Gryo?)
Well - I suppose it helps with this issue, but it's not as though it is
actively monitoring our code to do that obviously. In other words, it will help
if by convention we do all serialization testing through `gremlin-io-test`. If
we do that, then we register a type with the `Model` and it will immediately
force testing of both Gryo and GraphSON. If you didn't implement a serializer
for one or the other then its going to fail. If you just decide to add a new
class registration to Gryo and don't bother to write a test for it in
`gremlin-io-test` then obviously it's not going to have any knowledge of the
new registration.
I think that does lead to some issues which I'd brought up on the dev
mailing list in the last couple of weeks. Right now there is some disparity
between the types supported by GraphSON and those supported by Gryo. Right now
Gryo has a lot more supported classes (many of which are not specifically
tested under `gremlin-io-test) - I only covered those classes supported by
GraphSON so far. I think that we will want Gryo and GraphSON to support the
same types (for 3.0 of both formats) which may mean shrinking the types that
Gryo supports (I would think we would want the smallest type library possible),
but I'm not sure how practical that is. A lot of the Gryo registrations were
added to support OLAP stuff and remote traversal stuff (before we had
bytecode). I don't know how easy it is to just drop some of that stuff or which
ones are even candidates to be dropped.
> Each release should store Kryo/GraphSON/GraphML versions to ensure future
> compatibility
> ---------------------------------------------------------------------------------------
>
> Key: TINKERPOP-1130
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1130
> Project: TinkerPop
> Issue Type: Improvement
> Components: io, test-suite
> Affects Versions: 3.1.1-incubating
> Reporter: Marko A. Rodriguez
> Labels: breaking
>
> I think we should make a new toy data set that has all the graph structure
> features in it -- vertices, edges, vertex properties, multi-properties,
> meta-properties, graph variables, different edge labels with different
> property keys, etc. etc.
> The graph doesn't have to be big, it just needs to cover all the features.
> Next, we should then stamp out a version of that file at every release:
> {code}
> graph-test-x.y.z.xml
> graph-test-x.y.z.kryo
> graph-test-x.y.z.json
> graph-test-x.y.z-typed.json
> {code}
> Then we should have a test case that verifies that the current SNAPSHOT
> {{GryoReader}}, {{GraphSONReader}}, {{GraphMLReader}}, etc. can still read
> those files. If they can't, then we have introduced a change in our
> serialization format.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
