Thanks for looking into my suggested enhancements. I agree that it is not easy to see which Interfaces are really only renamed instances of the original API and which Interfaces are new or add features, so I will move the renamed Interfaces to another package. In the enhanced API there is indeed one thing (Element is a better word than NodeLike, so I'll change that). Elements can be connected to other Elements, there is no distinction on that level, though we do want to know what the type is of the Elements a relationship is referring to when we retrieve it. Each of the different Element types, has its own interface, extending Element. So there are interfaces defined for Node, Relationship, RelationshipType, Property and PropertType. Each of these Elements have the exact same function as their counterparts in the original API, with additional features that make them more alike. In fact the enhanced API has the exact same semantics as the original API, except for the enhanced features. As an example, createRelationshipTo, which is available on all Elements, doesn't accept just a Node as its first argument, but accepts any Element. This works well as a drop-in replacement, because every existing call of createRelationshipTo is called with a Node, which is an Element. When a relationship is created between two Nodes, the implementation simply calls the original createRelationshipTo method. So there is no semantic difference for that case. Only when a relationship is created between two elements of which at least one is not a Node, do the semantics change: ie. for the non-Node elements, the associated node is looked-up and the relationship is created on that associated node. The same applies for the setting of properties. In case of Node and Relationship, direct calls are made to setProperty, getProperty and hasProperty. Only in other cases is the semantics different, because then a look-up is done on an associated Node and the properties are set or retrieved on that associated Node. The layout of the graph doesn't change with the enhanced API, either. All additional nodes are disconnected from the main graph, so they don't influence existing traversals. Of course users can create relationships from the main graph to the associated nodes, but that is a deliberate decision. So all Elements keep their existing semantics, though they all now share a common interface, allowing hooks into every aspect of the graph. The enhanced API is nothing more or less than a reification facility for all elements already existing in neo4j (every element can be viewed as if a Node). See it as if it adds hooks for reflection (where fields, methods, classes etc., can be reified into Objects) and allows for the storage of annotations. The next step is to add features to Relationship, such that it keeps working the way it has always done for binary relationships, but adds the necessary features for n-ary relationships. (I already know how I want to do that). Niels
> From: [email protected] > Date: Sat, 23 Jul 2011 22:19:55 +0200 > To: [email protected] > Subject: Re: [Neo4j] Hyperedges and Objects > > Ok, makes sense. > > It just hard to see what is new in that package and what is just "hiding" the > original API. > > Probably I don't know enough about type/class system to understand the > implication of having properties and relationships on top of property-names, > properties, relationships, relationship-types and nodes. > > Wouldn't it be much simpler to say, there is just one thing (Element) and > this Element can be connected to other things ? > > So make no distinction between connections to other "nodes" or connection to > properties ? > > then you wouldn't need all the existing API at all. > I would also leave off all the management stuff here (.e.g shutdown > add/removeXXXHandler, etc). > > For interaction with the old API I would have a methods on Element (or a > Factory / Converter) > > Element.from(Node) > Element.from(Relationship) > Element.from(RelationshipType) > Element.from(PropertyContainer, propertyName) > Element.from(PropertyContainer, propertyName, type) > > But these interconnected elements are imho a really different approach to the > current API, so mixing things just makes it confusing. Especially keeping the > old names around when the things are actually unified carries too much > "wrong" semantics. > > Michael > > Am 23.07.2011 um 22:02 schrieb Niels Hoogeveen: > > > > > I had to wrap many of the classes, because of the return types of several > > methods. org.neo4j.collections.graphdb.Node does not extend > > org.neo4j.graphdb.Node, even though all methods of org.neo4j.graphdb.Node > > are also defined in org.neo4j.collections.graphdb.Node. > > I realize this makes the design more brittle (then again, major API changes > > to Neo4j core need to be reflected anyway), but it does allow the API to be > > a drop-in replacement of the old API. > > I like your suggestions, and would like to incorporate those to make the > > API more fluent. > > My first goal was to create a drop-in replacement API with extended > > features. > > PropertyType was the result of work I did on IndexedRelationships, where I > > wanted a simple interface for the indexing of relationships based on a > > property on the end node of the relationship. For that I needed a > > "comparator" which compared a property value to some supplied value. To > > hide the details of such "comparators", I created PropertyType. > > One of the goals of the enhanced API is to allow every item in the database > > (node, relationship, relationshiptype, relationship, property name, > > property value) to be treated as if it were like a Node. So every item > > implements the NodeLike interface, allowing for the maintenance of > > properties and relationships on those items. > > This serves several purposes. First of all for the types: RelationshipType > > and PropertyType, we now have the hooks for users to implement a type/class > > system on top of Neo4J. It also allows for relationships on relationships > > (needed to create n-ary relationships), and allows for relationships and > > properties on property values, without having to resort to a bipartite > > graph layout. > > Like I stated in an earlier message, the code is a first edit and still > > needs to be thoroughly tested and documented. I posted it on Github for two > > reasons: safety and to gather feedback about the design. > > Niels > > > >> From: [email protected] > >> Date: Sat, 23 Jul 2011 21:26:53 +0200 > >> To: [email protected] > >> Subject: Re: [Neo4j] Hyperedges and Objects > >> > >> I don't understand why you wrapped _all_ the classes, this will make this > >> package very brittle as it has to copy all API changes? > >> > >> I rather thought of a better / fluent / more usable API on top of the > >> existing one. > >> > >> something like > >> > >> new Graph(gbd).autoIndex("name") > >> .node("name","Niels") > >> > >> .relateTo("works_on","repo","graph_collections","language","Java").index("repo","language") > >> .node("name","Someone", > >> "age",20,"gender",Gender.MALE,"birthday",sdf.parse("01.01.1970")); > >> > >> int node.getProperty("age", int.class); > >> > >> etc. > >> > >> Just convenience for users. > >> > >> And I don't really understand what property-type is for. I thought you > >> were only working on n-ary relationships? > >> > >> Btw. you can replace those unholy if clauses with polymorphism in the enum. > >> > >> Are there some tests and/or examples that uses this API ? > >> > >> Cheers > >> > >> Michael > >> > >> Am 23.07.2011 um 21:02 schrieb Niels Hoogeveen: > >> > >>> > >>> Traverser and Index are wrapped too now, so the entire API as can be > >>> accessed from GraphDatabaseService is now available in enhanced API > >>> version. > >>> The API is a drop-in replacement, all that needs to be done to make an > >>> application work against the enhanced API, is to change imports. > >>> > >>> From: [email protected] > >>> To: [email protected] > >>> Subject: RE: [Neo4j] Hyperedges and Objects > >>> Date: Fri, 22 Jul 2011 04:00:18 +0200 > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> I uploaded the first edit (still needs to completely be tested - > >>> IS-PRE-ALPHA quality) of the enhanced API. > >>> > >>> See: > >>> https://github.com/peterneubauer/graph-collections/tree/master/src/main/java/org/neo4j/collections/graphdb > >>> > >>> The enhanced API has several new features: > >>> > >>> > >>> 1.) A new interface RelationshipContainer, supporting createRelation, > >>> getRelation, and hasRelation methods. > >>> > >>> > >>> 2.) Relationship implements RelationshipContainer so relationships to and > >>> from relationships can be created. > >>> A relationship property is maintained containing the node ID of an > >>> associated node. This node is lazily created (and with that the > >>> relationship property is set). > >>> Relationships are stored on the associated node, while properties are > >>> stored on the relationship itself. > >>> > >>> > >>> 3.) EnhancedRelationshipType implements RelationshipContainer (and with > >>> that PropertyContainer) so RelationshipTypes can have properties and > >>> relationships. > >>> > >>> A node associated with a RelationshipType is created lazily. From the > >>> reference node a RELTYPE_SUBREF relation points to a node, containing > >>> properties corresponding to RelationshipType names. The value of this > >>> property is the node ID of the RelationshipType associated node. > >>> > >>> All relationships creation to and from a EnhancedRelationshipType and all > >>> properties setting on a EnhancedRelationshipType are performed on the > >>> associated node. > >>> All retrievals are done from the associated node. > >>> > >>> The associated node is created the first time it is needed. > >>> > >>> 4.) PropertyType implements RelationshipContainer (and with that > >>> PropertyContainer). PropertyTypes can be used in getProperty, setProperty > >>> and hasProperty style operations. > >>> > >>> PropertyTypes are parameterized with the datatype of the property. This > >>> makes it possible to support methods such as: <T> > >>> getPropertyValue(PropertyType) > >>> and setProperty(PropertyType<T>, T) that guard property values added > >>> conform to the datatype presumed by the PropertyType and property values > >>> are returned typed. > >>> > >>> A node associated with a PropertyType is created lazily. From the > >>> reference node a PROPTYPE_SUBREF relation points to a node, which points > >>> to nodes per datatype. > >>> These datatype related nodes contain properties corresponding to > >>> PropertyType names. > >>> > >>> All relationships creation to and from a PropertyType and all properties > >>> setting on a PropertyType are performed on the associated node. All > >>> retrievals are done from the associated node. > >>> > >>> The associated node is created the first time it is needed. > >>> > >>> 5.) GraphDatabaseService allows for the lookup of PropertyTypes and > >>> RelationshipTypes and enhanced Nodes and Relationships. > >>> > >>> > >>> 6.) Property implements RelationshipContainer (and with that > >>> PropertyContainer). //TODO > >>> A node associated to a property is created lazily, allowing properties to > >>> have relationships and properties. > >>> The associated node is created the first time it is needed. > >>> > >>> With these enhancements, all elements of Neo4j (Node, Relationship, > >>> RelationshipType, property name and property value) are unified, sharing > >>> a common interface RelationshipContainer (and with that > >>> PropertyContainer). Node and Relationship also extend their original > >>> interfaces, so the API is a drop-in replacement for the original API. > >>> > >>> Both ProperyType and RelatioshipType now provide the necessary methods to > >>> implement a type/class system on top of Neo4j, but nothing imposes a > >>> type/class system. > >>> > >>> Being able to create Relationships to and from Relationships opens the > >>> way to a transparent implementation of n-ary relationships. > >>> > >>> > >>> This is all neat, I believe, but I would be really happy if there could > >>> be some support in core to hide the implementation a bit more. > >>> > >>> Now: > >>> > >>> 1.) Relationships can have a property containing the node ID of the > >>> associated node. This polutes the namespace, so it would be better if > >>> there were a setLong and getLong method on Relationship, where the node > >>> ID can be stored, keeping the property name away from the property > >>> namespace. > >>> > >>> 2.) Named persistent maps from String to PropertyContainer. This would > >>> eliminate the REL_SUBREF related node and the PROP_SUBREF related nodes. > >>> It would be nice if these enhancements didn't polute the node space. > >>> Named persistent maps from String to PropetyContainer would also be > >>> helpful for fast lookup of uris and generally for the creation of unique > >>> keys. > >>> > >>> Niels > >>>> Date: Thu, 21 Jul 2011 20:38:15 +0200 > >>>> From: [email protected] > >>>> To: [email protected] > >>>> Subject: Re: [Neo4j] Hyperedges and Objects > >>>> > >>>> Niels, > >>>> > >>>> sounds like fun. Looking forward to get a look at this > >>>> > >>>> 2011/7/21 Niels Hoogeveen <[email protected]> > >>>> > >>>>> > >>>>> i made a start on this. It's not all too difficult to enhance > >>>>> relationships > >>>>> such that relationships can be created upon them, which is a first step > >>>>> towards supporting hypergraphs. In fact hypergraphs are more constrained > >>>>> than an enhanced graph that supports the creation of relationships on > >>>>> relationships. > >>>>> As it stands now, the enhanced graph can work as a drop-in replacement > >>>>> of > >>>>> the standard neo4j API and has enhanced methods that mirror the standard > >>>>> methods. > >>>>> I also added support for typed properties, such that methods like <T> > >>>>> getPropertyValue<PropertyType<T>> are possible where <T> is one of the > >>>>> data > >>>>> types supported by PropertyContainer. > >>>>> I will upload this within the next few days. > >>>>> Niels > >>>>> > >>>>> > >>>>> From: [email protected] > >>>>> To: [email protected] > >>>>> Subject: RE: [Neo4j] Hyperedges and Objects > >>>>> Date: Mon, 18 Jul 2011 17:59:08 +0200 > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> I think a hypergraph would be an interesting collection to support in > >>>>> neo-graph-collections. If I find the time, I will make a start with it > >>>>> this > >>>>> week, unless there are of course other volunteers :wink: willing to do > >>>>> the > >>>>> lifting here. > >>>>> Niels > >>>>> > >>>>>> From: [email protected] > >>>>>> Date: Sat, 16 Jul 2011 23:37:48 +0200 > >>>>>> To: [email protected] > >>>>>> Subject: Re: [Neo4j] Hyperedges and Objects > >>>>>> > >>>>>> I completely agree, > >>>>>> > >>>>>> hyperedges and the accompanying traversers should be handled in a > >>>>> library. As you probably know the traversal framework currently also > >>>>> uses > >>>>> the core API under the hood to perform the traversals (and no black > >>>>> magic > >>>>> (yet)). > >>>>>> > >>>>>> So it should be fairly easy to take that approach/code and create a > >>>>> library that abstracts the hyper-edge issues (creation, deletion, > >>>>> traversal). The position semantics based approach sounds interesting. > >>>>>> > >>>>>> Would love to see that as community contribution. > >>>>>> > >>>>>> Cheers > >>>>>> > >>>>>> Michael > >>>>>> > >>>>>> Am 16.07.2011 um 23:08 schrieb Niels Hoogeveen: > >>>>>> > >>>>>>> > >>>>>>> The question is how much easier a traverser can become when there were > >>>>> dedicated hyper edges. In a binary relation it is fairly easy to define > >>>>> one > >>>>> end of the relation as the source and the other as the target (start > >>>>> and end > >>>>> node), > >>>>>>> but in n-ary relationships the roles of the attached nodes become more > >>>>> complicated. > >>>>>>> > >>>>>>> Suppose we have the GIVES relationship, where one attached node takes > >>>>> the role of subject (the giver), > >>>>>>> one node takes the role of direct object (the gift), and another node > >>>>> takes the role of indirect object (the recipient). > >>>>>>> To traverse such a graph, we need to know these different roles, > >>>>> otherwise we may end up traversing the wrong nodes. > >>>>>>> > >>>>>>> Suppose we the following statements: > >>>>>>> > >>>>>>> John gives Paul a servant. > >>>>>>> Paul visited Albania. > >>>>>>> Paul's servant visited Albania. > >>>>>>> > >>>>>>> We now want to know all people that received a gift from John and who > >>>>> visited Albania. > >>>>>>> > >>>>>>> Without properly denoting the roles in the ternary relationship stated > >>>>> by "John gives Paul a servant", > >>>>>>> the answer to the query may well be: Paul and Paul's servant. > >>>>>>> Both are persons, both have visited Albania and both are part of the > >>>>> GIVES relationship defined. > >>>>>>> > >>>>>>> When we have to define the exact roles of each part of an n-ary > >>>>> relationship for each traversal, > >>>>>>> it is just as complicated as defining a traversal based on binary > >>>>> relationships, > >>>>>>> where different relationship types denote the roles of each part of > >>>>>>> the > >>>>> n-ary relationship. > >>>>>>> > >>>>>>> If hyperedges were to be introduced, either as a library or in core, > >>>>>>> the entire notion of the graph and how traversals are performed need > >>>>>>> to > >>>>> be rethought. > >>>>>>> > >>>>>>> The concept of an edge as having a start and an end node doesn't > >>>>> translate well into the world of hyper edges. > >>>>>>> There is not necessarily a start node and an end node, > >>>>>>> instead there are various nodes that are distinctly attached to the > >>>>> hyper edge. > >>>>>>> > >>>>>>> One way to think about an edge in both the graph and in the hypergraph > >>>>> world is as a tuple. > >>>>>>> > >>>>>>> A binary relationship can be thought of as the tuple: > >>>>>>> > >>>>>>> (node1, node2, RelationshipType, Set(property)) > >>>>>>> > >>>>>>> A ternary relationship can be thought of as the tuple: > >>>>>>> > >>>>>>> (node1, node2, node3, RelationshipType, Set(property)) > >>>>>>> > >>>>>>> etc... > >>>>>>> > >>>>>>> Instead of marking a binary relationship as outgoing or incoming, we > >>>>> can use the position in the tuple to denote its role. > >>>>>>> We can say the first node in the tuple corresponds to the start node, > >>>>> and the second node in the tuple corresponds to the end node. > >>>>>>> Having two possible permutations relates to the two possible > >>>>>>> directions > >>>>> an edge can have. > >>>>>>> > >>>>>>> Position based definitions of relationships translate well into the > >>>>> domain of n-ary relationships, > >>>>>>> though the semantics of such relationships can easily become > >>>>>>> difficult. > >>>>>>> A ternary relationship already has 6 permutations for the attached > >>>>> nodes, > >>>>>>> while a 10-ary relationship has 3,628,800 possible permutations. > >>>>>>> > >>>>>>> It would be an interesting project to design a tuple-position-based > >>>>> traverser. > >>>>>>> For binary relationships it should have the exact same features as the > >>>>> current direction based traverser, > >>>>>>> but it would be possible to generalize that design to n-ary > >>>>> relationships. > >>>>>>> This can all be done as a libary. > >>>>>>> > >>>>>>> Only when perfomance can be really improved in core, does it make > >>>>>>> sense > >>>>> to request for native support. > >>>>>>> > >>>>>>> Niels > >>>>>>>> Date: Sat, 16 Jul 2011 19:31:22 +0200 > >>>>>>>> From: [email protected] > >>>>>>>> To: [email protected] > >>>>>>>> Subject: Re: [Neo4j] Hyperedges and Objects > >>>>>>>> > >>>>>>>> That's clear. Such mappings are more difficult to traverse since the > >>>>>>>> traverser has to know if there is such a hyper edge vertex. I'm > >>>>>>>> wondering if there is no need to provide an embedded solution for > >>>>>>>> such > >>>>> a > >>>>>>>> transformation. Each user who is confronted with hyper edges has to > >>>>>>>> implement some kind of such mapping. It would be helpful, for > >>>>>>>> example, > >>>>>>>> if each Neo4j relationship can manage hyper edges automatically and > >>>>>>>> provide an additional 'isHyper' method pointing out that there are > >>>>>>>> additional methods allowing to get all further incident nodes. This > >>>>>>>> is > >>>>>>>> also a matter of performance. I think such a handling is implemented > >>>>>>>> most efficiently within the Neo4j engine. > >>>>>>>> > >>>>>>>> Best regards > >>>>>>>> > >>>>>>>> Norbert Tausch > >>>>>>>> > >>>>>>>> Am 16.07.2011 00:52, schrieb Niels Hoogeveen: > >>>>>>>>> Hyper edges can be emulated. > >>>>>>>>> Suppose you want to store the fact "John gives Pete a book". > >>>>>>>>> This is indeed a ternary relationship, which would call for an hyper > >>>>> edge. > >>>>>>>>> This fact can be stored as follows: > >>>>>>>>> Some act of giving --Giver-->JohnSome act of giving --Recipient--> > >>>>> PeteSome act of giving --Gift--> book > >>>>>>>>> N-ary relationships can generally be decomposed into N binary > >>>>> relationships, where a central node is used to used as a placeholder > >>>>> for the > >>>>> N-ary relationship. > >>>>>>>>> Complex types can either be stored into a byte array or if > >>>>> appropriate in multiple properties. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> Date: Sat, 16 Jul 2011 00:04:15 +0200 > >>>>>>>>>> From: [email protected] > >>>>>>>>>> To: [email protected] > >>>>>>>>>> Subject: [Neo4j] Hyperedges and Objects > >>>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> > >>>>>>>>>> is there a possibility to store hyper edges in Neo4j? Is there a > >>>>> plan to > >>>>>>>>>> implement this? > >>>>>>>>>> > >>>>>>>>>> Will Neo4j get the ability to store complex types in addition to > >>>>>>>>>> primitve data types for properties of nodes and relationships? > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Best regards > >>>>>>>>>> > >>>>>>>>>> Norbert Tausch > >>>>>>>>>> > >>>>>>>>>> _______________________________________________ > >>>>>>>>>> Neo4j mailing list > >>>>>>>>>> [email protected] > >>>>>>>>>> https://lists.neo4j.org/mailman/listinfo/user > >>>>>>>>> > >>>>>>>>> _______________________________________________ > >>>>>>>>> Neo4j mailing list > >>>>>>>>> [email protected] > >>>>>>>>> https://lists.neo4j.org/mailman/listinfo/user > >>>>>>>> _______________________________________________ > >>>>>>>> Neo4j mailing list > >>>>>>>> [email protected] > >>>>>>>> https://lists.neo4j.org/mailman/listinfo/user > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> Neo4j mailing list > >>>>>>> [email protected] > >>>>>>> https://lists.neo4j.org/mailman/listinfo/user > >>>>>> > >>>>>> _______________________________________________ > >>>>>> Neo4j mailing list > >>>>>> [email protected] > >>>>>> https://lists.neo4j.org/mailman/listinfo/user > >>>>> > >>>>> _______________________________________________ > >>>>> Neo4j mailing list > >>>>> [email protected] > >>>>> https://lists.neo4j.org/mailman/listinfo/user > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Mattias Persson, [[email protected]] > >>>> Hacker, Neo Technology > >>>> www.neotechnology.com > >>>> _______________________________________________ > >>>> Neo4j mailing list > >>>> [email protected] > >>>> https://lists.neo4j.org/mailman/listinfo/user > >>> > >>> > >>> _______________________________________________ > >>> Neo4j mailing list > >>> [email protected] > >>> https://lists.neo4j.org/mailman/listinfo/user > >> > >> _______________________________________________ > >> Neo4j mailing list > >> [email protected] > >> https://lists.neo4j.org/mailman/listinfo/user > > > > > > _______________________________________________ > > Neo4j mailing list > > [email protected] > > https://lists.neo4j.org/mailman/listinfo/user > > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
