Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Am Sonntag, 13. November 2005 12:07 schrieb Jean-Marc Orliaguet: > Helmut Merz wrote: > >Anyway, what we are talking about are not "references". > > The approach is quite different: references start from the > objects themselves that they connect to other objects using > one-way relations (a pointer, an arrow). And this can be easily implemented by using assignments to object attributes. > The application has > to know how to interpret the references. I don't think that > you can build a robust relation engine only with that. > Relations start from "the top": you first define an ontology > (a set of general predicates) that you use to relate the > objects of your application. This is a conceptual schema. Here > is the cpsskins ontology: > http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo >-perspectives/ontology.py > > The relation engine then manages all the necessary references, > but the application does not need to know about the references > at all. The interaction with the relation engine is done only > via the ontology. OK, I see. I now also understand why you include monadic relations which could just be attributes or annotations... (I ignored them up to now but I think I should change this.) OTOH I'd like to have a relation management API that could as well be used for cases where the relations are controlled by the application or the objects involved. This is then just a convenience API that cares about the problems of keeping up-to-date references in both directions, deletion handling, the complexity arising with connecting more than two objects, attributed or annotated relations, etc. Helmut ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Helmut Merz wrote: >Anyway, what we are talking about are not "references". > > The approach is quite different: references start from the objects themselves that they connect to other objects using one-way relations (a pointer, an arrow). The application has to know how to interpret the references. I don't think that you can build a robust relation engine only with that. Relations start from "the top": you first define an ontology (a set of general predicates) that you use to relate the objects of your application. This is a conceptual schema. Here is the cpsskins ontology: http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo-perspectives/ontology.py The relation engine then manages all the necessary references, but the application does not need to know about the references at all. The interaction with the relation engine is done only via the ontology. To compare with python: references are to relations what methods are to classes. A set of unrelated methods doesn't make a model, similarly a set of loose references doesn't make an ontology. /JM ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Am Samstag, 12. November 2005 23:13 schrieb Jean-Marc Orliaguet: > Helmut Merz wrote: > >Maybe there is a better word for this kind of 'relation'? > > > >Helmut > > I don't know. I think you have to start from a definition that > is not dependent on zope in general and understand what the > concepts mean outside the computer world. This was not > invented here. > > The "relations" used in software are more than just relations > since they also assert something, and a relation by itself > asserts nothing, it is just an abstraction that connect > several things or aspects of things. So "relations" here are > rather "propositions". > > for instance: > "A blue box" involves a relation between the "box" and the > quality "blue", but to represent the relation, you would to > assert a proposition: > > "The box is blue" Thanks for the clarification, and I think I got it - though I have to admit that I got the feeling that my brain is not really built to work with formal logic in a practical way ;-) So my suggestion would be to go the pragmatic way an just keep the word "relation" for representing the assertion of such propositions in software. I'll be going to write a Zope 3 proposal on on the relation management stuff (in the first step only on the interfaces trying not to talk about implementation issues) and would gladly expect your comments then... I think we cannot alway avoid words defined in another problem domain in a different way, which may be problem especially if the problem domains are related (like formal logic and computer science); but I'd care not to use a nomenclature that might confuse or mislead people knowing more about some stuff than me. Anyway, what we are talking about are not "references". > The proposition can be analysed. It has a predicate and a > subject (or subjects if taken individually), the subject is an > ordered tuple and the predicate is what is left of the > proposition when the subjects have been removed (it is what is > "predicated", or asserted of the subjects of the proposition) > > for instance in the proposition: > > "The box is blue" > > the predicate "__ is blue" predicates "blueness" of the box > > but a predicate is not the same thing as a relation Yup - I'm at the moment using the word "predicate" to signify a type of a relation (using it as a homonym to "relationship") though I'm again not sure if this might be plainly wrong or at least confusing. Helmut ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Helmut Merz wrote: > >OK - then I'm not talking about references, though we still mean >different things when talking about 'relation': > >Mine goes like this: A relation object R providing >ITriadicRelation and connecting A, B and C involves three >references: > >R with A >R with B >R with C > >In addition the relation object has a sort of type, the >'relationship' (corresponding to a 'predicate') that may be >specified by the relation's class or - a special case - a >string. > >Maybe there is a better word for this kind of 'relation'? > >Helmut > > I don't know. I think you have to start from a definition that is not dependent on zope in general and understand what the concepts mean outside the computer world. This was not invented here. The "relations" used in software are more than just relations since they also assert something, and a relation by itself asserts nothing, it is just an abstraction that connect several things or aspects of things. So "relations" here are rather "propositions". for instance: "A blue box" involves a relation between the "box" and the quality "blue", but to represent the relation, you would to assert a proposition: "The box is blue" The proposition can be analysed. It has a predicate and a subject (or subjects if taken individually), the subject is an ordered tuple and the predicate is what is left of the proposition when the subjects have been removed (it is what is "predicated", or asserted of the subjects of the proposition) for instance in the proposition: "The box is blue" the predicate "__ is blue" predicates "blueness" of the box but a predicate is not the same thing as a relation /JM ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Am Samstag, 12. November 2005 20:00 schrieb Jean-Marc Orliaguet: > Helmut Merz wrote: > >Am Samstag, 12. November 2005 18:00 schrieb Jean-Marc Orliaguet: > >>My > >>impression is that you are thinking of a reference engine > >>rather than a relation engine > > > >Maybe I just don't see the difference... (There is one, of > >course, but I doubt it is really of relevance in a practical > >context. ) > > > >Helmut > > A relation contains several references: > > a relation between A, B and C involves 7 references: > > A with itself > B with itself > C with itself > A with B > B with C > C with A > A with B with C > > But a set of references don't make explicit relations, unless > the application know how to interpret the references. OK - then I'm not talking about references, though we still mean different things when talking about 'relation': Mine goes like this: A relation object R providing ITriadicRelation and connecting A, B and C involves three references: R with A R with B R with C In addition the relation object has a sort of type, the 'relationship' (corresponding to a 'predicate') that may be specified by the relation's class or - a special case - a string. Maybe there is a better word for this kind of 'relation'? Helmut ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Helmut Merz wrote: >Am Samstag, 12. November 2005 18:00 schrieb Jean-Marc Orliaguet: > > > >>My >>impression is that you are thinking of a reference engine >>rather than a relation engine >> >> > >Maybe I just don't see the difference... (There is one, of >course, but I doubt it is really of relevance in a practical >context. ) > >Helmut > > A relation contains several references: a relation between A, B and C involves 7 references: A with itself B with itself C with itself A with B B with C C with A A with B with C But a set of references don't make explicit relations, unless the application know how to interpret the references. /JM ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Am Samstag, 12. November 2005 18:00 schrieb Jean-Marc Orliaguet: > The good thing about IntIds is that it leaves the objects > untouched, the bad thing is that it creates a hard coupling > between the integer id that is used to *refer* to the object > and the key that is used to *represent* the object in the > relation. These are really two different aspects. > > In cpsskins when objects are set in relation, it is the > application's responsibility to tell the engine how objects > will be identified in the relation, i.e. with a unique id > (IntIds), or with a URI or a with a name (that may be share > between several objects) ... The relation engine then stores > this key as well as an internal reference to the object. > > so for each object put in relation we have: > - a key (used by the application to query the engine) > - a reference to the object (used internally by the relation > engine to get the object) > > the application only "sees" the keys until the objects get > dereferenced. > > But no 1:1 mapping between the key and the object is imposed > by any external IntIds utility. Which make it possible to ask > the engine: "give me all the portlets associated to the 'left' > slot" even though the 'left' slot is materialized in more than > one instance. Hm, I'm not sure I understand - so it's not the object (more precisely: its unique id) that's indexed but some value provided by the object - so this is indeed some sort of an attribute that's indexed. This I would prefer to solve using relation objects that explicitly handle additional attributes (provided by the objects that are connected by the relation) that may be indexed *in addition* to the unique ids. Or let's see this way: The keys (unique ids in the standard implementation) are retrieved by an adapter to the relation class (providing IIndexableRelation) so this adapter could provide something else if necessary. > In your implementation only objects that are identified > uniquely can be put in relation, and it doesn't seem to be a > design choice other than a limitation imposed by the catalog. Yes, it is a limitation of the default implementation but I'm still not sure it is bad, and it can be overcome easily. > http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches > /j mo -perspectives/storage/relations.py > >>> > >>>I read this, and it indeed gave me the impression that it > >>>might be a not so bad idea to use a catalog ;-) > >> > >>well, you haven't written the catalog indexes yet :-) > > > >I needn't because I just use the FieldIndex from zope.index. > > I understand, but the idea is to index relations, not simply > the references between in the objects put in relation. ACK > My > impression is that you are thinking of a reference engine > rather than a relation engine Maybe I just don't see the difference... (There is one, of course, but I doubt it is really of relevance in a practical context. ) Helmut ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Helmut Merz wrote: >Am Freitag, 11. November 2005 18:00 schrieb Jean-Marc Orliaguet: > > > >>I was thinking more about the policy of assigning unique ids >>to objects in a relation. It's the application that really >>should decide about that policy. >> >> > >in fact the unique ids aren't "assigned" to the objects (the >objects aren't touched at all) by the IntIds utility - if you >have registered a local IntIds utility they are just created in >the utility driven by the IObjectAdded event. So this IntIds >stuff should not interfere with anything else and the >application should not be concerned at all about it. > > The good thing about IntIds is that it leaves the objects untouched, the bad thing is that it creates a hard coupling between the integer id that is used to *refer* to the object and the key that is used to *represent* the object in the relation. These are really two different aspects. In cpsskins when objects are set in relation, it is the application's responsibility to tell the engine how objects will be identified in the relation, i.e. with a unique id (IntIds), or with a URI or a with a name (that may be share between several objects) ... The relation engine then stores this key as well as an internal reference to the object. so for each object put in relation we have: - a key (used by the application to query the engine) - a reference to the object (used internally by the relation engine to get the object) the application only "sees" the keys until the objects get dereferenced. But no 1:1 mapping between the key and the object is imposed by any external IntIds utility. Which make it possible to ask the engine: "give me all the portlets associated to the 'left' slot" even though the 'left' slot is materialized in more than one instance. In your implementation only objects that are identified uniquely can be put in relation, and it doesn't seem to be a design choice other than a limitation imposed by the catalog. http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/j mo -perspectives/storage/relations.py >>>I read this, and it indeed gave me the impression that it >>>might be a not so bad idea to use a catalog ;-) >>> >>> >>well, you haven't written the catalog indexes yet :-) >> >> > >I needn't because I just use the FieldIndex from zope.index. > > > I understand, but the idea is to index relations, not simply the references between in the objects put in relation. My impression is that you are thinking of a reference engine rather than a relation engine with the difference compared to the Archetype's engine that references are stored outside the objects. /JM ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Am Freitag, 11. November 2005 17:26 schrieb Jean-Marc Orliaguet:
> if the relation is symmetrical, there are 2 relations:
>
> A is like B
>
> means:
>
>A resembles B
>
> and
>
>B resembles A( or A __ is resembled by __ B )
>
> which you can express as a compound predicate:
>
> is_like = CompoundPredicate(Predicate('_ resembles _', '_
> is resembled by _'))
>
> and do a query with:
>
> relations.search(predicate=is_like, first=A)
>
> to get all the B:s in relation with A independently of the
> position.
Interesting, maybe I should learn more about predicate logic...
Nevertheless let me try it "my way":
Let's have an interface ISymmetricalRelation(IDyadicRelation)
that is implemented by a class
SymmetricalRelation(PredicateRelation) (PredicateRelation is an
example from the README.txt
http://svn.cy55.de/viewcvs/Zope3/src/cybertools/trunk/relation,
Concept is some imaginary class used for conceptual modelling).
>>> isSynonym = Predicate('__ is a synonym of __')
>>> crazy = Concept('crazy')
>>> mad = Concept('mad')
Then the contract for ISymmetricalRelation would look like this
(this is not strictly necessary but makes it easier to
understand):
>>> SymmetricalRelation(isSynonym, crazy, mad) ==
SymmetricalRelation(isSynonym, mad, crazy)
True
Now we register such a relation:
>>> relations.register(SymmetricalRelation(isSynonym, crazy,
mad))
and query in two ways:
>>> rels1 = relations.query(relationship=isSynonym,
... first=crazy)
>>> rels2 = relations.query(relationship=isSynonym,
... second=crazy)
then all the found relations must be identical:
>>> [r1 in rels2 for r1 in rels1]
[True]
(When I'd query for 'mad' I'd get the same results; I could also
omit the relationship parameter on the query() call.)
In addition to the standard IDyadicRelation interface,
ISymmetricalRelation has a method getOther():
>>> [r.getOther(crazy) for r in rels1]
[]
This is all I need to find all the synonyms of 'mad'.
Don't ask me about the implementation now - I just don't care at
the moment...
> Then most of the relations do have a direction ( __ has author
> __, __ was modified by __, ...)
Yes, a symmetrical relationship is a special case, and one could
even handle it on the application layer. Nevertheless I think
it's a good idea to consider it.
Helmut
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Helmut Merz wrote:
>Am Freitag, 11. November 2005 16:11 schrieb Jean-Marc Orliaguet:
>
>
>
>>Hi Helmut!
>>
>>
>
>Hi Jean-Marc,
>
>thanks for your remarks,
>
>just before going into more detail: My primary concern was the
>API - it would really fine if there could be a simple (as simple
>as possible but not simpler) standard set of (low-level)
>interfaces on which to build (defining semantically richer
>interfaces) and for which to provide implementations (depending
>on the needs of the application).
>
>The implementation with the catalog should serve as a (again
>fairly simple but working) example and a proof of concept; I
>think I would be just lucky if it would show up as really
>useful ;-) (but maybe...)
>
>
>
Hi,
a common interface could be useful indeed.
>>I can tell you about the design decisions made in the case of
>>the relation tool included in CPSSkins. They don't necessarily
>>appear in the code itself in an obvious way.
>>
>>1) separate storage from storage policy
>>
>>the relation storage stores what it is told to store, as long
>>as the objects are Relatable they can be stored. The storage
>>policy (using unique ids or not, etc..) is the responsibility
>>of the application itself. To impose a unique id policy when
>>storing elements would be a mistake in my opinion (in the case
>>of cpsskins it wouln't work either).
>>
>>
>
>The only prerequisite for using the IntIds utility is that the
>objects are persistent (provide IPersistent). If one wants to
>relate objects that are not persistent or have relations that
>for some reason can't be persistent you can't use the catalog
>approach because the catalog depends on IntIds.
>
>So the catalog-based implementation won't be usable for relations
>between in-memory objects (like views, adapters and related
>stuff), that's true.
>
>
I was thinking more about the policy of assigning unique ids to objects
in a relation. It's the application that really should decide about that
policy.
>>2) keep the relation storage index as small as possible.
>>
>>Do not index predicates, the same predicates are used in too
>>many relations, the size of the index ould just increase
>>dramatically. Instead only index the elements that are inside
>>the relation, the chances that the same elements are related
>>in many different ways are very low.
>>
>>cf.
>>
>>
>>
>http://www.z3lab.org/sections/blogs/jean-marc-orliaguet/2005_08_27_triadic-relations/
>
>
>>http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo
>>-perspectives/storage/relations.py
>>
>>
>
>I read this, and it indeed gave me the impression that it might
>be a not so bad idea to use a catalog ;-)
>
>
>
well, you haven't written the catalog indexes yet :-)
And Lennart wrote a piece about the kinds of problems you'll run into if
you don't optimize them for relations. You'll end up with intersections
of huge sets:
http://blogs.nuxeo.com/sections/blogs/lennart_regebro/2005_08_29_indexing-events
>>I don't know about using the zc.catalog for indexing
>>relations, you could end up in huge indexes and very slow
>>queries.
>>
>>
>
>This is one of my concerns, too, but I'm fairly optimistic: the
>catalog indexes store a common string to be indexed only once,
>so having identical ; I'm working with the Archetypes reference
>engine (that uses - at least in this respect - the same kind of
>catalog indexes) in situations with many thousands of objects
>and didn't get problems of this kind.
>
>
>
By looking at the code, Archetypes does not store relations, it stores
'references' (and backward references) which consist in a target object
and a predicate ('relationship') in the objects themselves . I guess
that objects are indexed in the catalog. So the relation is stored
implicitly but there is no explicit relation object to start with. So
the model is a bit different I guess.
>>3) don't make the API for querying the storage be too
>>intelligent,
>>
>>
>
>The query() method using the catalog's searchResults() / apply()
>methods was the dumbest one I could thing of ;-)
>
>
>
>> to create complex queries, create complex predicates instead,
>>i.e.
>>
>> - predicates that combine several predicates
>> - proxy predicates (when the predicate is evaluated at
>>runtime and a method is specifed instead)
>>
>>cf
>>
>>
>>
>http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo-perspectives/relations/__init__.py
>
>
>>if you need to do really complex queries, do several
>>queries and filter out the results afterwards in you
>>application unless you're fine with ending up with a huge
>>catalog index.
>>
>>
>
>To be honest, I never thought about complex queries as I just
>want to find e.g. the subtasks of a task and the resources
>allocated to it - maybe my use case is just somewhat simple.
>
>OTOH: An advantage of using a catalog are the - as I think -
>fairly efficient set operations on search results for the
>indexes..
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Reinoud van Leeuwen wrote:
>On Fri, Nov 11, 2005 at 04:44:47PM +0100, Jean-Marc Orliaguet wrote:
>
>
>>Reinoud van Leeuwen wrote:
>>
>>
>>
>>>On Fri, Nov 11, 2005 at 03:50:25PM +0100, Helmut Merz wrote:
>>>
>>>
>>>
>>>
A relation is an object providing the IRelation interface. This
has got two attributes ('first', 'second' - dyadic relation) or
>>>I've done this kind of thing in relational databases. Problem with 'first'
>>>and 'second' is that it seems to imply some order. And if I try to find
>>>all relations from an object I allways have to compare my ID to either
>>>first or second.
>>>
>>>I solved my problem by chopping a relation into three parts: the relation
>>>itself and both endpoints. In my database this generated an extra table
>>>(and some more work when writing queries), but the solution became more
>>>generic and flexible in the end.
>>>
>>>(See the database diagram on
>>>http://www.drbob.org/datamodel/drbob_datamodel.htm . The two endpoints of
>>>a relation are stored in two object_link records, the relation itself in
>>>one link record.)
>>>
>>>
>>>
>>>
>>>
>>Hi!
>>
>>I think that you want 1 relation that consists of:
>>- the elements in the relation (the relates)
>>- the predicate
>>
>>then you can chop the relations into piece when you index it, but not
>>before.
>>
>>a relation is by definition 1 entity, if you start storing 2 items to
>>represent a single relation, you will have problems looming ahead when
>>it comes to managing the different "parts" of the relation. For instance
>>if you store A -> B and B -> A to represent A <-> B, you will have to
>>destroy A -> B if you remove B -> A.
>>
>>This is why I use real triadic relations to avoid having complicated
>>constructions based on dyadic relations that only make sense when
>>they're taken together. When I destroy a triadic relation between 3
>>objects, this is done in one take, not three takes.
>>
>>then order in a relation is important:
>>
>>A kills B
>>
>>is not the same as:
>>
>>B kills A
>>
>>of course if the predicate is :
>>
>> __ is like __
>>
>>it appears as though order is not important, but it is a question of
>>semantics, not a question of logic. A relation engine is not supposed to
>>know about semantics.
>>
>>
>
>OK. In a relational database you can solve the 'one entity stored in
>different tables' with triggers. But this discussion is not about a RDBMS
>";-)
>
>As long as the relations you propose have a direction ('kills') it makes
>sense to stored them as 'first' and 'second' somewhere. But when the
>direction is symetrical ('is like') this implementation does not feel
>right. Of course it is possible in Python to make a class around it, but
>still if a piece of code wants to find all related objects it has to
>search both the 'first' and the 'second' field.
>
>That sounds less generic than it could be in my opinion.
>
>
In any case it is the responsibility of the application to define its
ontology. The relation engine shouldn't make any assumptions in that area.
if the relation is symmetrical, there are 2 relations:
A is like B
means:
A resembles B
and
B resembles A( or A __ is resembled by __ B )
which you can express as a compound predicate:
is_like = CompoundPredicate(Predicate('_ resembles _', '_ is
resembled by _'))
and do a query with:
relations.search(predicate=is_like, first=A)
to get all the B:s in relation with A independently of the position.
Then most of the relations do have a direction ( __ has author __, __
was modified by __, ...)
/JM
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Am Freitag, 11. November 2005 16:11 schrieb Jean-Marc Orliaguet: > Hi Helmut! Hi Jean-Marc, thanks for your remarks, just before going into more detail: My primary concern was the API - it would really fine if there could be a simple (as simple as possible but not simpler) standard set of (low-level) interfaces on which to build (defining semantically richer interfaces) and for which to provide implementations (depending on the needs of the application). The implementation with the catalog should serve as a (again fairly simple but working) example and a proof of concept; I think I would be just lucky if it would show up as really useful ;-) (but maybe...) > I can tell you about the design decisions made in the case of > the relation tool included in CPSSkins. They don't necessarily > appear in the code itself in an obvious way. > > 1) separate storage from storage policy > > the relation storage stores what it is told to store, as long > as the objects are Relatable they can be stored. The storage > policy (using unique ids or not, etc..) is the responsibility > of the application itself. To impose a unique id policy when > storing elements would be a mistake in my opinion (in the case > of cpsskins it wouln't work either). The only prerequisite for using the IntIds utility is that the objects are persistent (provide IPersistent). If one wants to relate objects that are not persistent or have relations that for some reason can't be persistent you can't use the catalog approach because the catalog depends on IntIds. So the catalog-based implementation won't be usable for relations between in-memory objects (like views, adapters and related stuff), that's true. > 2) keep the relation storage index as small as possible. > > Do not index predicates, the same predicates are used in too > many relations, the size of the index ould just increase > dramatically. Instead only index the elements that are inside > the relation, the chances that the same elements are related > in many different ways are very low. > > cf. > http://www.z3lab.org/sections/blogs/jean-marc-orliaguet/2005_08_27_triadic-relations/ > > http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo >-perspectives/storage/relations.py I read this, and it indeed gave me the impression that it might be a not so bad idea to use a catalog ;-) > I don't know about using the zc.catalog for indexing > relations, you could end up in huge indexes and very slow > queries. This is one of my concerns, too, but I'm fairly optimistic: the catalog indexes store a common string to be indexed only once, so having identical ; I'm working with the Archetypes reference engine (that uses - at least in this respect - the same kind of catalog indexes) in situations with many thousands of objects and didn't get problems of this kind. > 3) don't make the API for querying the storage be too > intelligent, The query() method using the catalog's searchResults() / apply() methods was the dumbest one I could thing of ;-) > to create complex queries, create complex predicates instead, > i.e. > >- predicates that combine several predicates >- proxy predicates (when the predicate is evaluated at > runtime and a method is specifed instead) > > cf > http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo-perspectives/relations/__init__.py > > if you need to do really complex queries, do several > queries and filter out the results afterwards in you > application unless you're fine with ending up with a huge > catalog index. To be honest, I never thought about complex queries as I just want to find e.g. the subtasks of a task and the resources allocated to it - maybe my use case is just somewhat simple. OTOH: An advantage of using a catalog are the - as I think - fairly efficient set operations on search results for the indexes... Helmut ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
On Fri, Nov 11, 2005 at 04:44:47PM +0100, Jean-Marc Orliaguet wrote:
> Reinoud van Leeuwen wrote:
>
> >On Fri, Nov 11, 2005 at 03:50:25PM +0100, Helmut Merz wrote:
> >
> >
> >>A relation is an object providing the IRelation interface. This
> >>has got two attributes ('first', 'second' - dyadic relation) or
> >>
> >>
> >
> >I've done this kind of thing in relational databases. Problem with 'first'
> >and 'second' is that it seems to imply some order. And if I try to find
> >all relations from an object I allways have to compare my ID to either
> >first or second.
> >
> >I solved my problem by chopping a relation into three parts: the relation
> >itself and both endpoints. In my database this generated an extra table
> >(and some more work when writing queries), but the solution became more
> >generic and flexible in the end.
> >
> >(See the database diagram on
> >http://www.drbob.org/datamodel/drbob_datamodel.htm . The two endpoints of
> >a relation are stored in two object_link records, the relation itself in
> >one link record.)
> >
> >
> >
>
> Hi!
>
> I think that you want 1 relation that consists of:
> - the elements in the relation (the relates)
> - the predicate
>
> then you can chop the relations into piece when you index it, but not
> before.
>
> a relation is by definition 1 entity, if you start storing 2 items to
> represent a single relation, you will have problems looming ahead when
> it comes to managing the different "parts" of the relation. For instance
> if you store A -> B and B -> A to represent A <-> B, you will have to
> destroy A -> B if you remove B -> A.
>
> This is why I use real triadic relations to avoid having complicated
> constructions based on dyadic relations that only make sense when
> they're taken together. When I destroy a triadic relation between 3
> objects, this is done in one take, not three takes.
>
> then order in a relation is important:
>
> A kills B
>
> is not the same as:
>
> B kills A
>
> of course if the predicate is :
>
>__ is like __
>
> it appears as though order is not important, but it is a question of
> semantics, not a question of logic. A relation engine is not supposed to
> know about semantics.
OK. In a relational database you can solve the 'one entity stored in
different tables' with triggers. But this discussion is not about a RDBMS
";-)
As long as the relations you propose have a direction ('kills') it makes
sense to stored them as 'first' and 'second' somewhere. But when the
direction is symetrical ('is like') this implementation does not feel
right. Of course it is possible in Python to make a class around it, but
still if a piece of code wants to find all related objects it has to
search both the 'first' and the 'second' field.
That sounds less generic than it could be in my opinion.
--
__
"Nothing is as subjective as reality"
Reinoud van Leeuwen[EMAIL PROTECTED]
http://www.xs4all.nl/~reinoud
__
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Reinoud van Leeuwen wrote:
>On Fri, Nov 11, 2005 at 03:50:25PM +0100, Helmut Merz wrote:
>
>
>>A relation is an object providing the IRelation interface. This
>>has got two attributes ('first', 'second' - dyadic relation) or
>>
>>
>
>I've done this kind of thing in relational databases. Problem with 'first'
>and 'second' is that it seems to imply some order. And if I try to find
>all relations from an object I allways have to compare my ID to either
>first or second.
>
>I solved my problem by chopping a relation into three parts: the relation
>itself and both endpoints. In my database this generated an extra table
>(and some more work when writing queries), but the solution became more
>generic and flexible in the end.
>
>(See the database diagram on
>http://www.drbob.org/datamodel/drbob_datamodel.htm . The two endpoints of
>a relation are stored in two object_link records, the relation itself in
>one link record.)
>
>
>
Hi!
I think that you want 1 relation that consists of:
- the elements in the relation (the relates)
- the predicate
then you can chop the relations into piece when you index it, but not
before.
a relation is by definition 1 entity, if you start storing 2 items to
represent a single relation, you will have problems looming ahead when
it comes to managing the different "parts" of the relation. For instance
if you store A -> B and B -> A to represent A <-> B, you will have to
destroy A -> B if you remove B -> A.
This is why I use real triadic relations to avoid having complicated
constructions based on dyadic relations that only make sense when
they're taken together. When I destroy a triadic relation between 3
objects, this is done in one take, not three takes.
then order in a relation is important:
A kills B
is not the same as:
B kills A
of course if the predicate is :
__ is like __
it appears as though order is not important, but it is a question of
semantics, not a question of logic. A relation engine is not supposed to
know about semantics.
/JM
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
On Fri, Nov 11, 2005 at 03:50:25PM +0100, Helmut Merz wrote:
>
> A relation is an object providing the IRelation interface. This
> has got two attributes ('first', 'second' - dyadic relation) or
I've done this kind of thing in relational databases. Problem with 'first'
and 'second' is that it seems to imply some order. And if I try to find
all relations from an object I allways have to compare my ID to either
first or second.
I solved my problem by chopping a relation into three parts: the relation
itself and both endpoints. In my database this generated an extra table
(and some more work when writing queries), but the solution became more
generic and flexible in the end.
(See the database diagram on
http://www.drbob.org/datamodel/drbob_datamodel.htm . The two endpoints of
a relation are stored in two object_link records, the relation itself in
one link record.)
--
__
"Nothing is as subjective as reality"
Reinoud van Leeuwen[EMAIL PROTECTED]
http://www.xs4all.nl/~reinoud
__
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Helmut Merz wrote: there are quite a few solutions out there in the Zope world that allow the management of relations or references between objects. Knowing as little as I do about this area, like the basics of this approach. Jean-Marc had some interesting things to say and I'd like for the essentials of relationship management to be distilled into a set of concepts that can be reused in most/all implementations, where only the details differ. -- Benji York Senior Software Engineer Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...
Helmut Merz wrote:
>Hi,
>
>there are quite a few solutions out there in the Zope world that
>allow the management of relations or references between objects.
>So I have been working with the reference engine provided by
>Archetypes (as part of Plone) for nearly two years now; and I
>also had a look at some proposals on zope.org and
>implementations that are available in the Zope 3 packages from
>SchoolTool and CPSSkins.
>
>I have to confess that I was not really happy with any of these
>proposals or solutions - a feeling that we may discuss in more
>detail, the main point being just that I wanted to have
>something simple that is nevertheless flexible enough to
>accommodate to various needs.
>
>So I came up with the following stuff (mainly combining concepts
>by Jean-Marc Orliaguet and implementation ideas from Archetypes
>1.3) :
>
>A relation is an object providing the IRelation interface. This
>has got two attributes ('first', 'second' - dyadic relation) or
>three ('first', 'second', 'third' - triadic relation). (You may
>ignore triadic relations if you don't think you need them.)
>
>If I now create corresponding classes (DyadicRelation,
>TriadicRelation) I can assign the objects taking part in a
>relation directly to these attributes of the relation. (Note
>that the objects taking part in a relation are not touched at
>all.) In fact I create a class for each kind of relation (each
>relationship or predicate or whatever you like to call it)
>I want to use.
>
>So the question arises what to do with the relation objects? The
>point here is not so much where to store them but how to make
>sure that we can find them. So we need a registry for relations
>- interface IRelationsRegistry.
>
>This has three methods:
>
>- register(relation)
>- unregister(relation)
>- query(**kw)
>
>The keyword arguments to the query() method may be:
>
>- relationship=... - a relation class (we can also work with
> named predicates - see below)
>- first=... - an object
>- second=... - an object
>- third=... - an object
>
>One can combine those arguments at will to search for all
>relations of the relation class specified (if given) with the
>given values for the attributes. The list may be extended if you
>use relation classes with additional attributes you want to
>search for.
>
>(For details have a look at the README.txt and other files on
>http://svn.cy55.de/viewcvs/Zope3/src/cybertools/trunk/relation
>or check it out via
>svn co svn://svn.cy55.de/Zope3/src/cybertools/trunk/relation)
>
>You see that this looks very much like a catalog search - so why
>not implement the relations registry as a subclass of
>zope.app.catalog.catalog.Catalog?
>
>OK, so the RelationsRegistry class is derived from Catalog, and
>of course it is a local utility. The indexes are just FieldIndex
>objects, using an adapter to provide unique ids for the objects
>involved via the IntIds utility; the same is done for providing
>ids for the relation objects themselves. The relationship is
>mapped to the full module path + class name of the relation
>class.
>
>An interesting consequence of this is that the relation objects
>are not explicitly stored somewhere, the only reference to them
>is via the IntIds utility. Thus you are free to store them in
>some container or even make them first class content objects if
>you want - but you needn't.
>
>The whole stuff is flexible enough to be extended as needed, e.g.
>with special registry classes used via named utilities,
>additional attributes on relations (easily indexable in the
>registry); it could be wrapped in a richer and more
>application-oriented API, etc.
>
>It is also possible to use named predicates (with URIs or strings
>like '_ is child of _') instead of different relation classes to
>represent relationships (this is shown by an example at the end
>of the README.txt).
>
>
>So what do you think? - any feedback and suggestions welcome...
>
>I'd also gladly write a proposal (if I get write access to the
>wiki ;-)) and of course change the license from GPL to ZPL if
>this would be considered useful.
>
>Helmut
>
>
>
Hi Helmut!
I can tell you about the design decisions made in the case of the
relation tool included in CPSSkins. They don't necessarily appear in the
code itself in an obvious way.
1) separate storage from storage policy
the relation storage stores what it is told to store, as long as the
objects are Relatable they can be stored. The storage policy (using
unique ids or not, etc..) is the responsibility of the application
itself. To impose a unique id policy when storing elements would be a
mistake in my opinion (in the case of cpsskins it wouln't work either).
2) keep the relation storage index as small as possible.
Do not index predicates, the same predicates are used in too many
relations, the size of the index ould just increase dramatically.
Instead only index the elements that are inside the relation, the
chances t
[Zope3-dev] Yet Another Relations (aka Reference) Engine...
Hi,
there are quite a few solutions out there in the Zope world that
allow the management of relations or references between objects.
So I have been working with the reference engine provided by
Archetypes (as part of Plone) for nearly two years now; and I
also had a look at some proposals on zope.org and
implementations that are available in the Zope 3 packages from
SchoolTool and CPSSkins.
I have to confess that I was not really happy with any of these
proposals or solutions - a feeling that we may discuss in more
detail, the main point being just that I wanted to have
something simple that is nevertheless flexible enough to
accommodate to various needs.
So I came up with the following stuff (mainly combining concepts
by Jean-Marc Orliaguet and implementation ideas from Archetypes
1.3) :
A relation is an object providing the IRelation interface. This
has got two attributes ('first', 'second' - dyadic relation) or
three ('first', 'second', 'third' - triadic relation). (You may
ignore triadic relations if you don't think you need them.)
If I now create corresponding classes (DyadicRelation,
TriadicRelation) I can assign the objects taking part in a
relation directly to these attributes of the relation. (Note
that the objects taking part in a relation are not touched at
all.) In fact I create a class for each kind of relation (each
relationship or predicate or whatever you like to call it)
I want to use.
So the question arises what to do with the relation objects? The
point here is not so much where to store them but how to make
sure that we can find them. So we need a registry for relations
- interface IRelationsRegistry.
This has three methods:
- register(relation)
- unregister(relation)
- query(**kw)
The keyword arguments to the query() method may be:
- relationship=... - a relation class (we can also work with
named predicates - see below)
- first=... - an object
- second=... - an object
- third=... - an object
One can combine those arguments at will to search for all
relations of the relation class specified (if given) with the
given values for the attributes. The list may be extended if you
use relation classes with additional attributes you want to
search for.
(For details have a look at the README.txt and other files on
http://svn.cy55.de/viewcvs/Zope3/src/cybertools/trunk/relation
or check it out via
svn co svn://svn.cy55.de/Zope3/src/cybertools/trunk/relation)
You see that this looks very much like a catalog search - so why
not implement the relations registry as a subclass of
zope.app.catalog.catalog.Catalog?
OK, so the RelationsRegistry class is derived from Catalog, and
of course it is a local utility. The indexes are just FieldIndex
objects, using an adapter to provide unique ids for the objects
involved via the IntIds utility; the same is done for providing
ids for the relation objects themselves. The relationship is
mapped to the full module path + class name of the relation
class.
An interesting consequence of this is that the relation objects
are not explicitly stored somewhere, the only reference to them
is via the IntIds utility. Thus you are free to store them in
some container or even make them first class content objects if
you want - but you needn't.
The whole stuff is flexible enough to be extended as needed, e.g.
with special registry classes used via named utilities,
additional attributes on relations (easily indexable in the
registry); it could be wrapped in a richer and more
application-oriented API, etc.
It is also possible to use named predicates (with URIs or strings
like '_ is child of _') instead of different relation classes to
represent relationships (this is shown by an example at the end
of the README.txt).
So what do you think? - any feedback and suggestions welcome...
I'd also gladly write a proposal (if I get write access to the
wiki ;-)) and of course change the license from GPL to ZPL if
this would be considered useful.
Helmut
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
