Re: [Zope3-dev] Extrinsic references

[Helmut Merz]

Am Samstag, 12. November 2005 17:15 schrieb Jim Fulton:
 We have a *simple* facility for managing references between
 objects extrinsically.  See:

   
 http://svn.zope.org/Sandbox/zc/extrinsicreference/extrinsicref
erence.txt?view=markup

 If there is sufficient interest, we plan to move this (tiny)
 project to the zope repository.

+1

... though it does not provide a complete solution for what I am 
thinking about concerning the relation management stuff; but it 
is a solution for a lot of use cases and may also provide the 
basis for an implementation of a richer relation management 
framework.

Helmut
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com

Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...

[Helmut Merz]

Am Sonntag, 13. November 2005 12:07 schrieb Jean-Marc Orliaguet:
 Helmut Merz wrote:
 Anyway, what we are talking about are not references.

 The approach is quite different: references start from the
 objects themselves that they connect to other objects using
 one-way relations (a pointer, an arrow). 

And this can be easily implemented by using assignments to object 
attributes.

 The application has 
 to know how to interpret the references. I don't think that 
 you can build a robust relation engine only with that.
 Relations start from the top: you first define an ontology
 (a set of general predicates) that you use to relate the
 objects of your application. This is a conceptual schema. Here
 is the cpsskins ontology:
 http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo
-perspectives/ontology.py

 The relation engine then manages all the necessary references,
 but the application does not need to know about the references
 at all. The interaction with the relation engine is done only
 via the ontology.

OK, I see. I now also understand why you include monadic 
relations which could just be attributes or annotations... 
(I ignored them up to now but I think I should change this.)

OTOH I'd like to have a relation management API that could as 
well be used for cases where the relations are controlled by the 
application or the objects involved.

This is then just a convenience API that cares about the problems 
of keeping up-to-date references in both directions, deletion 
handling, the complexity arising with connecting more than two 
objects, attributed or annotated relations, etc.

Helmut
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com

Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...

[Helmut Merz]

Am Samstag, 12. November 2005 18:00 schrieb Jean-Marc Orliaguet:

 The good thing about IntIds is that it leaves the objects
 untouched, the bad thing is that it creates a hard coupling
 between the integer id that is used to *refer* to the object
 and the key that is used to *represent* the object in the
 relation. These are really two different aspects.

 In cpsskins when objects are set in relation, it is the
 application's responsibility to tell the engine how objects
 will be identified in the relation, i.e. with a unique id
 (IntIds), or with a URI or a with a name (that may be share
 between several objects) ... The relation engine then stores
 this key as well as an internal reference to the object.

 so for each object put in relation we have:
 - a key (used by the application to query the engine)
 - a reference to the object (used internally by the relation
 engine to get the object)

 the application only sees the keys until the objects get
 dereferenced.

 But no 1:1 mapping between the key and the object is imposed
 by any external IntIds utility. Which make it possible to ask
 the engine: give me all the portlets associated to the 'left'
 slot even though the 'left' slot is materialized in more than
 one instance.

Hm, I'm not sure I understand - so it's not the object (more 
precisely: its unique id) that's indexed but some value provided 
by the object - so this is indeed some sort of an attribute 
that's indexed.

This I would prefer to solve using relation objects that 
explicitly handle additional attributes (provided by the objects 
that are connected by the relation) that may be indexed *in 
addition* to the unique ids.

Or let's see this way: The keys (unique ids in the standard 
implementation) are retrieved by an adapter to the relation 
class (providing IIndexableRelation) so this adapter could 
provide something else if necessary.

 In your implementation only objects that are identified
 uniquely can be put in relation, and it doesn't seem to be a
 design choice other than a limitation imposed by the catalog.

Yes, it is a limitation of the default implementation but I'm 
still not sure it is bad, and it can be overcome easily.

 http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches
 /j mo -perspectives/storage/relations.py
 
 I read this, and it indeed gave me the impression that it
 might be a not so bad idea to use a catalog ;-)
 
 well, you haven't written the catalog indexes yet :-)
 
 I needn't because I just use the FieldIndex from zope.index.

 I understand, but the idea is to index relations, not simply 
 the references between in the objects put in relation. 

ACK

 My 
 impression is that you are thinking of a reference engine
 rather than a relation engine 

Maybe I just don't see the difference... (There is one, of 
course, but I doubt it is really of relevance in a practical 
context. )

Helmut
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com

Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...

[Helmut Merz]

Am Freitag, 11. November 2005 16:11 schrieb Jean-Marc Orliaguet:

 Hi Helmut!

Hi Jean-Marc,

thanks for your remarks,

just before going into more detail: My primary concern was the 
API - it would really fine if there could be a simple (as simple 
as possible but not simpler) standard set of (low-level) 
interfaces on which to build (defining semantically richer 
interfaces) and for which to provide implementations (depending 
on the needs of the application).

The implementation with the catalog should serve as a (again 
fairly simple but working) example and a proof of concept; I 
think I would be just lucky if it would show up as really 
useful ;-) (but maybe...)

 I can tell you about the design decisions made in the case of
 the relation tool included in CPSSkins. They don't necessarily
 appear in the code itself in an obvious way.

 1) separate storage from storage policy

 the relation storage stores what it is told to store, as long
 as the objects are Relatable they can be stored. The storage
 policy (using unique ids or not, etc..) is the responsibility
 of the application itself. To impose a unique id policy when
 storing elements would be a mistake in my opinion (in the case
 of cpsskins it wouln't work either).

The only prerequisite for using the IntIds utility is that the 
objects are persistent (provide IPersistent). If one wants to 
relate objects that are not persistent or have relations that 
for some reason can't be persistent you can't use the catalog 
approach because the catalog depends on IntIds.

So the catalog-based implementation won't be usable for relations 
between in-memory objects (like views, adapters and related 
stuff), that's true.

 2) keep the relation storage index as small as possible.

 Do not index predicates, the same predicates are used in too
 many relations, the size of the index ould just increase
 dramatically. Instead only index the elements that are inside
 the relation, the chances that the same elements are related
 in many different ways are very low.

 cf.
 
http://www.z3lab.org/sections/blogs/jean-marc-orliaguet/2005_08_27_triadic-relations/

 http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo
-perspectives/storage/relations.py

I read this, and it indeed gave me the impression that it might 
be a not so bad idea to use a catalog ;-)

 I don't know about using the zc.catalog for indexing
 relations, you could end up in huge indexes and very slow
 queries.

This is one of my concerns, too, but I'm fairly optimistic: the 
catalog indexes store a common string to be indexed only once, 
so having identical ; I'm working with the Archetypes reference 
engine (that uses - at least in this respect - the same kind of 
catalog indexes) in situations with many thousands of objects 
and didn't get problems of this kind.

 3) don't make the API for querying the storage be too
 intelligent,

The query() method using the catalog's searchResults() / apply() 
methods was the dumbest one I could thing of ;-)

  to create complex queries, create complex predicates instead,
 i.e.

- predicates that combine several predicates
- proxy predicates (when the predicate is evaluated at
 runtime and a method is specifed instead)

 cf
 
http://svn.nuxeo.org/trac/pub/file/z3lab/cpsskins/branches/jmo-perspectives/relations/__init__.py

 if you need to do really complex queries, do several
 queries and filter out the results afterwards  in you
 application unless you're fine with ending up with a huge
 catalog index.

To be honest, I never thought about complex queries as I just 
want to find e.g. the subtasks of a task and the resources 
allocated to it - maybe my use case is just somewhat simple.

OTOH: An advantage of using a catalog are the - as I think - 
fairly efficient set operations on search results for the 
indexes...

Helmut
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com

Re: [Zope3-dev] Yet Another Relations (aka Reference) Engine...

[Helmut Merz]

Am Freitag, 11. November 2005 17:26 schrieb Jean-Marc Orliaguet:

 if the relation is symmetrical, there are 2 relations:

 A is like B

 means:

A resembles B

 and

B resembles A( or A __ is resembled by __ B )

 which you can express as a compound predicate:

 is_like = CompoundPredicate(Predicate('_ resembles _', '_
 is resembled by _'))

 and do a query with:

 relations.search(predicate=is_like, first=A)

 to get all the B:s in relation with A independently of the
 position.

Interesting, maybe I should learn more about predicate logic...

Nevertheless let me try it my way:

Let's have an interface ISymmetricalRelation(IDyadicRelation) 
that is implemented by a class 
SymmetricalRelation(PredicateRelation) (PredicateRelation is an 
example from the README.txt 
http://svn.cy55.de/viewcvs/Zope3/src/cybertools/trunk/relation, 
Concept is some imaginary class used for conceptual modelling).

   isSynonym = Predicate('__ is a synonym of __')
   crazy = Concept('crazy')
   mad = Concept('mad')

Then the contract for ISymmetricalRelation would look like this 
(this is not strictly necessary but makes it easier to 
understand):

   SymmetricalRelation(isSynonym, crazy, mad) == 
SymmetricalRelation(isSynonym, mad, crazy)
  True

Now we register such a relation:

   relations.register(SymmetricalRelation(isSynonym, crazy, 
mad))

and query in two ways:

   rels1 = relations.query(relationship=isSynonym, 
  ... first=crazy) 
   rels2 = relations.query(relationship=isSynonym, 
  ... second=crazy) 

then all the found relations must be identical:

   [r1 in rels2 for r1 in rels1]
  [True]

(When I'd query for 'mad' I'd get the same results; I could also 
omit the relationship parameter on the query() call.)

In addition to the standard IDyadicRelation interface, 
ISymmetricalRelation has a method getOther():

   [r.getOther(crazy) for r in rels1]
  [Concept 'mad']

This is all I need to find all the synonyms of 'mad'.

Don't ask me about the implementation now - I just don't care at 
the moment...

 Then most of the relations do have a direction ( __ has author
 __, __ was modified by __,  ...)

Yes, a symmetrical relationship is a special case, and one could 
even handle it on the application layer. Nevertheless I think 
it's a good idea to consider it.

Helmut


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com