Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping
Casey Duncan [EMAIL PROTECTED] writes: I am not arguing necessarily for SQL as a query language for the ZODB. Although it is an accepted standard, but not a perfect one by any means especially for OODBs. Its appeal lies mainly in the high level of community familiarity and the plethora of SQL software to borrow from. Does anyone have an opinion on the possible usefulness of XPath, XQuery, and other XML standards for this? Someone suggested (on the zope-xml wiki) that it would be nice to be able to drop in a cataloger that supported a presumably standard and presumably well-known XML query syntax, and which would work throughout the database because Zope objects would support DOM. This is all speculation, and I personally don't know much right now about XML database interfaces and how finished or well-regarded they are. -- Karl Anderson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
RE: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping
[Karl Anderson]
Casey Duncan [EMAIL PROTECTED] writes:
I am not arguing necessarily for SQL as a query language for the ZODB.
Although it is an accepted standard, but not a perfect one by any means
especially for OODBs. Its appeal lies mainly in the high level of
community familiarity and the plethora of SQL software to borrow from.
Does anyone have an opinion on the possible usefulness of XPath,
XQuery, and other XML standards for this? Someone suggested (on the
zope-xml wiki) that it would be nice to be able to drop in a cataloger
that supported a presumably standard and presumably well-known XML
query syntax, and which would work throughout the database because
Zope objects would support DOM.
This is all speculation, and I personally don't know much right now
about XML database interfaces and how finished or well-regarded they
are.
[Albert]
An excellent introduction to this topic is:
Putting XML in context with hierarchical, relational, and
object-oriented models by David Mertz.
ftp://www6.software.ibm.com/software/developer/library/x-matters8.pdf
Author is a python developer with lots of interesting XML stuff.
See also his xml_matters 1 and 2 for xml_object and xml_pickle with
much nicer pythonic syntax instead of using DOM directly.
Article is also *essential* background for the distinction between
Object Mapping and Object Relational Mapping which needs to be
understood by anyone participating in this discussion.
An example of a python ODBMS with some partial support for OQL is 4ODS
from 4 Suite, which uses a very natural pythonic syntax for objects
stored in and queried from PostgreSQL:
Following is from 4Suite-docs-0.11/4Suite-0.11/html/4ODS-userguide.html
available via:
http://4suite.org/download.html#4Suite_Documentation
How to use the system (a very basic walk through)
First create a ODL file that represents what you want to store in test.odl
module simple {
class person {
attribute string name;
attribute double weight;
relationship Person spouse inverse Person ::spouse_of;
relationship Person spouse_of inverse Person ::spouse;
relationship listPerson children inverse Person ::child_of;
relationship Person child_of inverse Person ::children;
};
class employee (extends person) {
attribute string id;
};
};
Now create a new database and initialize
#OdlParse -ifp test test.odl
Now write some python code to do stuff with these people
#!/usr/bin/python
#Every thing that is persisten must be done inside a transaction and open
database
from Ft.Ods import Database
db = Database.Database()
db.open('test')
tx = db.new()
tx.begin()
#Create a new instance of some objects
import person
import employee
dad = employee.new()
mom = person.new()
son1 = person.new()
son2 = person.new()
daughter = person.new()
#Set some attributes
dad.name = Pops
mom.name = Ma
son1.name = Joey
son2.name = Bobby
daughter.name = Betty
dad.weight = 240.50
#We can set attributes not defined in the ODL but they will not persist
mom.address = 1234 Error Way
#Set some relationships
#First set a one to one relationship
dad.spouse = mom
#Or we could have done it via the ODMG spec
#dad.form_spouse(mom)
#Add some children to the dad (our data model does not let mom have
children. We'd need a family struct (left up to the reader)
dad.add_children(son1)
#We can create relationships both ways
son2.form_child_of(dad)
#Shortcut for adding
dad.children = daughter
#Now root the family to some top level object.
db.bind(dad,The Fam)
#Make it so
tx.commit()
#Out side of a transaction we can still access the objects.
#However, any changes we make will not persist.
#NOTE, because 4ODS caches relationships, any relationships that were not
traversed during the
#transaction, cannot be traversed now because an object cannot be loaded
from the db outside
#of a transaction.
print dad.name
#Start a new tx to fetch
tx = db.new()
tx.begin()
newDad = db.lookup(The Fam)
print newDad.name
print newDad.children[0].name
print newDad.spouse
#Discard this transaction
tx.abort()
Ft/Ods/test_suite and Ft/Ods/demo are good places to look for more examples
^^
See also:
http://www.xml.com/pub/a/2000/10/11/rdf/
Some other relevant references are:
Extraction of DBMS catalogs to XML using python.
http://hyperschema.sourceforge.net./
PostgreSQL as XML repository
http://zvon.org/index.php?nav_id=61
http://hopla.sourceforge.net/doc/README
Note that none of this has much to do with the original topic of
Object-*Relational* Mapping.
*Essential* background for understanding what an object-relational
persistence layer looks like is:
http://www.ambysoft.com/persistenceLayer.html
It isn't very long and there *absolutely* isn't any point discussing
how to design such an OR persistence layer without first reading
and fully understanding it. (I say that after having carefully
studied all the messages in this discussion - though I also said
so before ;-)
The rest of that web site has
Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping
On friday, 11 May, Joachim Warner wrote: The other motivations for an RDBMS are (1) people have existing schemas and want Zope to access the same data as their existing apps, and they want it to be transparent, and (2) tables with millions of entries are easily stored in Zope but the perception is that the catalog isn't as fast as a database index. No one has done any tests AFAIK. Then we should do these tests. E.g. I'd like to see: - 20 GB of Word and PDF documents stored in the ZODB and full-text + metadata indexed in ZCatalog - the complete [EMAIL PROTECTED] mailing list archive in the ZODB If Zope can handle those without the help of external tools (RDBMS etc.), I'll use it for all our Document Management ... As a matter of fact, we did a quick CMF demo that has the content of the zope list, zope-dev, and many of the other zope.org lists, and the comp.lang.python list for the past few years. The catalog searches are very very fast, i can't recall if the demo was set up with some interesting canned CMF topics, but the things works well. The picture isn't altogether rosy - the process of loading the objects was arduous. On the other hand, the exercise (actually, a subsequent one with simpler article objects) served as the basis for tuning the cataloging process, and may have helped it get a lot better. If i have time next week, i'll see if we have the corpus online somewhere. (The lists were complete up to a few months ago.) Eventually we'd like to be incrementally stuffing new messages into the database as they arrive. The catalog has required some substantial work investment, but from my viewpoint (particularly since i haven't had to work on it!-), it's well worth it. Actually OracleStorage and bsddbstorage, recently released, are designed to address concerns about performance and reliability, and they do an excellent job at it. And I consider ZODB as real an OODB as anything else. (In some ways it's the best out there IMHO.) Agreed. ZODB has a much longer proven history of success than most other OODBs. It is quite useful! I have no experience programming with other object databases, and very little with relational databases, so i have no basis for comparison. What i do know, as a python and zope programmer, is that ZODB is spectacularly useful as a persistent store - as flexible *and* as powerful as i could want. The addition of ZEO manages to significantly increase that usefulness! The work we/pythonlabs (and andrew kuchling, etc) is doing to enable use of it as an independent entity can only help improve it's usefulness for everyone. Ken Manheimer [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping
As a matter of fact, we did a quick CMF demo that has the content of the zope list, zope-dev, and many of the other zope.org lists, and the comp.lang.python list for the past few years. The catalog searches are very very fast, i can't recall if the demo was set up with some interesting canned CMF topics, but the things works well. Sounds very promising ... The picture isn't altogether rosy - the process of loading the objects was arduous. What exactly were the problems? I mean, uploading and indexing thousands of documents IS a big deal. I'm sure that any other indexing system will take a lot of time for this, too. Joachim ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping
David Brown wrote: At 11:45 AM 5/11/2001 -0600, Casey Duncan wrote: One of the biggest limitations in my mind is the lack of a general query language for the ZODB like what you get with most OODBMS and all RDBMS. I used to think this as well. But isn't Python a decent query language? Isn't it nice to be able to have all of the facilities of Python at your disposal when manipulating data, rather than hoping that whatever database you are using doesn't have a brain-damaged implementation of SQL? No and yes. Python is a great oop language, it has no inherent querying capabilites though outside of namespace lookups (with acquisition when in Zope) and dictionaries. You would need to build in all possible queries as Python methods none of which would be very general. This is something you need not do with a general query language where you can make arbitrary queries at will. Isn't it nice not to have to convert back and forth between SQL types and native types? Isn't it nice not to have to swap in your SQL mind in the middle of your Python program? I am not arguing necessarily for SQL as a query language for the ZODB. Although it is an accepted standard, but not a perfect one by any means especially for OODBs. Its appeal lies mainly in the high level of community familiarity and the plethora of SQL software to borrow from. Having a general query language makes it easy for people who know that particular general query language to write programs. It makes it easy to access a bunch of different data sources, at least until the monster named implementation differences rears it's ugly head. We've all spent years learning to make our programs interface with databases, learning how to jump the mental chasm between our programs and they way they want to manipulate data, and the way that the database wants to manipulate data. Isn't it nice not to have to do that any more? Yes, I would argue for a tight integration between the query mechanism and regular Python, something that Catalogs have begun to implement. However their query language is a very limiting argument list. Perhaps a more general Python based query language of some kind needs to be developed. Something like an expression that returns the set of objects that meet its criteria. Exactly what this would be would need much community (Python and Zope) discussion. A start to this discussion can be found here: http://dev.zope.org/Wikis/DevSite/Proposals/UnionAndIntersectionOperations Don't get me wrong, I believe I get your point. SQL implementations are getting more and more compatible. There are OODBMS query languages specified. There's no really good way of making different programming languages and programming environments interoperate without some sort of common meeting ground, like a general query language. A be all query language for all databases (Which would be SQL at this point, which is not necessarily tailored to OODBs) will always have compromises and flaws. I look at something like CORBA which is can be used to tie programs of all different languages together. Unfortunately the result of reconsiling COBOL, C, Python, Java, Perl, BASIC, etc. to some common ground is not always very pretty. I would really be happy just to get Python (and possibly C) to be able to perform general ZODB queries without resorting to kludges like spam_usage='range:min:max'. So what I am talking about is not really general, but mostly Python/ZODB specific. And perhaps I'm overdoing the response, perhaps I've gone off in a different direction. I've just been thinking about this quite a bit lately. dave I think this is a very important topic of discussion. Thanks for adding your thoughts! -- | Casey Duncan | Kaivo, Inc. | [EMAIL PROTECTED] `-- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
