[Zope-dev] python scripts
Hi, This weekend I did some work on python scripts and found some things which suprised me, can someone enlighten me on this why these 'strange' things are needed? : 1. When calling a method from a python script you've to do this : (assume x is the value to pass and name is a parameter of mymethod) context.mymethod(name=x) The following doesn't works : context.mymethod(x) (this gives an error) Why 2. Assume you've an sqlmethod which expects 2 parametes, but one is optional : eg select * from invoice dtml-sqlgroup where required dtml-sqlgroup dtml-sqltest customersid column=customers_id op=eq type=int dtml-and dtml-sqltest productsid column=products_id op=eq type=int optional /dtml-sqlgroup /dtml-sqlgroup So, you could run this sql with or without the productsid No, calling from python script doesn't works since it expects: context.mymethod(customerid=..., productsid=...) while context.mymethod(customerid=...) doesn't work.. Now, this implies that I've to write 2 sqlmethods (one with 2 parameters and one with 1), which is pretty stupid, since sqlmethods perfect allow to ignore a parameter when not passed... Can somebody tell me more on this? Why this is the case and perhaps a solution for (2) Thanks in advance, Tom Deprez ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping
From: Joachim Werner [EMAIL PROTECTED]
This is true in the ZODB, but can be complicated by acquisition. If an
object can acquire itself, it can cause issues. Plus it becomes
difficult to know whether objects are clones or just identical
instances, although this can be mitigated by exposing their Python
instance id.
Acquisition is very cool, but it sometimes really sucks ... AFAIK you can
easily switch it off in your own Python products. But I am still
fighting
with only getting private variables (i.e. not acquired ones) in DTML ...
From DTML I have used 2 different methods for this:
1) dtml-with expr=object only
.
/dtml-with
or
2) dtml-if expr=_.hostattr(object.aq_explicit('attribute')
...
/dtml-if
In both cases 'object' is the thing with the 'private variables' and in 2),
'attribute' is the 'variable' name.
Jeff
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
Phillip J. Eby wrote: At 05:42 PM 5/11/01 -0400, Shane Hathaway wrote: Phillip J. Eby wrote: I'm not quite clear on how exactly you suggest mapping from RDMBS - ZODB. There's a *significant* (IMHO) impedance mismatch between ZODB's arbitrarily identified variably structured single records and SQL's content-identified fixed-structure record sets. This is what application frameworks/toolkits (such as your own DBAPI product) are needed for. If you implement this at the Storage level, yes, there is a major mismatch. But at the Connection level it makes a lot of sense. Connection basically exposes a pile of pickles as objects; an OR mapping exposes a complex schema as objects. I think that understanding will change the rest of your response. :-) Nope. As far as I can see, all that does is leverage ZODB Connections as a crude sort of Rack that only provides a mapping-like interface for retrieving objects, without helping any of the higher-order needs of an application. I guess if you define O-R mapping as can you store and retrieve an object's properties from a database, then I guess you've succeeded at that point. But if that was all my O-R apps needed, there would've been no reason to use the RDBMS; I could've just used ZODB and a BTree with less bother. I'm telling you there's a lot more you can do with the code that makes up ZODB. The mapping interface is not the key to the way Connection does its work. OIDs don't have to be strings. If we just use cPersistent, cPickleCache (which is misnamed), and Transaction to implement an OR mapping, here's what we get for free: - A transparent, transactional database. - Potential interoperability with everything written for ZODB. - Robust, tested code. - Optimizations in C. Since you challenged me :-) I decided to put together a prototype. What I came up with was a database connection that sets a different _p_jar for each persistent object. The _p_jar is an object that manages the storage for only one persistent object, making it possible to customize the storage. There was only one hurdle in this approach but I hacked around it: the tpc_* messages get sent to each _p_jar. Then I wrote a very minimal test that doesn't connect to an RDBMS but stores and retrieves simple objects in a dictionary. OIDs are still necessary to support multithreaded invalidation messages. But the OIDs in the prototype are a tuple consisting of a schema ID and a record ID. That way the record IDs only need to be unique within an RDBMS table (or combination of tables.) I didn't think anything like this was possible before I got to know Jim. I still didn't understand when he presented the idea months ago. But now I see the idea really is workable. The advantage is that it lets us completely isolate RDBMS storage details from the application. The next thing to do is to write a fishbowl proposal. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping
Joachim Werner wrote: Probably I'm daft because it is Friday night, but AFAIK ZODB and most OODB's store an object only once, keyed by its object id. The rest is just references through that oid, so objects that belong to more than one container can be added to all these containers and n:m relations are implemented by having a list of objects on both sides. Yes, but these references (let's call them symlinks ...) are not in the standard Zope. Of course it should be easy to do these things, but except for this from Shane I haven't seen anything so far: http://www.zope.org/Members/hathawsh/Symlink/index_html Actually I was referring to the OODB-equivalent of hard links. It's only possible from Python products / external methods / filesystem scripts / Zope core code, but here's how you do it: ob = some_folder.some_child some_other_folder.some_child = ob In other words, if you just assign an object to have multiple parents then it will just work. Changes to the object will appear to occur in both places, but really they're only occurring in one place. This behavior is fully persistent, works over ZEO, and ZODB won't even see there's anything unusual. P.S.: Shane, have you developed Symlinks any further? I think they could be extremely useful. I tried out the initial release and liked it, except for the fact that the symlinks looked EXACTLY like real ones, so they can be very irritating ... I'm not sure what to do with symlinks. How should security be applied? How are they most useful? It's neat to see it works, though. :-) Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping
From: Joachim Werner [EMAIL PROTECTED]
This is true in the ZODB, but can be complicated by acquisition. If an
object can acquire itself, it can cause issues. Plus it becomes
difficult to know whether objects are clones or just identical
instances, although this can be mitigated by exposing their Python
instance id.
Acquisition is very cool, but it sometimes really sucks ... AFAIK you can
easily switch it off in your own Python products. But I am still
fighting
with only getting private variables (i.e. not acquired ones) in DTML ...
From DTML I have used 2 different methods for this:
1)
dtml-with expr=object only
.
/dtml-with
or
2)
dtml-if expr=_.hostattr(object.aq_explicit('attribute')
...
/dtml-if
In both cases 'object' is the thing with the 'private variables' and in 2),
'attribute' is the 'variable' name.
Jeff
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
Shane Hathaway wrote: I'm telling you there's a lot more you can do with the code that makes snip The next thing to do is to write a fishbowl proposal. This sounds cool but made my head hurt :-S Can you try and bring this back down to the level of us mere mortals by explaining how your OR stuff would let me take a table of data in an RDBMS table and have it appear as objects in the Management Inteferace? I know that's horribly oversimplified, but some of us only have small brains ;-) cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping
P.S.: Shane, have you developed Symlinks any further? I think they could be extremely useful. I tried out the initial release and liked it, except for the fact that the symlinks looked EXACTLY like real ones, so they can be very irritating ... I'm not sure what to do with symlinks. How should security be applied? How are they most useful? It's neat to see it works, though. :-) Shane Last year I needed symlinks and developed a version of your product where the symlink had a different id than the real object. But the general solution of a symlink where an arbitrary set of attributes are from the symlink instance instead of from the real object was too much difficult for me, so this year we've changed the dessign of the application and don't need symlinks anymore. jdavid ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
Chris Withers wrote: Shane Hathaway wrote: I'm telling you there's a lot more you can do with the code that makes snip The next thing to do is to write a fishbowl proposal. This sounds cool but made my head hurt :-S Can you try and bring this back down to the level of us mere mortals by explaining how your OR stuff would let me take a table of data in an RDBMS table and have it appear as objects in the Management Inteferace? Sorry, this is at a pretty low level and I do need to explain it better. One would define an ObjectMappingSchema whose job it is to store and retrieve objects of a specific type and in a specific location. It would usually grab a database connection object to do its work. When loading, it would perform a query then manually put attributes into a persistent object. When storing, it would grab specific attributes from the persistent object and execute a statement to store those attributes. So let's say you want a ZODB to store and retrieve users in a specific table while putting everything else in pickles. You would create an instance of PickleSchema, which implements the ObjectMappingSchema interface, and tell it to manage everything *except* the users mapping in BasicUserFolder objects. You would tell it to store and retrieve this object using your UserFolderSchema instead. Your UserFolderSchema would store and retrieve the users from the USERS and USER_PREFS tables. The user table wouldn't require the use of OIDs but would require unique user IDs. So in the management interface nothing would change. Nor would the application-level Python code. You would only define a layer that maps objects to a relational database. You would still see the user folder as you do now. Now, it may be useful to provide a management interface for defining the schema mapping. I haven't approached that yet; AFAICT this is where the work done on SmartObjects and DBObjects would be very useful. Initially I was planning for people to code the mapping purely in Python so we could gain experience and find common patterns before inventing a UI. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
Shane Hathaway wrote: One would define an ObjectMappingSchema whose job it is to store and retrieve objects of a specific type and in a specific location. It would usually grab a database connection object to do its work. When loading, it would perform a query then manually put attributes into a persistent object. When storing, it would grab specific attributes from the persistent object and execute a statement to store those attributes. How does this differ from ZPatterns? Phil? Now, it may be useful to provide a management interface for defining the schema mapping. I haven't approached that yet; AFAICT this is where the work done on SmartObjects and DBObjects would be very useful. Initially I was planning for people to code the mapping purely in Python so we could gain experience and find common patterns before inventing a UI. Sounds good to me :-) Thanks for the explanation... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
I think this is a great idea! I would definetely like to use and contribute to this effort. Having this kind of flexibily would be fantastic. After demonstratable Python code is working I would request that usability issues (UI Schema mapper, data migration/schema evolution tools, ZEO integration, multi-Storage uses) be addressed sooner than later. John On Monday 14 May 2001 10:47, Shane Hathaway wrote: Chris Withers wrote: Shane Hathaway wrote: I'm telling you there's a lot more you can do with the code that makes snip The next thing to do is to write a fishbowl proposal. This sounds cool but made my head hurt :-S Can you try and bring this back down to the level of us mere mortals by explaining how your OR stuff would let me take a table of data in an RDBMS table and have it appear as objects in the Management Inteferace? Sorry, this is at a pretty low level and I do need to explain it better. One would define an ObjectMappingSchema whose job it is to store and retrieve objects of a specific type and in a specific location. It would usually grab a database connection object to do its work. When loading, it would perform a query then manually put attributes into a persistent object. When storing, it would grab specific attributes from the persistent object and execute a statement to store those attributes. So let's say you want a ZODB to store and retrieve users in a specific table while putting everything else in pickles. You would create an instance of PickleSchema, which implements the ObjectMappingSchema interface, and tell it to manage everything *except* the users mapping in BasicUserFolder objects. You would tell it to store and retrieve this object using your UserFolderSchema instead. Your UserFolderSchema would store and retrieve the users from the USERS and USER_PREFS tables. The user table wouldn't require the use of OIDs but would require unique user IDs. So in the management interface nothing would change. Nor would the application-level Python code. You would only define a layer that maps objects to a relational database. You would still see the user folder as you do now. Now, it may be useful to provide a management interface for defining the schema mapping. I haven't approached that yet; AFAICT this is where the work done on SmartObjects and DBObjects would be very useful. Initially I was planning for people to code the mapping purely in Python so we could gain experience and find common patterns before inventing a UI. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) -- . . . . . . . . . . . . . . . . . . . . . . . . John D. Heintz | Senior Engineer 1016 La Posada Dr. | Suite 240 | Austin TX 78752 T 512.633.1198 | [EMAIL PROTECTED] w w w . d a t a c h a n n e l . c o m ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
Chris Withers wrote: Shane Hathaway wrote: One would define an ObjectMappingSchema whose job it is to store and retrieve objects of a specific type and in a specific location. It would usually grab a database connection object to do its work. When loading, it would perform a query then manually put attributes into a persistent object. When storing, it would grab specific attributes from the persistent object and execute a statement to store those attributes. How does this differ from ZPatterns? Phil? ZPatterns implements storage logic on the application level. Applications have to be aware of (in fact they have to be centered around) ZPatterns. This alternate approach keeps storage logic independent of application logic. It lets you take any code written for the ZODB and move it to your RDBMS schema without changing the code (most of the time :-) ). Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
Shane Hathaway wrote: ZPatterns implements storage logic on the application level. Applications have to be aware of (in fact they have to be centered around) ZPatterns. This alternate approach keeps storage logic independent of application logic. It lets you take any code written for the ZODB and move it to your RDBMS schema without changing the code (most of the time :-) ). Now that sounds really f^%$^ing cool :-) How can I help? Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
Chris Withers wrote: Shane Hathaway wrote: ZPatterns implements storage logic on the application level. Applications have to be aware of (in fact they have to be centered around) ZPatterns. This alternate approach keeps storage logic independent of application logic. It lets you take any code written for the ZODB and move it to your RDBMS schema without changing the code (most of the time :-) ). Now that sounds really f^%$^ing cool :-) How can I help? For now, I can make public the code I wrote for experimentation: http://www.zope.org/Members/hathawsh/orconn.tar.gz The archive contains two Python files and a patch. Put them in the ZODB directory. Perform the patch to DB.py. (It's very minimal.) Then run testOR.py. It just changes a mapping, but it does it transparently and transactionally. :-) I need to fishbowl this before going any further. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Disabling anonymous webdav access
Hi All,
As someone pointed out on #zope, it is possible to view folder contents
using a webdav client as an anonymous user.
I.e. download cadaver (http://www.webdav.org/cadaver/), open
yourzopeserver:8080 and do ls. Then decide if you want anyone to be
able to access this. Eventhough hiding this information may be security
by obscurity, there are some things you just don't want everyone to see.
This allows you to see, for example, the installed products on the server.
A hacker might use this knowledge to exploit some known bug in a zope product
if one exists.
Most people (like me) probably think it's harmless to let old
objects, documents etc linger around as you can't view them in listings
through ftp or http. They don't realize webdav is running by default. Actually,
it can't even be disabled! (z2.py -X -w80 won't do the trick!)
Personally I'd rather see this secured. It's not possible to disable
'view contents information' for anonymous users in zope, as this will ruin
your entire site (all anonymous access will then be disabled), so the solution
would be to create a new permission for access contents through webdav.
And that's what the following (trivial) patch does.
After applying you'll get a new permission in your security tab, which
is set to manager by default. To get the old behaviour back, just set the
permission back to anonymous.
Apply it using patch -p1 ../webdav.patch in your SOFTWARE_HOME (i.e. the
Zope-2.3.2-src dir).
Or just edit lib/python/webdav/Resource.py by hand :)
I've tested it with Zope 2.3.2, I can't guarantee it will work with other
versions (use at your own risk anyway).
-- cut here --
*** Zope-2.3.2-orig/lib/python/webdav/Resource.py Tue Mar 27 21:50:37 2001
--- Zope-2.3.2-src/lib/python/webdav/Resource.pyMon May 14 19:16:46 2001
***
*** 109,115
__ac_permissions__=(
('View', ('HEAD',)),
! ('Access contents information', ('PROPFIND',)),
('Manage properties',('PROPPATCH',)),
('Delete objects', ('DELETE',)),
)
--- 109,115
__ac_permissions__=(
('View', ('HEAD',)),
! ('Access contents information through WebDav', ('PROPFIND',)),
('Manage properties',('PROPPATCH',)),
('Delete objects', ('DELETE',)),
)
-- cut here --
Cheers,
Ivo
--
Drs. I.R. van der Wijk -=-
Brouwersgracht 132 Amaze Internet Services V.O.F.
1013 HA Amsterdam -=-
Tel: +31-20-4688336 Linux/Web/Zope/SQL
Fax: +31-20-4688337 Network Solutions
Web: http://www.amaze.nl/Consultancy
Email: [EMAIL PROTECTED] -=-
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
At 12:26 PM 5/14/01 -0400, Shane Hathaway wrote: Chris Withers wrote: Shane Hathaway wrote: One would define an ObjectMappingSchema whose job it is to store and retrieve objects of a specific type and in a specific location. It would usually grab a database connection object to do its work. When loading, it would perform a query then manually put attributes into a persistent object. When storing, it would grab specific attributes from the persistent object and execute a statement to store those attributes. How does this differ from ZPatterns? Phil? ZPatterns implements storage logic on the application level. Applications have to be aware of (in fact they have to be centered around) ZPatterns. This alternate approach keeps storage logic independent of application logic. It lets you take any code written for the ZODB and move it to your RDBMS schema without changing the code (most of the time :-) ). I am reminded of a passage from Dirk Gently's Holistic Detective Agency, wherein Dirk decides to cut to the heart of the problem, and simply writes down the answer. Now, he declares, he is faced with only a relatively much easier problem: What strange language did he write the answer down in? While I'm not saying that what you're doing isn't possible, I *will* say that I believe you are replacing a small amount of API-specificity in the code with a similar amount of specificity, coupled with a much larger complexity in the mapping layers. It is, IMHO, much harder to write generic mappings at the schema layer where you propose, than to do so at the domain logic layer where ZPatterns operates. Ty and I spent many weeks chewing over designs in the space you're talking about, back before ZPatterns existed, after consulting at length with Jim Fulton and Michel Pelletier. Our conclusion was that yes, it was *possible* to do the mapping you're talking about, but that it was very high impedance for the kinds of applications we had in mind. I'll give a simple example to show what I'm talking about. Consider a task framework where TaskPerformers are assigned Tasks, in a simple one-to-many relationship. TaskPerformers, in this framework, perform up to hundreds of tasks per day, thousands per month. Assuming an RDBMS backend, how do you propose to map this in your storage model? My thought on this, is that you will be forced to explicitly consider the nature of this relationship and its storage at the application level. If you write explicitly for ZODB, you might use a BTree, for example. Or perhaps you'd have some kind of global container for the Tasks, with a Catalog to index them, and a method on TaskPerformer to query the Catalog. How would you propose to map this to an RDMS? Well, substituting catalog queries might be relatively straightforward, but perhaps rather time consuming to develop in a generic way. The BTree might give you a bit more trouble. Probably you'd have to end up using some sort of generic queryable containers that translated to BTrees or Catalogs or RDBMS tables, depending... And lo! You now have an application framework. Oops. We haven't even addressed performance issues yet, which are the real killer. The only way (again IMHO) to get reasonably high performance that is portable across different databases is to be able to write code that is optimized based on the back-end. The only way you can push this code under the hood is to create an all-purpose query mechanism. Otherwise, you must expose enough of the inner workings to the application developer that they can include platform-specific code for each circumstance. All this is of course just my opinion, and quite possibly wrong. But I think that an application developer would get more mileage out of your DBAPI product (coupled with some utility classes to replace Racks) or from ZPatterns than they would from this approach to O-R mapping. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping
Casey Duncan [EMAIL PROTECTED] writes: I am not arguing necessarily for SQL as a query language for the ZODB. Although it is an accepted standard, but not a perfect one by any means especially for OODBs. Its appeal lies mainly in the high level of community familiarity and the plethora of SQL software to borrow from. Does anyone have an opinion on the possible usefulness of XPath, XQuery, and other XML standards for this? Someone suggested (on the zope-xml wiki) that it would be nice to be able to drop in a cataloger that supported a presumably standard and presumably well-known XML query syntax, and which would work throughout the database because Zope objects would support DOM. This is all speculation, and I personally don't know much right now about XML database interfaces and how finished or well-regarded they are. -- Karl Anderson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] (LONG) RE: [SmartObjects] Wrap-up of the discussion going on on zope-dev?
[Joachim Werner] As most of you might have recognized, there is a very active thread (actually two interwoven ones) about RO-mapping etc. going on on zope-dev. I'd be very happy if someone could wrap up the stuff from there on the SmartObjects list. ;-) [Albert] Thanks for the pointer. As I read them all through at once, after seeing your pointer, I made notes before joining the discussion, which may help, though far from a wrap-up. * SUMMARY * There seem to be 6 separate topics: 1) Nature of SmartObjects, ZPatterns and Transwarp frameworks and relation to Zope/ZODB framework. 2) Improvements to ZCatalog search efficiency 3) Query syntax - see Zwiki: http://dev.zope.org/Wikis/DevSite/Proposals/UnionAndIntersectionOperations 4) Schema transparency (an oxymoron?) 5) Storage of ZODB objects in separate DBMS tables per class - Object Mapping. 6) Requirements for Object Relational Mapping. * DETAILED NOTES * Here's a list of all the items I found in the May archives with the term ORMapping in the subject line, from thread Experiments in ORMapping and interwoven thread oodb philosophics was [above] until I subscribed. Will comment later on subsequent messages rather than making notes. If there are any other subject headings missed, or other postings (including earlier months), please let me know. Highlights before links attempt to be objective but are limited by space, by my specific interests in the topic and by my lack of thorough understanding of Zope internals and consequent previous non-participation in [zope-dev] with consequent lack of appreciation of where various posters are coming from. Quotes following links are verbatim extracts (not necessarily in same order or in context). Must read means important message with no or inadequate highlights or extracts given here. I have omitted numerous affirmations that ZODB meets most current Zope requirements (perhaps with improvements to ZCatalog) in view of the fact that nobody appeared to be arguing otherwise. BTW the date order used in list archives could be better in numeric order as that might more closely reflect which messages are likely to have been seen before replying. I have re-arranged. Starting from 10 May 2001: Shane Hathaway (DC) - start of discussion - must read http://lists.zope.org/pipermail/zope-dev/2001-May/011101.html includes link http://www.zope.org/Members/hathawsh/ormapping.tar.gz overview in file: sketch Tino Wildenhain - asks about ZPatterns, ZClasses, doesn't want purely relational application server. http://lists.zope.org/pipermail/zope-dev/2001-May/011102.html SH - answers TW, says not hard to implement by mapping classes to a table. http://lists.zope.org/pipermail/zope-dev/2001-May/011103.html TW - answers SH, prefers improved OODB because doesn't like mapping classes to schema. http://lists.zope.org/pipermail/zope-dev/2001-May/011105.html SH - answers TW, enhanced ZODB storage (RDBMS and BerkelyDBM) is in some ways the best OODB. http://lists.zope.org/pipermail/zope-dev/2001-May/011107.html The other motivations for an RDBMS are (1) people have existing schemas and want Zope to access the same data as their existing apps, and they want it to be transparent, and (2) tables with millions of entries are easily stored in Zope but the perception is that the catalog isn't as fast as a database index. No one has done any tests AFAIK. [...] That's one reason ZODB is so nice. You can write an application without writing a formal schema. Casey Duncan (Kaivo) - suggests Matisse or Objectivity but limited by not supporting ZODB versions. BerkelyDBM best soon. Mentions slow XML storage startup. http://lists.zope.org/pipermail/zope-dev/2001-May/011108.html JW - already 2 OR mapping projects, SmartObjects and TransWarp, no duplication because Phillip from Transwarp participating in SmartObjects list. http://lists.zope.org/pipermail/zope-dev/2001-May/011109.html TW - answers CD, XML just another pickle format. http://lists.zope.org/pipermail/zope-dev/2001-May/00.html SH - answers JW. SmartObjects, ZPatterns and Transwarp require new database API instead of maintaining transparency like ZODB. Projects should look at replacing parts of ZODB instead of adding complexity. http://lists.zope.org/pipermail/zope-dev/2001-May/01.html ZODB has pieces that can be split apart and replaced as needed, such as caching, persistence, transactions, the pickle jar, the multi-threaded connection factory, and the storage layer. I'm hoping we can achieve OR mapping by only replacing the pickle jar, i.e. Connection.py. Cees de Groot - answers SH. Confirms advantage of not having to write formal schema, migrating from PostgreSQL to ZODB for that. File Storage faster than Oracle and is basic transaction log so nothing could be more reliable. http://lists.zope.org/pipermail/zope-dev/2001-May/011124.html Are people using ZODB for non-Zope data? I'd be very interested to discuss things like emulating extents, patterns for indexing,
RE: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping
[Karl Anderson]
Casey Duncan [EMAIL PROTECTED] writes:
I am not arguing necessarily for SQL as a query language for the ZODB.
Although it is an accepted standard, but not a perfect one by any means
especially for OODBs. Its appeal lies mainly in the high level of
community familiarity and the plethora of SQL software to borrow from.
Does anyone have an opinion on the possible usefulness of XPath,
XQuery, and other XML standards for this? Someone suggested (on the
zope-xml wiki) that it would be nice to be able to drop in a cataloger
that supported a presumably standard and presumably well-known XML
query syntax, and which would work throughout the database because
Zope objects would support DOM.
This is all speculation, and I personally don't know much right now
about XML database interfaces and how finished or well-regarded they
are.
[Albert]
An excellent introduction to this topic is:
Putting XML in context with hierarchical, relational, and
object-oriented models by David Mertz.
ftp://www6.software.ibm.com/software/developer/library/x-matters8.pdf
Author is a python developer with lots of interesting XML stuff.
See also his xml_matters 1 and 2 for xml_object and xml_pickle with
much nicer pythonic syntax instead of using DOM directly.
Article is also *essential* background for the distinction between
Object Mapping and Object Relational Mapping which needs to be
understood by anyone participating in this discussion.
An example of a python ODBMS with some partial support for OQL is 4ODS
from 4 Suite, which uses a very natural pythonic syntax for objects
stored in and queried from PostgreSQL:
Following is from 4Suite-docs-0.11/4Suite-0.11/html/4ODS-userguide.html
available via:
http://4suite.org/download.html#4Suite_Documentation
How to use the system (a very basic walk through)
First create a ODL file that represents what you want to store in test.odl
module simple {
class person {
attribute string name;
attribute double weight;
relationship Person spouse inverse Person ::spouse_of;
relationship Person spouse_of inverse Person ::spouse;
relationship listPerson children inverse Person ::child_of;
relationship Person child_of inverse Person ::children;
};
class employee (extends person) {
attribute string id;
};
};
Now create a new database and initialize
#OdlParse -ifp test test.odl
Now write some python code to do stuff with these people
#!/usr/bin/python
#Every thing that is persisten must be done inside a transaction and open
database
from Ft.Ods import Database
db = Database.Database()
db.open('test')
tx = db.new()
tx.begin()
#Create a new instance of some objects
import person
import employee
dad = employee.new()
mom = person.new()
son1 = person.new()
son2 = person.new()
daughter = person.new()
#Set some attributes
dad.name = Pops
mom.name = Ma
son1.name = Joey
son2.name = Bobby
daughter.name = Betty
dad.weight = 240.50
#We can set attributes not defined in the ODL but they will not persist
mom.address = 1234 Error Way
#Set some relationships
#First set a one to one relationship
dad.spouse = mom
#Or we could have done it via the ODMG spec
#dad.form_spouse(mom)
#Add some children to the dad (our data model does not let mom have
children. We'd need a family struct (left up to the reader)
dad.add_children(son1)
#We can create relationships both ways
son2.form_child_of(dad)
#Shortcut for adding
dad.children = daughter
#Now root the family to some top level object.
db.bind(dad,The Fam)
#Make it so
tx.commit()
#Out side of a transaction we can still access the objects.
#However, any changes we make will not persist.
#NOTE, because 4ODS caches relationships, any relationships that were not
traversed during the
#transaction, cannot be traversed now because an object cannot be loaded
from the db outside
#of a transaction.
print dad.name
#Start a new tx to fetch
tx = db.new()
tx.begin()
newDad = db.lookup(The Fam)
print newDad.name
print newDad.children[0].name
print newDad.spouse
#Discard this transaction
tx.abort()
Ft/Ods/test_suite and Ft/Ods/demo are good places to look for more examples
^^
See also:
http://www.xml.com/pub/a/2000/10/11/rdf/
Some other relevant references are:
Extraction of DBMS catalogs to XML using python.
http://hyperschema.sourceforge.net./
PostgreSQL as XML repository
http://zvon.org/index.php?nav_id=61
http://hopla.sourceforge.net/doc/README
Note that none of this has much to do with the original topic of
Object-*Relational* Mapping.
*Essential* background for understanding what an object-relational
persistence layer looks like is:
http://www.ambysoft.com/persistenceLayer.html
It isn't very long and there *absolutely* isn't any point discussing
how to design such an OR persistence layer without first reading
and fully understanding it. (I say that after having carefully
studied all the messages in this discussion - though I also said
so before ;-)
The rest of that web site has
Re: [Zope-dev] Experiments with ORMapping
Now, it may be useful to provide a management interface for defining the schema mapping. I haven't approached that yet; AFAICT this is where the work done on SmartObjects and DBObjects would be very useful. Initially I was planning for people to code the mapping purely in Python so we could gain experience and find common patterns before inventing a UI. Our initial plans for SmartObjects (if I get Stephan right) are to define all objects in Python code (as DBObjects do now), but to also provide two alternative ways of creating the necessary code: a) by importing and compiling an XML object description (which might come from a UML tool) b) via a ZMI interface similar to the ZClass GUI (which may or may not use the XML schemas, depending on the actual implementation) Of course it would be necessary to dynamically refresh the newly generated filesystem-based code. The other issue that applies in general to all filesystem-based code is that it is not directly available over ZEO as it is not in the ZODB. But that should not be a very large problem ... ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
My thought on this, is that you will be forced to explicitly consider the nature of this relationship and its storage at the application level. If you write explicitly for ZODB, you might use a BTree, for example. Or perhaps you'd have some kind of global container for the Tasks, with a Catalog to index them, and a method on TaskPerformer to query the Catalog. How would you propose to map this to an RDMS? Well, substituting catalog queries might be relatively straightforward, but perhaps rather time consuming to develop in a generic way. The BTree might give you a bit more trouble. Probably you'd have to end up using some sort of generic queryable containers that translated to BTrees or Catalogs or RDBMS tables, depending... And lo! You now have an application framework. Oops. I'll try to somehow get your and Shane's opinions together a bit: I think Shane's approach does most of the job (of OR mapping) if the objects we are talking about are relatively simple and the application domain is relatively typical for what Zope can do well. E.g. the composite objects we have in mind for the groupware (user credentials coming from LDAP, rest in the ZODB, or file stored in the filesystem metadata in SQL DB, rest in ZODB) can easily be modelled this way. And the existing Zope API (manage_AddX, manage_changeProperties etc.) would do most of the job. But the problems start with querying. Can the ZCatalog do it? Does it make sense to implement a SQLZCatalog instead of just offering native SQL query objects or an abstract query language like SkinSkript? To put it simply: The reason why we want to develop a framework (or framework extension) is that the Zope framework just doesn't do the stuff we need yet. BTW, the reason why we didn't just use ZPatterns was very similar: ZPatterns makes it easy to write an abstract implementation of a project, but you still have to write all the glue code between the storage and the application layer by hand. What we want (and what TransWarp is about, with a slightly different focus) is that we can just map an attribute to a data source and all the glue code (either ZODB or SQL statements) is auto-generated. If the new framework just extends the existing Zope API, I don't see a problem. ZPatterns did not, but maybe it is just not that easy to do so (and not all parts of the Zope API really should bear this name ...). The really good thing about the whole discussion is that at the end we will see much better where we should put things. We (the SmartObjects people at iuveno) do not really want to have a completely new world of SmartObjects that will require a new API to learn. We just want to add new functionality to Zope, be it in the core or as mix-in classes, or through-the-web products. Joachim ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiments with ORMapping
My thought on this, is that you will be forced to explicitly consider the nature of this relationship and its storage at the application level. If you write explicitly for ZODB, you might use a BTree, for example. Or perhaps you'd have some kind of global container for the Tasks, with a Catalog to index them, and a method on TaskPerformer to query the Catalog. How would you propose to map this to an RDMS? Well, substituting catalog queries might be relatively straightforward, but perhaps rather time consuming to develop in a generic way. The BTree might give you a bit more trouble. Probably you'd have to end up using some sort of generic queryable containers that translated to BTrees or Catalogs or RDBMS tables, depending... And lo! You now have an application framework. Oops. I'll try to somehow get your and Shane's opinions together a bit: I think Shane's approach does most of the job (of OR mapping) if the objects we are talking about are relatively simple and the application domain is relatively typical for what Zope can do well. E.g. the composite objects we have in mind for the groupware (user credentials coming from LDAP, rest in the ZODB, or file stored in the filesystem metadata in SQL DB, rest in ZODB) can easily be modelled this way. And the existing Zope API (manage_AddX, manage_changeProperties etc.) would do most of the job. But the problems start with querying. Can the ZCatalog do it? Does it make sense to implement a SQLZCatalog instead of just offering native SQL query objects or an abstract query language like SkinSkript? To put it simply: The reason why we want to develop a framework (or framework extension) is that the Zope framework just doesn't do the stuff we need yet. BTW, the reason why we didn't just use ZPatterns was very similar: ZPatterns makes it easy to write an abstract implementation of a project, but you still have to write all the glue code between the storage and the application layer by hand. What we want (and what TransWarp is about, with a slightly different focus) is that we can just map an attribute to a data source and all the glue code (either ZODB or SQL statements) is auto-generated. If the new framework just extends the existing Zope API, I don't see a problem. ZPatterns did not, but maybe it is just not that easy to do so (and not all parts of the Zope API really should bear this name ...). The really good thing about the whole discussion is that at the end we will see much better where we should put things. We (the SmartObjects people at iuveno) do not really want to have a completely new world of SmartObjects that will require a new API to learn. We just want to add new functionality to Zope, be it in the core or as mix-in classes, or through-the-web products. Joachim ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] manage_workspace = index_html
On Mon, 14 May 2001, The Doctor What wrote: What's going on? Is this a Mozilla problem? Yes. If so, why is lynx doing the same? Because lynx is broken too. How should I go about trouble shooting it? The problem is the client does not provide the right Basic authentication credentials for the frame like it's supposed to. Zope then assumes that you are just an anonymous user, and shows you the only 'managment tab' you have permission to see, 'View'. If I recall, this have been broken an fixed in mozilla a few times now. Other browsers do this wrong too, like w3m. -Michel ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Proposal: AuthenticatedRole
Hi all, If you're concerned or curious, please review this proposal on the fishbowl regarding adding a default role to Zope named 'Authenticated': http://dev.zope.org/Wikis/DevSite/Proposals/AuthenticatedRole ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
