RE: [Zope-dev] Boost.Python
[...James Treleaven] > but > are there any complications with respect to my using Boost.Python to bind > any Zope Python scripts that I write to C++ code? [Michel] You've _really_ lost me, maybe someone else knows what you mean. I don't know what you mean by 'bind' (do you mean writing python extension modules in C?), or what Boost.Python is, or what you are referring to as 'Zope Python scripts' (do you mean the python modules that come with Zope?). [Albert] I don't know the answers to James' questions but here's the links for Boost Python: http://www.boost.org/libs/python/doc/index.html and comparison with Zope Extension Classes and other approaches: http://www.boost.org/libs/python/doc/comparisons.html Looks very interesting. Check it out. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] ANNOUNCE: Zope 2.4.0 alpha 1 released
[Brian] Zope 2.4.0 alpha 1 has been released - you can download it from Zope.org: http://www.zope.org/Products/Zope/2.4.0a1/ [...] Note that this is an alpha release, so it is available as a source distribution only. We will make binary releases available with the first beta release. [Albert] Congratulations!!! Two questions: 1. Any guesstimate (vague upper and lower bounds) on first beta release for the binaries, (or even just a vague lower bound - "probably not before ...")? 2. In the Migration Guide: "A note to component developers - as of Zope 2.4 ExtensionClass has not been updated to support all of the new "magic protocols" that Python classes support (we're hoping that EC will go away soon)." What does "hoping that EC will go away soon" mean? I thought that ExtensionClass was pretty fundamental to Zope?? Does this just mean that the limitation on "magic protocols" will go away soon, or is there some URL I should look at re replacement for EC? Thanks, and congratulations again. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
RE: Randomness (RE: [Zope-dev] CoreSessionTracking 0.8)
It's obvious. This is just Zope's way of telling you not live on hamburgers and coke. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Bjorn Stabell Sent: Thursday, May 24, 2001 12:33 AM To: Chris McDonough; Howard Zhang Cc: [EMAIL PROTECTED] Subject: Randomness (RE: [Zope-dev] CoreSessionTracking 0.8) Allright, let me try again. I wish I had a small piece of code to give you so you can reproduce it, but right now you'd have to get our entire CMF-based website. The bug basically manifests itself in that there are two versions of the variable we put in the session (a shopping cart dict). When I browse through the site (not even updating the shopping cart) it'll show one version for some links (1-40) before it switches to show the other, and so on. It looks like the website has two shopping carts that it switches back and forth between. You can see the shopping cart on every page in the website (it's embedded into the template). We were using frames, but I tried it several times without frames now and the bug remains. I even noticed that other variables disappeared randomly as well, e.g., USER_PREF_LANGUAGES which is set by the Localizer, resulting in a key error (I've probably seen 300 pages views, and then suddenly one going back to another page gives a key error?). I'm very curious what could possibly be causing such problems. I thought there might be something wrong in the shared memory between threads, as I can't see anything else changing but the threads (is there a way to display which thread is doing the publishing?). I've seen similar randomness displayed in other situations where I've been reloading pages that would sometimes (same interval, about every 1-40 times) show one character set, and other times another. I think nobody likes to see that kind of randomness. It gives me a very bad stomach feeling. I definately think it's something deeper than a CoreSessionTracking problem. Bye, -- Bjorn Stabell <[EMAIL PROTECTED]> -Original Message- From: Chris McDonough [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 23, 2001 20:34 To: Howard Zhang Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; Exoweb Subject: Re: [Zope-dev] CoreSessionTracking 0.8 I remember this problem, but I haven't been able to reproduce it. But maybe it's because I'm not understanding the steps to reproduce it. The sentence "user adds coke to shopping cart and click link to add coke again before request finished" is hard to understand. Can you explain? Are you using frames? Howard Zhang wrote: > > Hi > The problem about CoreSessionTracking we describe before we can > repeat again now. > The step is: >( 1 ) User adds Burger to shopping cart >( 2 ) User adds coke to shopping cart and click link to add coke > again before request finished >( 3 ) The Burger is disappear in shopping cart and just one coke > ( not two ) >( 4 ) Repeat the step 2,Burger is back > > Anything you could tell me would be helpful. > > Regards, > > howard > > ___ > Zope-Dev maillist - [EMAIL PROTECTED] > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) winmail.dat
[Zope-dev] RE: [Zope] xmlrpc slowness
[Shilad] The new release is up on sourceforge. It should be compatible with the Zope client/server it was tested against. It is at: http://www.sourceforge.net/projects/py-xmlrpc Let me know how it goes. I'm curious to see what kind of speed increase you see. My guess is that the implementation at the other xmlrpc end will be the bottle-neck pretty soon. [Albert] Thanks *very* much for this!!! 1000 xml-rpc calls per second with commodity hardware sounds pretty attractive! I've had to just pass it on to a friend to checkout at the moment, but will try to get back to you soon with speed results, as I'm sure the other 3 CCs will. I've added the Zope list back to the addresses as there may be others there interested in trying it out now that Phil Harris has confirmed the new version interworks correctly with the Zope implementation. Also added [Zope-dev] list as Zope developers should be more interested and Matt Hamilton as issues below may be closely related to his posting "[Zope-dev] Asyncore in an external method". http://lists.zope.org/pipermail/zope-dev/2001-May/011274.html In a previous email CC you said: "I'm really excited that zope people may be using this. Let me know if you have any questions/concerns/requests." As you've also done binary releases for Windows and the various unixen that Zope does releases for, it should be suitable for actually integrating into Zope rather than just as an add-on "Product" or it's present form as just a separate python process talking to Zope rather than implementing Zope's xml-rpc itself. So here goes with some questions/concerns/requests... In the README you mention: "Some time I should document everything better, especially the nonblocking aspects of the server and client." and " * Non-Blocking IO: This becomes important when you have critical applications (especially servers) that CANNOT be held up because a client is trickling a request in or is reading a response at 1 byte per second. * Event based: This goes hand in hand with non-blocking IO. It lets you use an event based model for your client/server application, where the main body is basically the select loop on the file descriptors you are interested in. The work is done by registering callbacks for different events you received on file descriptors. This can be very nice. There is some work to be done here, but all the hooks are in place. See TO DO for a list of the features that have yet to be incorporated." (BTW there is currently no "TO DO" file.) These aspects are a very big advantage. For Zope as a client, I suspect that the trickling issue may also be very important since it could be blocking an expensive Zope thread while waiting for a response from a slow remote server to pass on some information in response to relayed request (ie http/xmlrpc relay mode or even straight xmlrpc/xmlrpc proxy mode). As you also mentioned, "the implementation at the other xmlrpc end will be the bottle-neck pretty soon". Sorry I'm not very familiar with how to do this stuff myself, so I have 3 questions which maybe you could answer when you get around to doing those docs (or even better by also providing implementations in the next release ;-) or perhaps someone else on CC list knows the answers or is planning to do something about it? 1) I'm not sure if I've got it right, but my understanding is that despite being based on a Medusa Reactor design that can handle many web hits with a single thread, Zope also maintains a small pool of threads to allow for (brief) blocking on calls to the file system for ZODB and for (also brief) external connections. I suspect these threads are "expensive" because they each end up keeping separate copies of cached web pages etc (to avoid slow python thread switching). So simply increasing the number of such threads is not recommended for improving Zope performance - performance relies on the non-blocking async Medusa Reactor design of Zserver, not on the threading, which just seems to be a sort of extra workaround. If that is correct, then a few concurrent external calls to slow external xmlrpc servers (eg for credit card authorization taking 30 seconds or more) could easily tie up a lot of Zope resources. The non-blocking py-xmlrpc client could presumably surrender it's turn in the main event loop for it's thread until a response is received and then be woken up by the response, thus improving things greatly. Unfortunately I have no idea how to do this - whether it would just happen automatically or there are built in facilities for doing that sort of thing easily in Zope already, or whether it is difficult to do. I am just guessing that there would be some special tricks needed to wake up a channel when a response comes back (eg using the stuff in Zserver/medusa/select_trigger.py and Zserver/PubCore/. which I don't fully understand, but looks relevant). Maybe I have misunderstood, but it looks to me like existing use of xmlrpc clients *from* Zope to
RE: [Zope-dev] OR mapping proposal
[Phillip] http://dev.zope.org/Wikis/DevSite/Proposals/ORMappingDB Comments encouraged! [Albert] I've added some there. Jim highlighted a project Risk there: "Updates to RDBMS data outside of the OR mapping could cause cached data to be inconsistent." This strikes me as rather fundamental. Unless the goals include actual *sharing* of RDBMS data with other applications completely independent of Zope I doubt that the most important benefits of an OR mapping could be achieved. Essentially SQL RDBM Systems are *about* sharing data among applications. When "customers want SQL" that is often what they actually want. An SQL RDBMS can be overkill for other purposes which may be just as well achieved by an embedded ODBMS like ZODB, an SQL file system like MySQL or an LDAP directory. Alternative goals for *exporting* ZODB data to an RDBMS continuously, *importing* data from an RDBMS at regular intervals and *embedding* an RDBMS database for exclusive use by Zope with no write access for other applications could all be met more easily. There is certainly no major difficulty on the RDBMS side, giving a Zope instance control over a set of tables for it's own use and providing append only and read only access to export and import tables or views for regular or continuous replication. But the combination of all 3 (which could be delivered incrementally in any order) is *not* the same as *sharing*. As I understand it, Zope's approach to cacheing inherently prevents support for the Isolation part of ACID. Conflicting writes to the same object are detected by version stamps but the objects used by a transaction in one thread may have been inconsistently changed by transactions in other threads. This will not be detected unless those objects used are also changed. Similar problems are inherent in LDAP directories, which are also designed for relatively static data with a low rate of updates. This is acceptable for many applications. Scope can and should be limited to sharing that works with optimistic checkout and does not require pessimistic locking. It is common for an "Enterprise Object" to be read from an RDBMS with it's stamp noted, modified independently by an application and then updated iff the stamp was not changed. Only the simultaneous checking of the stamp and update of the object needs to be wrapped within a short ACID RDBMS transaction. For example ACS 4 maintains a timestamp on every object which can be used for this purpose. This is similar to the ZODB approach. Note however that: 1) The application must be prepared to deal with an exception that cannot just be handled as a lower layer "ConflictError" by retrying. 2) The object will often be a composite - eg an order header *and* all it's line items, and fulfilments. Entanglement with other objects such as products (for pricing) is avoided by specific application programming (which may also be done in stored procedures within the DBMS). 3) This does not support *any* cacheing of objects outside of a transaction. The RDBMS itself provides internal cacheing (often of the entire database for efficient queries with web applications). This leads to the ACS paradigm of "the web server is the database client", which is actually rather similar to Zope's "Zserver is the ZODB client". Both ACS and Zope involve complex issues for database client side cacheing Both 1 and 2 completely preclude any possibility of the same level of "transparency" as for ZODB, while in no way hindering use of "pythonic" syntax. For most Zope web object publishing purposes cached objects just need to be kept reasonably up to date rather than synchronized with RDBMS transactions. The only viable mechanism I can think of for dealing with item 3 in a Zope context would involve the RDBMS maintaining a "Changes" table which it appends to whenever any object that has a special column for "ZeoItem" is changed without also changing the value of "ZeoItem". (ACS does not do this and I'm not sure what it does do). Zeo would monitor that table, either by regular polling or continuously (eg with PostgreSQL as a "LISTENer" responding to NOTIFY commands issued automatically whenever the triggers append to the Changes table). For each change Zeo would notify it's Zope instances to invalidate their caches for that item. I'm not familiar enough with Zope cacheing internals to know whether some other approach is feasible. Requiring such changes in a shared database is certainly undesirable. Q1. Could somebody please post specific URLs for relevant documentation of Zope cacheing? Q2. I have a related question about the Zope design overall. As far as I can make out Zope actually keeps separate copies of persistent objects in RAM for each thread, and relies on the fact that there is a tree structure corresponding to the URL paths that ensures objects from which attributes will be acquired tend to already be in RAM when the acquisition occurs. I assume this is trading off the horrendous inef
RE: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping
[Karl Anderson]
Casey Duncan <[EMAIL PROTECTED]> writes:
> I am not arguing necessarily for SQL as a query language for the ZODB.
> Although it is an accepted standard, but not a perfect one by any means
> especially for OODBs. Its appeal lies mainly in the high level of
> community familiarity and the plethora of SQL software to borrow from.
Does anyone have an opinion on the possible usefulness of XPath,
XQuery, and other XML standards for this? Someone suggested (on the
zope-xml wiki) that it would be nice to be able to drop in a cataloger
that supported a presumably standard and presumably well-known XML
query syntax, and which would work throughout the database because
Zope objects would support DOM.
This is all speculation, and I personally don't know much right now
about XML database interfaces and how finished or well-regarded they
are.
[Albert]
An excellent introduction to this topic is:
"Putting XML in context with hierarchical, relational, and
object-oriented models" by David Mertz.
ftp://www6.software.ibm.com/software/developer/library/x-matters8.pdf
Author is a python developer with lots of interesting XML stuff.
See also his xml_matters 1 and 2 for xml_object and xml_pickle with
much nicer "pythonic" syntax instead of using DOM directly.
Article is also *essential* background for the distinction between
"Object Mapping" and "Object Relational Mapping" which needs to be
understood by anyone participating in this discussion.
An example of a python ODBMS with some partial support for OQL is 4ODS
from 4 Suite, which uses a very natural "pythonic" syntax for objects
stored in and queried from PostgreSQL:
Following is from 4Suite-docs-0.11/4Suite-0.11/html/4ODS-userguide.html
available via:
http://4suite.org/download.html#4Suite_Documentation
How to use the system (a very basic walk through)
First create a ODL file that represents what you want to store in test.odl
module simple {
class person {
attribute string name;
attribute double weight;
relationship Person spouse inverse Person ::spouse_of;
relationship Person spouse_of inverse Person ::spouse;
relationship list children inverse Person ::child_of;
relationship Person child_of inverse Person ::children;
};
class employee (extends person) {
attribute string id;
};
};
Now create a new database and initialize
#OdlParse -ifp test test.odl
Now write some python code to do stuff with these people
#!/usr/bin/python
#Every thing that is persisten must be done inside a transaction and open
database
from Ft.Ods import Database
db = Database.Database()
db.open('test')
tx = db.new()
tx.begin()
#Create a new instance of some objects
import person
import employee
dad = employee.new()
mom = person.new()
son1 = person.new()
son2 = person.new()
daughter = person.new()
#Set some attributes
dad.name = "Pops"
mom.name = "Ma"
son1.name = "Joey"
son2.name = "Bobby"
daughter.name = "Betty"
dad.weight = 240.50
#We can set attributes not defined in the ODL but they will not persist
mom.address = "1234 Error Way"
#Set some relationships
#First set a one to one relationship
dad.spouse = mom
#Or we could have done it via the ODMG spec
#dad.form_spouse(mom)
#Add some children to the dad (our data model does not let mom have
children. We'd need a family struct (left up to the reader)
dad.add_children(son1)
#We can create relationships both ways
son2.form_child_of(dad)
#Shortcut for adding
dad.children = daughter
#Now root the family to some top level object.
db.bind(dad,"The Fam")
#Make it so
tx.commit()
#Out side of a transaction we can still access the objects.
#However, any changes we make will not persist.
#NOTE, because 4ODS caches relationships, any relationships that were not
traversed during the
#transaction, cannot be traversed now because an object cannot be loaded
from the db outside
#of a transaction.
print dad.name
#Start a new tx to fetch
tx = db.new()
tx.begin()
newDad = db.lookup("The Fam")
print newDad.name
print newDad.children[0].name
print newDad.spouse
#Discard this transaction
tx.abort()
Ft/Ods/test_suite and Ft/Ods/demo are good places to look for more examples
^^
See also:
http://www.xml.com/pub/a/2000/10/11/rdf/
Some other relevant references are:
Extraction of DBMS catalogs to XML using python.
http://hyperschema.sourceforge.net./
PostgreSQL as XML repository
http://zvon.org/index.php?nav_id=61
http://hopla.sourceforge.net/doc/README
Note that none of this has much to do with the original topic of
Object-*Relational* Mapping.
*Essential* background for understanding what an object-relational
persistence layer looks like is:
http://www.ambysoft.com/persistenceLayer.html
It isn't very long and there *absolutely* isn't any point discussing
how to design such an OR persistence layer without first reading
and fully understanding it. (I say that after having carefully
studied all the messages in this discussion - though I also said
so before ;-)
The rest
[Zope-dev] (LONG) RE: [SmartObjects] Wrap-up of the discussion going on on zope-dev?
[Joachim Werner] As most of you might have recognized, there is a very active thread (actually two interwoven ones) about RO-mapping etc. going on on zope-dev. I'd be very happy if someone could wrap up the stuff from there on the SmartObjects list. ;-) [Albert] Thanks for the pointer. As I read them all through at once, after seeing your pointer, I made notes before joining the discussion, which may help, though far from a "wrap-up". * SUMMARY * There seem to be 6 separate topics: 1) Nature of SmartObjects, ZPatterns and Transwarp frameworks and relation to Zope/ZODB framework. 2) Improvements to ZCatalog search efficiency 3) Query syntax - see Zwiki: http://dev.zope.org/Wikis/DevSite/Proposals/UnionAndIntersectionOperations 4) Schema transparency (an oxymoron?) 5) Storage of ZODB objects in separate DBMS tables per class - "Object Mapping". 6) Requirements for "Object Relational Mapping". * DETAILED NOTES * Here's a list of all the items I found in the May archives with the term "ORMapping" in the subject line, from thread "Experiments in ORMapping" and interwoven thread "oodb philosophics was [above]" until I subscribed. Will comment later on subsequent messages rather than making notes. If there are any other subject headings missed, or other postings (including earlier months), please let me know. Highlights before links attempt to be objective but are limited by space, by my specific interests in the topic and by my lack of thorough understanding of Zope internals and consequent previous non-participation in [zope-dev] with consequent lack of appreciation of where various posters are "coming from". Quotes following links are verbatim extracts (not necessarily in same order or in context). "Must read" means important message with no or inadequate highlights or extracts given here. I have omitted numerous affirmations that ZODB meets most current Zope requirements (perhaps with improvements to ZCatalog) in view of the fact that nobody appeared to be arguing otherwise. BTW the date order used in list archives could be better in numeric order as that might more closely reflect which messages are likely to have been seen before replying. I have re-arranged. Starting from 10 May 2001: Shane Hathaway (DC) - start of discussion - must read http://lists.zope.org/pipermail/zope-dev/2001-May/011101.html includes link http://www.zope.org/Members/hathawsh/ormapping.tar.gz overview in file: sketch Tino Wildenhain - asks about ZPatterns, ZClasses, doesn't want "purely relational application server". http://lists.zope.org/pipermail/zope-dev/2001-May/011102.html SH - answers TW, says not hard to implement by mapping classes to a table. http://lists.zope.org/pipermail/zope-dev/2001-May/011103.html TW - answers SH, prefers improved OODB because doesn't like mapping classes to schema. http://lists.zope.org/pipermail/zope-dev/2001-May/011105.html SH - answers TW, enhanced ZODB storage (RDBMS and BerkelyDBM) is in some ways the best OODB. http://lists.zope.org/pipermail/zope-dev/2001-May/011107.html "The other motivations for an RDBMS are (1) people have existing schemas and want Zope to access the same data as their existing apps, and they want it to be transparent, and (2) tables with millions of entries are easily stored in Zope but the perception is that the catalog isn't as fast as a database index. No one has done any tests AFAIK." [...] "That's one reason ZODB is so nice. You can write an application without writing a formal schema." Casey Duncan (Kaivo) - suggests Matisse or Objectivity but limited by not supporting ZODB versions. BerkelyDBM best soon. Mentions slow XML storage startup. http://lists.zope.org/pipermail/zope-dev/2001-May/011108.html JW - already 2 OR mapping projects, SmartObjects and TransWarp, no duplication because Phillip from Transwarp participating in SmartObjects list. http://lists.zope.org/pipermail/zope-dev/2001-May/011109.html TW - answers CD, XML just another pickle format. http://lists.zope.org/pipermail/zope-dev/2001-May/00.html SH - answers JW. SmartObjects, ZPatterns and Transwarp require new database API instead of maintaining transparency like ZODB. Projects should look at replacing parts of ZODB instead of adding complexity. http://lists.zope.org/pipermail/zope-dev/2001-May/01.html "ZODB has pieces that can be split apart and replaced as needed, such as caching, persistence, transactions, the "pickle jar", the multi-threaded connection factory, and the storage layer. I'm hoping we can achieve OR mapping by only replacing the "pickle jar", i.e. Connection.py." Cees de Groot - answers SH. Confirms advantage of not having to write formal schema, migrating from PostgreSQL to ZODB for that. File Storage faster than Oracle and is basic transaction log so nothing could be more reliable. http://lists.zope.org/pipermail/zope-dev/2001-May/011124.html "Are people using ZODB for non-Zope data? I'd be very interested to discuss things like emulating exten
[Zope-dev] RE: [Zope] REQUIRING Python 2.1??
[anser] I can't quite help wondering whether someone at DC has maybe gotten so "into" the development of Py 2.1 that they just can't wait to use its new stuff, whether it's objectively what's best for Zope or not. The prudent thing to do would have been to add features as needed using 1.5.2-compatible code, or at best to offer a "new18n" branch that requires 2.1, which people who are THAT desperate for i18n could choose to follow if they wanted. Then, say 6-12 months after 2.1 is gold, you could unify and require it for 3.0. Instead, for the sake of being able to let the Python developers stick a Zope logo on the 2.1 release, we are risking a boatload of trouble. [albert] As far as I can make out the strategy you advocate is more or less exactly what they *did* do - so smoothly you didn't even notice. The *big* leap is from 1.5.2 to 2.0 which has been out for quite a while. I18N is *desperately* needed but had to be delayed because of the compatability problems you are rightly concerned about. So even after I18N became feasible with 2.0 the main branch was made compatible with using 2.0 but binaries released with 1.5.2 to avoid risking a boatload of trouble while enabling people desperate for I18N to start using 2.0 and at the same time discover as much as possible of the hiccups before general switchover. Waiting for the "odd numbered release" is also a generally sound policy. Essentially you are confusing that prudent delay in completing the smoothly planned (and very clearly announced long ago) switch from 1.5.2 to 2.x with a sudden rush to 2.1. Whatever problems do occur will be overwhelmingly from the 2.x, not from it being 2.1 in particular. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
