[Zope3-dev] ZOPE3 Maildir implementation
I have a few questions regarding the ZOPE mailer implementation: 1. Why did you go for a file system implementation - as ZODB objects are being made into a mail queue anyway? The Maildir step could have been skipped, I suppose, just the connection to a mail server directly over the network interface would suffice. Is there a specific reason for the Maildir implementation? 2. The current implementation can only handle between 30-60 E-mail's a minute. This is mostly due to the naming convention used for maildir files (one a second - actually the Maildir naming allows in between numbers, but you don't use that). 30 E-mails a minute is not really scalable and has the (apparent) downside it (effectively) blocks the Zope webserver for the duration (it should not, but it does, possibly a locked ZODB?). We have that experience sending out bulk E-mail reminders using that module. Sending out 200 mails made the main ZOPE server inaccessible for 3-4 minutes. The reason I am asking is not so much an interest in fixing it for mail, as we can live with that performance issue, but I am looking to implement a similar functionality for storing xParrot/ZOPE objects in version control in a seperate thread. So I would like to understand the reason behind the Maildir choice. Regards, Pjotr Prins ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: ZOPE3 Maildir implementation
On Sat, Apr 22, 2006 at 01:27:36PM +0300, Marius Gedminas wrote: Actually, the DirectMailDelivery utility also aborts emails if the transaction is aborted. The reason for using Maildirs was to speed up the transaction commit -- if you send 100 emails in a transaction, you do not want to wait for 100 SMTP connections while the user is sitting in front of his browser waiting. Creating a 100 files in the outgoing mail queue was supposed to be faster. Thanks for fixing it Marius! I think your XP approach is fine ;-). I agree you don't want blocking. Nevertheless should we not store it in the ZODB instead - at least when ZODB is around? I don't know what the policy is with ZOPE3, but I would think that we should minimise deployment dependencies. Locating a directory is such a dependency - and people using, for example, NFS may get into trouble (though I think maildir tried to address that with that naming and file moving scheme - it is ugly, arguably fragile and slow!). Correct me if I am wrong. Someone wrote that you can't pass the ZODB access from a web server thread to a service/utility thread - The mailer utility is global and has no convenient access to the ZODB. Is that really true? Is there no way to pass this info? Pj. ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Version control of ZOPE3 objects
Hi, I need to build a versioning module for the xParrot project (http://xparrot.thebird.nl/). I have some questions and wondered who would be inclined to help with the architecture phase. The idea is to save objects that implement IVersioning on a modification event to either ZODB, or to an external version control system like subversion. The latter would be nice as it could handle for example diffs through Websvn (a reverse proxy may be needed to handle access control through ZOPE). Using an existing version system saves a lot of code and would allow for functionality like branching etc. On the other hand commits have to be run on a separate thread (which I can model on the current mailqueue implementation). Also deployment would require extra system administrator skills. Before I go down this route I would like to ask if anyone has a better idea? And who would take an interest in this little project, and doesn't mind bouncing off a few ideas? Pj. ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Version control of ZOPE3 objects
Thanks Tres for the reply. It really helps. I agree transactions are a problem. If SVN fails there is no clear route to notify the user. On the other hand we merely would use it for 'historic' storage - a one way system for moving objects. As we have to have a queue anyway, on failure an object should not be removed from the queue. That allows the administrator to resolve the SVN problem and rerun the queue. BTW I think there are two ways to make a queue - through the filesystem (like the mailer does), or through ZODB - accessing the queue through XML/RPC on a separate thread. If anyone knows how to use ZODB from an independent thread outside ZOPE I am all ears. I would like to drop the XML/RPC. You made me think of an alternative approach though. I don't know how you resolved your content repositories, but how about mirroring a site into a versioned repository in ZODB. When we have a site with objects we could create a versioned site with versioned objects (inheriting security per object): ZODB: mysite: objectA objectB etc. mysite_versioned: objectA_1 objectA_2 objectA_3 objectB_1 objectB_2 etc. An XML/RPC interface could be created to store the versioned objects offline - as objectA and objectB - through an external command. That would allow an admin to do that through a CRON job with feedback on the transaction. It could also purge old objects. The nice thing is that I can quite easily create a view of older objects in mysite_versioned (all XML/XSLT with xParrot) and even do a simple diff, indeed. Recovery is straightforward too. What do you think? Pj. ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: [DraftProposals] (edit)
On Mon, Dec 18, 2006 at 11:11:21PM -0800, Christian Theune wrote: I don't think so. The Zope 3 wiki was built up as, and still is, a resource for the development *of* Zope 3. We shouldn't So according to you ZOPE3 is still an engineering project as the main Wiki/Documentation site is basically a resource for the development of ZOPE. Indeed, this is the way ZOPE3 is presented to the outside world now. It is my impression and I am acting on that. People starting with ZOPE, or implementing with ZOPE, get lost in documentation about *proposals* on ZOPE. Place yourself in the shoes of someone new to ZOPE3 and trying to find his/her way. That should matter to you. Let me phrase it as a technical challenge (so the engineer can understand it): How do you create a web resource for ZOPE3 that allows people to find and resolve implementation issues easily? Without them having to wade through (alive or dead) proposals? And how do you trick search engines not to expose these proposals? Pj. ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Some thoughts on Zope 3, Zope 3 applications, and Zope 3 instances
Hi Jim Martijn, For xParrot we have a case in point where the ZMI both helped and confused us. Because of the way the documentation is 'framed' (where available) it lead us to believe, initially, that ZOPE was the ZMI. The first incarnation of xParrot (an XML/XSLT provider) was intermingled with the ZMI. The next version two was mostly about driving the ZMI out of the equation. The OFS (if I understand correctly we are referring to the file/object representation of ZOPE) is still very valuable for reaching resources. In fact xParrot2 does not use the ZMI at all now but would not be what it was without the OFS. On the other hand for another project I use the ZMI and the Rotterdam skin to develop faster - providing a useful tree interface. The tree is key here. It is obvious reflection/inspection etc. are quite useful too, during development. I would suggest to accentuate the library side of ZOPE and provide the ZMI as a (lesser) example. For ZOPE3 to replace ZOPE2 a new application engine may be an idea. A powerful engine that is better than Rails - with some code generation - should be an enticing goal for some. Because, take it or leave it, programming with ZOPE is still (too) hard. I learnt Ruby on Rails in a few days. With ZOPE3 I often still am in the dire straights (after a year). We use ZOPE3 because of the superior component model and because it looked better for xParrot, at the time. And in fact, we have no regrets there as Andrew designed xParrot so it can be used without any understanding of Python/ZOPE now. So, there you have one great application without the ZMI. Let me wrap up saying that despite my criticisms I think you all did a great job. The stability and robustness of ZOPE3 is legendary. Deployment is a bit odd, but one gets used to that (I agree about the duplication - but then that is configuration too, it is not that bad). A strong 2007 wished for. Pj. ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
