[Zope-dev] Zope Tests: 7 OK
Summary of messages to the zope-tests list. Period Tue Mar 6 12:00:00 2007 UTC to Wed Mar 7 12:00:00 2007 UTC. There were 7 messages: 7 from Zope Unit Tests. Tests passed OK --- Subject: OK : Zope-2.6 Python-2.1.3 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:08:30 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007398.html Subject: OK : Zope-2.6 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:10:00 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007399.html Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:11:30 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007400.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:13:00 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007401.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:14:30 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007402.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:16:01 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007403.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:17:31 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007404.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Proposal for optimized Blob handling
Hi, [modified slightly from a similar proposal to zope3-dev to match Zope 2's publisher] I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. However, the HTTPRequest class currently uses the default implementation of the cgi module's FieldStorage. I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. Notice: The FieldStorage explicitly provides the `make_file` method to allow overriding in this sense. Does anybody feel like this would be a bad idea? Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Caching ZCatalog results
On 3/6/07, Dieter Maurer [EMAIL PROTECTED] wrote: Lennart Regebro wrote at 2007-2-23 21:25 +0100: ... Compared with Lucene for example, which instead will create iterators who will only resturn the next match. This saves you from a lot of index searching when you have big results. I don't know if it is feasible to do something like that, but it would be interesting to look into it. It is done in IncrementalSearch2. http://www.dieter.handshake.de/pyprojects/zope Cool! This also needs to be in core. As most of your stuff. ;-) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 14:01 -0300 schrieb Sidnei da Silva: What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? No, I'd create a new hard link into the blob directory so the link to the temporary file can go away without making the inode go away. For the purposes of storing blobs the TMP directory should be on the same partition as the blobs directory anyway. Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
On Wed, Mar 07, 2007 at 06:31:25PM +0100, Christian Theune wrote: Am Mittwoch, den 07.03.2007, 14:01 -0300 schrieb Sidnei da Silva: What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? No, I'd create a new hard link into the blob directory so the link to the temporary file can go away without making the inode go away. For the purposes of storing blobs the TMP directory should be on the same partition as the blobs directory anyway. I like the idea, but what will you do if this fails? (eg. the admin has put TMP on a different mount, or we're running on Windows). -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
Christian Theune-2 wrote: Am Mittwoch, den 07.03.2007, 14:01 -0300 schrieb Sidnei da Silva: What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? No, I'd create a new hard link into the blob directory so the link to the temporary file can go away without making the inode go away. For the purposes of storing blobs the TMP directory should be on the same partition as the blobs directory anyway. Does this work on Windows? Martin -- View this message in context: http://www.nabble.com/Proposal-for-optimized-Blob-handling-tf3363320.html#a9357888 Sent from the Zope - Dev mailing list archive at Nabble.com. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 09:34 -0800 schrieb Martin Aspeli: Christian Theune-2 wrote: Am Mittwoch, den 07.03.2007, 14:01 -0300 schrieb Sidnei da Silva: What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? No, I'd create a new hard link into the blob directory so the link to the temporary file can go away without making the inode go away. For the purposes of storing blobs the TMP directory should be on the same partition as the blobs directory anyway. Does this work on Windows? Link does not work on Windows using the link() function from the os module. I don't know whether Windows has any API for doing this kind of operation. In any case we can fall back (e.g. if the link() call fails) to copying the data as this is just an optimization. Please consider looking at my upcoming proposal for discussion. In this thread I'd like to keep the focus on the change of the publisher to use NamedTemporaryFile. Christian PS: Thanks for the input though. -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Early processing of request body (was: Proposal for optimized Blob handling)
I was looking through some publisher code and found that the `process_request` method which takes the request body as a file-like object and processes it as a FieldStorage happens within the application thread. This would be better if it happened beforehand because it can takes up time while a transaction is running and a thread is used although it doesn't require any application-specific code. AFAICT a modified version of FieldStorage would be required to allow line-wise consumption and parsing of data while it is being uploaded and then hand this into the request instead of stdin. However, the FieldStorage implementation is recursive and wasn't obvious to me at a first glance how much work it would be to replace this. Are there similar feelings it would be a good idea to do this kind of early line-wise processing of request bodies? Christian Am Mittwoch, den 07.03.2007, 17:44 +0100 schrieb Christian Theune: Hi, [modified slightly from a similar proposal to zope3-dev to match Zope 2's publisher] I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. However, the HTTPRequest class currently uses the default implementation of the cgi module's FieldStorage. I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. Notice: The FieldStorage explicitly provides the `make_file` method to allow overriding in this sense. Does anybody feel like this would be a bad idea? Christian ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] anon http svn access status?
So: will anon http svn access happen? I'm late to this round of the discussion, but +1. I would suggest https, though, to make it firewall-friendly. Even if this is an anonymous mirror (sounds more likely), it would make sense for community bundle builders (e.g. Plone bundles) to use https URLs for everything, which supposes upstream can provide https. Neither http, svnserve, or svn+ssh are friendly to corporate firewalls - only https (where the firewall can't see the http method verbs because they are inside a TLS session). I've griped about this before: http://www.mostscript.com/weblog/?p=26 Today, I have to put in requests to my company network admins to open up to svn.zope.org:3690 out on a workstation by workstation basis. Their firewall can do a generic TCP socket proxy (network security folks want to run an app-level proxy, so we can only set this up one source IP at a time), and doesn't deal with DeltaV DAV verbs over plain-old http - so https is the only reasonable option for anonymous checkouts. Thanks, Sean +--+ Sean Upton SignOnSanDiego.com Site Technology Supervisor The San Diego Union-Tribune 619.718.5241 [EMAIL PROTECTED] 350 Camino De La Reina San Diego, CA 92108 Plone Powered! plone.org ++ python.org ++ zope.org +--+ ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Early processing of request body (was: Proposal for optimized Blob handling)
Christian Theune wrote at 2007-3-7 20:09 +0100: I was looking through some publisher code and found that the `process_request` method which takes the request body as a file-like object and processes it as a FieldStorage happens within the application thread. This would be better if it happened beforehand because it can takes up time while a transaction is running and a thread is used although it doesn't require any application-specific code. In my view, it already now happens far too early, because it may raise exceptions and those exceptions are not handled by the standard_error_message usually used for error processing depending on the url. Therefore, if you move out things, you should take care that you move out only parts that cannot raise exceptions. Furthermore, you seem to propose to move work from a worker thread to the IO (i.e. ZServer) thread. I do not think that it is a good idea to put significant work on the IO thread. Note, that the IO thread is responsible to handle all IO. When you keep it busy with other tasks, it will not handle IO... -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Early processing of request body (was: Proposal for optimized Blob handling)
Hi, Am Mittwoch, den 07.03.2007, 21:48 +0100 schrieb Dieter Maurer: Christian Theune wrote at 2007-3-7 20:09 +0100: I was looking through some publisher code and found that the `process_request` method which takes the request body as a file-like object and processes it as a FieldStorage happens within the application thread. This would be better if it happened beforehand because it can takes up time while a transaction is running and a thread is used although it doesn't require any application-specific code. In my view, it already now happens far too early, because it may raise exceptions and those exceptions are not handled by the standard_error_message usually used for error processing depending on the url. Therefore, if you move out things, you should take care that you move out only parts that cannot raise exceptions. Ah. Interesting point! Furthermore, you seem to propose to move work from a worker thread to the IO (i.e. ZServer) thread. I do not think that it is a good idea to put significant work on the IO thread. Note, that the IO thread is responsible to handle all IO. When you keep it busy with other tasks, it will not handle IO... Right. This optimization is about leveraging the fact that in many situations the upstream bandwith is *much* lower than the IO bandwith to the disk. Another condition that I have (and I think this is the general pattern) is that application threads should be given back to the pool as quickly as possible. If 5 seconds are spend in the application thread to untangle mime data which has nothing application-specific about it and then only 100ms or so in the application itself, I'd say there is a major overhead problem. Christiabn -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] Google Summer of Code
On 3/5/07, Philipp von Weitershausen [EMAIL PROTECTED] wrote: a) mentors. It'd be great if some of the Zope core committers would volunteer to mentor a student. This doesn't mean you will definitely end up mentoring one, just show your willingness. Yeah, I could do that. b) project suggestions. I'm sure I can come up with several. Finishing the twisted integration in Zope2, for example. It only does http yet we need ftp too, and maybe more. So, I think this is a good idea. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Tests: 7 OK
Am Mittwoch, den 07.03.2007, 13:00 +0100 schrieb Zope Tests Summarizer: Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:17:31 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007404.html I'm questioning those tests. I have two tests failing on a fresh checkout of the Zope 2 trunk on my machine in this setting (Linux/Python 2.4.4) Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Road to Zope 2.11
Hi, few questions: a) I want to do the switch to ZODB 3.8 (currently trunk) b) That needs zope.proxy as of Zope 3.4 (currently trunk) Is there anything that would block this and if something how I can contribute unblocking it? Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZopeTestCase.doctest is not aware of local sites
Hi again, I just noticed that the publish method from the Functional base class from ZopeTestCase has the same problem. You can find the adjusted test attached. Hanno Hanno Schlichting wrote: Hi all, while pushing local site support in CMF/Plone I came across some weird test failures. After a while of debbuging I could track this down to the http method in ZopeTestCase's doctest support. The method in its Zope2 incarnation is not aware of local sites, while the one in zope.app.testing is. I have attached a patch that mirrors the Zope3 behavior and saves the site information before publishing a module and restores it back, just like the security manager is saved/restored. As this is causing some test failures in Plone 3 now, I would be happy if somebody could take a look at this. Thank you, Hanno Index: /opt/plone30/parts/zope2/lib/python/Testing/ZopeTestCase/functional.py === --- /opt/plone30/parts/zope2/lib/python/Testing/ZopeTestCase/functional.py (revision 73024) +++ /opt/plone30/parts/zope2/lib/python/Testing/ZopeTestCase/functional.py (working copy) @@ -37,6 +37,7 @@ request_method='GET', stdin=None, handle_errors=True): '''Publishes the object at 'path' returning a response object.''' +from zope.app.component.hooks import setSite, getSite from StringIO import StringIO from ZPublisher.Response import Response from ZPublisher.Test import publish_module @@ -47,6 +48,10 @@ # Save current security manager sm = getSecurityManager() +# And we need to store the old site +old_site = getSite() +setSite(None) + # Commit the sandbox for good measure transaction.commit() @@ -89,6 +94,9 @@ # Restore security manager setSecurityManager(sm) +# And we need to restore the site again +setSite(old_site) + return ResponseWrapper(response, outstream, path) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Road to Zope 2.11
Christian Theune wrote: a) I want to do the switch to ZODB 3.8 (currently trunk) b) That needs zope.proxy as of Zope 3.4 (currently trunk) Is there anything that would block this and if something how I can contribute unblocking it? The only way to really find out is to try it. Zope 3.3 and 3.4 are close enough not to cause too much trouble, I think. -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
On 3/7/07, Christian Theune [EMAIL PROTECTED] wrote: Does this work on Windows? Link does not work on Windows using the link() function from the os module. I don't know whether Windows has any API for doing this kind of operation. Yes [1] it [2] does [3]. The omission on os.link() is just the lack of a good soul to contribute it apparently [4]. A good task for a friday evening. [1] http://msdn2.microsoft.com/en-us/library/aa363860.aspx [2] http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/fsutil_hardlink.mspx?mfr=true [3] http://www.flexhex.com/docs/articles/hard-links.phtml [4] http://www.thescripts.com/forum/thread537011.html -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Early processing of request body (was: Proposal for optimized Blob handling)
On 3/7/07, Christian Theune [EMAIL PROTECTED] wrote: Right. This optimization is about leveraging the fact that in many situations the upstream bandwith is *much* lower than the IO bandwith to the disk. Another condition that I have (and I think this is the general pattern) is that application threads should be given back to the pool as quickly as possible. If 5 seconds are spend in the application thread to untangle mime data which has nothing application-specific about it and then only 100ms or so in the application itself, I'd say there is a major overhead problem. I had came to the same conclusion a couple weeks ago, somehow *wink*. Maybe we've been influenced by the same person. :) So if the uploaded file shouldn't be handled by the application thread, neither by the IO layer, then I guess we need a 'upload handling thread pool' of some sorts, whose sole purpose is to handle incoming requests that are large before it gets to the application thread while still outside the async IO layer. Hopefully something similar could be done for files being sent *out* of the application when they don't need any application processing anymore (ie, Blobs!). -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] Google Summer of Code
On 7 Mar 2007, at 12:24 , Lennart Regebro wrote: On 3/5/07, Philipp von Weitershausen [EMAIL PROTECTED] wrote: a) mentors. It'd be great if some of the Zope core committers would volunteer to mentor a student. This doesn't mean you will definitely end up mentoring one, just show your willingness. Yeah, I could do that. Great. Please add yourself with your Google account to http:// wiki.zope.org/zope3/SummerOfCode2007. b) project suggestions. I'm sure I can come up with several. Finishing the twisted integration in Zope2, for example. It only does http yet we need ftp too, and maybe more. Very good idea! Actually, this sort of goes along with the WSGI idea: making WSGI a first-class citizen in Zope 2. Feel free to either amend Sidnei's and my original suggestion or create a new one at http://wiki.zope.org/zope3/SummerOfCode2007. Philipp ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Proposal for optimized Blob handling
Christian Theune wrote: Hi, [modified slightly from a similar proposal to zope3-dev to match Zope 2's publisher] I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. I think this is a great idea. Am I the only person here who immediately associated link with the POSIX? Also, am I the only one who read when possible as when on a POSIX system where link is available, in other words, when not on Windows? One starts to wonder... However, the HTTPRequest class currently uses the default implementation of the cgi module's FieldStorage. I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. +1 -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Proposal for optimized Blob handling
Note that one micro-optimization for PUT requests is to not use a
FieldStorage at all because the body is never mime-encoded anyway in
practice.
I have a monkey patch to do this now, which I turned into a patch for
the core, but took out because Phillipp whined at a sprint once. ;-)
Here's the monkey patch...
def patch_httprequest_processinputs():
Patch HTTPRequest.processInputs to not do any processing on a
PUT request (it's pointless, and foils our on-the-fly encryption,
as it creates a new tempfile via FieldStorage).
# note that OTF encryption support only works for PUT requests
import re
from ZPublisher.HTTPRequest import HTTPRequest
oldProcessInputs = HTTPRequest.processInputs
def newProcessInputs(
self,
# static variables that we want to be local for speed
SEQUENCE=1,
DEFAULT=2,
RECORD=4,
RECORDS=8,
REC=12, # RECORD|RECORDS
EMPTY=16,
CONVERTED=32,
hasattr=hasattr,
getattr=getattr,
setattr=setattr,
search_type=re.compile('(:[a-zA-Z][-a-zA-Z0-9_]+|\\.[xy])
$').search,
):
Process request inputs
We need to delay input parsing so that it is done under
publisher control for error handling purposes.
method=self.environ.get('REQUEST_METHOD','GET')
if method == 'PUT':
# we don't need to do any real input processing if we
are handling
# a PUT request. This is an optimization especially
because
# FieldStorage creates an additional tempfile if we
allow it to
# parse the body, and PUT uploads can tend to be large.
self._file = self.stdin
return
return oldProcessInputs(self)
HTTPRequest.processInputs = newProcessInputs
- C
On Mar 7, 2007, at 9:57 PM, Philipp von Weitershausen wrote:
Christian Theune wrote:
Hi,
[modified slightly from a similar proposal to zope3-dev to match Zope
2's publisher]
I'm writing up a proposal for the ZODB to make even more efficient
Blob
handling possible.
This includes not copying the data from an uploaded file, but using a
`link` operation when possible.
I think this is a great idea.
Am I the only person here who immediately associated link with
the POSIX? Also, am I the only one who read when possible as
when on a POSIX system where link is available, in other words,
when not on Windows? One starts to wonder...
However, the HTTPRequest class currently uses the default
implementation
of the cgi module's FieldStorage.
I propose to create a small subclass to override the `make_file`
method
to use `NamedTemporaryFile` instead of `TemporaryFile` to allow
the file
being accessible from a filename so I can apply a `link` operation.
+1
--
http://worldcookery.com -- Professional Zope documentation and
training
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] Re: Proposal for optimized Blob handling
On 3/7/07, Philipp von Weitershausen [EMAIL PROTECTED] wrote: Am I the only person here who immediately associated link with the POSIX? Also, am I the only one who read when possible as when on a POSIX system where link is available, in other words, when not on Windows? One starts to wonder... NTFS does support hard links since version 5, which means Windows 2000+. It does not support hard links to directories though, only soft links (which are called junctions or junction points). The version of NTFS shipped with Vista supports hard links for directories I believe. -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] Re: Proposal for optimized Blob handling
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sidnei da Silva wrote: On 3/7/07, Philipp von Weitershausen [EMAIL PROTECTED] wrote: Am I the only person here who immediately associated link with the POSIX? Also, am I the only one who read when possible as when on a POSIX system where link is available, in other words, when not on Windows? One starts to wonder... NTFS does support hard links since version 5, which means Windows 2000+. It does not support hard links to directories though, only soft links (which are called junctions or junction points). The version of NTFS shipped with Vista supports hard links for directories I believe. POSIX systems don't allow hard links to directories, either, in practice:: $ man ln ... -d, -F, --directory allow the superuser to attempt to hard link directories (note: will probably fail due to system restrictions, even for the superuser) Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF76hn+gerLs4ltQ4RArm3AJ9o6Vw63qc6cJT3GPJOVebCFhDtiACfViDa EI1MxpxIIwEQl3uXVxly7M4= =NZUy -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [ZWeb] wiki.zope.org news
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - Yesterday, an automated zope restart failed because of an error related to the (new) permanent zeo cache. I have gone back to temporary zeo cache. Persistent ZEO caches usually create more problems than they solve. There are deeply hidden bugs that can cause inconsistencies between the cache state and the real state. I'd never recommend anyone use them. - Today, zope stopped responding as both threads blocked trying to send mail. (I was able to see this from the traceback in event.log when I killed them.) Some sort of deadlock with qmail. This has been happening before so it's good to pinpoint it. I will switch to MaildropHost asap which may help. It will help in this specific situation. Mail will not block any Zope thread at that point. Edit operations, especially those touching many pages, can still be rather slow as we are still memory-constrained (~250M quota). The server is currently running two zope threads with cache-size 3000. When it exceeds quota (be it through normal growth or memory leak) it is restarted; this is still happening one or more times per day. Restarting and warming up the caches is quite slow (can be 5-10 minutes if I'm not mistaken). I have gone from 7-day to 2-day nightly packs to keep zodb size around the 1G mark (and within the 8G disk quota). Martijn Faassen is pursuing a possible new, beefier hosting situation. That is a pretty sad hosting situations. Good luck finding something better. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF70ymRAx5nvEhZLIRAqN9AJ0UtD5IJf5l7rELaeQ+L4/bT29kSACgo4Sl l2l+7hPF2nw/RZ8Q88RduBU= =U9by -END PGP SIGNATURE- ___ Zope-web maillist - Zope-web@zope.org http://mail.zope.org/mailman/listinfo/zope-web
[Zope] Re: Zope hangs in FreeBSD with plenty of memory and CPU resources
I know that the kqread state has to do with the kqueue or kevent system call, and assume that Zope being in that waiting state for a long time is abnormal. Asking here in case someone has seen this behavior before and knows what might be the cause. As for the last and only other time this happened, Zope regained normal behavior by itself after a little while (in the order of half an hour). I don't see Maybe your hanging server threads are related to that mysql problem you wrote about. I remember that I had an issue with mysql running out of connections and so a after a while all zope threads were waiting for mysql answers and Zope was practically unresponsive and not using much cpu. You might want to check your MySQL Logs... Kind Regards Maik ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: I've just installed Zope 2.8 with Apache 2 and I made a download library (a CMS site is too much for me and I can't find a simple "download area" product, so I've done it!) What about railorad: http://www.infrae.com/products/railroad How can i manage downloads uploads? In general it is bad idea to hold and serve large files directly from ZODB. This should be done by Apache etc. Large files aren't store in ZODB but in a shared folder connected with LOCALFS. Maybe Zope load a file in zodb before begin the download? (Are there other storage product that work only in zope - without plone?) Tramline is an Apache add-on but my problem still remain without Apache so this product can help me after I resolved the Zope issue. Thank you Illorca ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Large files aren't store in ZODB but in a shared folder connected with LOCALFS. Maybe Zope load a file in zodb before begin the download? (Are there other storage product that work only in zope - without plone?) Tramline is an Apache add-on but my problem still remain without Apache so this product can help me after I resolved the Zope issue. I think that if you'll use tramline you'll have no 'Zope issue' anymore. -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: Large files aren't store in ZODB but in a shared folder connected with LOCALFS. Maybe Zope load a file in zodb before begin the download? (Are there other storage product that work only in zope - without plone?) Tramline is an Apache add-on but my problem still remain without Apache so this product can help me after I resolved the Zope issue. I think that if you'll use tramline you'll have no 'Zope issue' anymore. Looking LocalFS specs I find the issue: This version of LocalFS takes advantanges of Zope 2.7.1 's new filestream_iterator feature and is believed to server large files better than before. However, my initial testing showed that it still doesn't prevent the whole file being loaded into memory. When I opened a URL pointing to a mpg file served by LocalFS, the file was loaded into memory and then immediately freeed twice. I think that's the problem... load 300Mb into memory Maybe I look for other fs products. Illorca ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] safe read-only access to acquisition parent objects
Hello, Thank you for your answer Dieter. Indeed, I have been looking in all the restricted interpreter things. I have been reading zope\security\untristedinterpreter.txt, and I think it is probably what I need. The thing is I don't know how to produce a security proxy which would allow any 'get' access and forbide any 'set' access. I understand I have to use ProxyFactory, but I don't understand how I can configure my own Checker that would grant the access policy I want. If anybody have some knowledge about it, any help would be appreciated. Thanks On 3/6/07, Dieter Maurer [EMAIL PROTECTED] wrote: Eric Bréhault wrote at 2007-3-5 13:14 +0100: I have build a Plone product which allows users to enter a piece of Python code. This way, users can easily define their own actions without changing the product source code. Those pieces of code are executed using the exec Python command. I would instead use TALES expressions of type python. There are restricted -- which is very essential if you cannot fully trust your users. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Looking LocalFS specs I find the issue: This version of LocalFS takes advantanges of Zope 2.7.1 http://www.zope.org/Members/hewei/zoperpms/zope2.7 's new /filestream_iterator/ feature and is believed to server large files better than before. However, my initial testing showed that *it still doesn't prevent the whole file being loaded into memory*. When I opened a URL pointing to a mpg file served by LocalFS, the file was loaded into memory and then immediately freeed twice. I think that's the problem... load 300Mb into memory Maybe I look for other fs products. AFAIK it uses tempfile for serving such files. This allows Zope resources to be not blocked while returning big amounts of data. Although creation of tempfile or something like that (I'm not sure how this works in details) may still be slow. In general serving static files is best done by servers like Apache and I think this shouldn't be done by application server like Zope (possibly until it has blobs support). I don't understand why you can't use tramline. It may be connected with zope easily. It gives you very fast filesystem storage. Even more, seems for me that it is designed to do exacly what you're asking for. You say that tramline may help you after resolving zope issues, but I don't get it... By using tramline you will have no files in Zope (ZODB or LocalFS) so you will have no problem to eliminate. But maybe I missed something in your's use case. -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: Looking LocalFS specs I find the issue: This version of LocalFS takes advantanges of Zope 2.7.1 http://www.zope.org/Members/hewei/zoperpms/zope2.7 's new /filestream_iterator/ feature and is believed to server large files better than before. However, my initial testing showed that *it still doesn't prevent the whole file being loaded into memory*. When I opened a URL pointing to a mpg file served by LocalFS, the file was loaded into memory and then immediately freeed twice. I think that's the problem... load 300Mb into memory Maybe I look for other fs products. AFAIK it uses tempfile for serving such files. This allows Zope resources to be not blocked while returning big amounts of data. Although creation of tempfile or something like that (I'm not sure how this works in details) may still be slow. In general serving static files is best done by servers like Apache and I think this shouldn't be done by application server like Zope (possibly until it has blobs support). I don't understand why you can't use tramline. It may be connected with zope easily. It gives you very fast filesystem storage. Even more, seems for me that it is designed to do exacly what you're asking for. You say that tramline may help you after resolving zope issues, but I don't get it... By using tramline you will have no files in Zope (ZODB or LocalFS) so you will have no problem to eliminate. But maybe I missed something in your's use case. I want to use localfs instead of Apache becouse: 1) upload is done by web too 2) upload/download operations cause db operation (files names and descriptions are stored in sql server) Now I find a product (StreamingFS) that patch LocalFS for do this but doesn't work. At the moment I'm trying to fix its code. Illorca ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
I want to use localfs instead of Apache becouse: 1) upload is done by web too tramline supports uploads and downloads 2) upload/download operations cause db operation (files names and descriptions are stored in sql server) With tramline: The only thing the appserver sees is a unique identifier of the uploaded file. I don't know whether it is a file name, but seems that while uploading/downloading you may do some writes into db. Good luck! -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] form-script-zsql
I´m doing a form with username and email, these values go script
python and with a zsql method inside script, it stored in DB
I have a zpt with a form
I have a script python::
Connection DB::
db_system_type = REQUEST.get('db_system_type', 'mysql')
from Products.ZMySQLDA import DA
DA.manage_addZMySQLConnection(self, db_conn_id, '',
REQUEST.get('db_conn_str', '[EMAIL PROTECTED] zzz zzz'))
#now i have to do zsql method to store the values in DB
how can i do it??? Can you help me???Thanks
When I fill in the form and i push send button, it store username
and email in DB
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] form-script-zsql
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
result = context.db_conn_id.db.query('YOUR QUERY HERE')
robert
javi lopez schrieb:
I´m doing a form with username and email, these values go script
python and with a zsql method inside script, it stored in DB
I have a zpt with a form
I have a script python::
Connection DB::
db_system_type = REQUEST.get('db_system_type', 'mysql')
from Products.ZMySQLDA import DA
DA.manage_addZMySQLConnection(self, db_conn_id, '',
REQUEST.get('db_conn_str', '[EMAIL PROTECTED] zzz zzz'))
#now i have to do zsql method to store the values in DB
how can i do it??? Can you help me???Thanks
When I fill in the form and i push send button, it store username
and email in DB
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.5 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org
iD8DBQFF7qxVGaryJ0T9kUYRAkSHAJ9DrgVCp5K79z1NAsQv6hZQtLZMHQCeP8i/
7ewGRmvUrDkvr0zBDI1tHas=
=Tlfx
-END PGP SIGNATURE-
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
I don't understand your paste. I'll try to describe better my situation - server A with Apache(80) and a rewrite rule to Zope(8080) - server B with a shared folder storage - *without any webserver* - server C with MS SQL Server - *without any webserver* - connected with ZODBC - firewall that send any incoming connection for www.mydomain.it to server A when protocol is http or https - in every server is active a ZOPEUSER (local administrator) with the same password. An incoming request to download will cause: - server A ask server C for filename - server A ask server B for file (using LocalFS object that stay in ZODB linked to server B's shared folder) I really can't understand how implement the same process using Apache for file storage. Please stay on zope list too! I've never used tramline, but after reading short description on their page seems that you'll have simpler configuration: - server A with Apache(and tramline) and a rewrite rule to Zope(8080) - server C with MS SQL Server - *without any webserver* - connected with ZODBC - firewall that send any incoming connection for www.mydomain.it to server A when protocol is http or https In general seems that instead of real, large files, app server receives only small files that contain only unique identifiers of these real ones. Real files are handled by tramline and stored in specific filesystem folders. When someone requests a file for download, application server sets special header that contains unique file identifier and this causes that tramline will return real, big file to the browser. Upload similarly uses headers to allow or disallow uploading. -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] database connection in script
how to create a database connection in a script python ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] database connection in script
- Original Message - From: javi lopez [EMAIL PROTECTED] To: zope@zope.org Sent: Wednesday, March 07, 2007 7:48 AM Subject: [Zope] database connection in script how to create a database connection in a script python What kind of database are you trying to connect to? Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] database connection in script
my database is Mysql. can you help me?Thanks ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] database connection in script
- Original Message - From: javi lopez [EMAIL PROTECTED] To: Jonathan [EMAIL PROTECTED]; zope@zope.org Sent: Wednesday, March 07, 2007 7:54 AM Subject: Re: [Zope] database connection in script my database is Mysql. can you help me?Thanks The easiest way to do this is to: 1- create a database/table in MySQL 2- use the ZMI to add a Z SQL Method 3- you can then have your python script call the Z SQL Method to access (ie add/modify/delete/search) the MySQL db Here is a link to the relational database connectivity section of the Zope Book: http://www.plope.com/Books/2_7Edition/RelationalDatabases.stx Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] database connection in script
how to create a database connection in a script python
This is your third (or more) thread about this during few hours!
Ask a specific questions and be patient please.
In general seems that you have no idea how it all works.
So my advice is:
1. Read abour basics of relational database connectivity in zope
2. Go to your portal folder via ZMI and create database
connection to your RDBMS
3. Still in ZMI create ZSQLMethods you need
4. In your product just call these ZSQLMethods via their id's like
result = self.mymethod1()
If this works then you may:
1. Change addform of your product to be able
to select specific database connection from the list
of available connections. Your product won't have
hardcoded connection id.
2. Create ZSQLMethods in your product as typical objects.
Instead of using ZSQLMethods you may use:
context.db_conn_id.db.query('YOUR QUERY HERE')
as Robert Rottermann suggested.
--
Maciej Wisniowski
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: I don't understand your paste. I'll try to describe better my situation - server A with Apache(80) and a rewrite rule to Zope(8080) - server B with a shared folder "storage" - *without any webserver* - server C with MS SQL Server - *without any webserver* - connected with ZODBC - firewall that send any incoming connection for www.mydomain.it to server A when protocol is http or https - in every server is active a ZOPEUSER (local administrator) with the same password. An incoming request to download will cause: - server A ask server C for filename - server A ask server B for file (using LocalFS object that stay in ZODB linked to server B's shared folder) I really can't understand how implement the same process using Apache for file storage. Please stay on zope list too! I've never used tramline, but after reading short description on their page seems that you'll have simpler configuration: - server A with Apache(and tramline) and a rewrite rule to Zope(8080) - server C with MS SQL Server - *without any webserver* - connected with ZODBC - firewall that send any incoming connection for www.mydomain.it to server A when protocol is http or https In general seems that instead of real, large files, app server receives only small files that contain only unique identifiers of these real ones. Real files are handled by tramline and stored in specific filesystem folders. When someone requests a file for download, application server sets special header that contains unique file identifier and this causes that tramline will return real, big file to the browser. Upload similarly uses headers to allow or disallow uploading. Sorry, I push the wrong button when I reply I'll try your configuration, but - where are files stored? (I dont't see server B in your post) - it seems I'll lose Zope User Restrictions ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Sorry, I push the wrong button when I reply I'll try your configuration, but - where are files stored? (I dont't see server B in your post) Read http://www.infrae.com/products/tramline There is description where files are stored - it seems I'll lose Zope User Restrictions Possibly not. This is application server that decides if user is allowed to upload/download file (by setting proper headers). -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Jary Busato schrieb: Maciej Wisniowski ha scritto: Large files aren't store in ZODB but in a shared folder connected with LOCALFS. Maybe Zope load a file in zodb before begin the download? (Are there other storage product that work only in zope - without plone?) Tramline is an Apache add-on but my problem still remain without Apache so this product can help me after I resolved the Zope issue. I think that if you'll use tramline you'll have no 'Zope issue' anymore. Looking LocalFS specs I find the issue: This version of LocalFS takes advantanges of Zope 2.7.1 http://www.zope.org/Members/hewei/zoperpms/zope2.7 's new /filestream_iterator/ feature and is believed to server large files better than before. However, my initial testing showed that *it still doesn't prevent the whole file being loaded into memory*. When I opened a URL pointing to a mpg file served by LocalFS, the file was loaded into memory and then immediately freeed twice. I think that's the problem... load 300Mb into memory Maybe I look for other fs products. However download of files off ZODB isnt such a big problem (you should mount an extra storage for the files to avoid overly cache trashing) but the performance is more then acceptable. ZODB/Filestorage is currely only a problem if you regulary update those files (or objects anyway) because it would add a complete new version. Regards Tino Wildenhain ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] zexp viewer
Hi, I'm using plone and one of my page got corrupted and displaying blank page. I tried to undo but failed and got error. and in history I'm unable to copy to present. I can see zexp file is of 137kb, that shows information is available in that file. Please let me know if any third party tool is available to retrieve the data or another way is available. Please suggest. Thanks Regards Sunil Sagar Work (Direct): +91-22-6750 6001 email: [EMAIL PROTECTED] +=+ This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. +=+ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Download Large Files in Zope
Maciej Wisniowski ha scritto: Sorry, I push the wrong button when I reply I'll try your configuration, but - where are files stored? (I dont't see server B in your post) Read http://www.infrae.com/products/tramline There is description where files are stored - it seems I'll lose Zope User Restrictions Possibly not. This is application server that decides if user is allowed to upload/download file (by setting proper headers). [SOLVED] http://www.slinkp.com/code/zopestuff/blobnotes Your solution was good but I don't want to do all again so I tried all products mentioned above. The new version of ExtFile (1.54) have resolved the problem. It implements files download streaming as well as I want. Thank you Illorca ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zexp viewer
- Original Message - From: [EMAIL PROTECTED] To: zope@zope.org Sent: Wednesday, March 07, 2007 11:59 AM Subject: [Zope] zexp viewer Hi, I'm using plone and one of my page got corrupted and displaying blank page. I tried to undo but failed and got error. and in history I'm unable to copy to present. I can see zexp file is of 137kb, that shows information is available in that file. Please let me know if any third party tool is available to retrieve the data or another way is available. Please suggest. Using the ZMI you can create a new folder and import the zexp file into that folder. Then copy what you need from there. Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zexp viewer
Still getting bank page. --Sunil sagar Jonathan [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 03/07/2007 10:38 PM To zope@zope.org, [EMAIL PROTECTED] cc Subject Re: [Zope] zexp viewer - Original Message - From: [EMAIL PROTECTED] To: zope@zope.org Sent: Wednesday, March 07, 2007 11:59 AM Subject: [Zope] zexp viewer Hi, I'm using plone and one of my page got corrupted and displaying blank page. I tried to undo but failed and got error. and in history I'm unable to copy to present. I can see zexp file is of 137kb, that shows information is available in that file. Please let me know if any third party tool is available to retrieve the data or another way is available. Please suggest. Using the ZMI you can create a new folder and import the zexp file into that folder. Then copy what you need from there. Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) +=+ This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. +=+ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zexp viewer
- Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: zope@zope.org; [EMAIL PROTECTED] Sent: Wednesday, March 07, 2007 12:38 PM Subject: Re: [Zope] zexp viewer Still getting bank page. --Sunil sagar If you have: 1- deleted the problem routine (script? zpt? dtml method?) 2- imported the zexp into a new folder 3- copied the problem routine from the imported area into the 'production' area and still have the same problem then: 1- there is a caching issue or 2- the routine you deleted/imported/copied is not what is causing the problem or 3- the routine in the zexp is also corrupted. Jonathan I'm using plone and one of my page got corrupted and displaying blank page. I tried to undo but failed and got error. and in history I'm unable to copy to present. I can see zexp file is of 137kb, that shows information is available in that file. Please let me know if any third party tool is available to retrieve the data or another way is available. Please suggest. Using the ZMI you can create a new folder and import the zexp file into that folder. Then copy what you need from there. Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] safe read-only access to acquisition parent objects
Eric Bréhault wrote at 2007-3-7 10:32 +0100: ... Indeed, I have been looking in all the restricted interpreter things. I have been reading zope\security\untristedinterpreter.txt, and I think it is probably what I need. The thing is I don't know how to produce a security proxy which would allow any 'get' access and forbide any 'set' access. Security proxies are a Zope 3 thing. In Zope 2 land, set accesses are forbidden by default, get accesses follow the standard Zope security pattern. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] invalid property value: wrapped object
Please stay on the list! I usually do not respond to private email... List readded. Anthony Monta wrote at 2007-3-6 13:09 -0800: All I have is a form passing stuff through REQUEST to a method that maps REQUEST variables to properties explicitly, and passing that to the ZClass constructor method. Is there any error-message I could parse to see where exactly the process goes wrong? Usually, you should look at the traceback you find in the so called error_log object (in your Zope Root Folder im the ZMI). It precisely tells you where exceptions are raised in the code and how you came there -- an incredibly helpful tool to understand any kind of exception. In your special case, the traceback may not yet tell you which property makes the problem. If you are ready to modify your Zope code (unless you put Zope into a revision control system, your changes will be lost with the next Zope version), then you can look at the traceback and find the place where the exception is raised. Before this place, you can add an __traceback_info__ = whatever information your are interested in In case of a traceback, this information is then included in it. In your special case, you will use the property id in __traceback_info__ to find out which property gives you the problem. I would approach this problem differently. I would try to reproduce the problem in an interactive Python interpreter (under *nix started with bin/zopectl debug; for Windows, there are alternative ways) and use from pdb import pm; pm() and then analyse the exeption state in pdb. There is also a Zope product that allows you to do that in a life Zope system -- but I forgot its name. You need to search for it. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zexp viewer
[EMAIL PROTECTED] wrote at 2007-3-7 22:29 +0530: I'm using plone and one of my page got corrupted and displaying blank page. I tried to undo but failed and got error. and in history I'm unable to copy to present. I can see zexp file is of 137kb, that shows information is available in that file. Please let me know if any third party tool is available to retrieve the data or another way is available. I have several times posted code to Zope mailing lists that allows to retrieve older versions of an object. Search the archives for getHistory. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Problems starting Zope 2.9.6 (ZEO starts fine)
I have setup Zope with ZEO on CentOS4.4, with the intention of running a Plone site. I previously used the same configuration to get Zope 2.9.3 with ZEO running on CentOS 3.7 * CentOS is setup without SELinux. * Python 2.5 is installed in /opt/python * Zeo starts fine, using /opt/zope/zeo/bin/zeoctl start I have created three zope instances in /opt/zope/pub0 /opt/zope/mem0 /opt/zope/sec0 When I try to start any of them, I get: [EMAIL PROTECTED] zope]$ pub0/bin/zopectl start Traceback (most recent call last): File /opt/zope/zope/lib/python/Zope2/Startup/zopectl.py, line 322, in module main() File /opt/zope/zope/lib/python/Zope2/Startup/zopectl.py, line 280, in main options.realize(args) File /opt/zope/zope/lib/python/Zope2/Startup/zopectl.py, line 91, in realize ZDOptions.realize(self, *args, **kw) File /opt/zope/z2.9.6-p2.5/lib/python/zdaemon/zdoptions.py, line 273, in realize self.load_schema() File /opt/zope/z2.9.6-p2.5/lib/python/zdaemon/zdoptions.py, line 321, in load_schema self.schema = ZConfig.loadSchema(self.schemafile) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/loader.py, line 31, in loadSchema return SchemaLoader().loadURL(url) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/loader.py, line 65, in loadURL return self.loadResource(r) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/loader.py, line 159, in loadResource schema = ZConfig.schema.parseResource(resource, self) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/schema.py, line 27, in parseResource xml.sax.parse(resource.file, parser) File /opt/python/lib/python2.5/xml/sax/__init__.py, line 33, in parse parser.parse(source) File /opt/python/lib/python2.5/xml/sax/expatreader.py, line 107, in parse xmlreader.IncrementalParser.parse(self, source) File /opt/python/lib/python2.5/xml/sax/xmlreader.py, line 123, in parse self.feed(buffer) File /opt/python/lib/python2.5/xml/sax/expatreader.py, line 207, in feed self._parser.Parse(data, isFinal) File /opt/python/lib/python2.5/xml/sax/expatreader.py, line 301, in start_element self._cont_handler.startElement(name, AttributesImpl(attrs)) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/schema.py, line 99, in startElement getattr(self, start_ + name)(attrs) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/schema.py, line 475, in start_schema keytype, valuetype, datatype = self.get_sect_typeinfo(attrs) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/schema.py, line 201, in get_sect_typeinfo datatype = self.get_datatype(attrs, datatype, null, base) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/schema.py, line 194, in get_datatype return self._registry.get(dtname) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/datatypes.py, line 398, in get t = self.search(name) File /opt/zope/z2.9.6-p2.5/lib/python/ZConfig/datatypes.py, line 423, in search package = __import__(n, g, g, component) File /opt/zope/z2.9.6-p2.5/lib/python/Zope2/Startup/datatypes.py, line 21, in module import OFS.Uninstalled File /opt/zope/z2.9.6-p2.5/lib/python/OFS/Uninstalled.py, line 20, in module import SimpleItem, Globals, Acquisition File /opt/zope/z2.9.6-p2.5/lib/python/OFS/SimpleItem.py, line 26, in module import AccessControl.Role, AccessControl.Owned, App.Common File /opt/zope/z2.9.6-p2.5/lib/python/AccessControl/__init__.py, line 17, in module from Implementation import setImplementation File /opt/zope/z2.9.6-p2.5/lib/python/AccessControl/Implementation.py, line 98, in module setImplementation(C) File /opt/zope/z2.9.6-p2.5/lib/python/AccessControl/Implementation.py, line 51, in setImplementation from AccessControl import ImplC as impl File /opt/zope/z2.9.6-p2.5/lib/python/AccessControl/ImplC.py, line 18, in module from cAccessControl import rolesForPermissionOn, \ File /opt/zope/z2.9.6-p2.5/lib/python/AccessControl/SimpleObjectPolicies.py, line 82, in module from DocumentTemplate.DT_Util import TemplateDict File /opt/zope/z2.9.6-p2.5/lib/python/DocumentTemplate/__init__.py, line 21, in module from DocumentTemplate import String, File, HTML, HTMLDefault, HTMLFile File /opt/zope/z2.9.6-p2.5/lib/python/DocumentTemplate/DocumentTemplate.py, line 112, in module from DT_String import String, File File /opt/zope/z2.9.6-p2.5/lib/python/DocumentTemplate/DT_String.py, line 19, in module from DT_Util import ParseError, InstanceDict, TemplateDict, render_blocks, str File /opt/zope/z2.9.6-p2.5/lib/python/DocumentTemplate/DT_Util.py, line 19, in module from html_quote import html_quote, ustr # for import by other modules, dont remove! File /opt/zope/z2.9.6-p2.5/lib/python/DocumentTemplate/html_quote.py, line 4, in module from ustr import ustr File /opt/zope/z2.9.6-p2.5/lib/python/DocumentTemplate/ustr.py, line 18, in module nasty_exception_str = Exception.__str__.im_func AttributeError: 'wrapper_descriptor' object has no attribute 'im_func' ___ Zope maillist - Zope@zope.org
Re: [Zope] Problems starting Zope 2.9.6 (ZEO starts fine)
On 3/8/07, Jens Vagelpohl [EMAIL PROTECTED] wrote: This is your problem. Zope 2.9 requires Python 2.4, and specifically does not want Python 2.5. Whoops! Sorry about that. Thankyou, Ben ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope-DB] form-controllerPythonScript-ZSQL
Hello everyone:
i´m doing a form where i get email and one ask. when I check that
email is valid, I do Controller Python Script where i want to create:
- DB conecction
- ZSQL method to modify DB
- get user session
and after i call a zpt, it tell me that all it´s ok
DB conection:
db_system_type = REQUEST.get('db_system_type', 'mysql')
from Products.ZMySQLDA import DA
DA.manage_addZMySQLConnection(self, db_conn_id, '',
REQUEST.get('db_conn_str', '[EMAIL PROTECTED] zforum zforum'))
zsql:
how to do it
Thanks for your help
___
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db
Re: [Zope-DB] form-controllerPythonScript-ZSQL
zsql: how to do it Have you read 'Relational database connectivity' chapter of Zope Book? -- Maciej Wisniowski ___ Zope-DB mailing list Zope-DB@zope.org http://mail.zope.org/mailman/listinfo/zope-db
