[Zope-dev] Zope Tests: 7 OK
Summary of messages to the zope-tests list. Period Tue Mar 6 12:00:00 2007 UTC to Wed Mar 7 12:00:00 2007 UTC. There were 7 messages: 7 from Zope Unit Tests. Tests passed OK --- Subject: OK : Zope-2.6 Python-2.1.3 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:08:30 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007398.html Subject: OK : Zope-2.6 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:10:00 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007399.html Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:11:30 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007400.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:13:00 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007401.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:14:30 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007402.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:16:01 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007403.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:17:31 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007404.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Proposal for optimized Blob handling
Hi, [modified slightly from a similar proposal to zope3-dev to match Zope 2's publisher] I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. However, the HTTPRequest class currently uses the default implementation of the cgi module's FieldStorage. I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. Notice: The FieldStorage explicitly provides the `make_file` method to allow overriding in this sense. Does anybody feel like this would be a bad idea? Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Caching ZCatalog results
On 3/6/07, Dieter Maurer [EMAIL PROTECTED] wrote: Lennart Regebro wrote at 2007-2-23 21:25 +0100: ... Compared with Lucene for example, which instead will create iterators who will only resturn the next match. This saves you from a lot of index searching when you have big results. I don't know if it is feasible to do something like that, but it would be interesting to look into it. It is done in IncrementalSearch2. http://www.dieter.handshake.de/pyprojects/zope Cool! This also needs to be in core. As most of your stuff. ;-) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 14:01 -0300 schrieb Sidnei da Silva: What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? No, I'd create a new hard link into the blob directory so the link to the temporary file can go away without making the inode go away. For the purposes of storing blobs the TMP directory should be on the same partition as the blobs directory anyway. Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
On Wed, Mar 07, 2007 at 06:31:25PM +0100, Christian Theune wrote: Am Mittwoch, den 07.03.2007, 14:01 -0300 schrieb Sidnei da Silva: What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? No, I'd create a new hard link into the blob directory so the link to the temporary file can go away without making the inode go away. For the purposes of storing blobs the TMP directory should be on the same partition as the blobs directory anyway. I like the idea, but what will you do if this fails? (eg. the admin has put TMP on a different mount, or we're running on Windows). -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
Christian Theune-2 wrote: Am Mittwoch, den 07.03.2007, 14:01 -0300 schrieb Sidnei da Silva: What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? No, I'd create a new hard link into the blob directory so the link to the temporary file can go away without making the inode go away. For the purposes of storing blobs the TMP directory should be on the same partition as the blobs directory anyway. Does this work on Windows? Martin -- View this message in context: http://www.nabble.com/Proposal-for-optimized-Blob-handling-tf3363320.html#a9357888 Sent from the Zope - Dev mailing list archive at Nabble.com. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 09:34 -0800 schrieb Martin Aspeli: Christian Theune-2 wrote: Am Mittwoch, den 07.03.2007, 14:01 -0300 schrieb Sidnei da Silva: What exactly do you mean by 'link'? As in 'soft links'? The uploaded file usually is a temporary file, so you are saying you would create a soft link on the 'blobs' directory to a file in the $TMP directory? Or maybe the other way around? No, I'd create a new hard link into the blob directory so the link to the temporary file can go away without making the inode go away. For the purposes of storing blobs the TMP directory should be on the same partition as the blobs directory anyway. Does this work on Windows? Link does not work on Windows using the link() function from the os module. I don't know whether Windows has any API for doing this kind of operation. In any case we can fall back (e.g. if the link() call fails) to copying the data as this is just an optimization. Please consider looking at my upcoming proposal for discussion. In this thread I'd like to keep the focus on the change of the publisher to use NamedTemporaryFile. Christian PS: Thanks for the input though. -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Early processing of request body (was: Proposal for optimized Blob handling)
I was looking through some publisher code and found that the `process_request` method which takes the request body as a file-like object and processes it as a FieldStorage happens within the application thread. This would be better if it happened beforehand because it can takes up time while a transaction is running and a thread is used although it doesn't require any application-specific code. AFAICT a modified version of FieldStorage would be required to allow line-wise consumption and parsing of data while it is being uploaded and then hand this into the request instead of stdin. However, the FieldStorage implementation is recursive and wasn't obvious to me at a first glance how much work it would be to replace this. Are there similar feelings it would be a good idea to do this kind of early line-wise processing of request bodies? Christian Am Mittwoch, den 07.03.2007, 17:44 +0100 schrieb Christian Theune: Hi, [modified slightly from a similar proposal to zope3-dev to match Zope 2's publisher] I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. However, the HTTPRequest class currently uses the default implementation of the cgi module's FieldStorage. I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. Notice: The FieldStorage explicitly provides the `make_file` method to allow overriding in this sense. Does anybody feel like this would be a bad idea? Christian ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] anon http svn access status?
So: will anon http svn access happen? I'm late to this round of the discussion, but +1. I would suggest https, though, to make it firewall-friendly. Even if this is an anonymous mirror (sounds more likely), it would make sense for community bundle builders (e.g. Plone bundles) to use https URLs for everything, which supposes upstream can provide https. Neither http, svnserve, or svn+ssh are friendly to corporate firewalls - only https (where the firewall can't see the http method verbs because they are inside a TLS session). I've griped about this before: http://www.mostscript.com/weblog/?p=26 Today, I have to put in requests to my company network admins to open up to svn.zope.org:3690 out on a workstation by workstation basis. Their firewall can do a generic TCP socket proxy (network security folks want to run an app-level proxy, so we can only set this up one source IP at a time), and doesn't deal with DeltaV DAV verbs over plain-old http - so https is the only reasonable option for anonymous checkouts. Thanks, Sean +--+ Sean Upton SignOnSanDiego.com Site Technology Supervisor The San Diego Union-Tribune 619.718.5241 [EMAIL PROTECTED] 350 Camino De La Reina San Diego, CA 92108 Plone Powered! plone.org ++ python.org ++ zope.org +--+ ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Early processing of request body (was: Proposal for optimized Blob handling)
Christian Theune wrote at 2007-3-7 20:09 +0100: I was looking through some publisher code and found that the `process_request` method which takes the request body as a file-like object and processes it as a FieldStorage happens within the application thread. This would be better if it happened beforehand because it can takes up time while a transaction is running and a thread is used although it doesn't require any application-specific code. In my view, it already now happens far too early, because it may raise exceptions and those exceptions are not handled by the standard_error_message usually used for error processing depending on the url. Therefore, if you move out things, you should take care that you move out only parts that cannot raise exceptions. Furthermore, you seem to propose to move work from a worker thread to the IO (i.e. ZServer) thread. I do not think that it is a good idea to put significant work on the IO thread. Note, that the IO thread is responsible to handle all IO. When you keep it busy with other tasks, it will not handle IO... -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Early processing of request body (was: Proposal for optimized Blob handling)
Hi, Am Mittwoch, den 07.03.2007, 21:48 +0100 schrieb Dieter Maurer: Christian Theune wrote at 2007-3-7 20:09 +0100: I was looking through some publisher code and found that the `process_request` method which takes the request body as a file-like object and processes it as a FieldStorage happens within the application thread. This would be better if it happened beforehand because it can takes up time while a transaction is running and a thread is used although it doesn't require any application-specific code. In my view, it already now happens far too early, because it may raise exceptions and those exceptions are not handled by the standard_error_message usually used for error processing depending on the url. Therefore, if you move out things, you should take care that you move out only parts that cannot raise exceptions. Ah. Interesting point! Furthermore, you seem to propose to move work from a worker thread to the IO (i.e. ZServer) thread. I do not think that it is a good idea to put significant work on the IO thread. Note, that the IO thread is responsible to handle all IO. When you keep it busy with other tasks, it will not handle IO... Right. This optimization is about leveraging the fact that in many situations the upstream bandwith is *much* lower than the IO bandwith to the disk. Another condition that I have (and I think this is the general pattern) is that application threads should be given back to the pool as quickly as possible. If 5 seconds are spend in the application thread to untangle mime data which has nothing application-specific about it and then only 100ms or so in the application itself, I'd say there is a major overhead problem. Christiabn -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] Google Summer of Code
On 3/5/07, Philipp von Weitershausen [EMAIL PROTECTED] wrote: a) mentors. It'd be great if some of the Zope core committers would volunteer to mentor a student. This doesn't mean you will definitely end up mentoring one, just show your willingness. Yeah, I could do that. b) project suggestions. I'm sure I can come up with several. Finishing the twisted integration in Zope2, for example. It only does http yet we need ftp too, and maybe more. So, I think this is a good idea. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Tests: 7 OK
Am Mittwoch, den 07.03.2007, 13:00 +0100 schrieb Zope Tests Summarizer: Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Mar 6 21:17:31 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-March/007404.html I'm questioning those tests. I have two tests failing on a fresh checkout of the Zope 2 trunk on my machine in this setting (Linux/Python 2.4.4) Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Road to Zope 2.11
Hi, few questions: a) I want to do the switch to ZODB 3.8 (currently trunk) b) That needs zope.proxy as of Zope 3.4 (currently trunk) Is there anything that would block this and if something how I can contribute unblocking it? Christian -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZopeTestCase.doctest is not aware of local sites
Hi again, I just noticed that the publish method from the Functional base class from ZopeTestCase has the same problem. You can find the adjusted test attached. Hanno Hanno Schlichting wrote: Hi all, while pushing local site support in CMF/Plone I came across some weird test failures. After a while of debbuging I could track this down to the http method in ZopeTestCase's doctest support. The method in its Zope2 incarnation is not aware of local sites, while the one in zope.app.testing is. I have attached a patch that mirrors the Zope3 behavior and saves the site information before publishing a module and restores it back, just like the security manager is saved/restored. As this is causing some test failures in Plone 3 now, I would be happy if somebody could take a look at this. Thank you, Hanno Index: /opt/plone30/parts/zope2/lib/python/Testing/ZopeTestCase/functional.py === --- /opt/plone30/parts/zope2/lib/python/Testing/ZopeTestCase/functional.py (revision 73024) +++ /opt/plone30/parts/zope2/lib/python/Testing/ZopeTestCase/functional.py (working copy) @@ -37,6 +37,7 @@ request_method='GET', stdin=None, handle_errors=True): '''Publishes the object at 'path' returning a response object.''' +from zope.app.component.hooks import setSite, getSite from StringIO import StringIO from ZPublisher.Response import Response from ZPublisher.Test import publish_module @@ -47,6 +48,10 @@ # Save current security manager sm = getSecurityManager() +# And we need to store the old site +old_site = getSite() +setSite(None) + # Commit the sandbox for good measure transaction.commit() @@ -89,6 +94,9 @@ # Restore security manager setSecurityManager(sm) +# And we need to restore the site again +setSite(old_site) + return ResponseWrapper(response, outstream, path) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Road to Zope 2.11
Christian Theune wrote: a) I want to do the switch to ZODB 3.8 (currently trunk) b) That needs zope.proxy as of Zope 3.4 (currently trunk) Is there anything that would block this and if something how I can contribute unblocking it? The only way to really find out is to try it. Zope 3.3 and 3.4 are close enough not to cause too much trouble, I think. -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Proposal for optimized Blob handling
On 3/7/07, Christian Theune [EMAIL PROTECTED] wrote: Does this work on Windows? Link does not work on Windows using the link() function from the os module. I don't know whether Windows has any API for doing this kind of operation. Yes [1] it [2] does [3]. The omission on os.link() is just the lack of a good soul to contribute it apparently [4]. A good task for a friday evening. [1] http://msdn2.microsoft.com/en-us/library/aa363860.aspx [2] http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/fsutil_hardlink.mspx?mfr=true [3] http://www.flexhex.com/docs/articles/hard-links.phtml [4] http://www.thescripts.com/forum/thread537011.html -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Early processing of request body (was: Proposal for optimized Blob handling)
On 3/7/07, Christian Theune [EMAIL PROTECTED] wrote: Right. This optimization is about leveraging the fact that in many situations the upstream bandwith is *much* lower than the IO bandwith to the disk. Another condition that I have (and I think this is the general pattern) is that application threads should be given back to the pool as quickly as possible. If 5 seconds are spend in the application thread to untangle mime data which has nothing application-specific about it and then only 100ms or so in the application itself, I'd say there is a major overhead problem. I had came to the same conclusion a couple weeks ago, somehow *wink*. Maybe we've been influenced by the same person. :) So if the uploaded file shouldn't be handled by the application thread, neither by the IO layer, then I guess we need a 'upload handling thread pool' of some sorts, whose sole purpose is to handle incoming requests that are large before it gets to the application thread while still outside the async IO layer. Hopefully something similar could be done for files being sent *out* of the application when they don't need any application processing anymore (ie, Blobs!). -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] Google Summer of Code
On 7 Mar 2007, at 12:24 , Lennart Regebro wrote: On 3/5/07, Philipp von Weitershausen [EMAIL PROTECTED] wrote: a) mentors. It'd be great if some of the Zope core committers would volunteer to mentor a student. This doesn't mean you will definitely end up mentoring one, just show your willingness. Yeah, I could do that. Great. Please add yourself with your Google account to http:// wiki.zope.org/zope3/SummerOfCode2007. b) project suggestions. I'm sure I can come up with several. Finishing the twisted integration in Zope2, for example. It only does http yet we need ftp too, and maybe more. Very good idea! Actually, this sort of goes along with the WSGI idea: making WSGI a first-class citizen in Zope 2. Feel free to either amend Sidnei's and my original suggestion or create a new one at http://wiki.zope.org/zope3/SummerOfCode2007. Philipp ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Proposal for optimized Blob handling
Christian Theune wrote: Hi, [modified slightly from a similar proposal to zope3-dev to match Zope 2's publisher] I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. I think this is a great idea. Am I the only person here who immediately associated link with the POSIX? Also, am I the only one who read when possible as when on a POSIX system where link is available, in other words, when not on Windows? One starts to wonder... However, the HTTPRequest class currently uses the default implementation of the cgi module's FieldStorage. I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. +1 -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Proposal for optimized Blob handling
Note that one micro-optimization for PUT requests is to not use a
FieldStorage at all because the body is never mime-encoded anyway in
practice.
I have a monkey patch to do this now, which I turned into a patch for
the core, but took out because Phillipp whined at a sprint once. ;-)
Here's the monkey patch...
def patch_httprequest_processinputs():
Patch HTTPRequest.processInputs to not do any processing on a
PUT request (it's pointless, and foils our on-the-fly encryption,
as it creates a new tempfile via FieldStorage).
# note that OTF encryption support only works for PUT requests
import re
from ZPublisher.HTTPRequest import HTTPRequest
oldProcessInputs = HTTPRequest.processInputs
def newProcessInputs(
self,
# static variables that we want to be local for speed
SEQUENCE=1,
DEFAULT=2,
RECORD=4,
RECORDS=8,
REC=12, # RECORD|RECORDS
EMPTY=16,
CONVERTED=32,
hasattr=hasattr,
getattr=getattr,
setattr=setattr,
search_type=re.compile('(:[a-zA-Z][-a-zA-Z0-9_]+|\\.[xy])
$').search,
):
Process request inputs
We need to delay input parsing so that it is done under
publisher control for error handling purposes.
method=self.environ.get('REQUEST_METHOD','GET')
if method == 'PUT':
# we don't need to do any real input processing if we
are handling
# a PUT request. This is an optimization especially
because
# FieldStorage creates an additional tempfile if we
allow it to
# parse the body, and PUT uploads can tend to be large.
self._file = self.stdin
return
return oldProcessInputs(self)
HTTPRequest.processInputs = newProcessInputs
- C
On Mar 7, 2007, at 9:57 PM, Philipp von Weitershausen wrote:
Christian Theune wrote:
Hi,
[modified slightly from a similar proposal to zope3-dev to match Zope
2's publisher]
I'm writing up a proposal for the ZODB to make even more efficient
Blob
handling possible.
This includes not copying the data from an uploaded file, but using a
`link` operation when possible.
I think this is a great idea.
Am I the only person here who immediately associated link with
the POSIX? Also, am I the only one who read when possible as
when on a POSIX system where link is available, in other words,
when not on Windows? One starts to wonder...
However, the HTTPRequest class currently uses the default
implementation
of the cgi module's FieldStorage.
I propose to create a small subclass to override the `make_file`
method
to use `NamedTemporaryFile` instead of `TemporaryFile` to allow
the file
being accessible from a filename so I can apply a `link` operation.
+1
--
http://worldcookery.com -- Professional Zope documentation and
training
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] Re: Proposal for optimized Blob handling
On 3/7/07, Philipp von Weitershausen [EMAIL PROTECTED] wrote: Am I the only person here who immediately associated link with the POSIX? Also, am I the only one who read when possible as when on a POSIX system where link is available, in other words, when not on Windows? One starts to wonder... NTFS does support hard links since version 5, which means Windows 2000+. It does not support hard links to directories though, only soft links (which are called junctions or junction points). The version of NTFS shipped with Vista supports hard links for directories I believe. -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] Re: Proposal for optimized Blob handling
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sidnei da Silva wrote: On 3/7/07, Philipp von Weitershausen [EMAIL PROTECTED] wrote: Am I the only person here who immediately associated link with the POSIX? Also, am I the only one who read when possible as when on a POSIX system where link is available, in other words, when not on Windows? One starts to wonder... NTFS does support hard links since version 5, which means Windows 2000+. It does not support hard links to directories though, only soft links (which are called junctions or junction points). The version of NTFS shipped with Vista supports hard links for directories I believe. POSIX systems don't allow hard links to directories, either, in practice:: $ man ln ... -d, -F, --directory allow the superuser to attempt to hard link directories (note: will probably fail due to system restrictions, even for the superuser) Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF76hn+gerLs4ltQ4RArm3AJ9o6Vw63qc6cJT3GPJOVebCFhDtiACfViDa EI1MxpxIIwEQl3uXVxly7M4= =NZUy -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
