Re: [Zope3-dev] Re: [Zope-dev] Re: Proposal for optimized Blob handling
Good point.
I'll re-read the spec and will try to integrate that.
Am Mittwoch, den 07.03.2007, 22:46 -0500 schrieb Chris McDonough:
Note that one micro-optimization for PUT requests is to not use a
FieldStorage at all because the body is never mime-encoded anyway in
practice.
I have a monkey patch to do this now, which I turned into a patch for
the core, but took out because Phillipp whined at a sprint once. ;-)
Here's the monkey patch...
def patch_httprequest_processinputs():
Patch HTTPRequest.processInputs to not do any processing on a
PUT request (it's pointless, and foils our on-the-fly encryption,
as it creates a new tempfile via FieldStorage).
# note that OTF encryption support only works for PUT requests
import re
from ZPublisher.HTTPRequest import HTTPRequest
oldProcessInputs = HTTPRequest.processInputs
def newProcessInputs(
self,
# static variables that we want to be local for speed
SEQUENCE=1,
DEFAULT=2,
RECORD=4,
RECORDS=8,
REC=12, # RECORD|RECORDS
EMPTY=16,
CONVERTED=32,
hasattr=hasattr,
getattr=getattr,
setattr=setattr,
search_type=re.compile('(:[a-zA-Z][-a-zA-Z0-9_]+|\\.[xy])
$').search,
):
Process request inputs
We need to delay input parsing so that it is done under
publisher control for error handling purposes.
method=self.environ.get('REQUEST_METHOD','GET')
if method == 'PUT':
# we don't need to do any real input processing if we
are handling
# a PUT request. This is an optimization especially
because
# FieldStorage creates an additional tempfile if we
allow it to
# parse the body, and PUT uploads can tend to be large.
self._file = self.stdin
return
return oldProcessInputs(self)
HTTPRequest.processInputs = newProcessInputs
- C
On Mar 7, 2007, at 9:57 PM, Philipp von Weitershausen wrote:
Christian Theune wrote:
Hi,
[modified slightly from a similar proposal to zope3-dev to match Zope
2's publisher]
I'm writing up a proposal for the ZODB to make even more efficient
Blob
handling possible.
This includes not copying the data from an uploaded file, but using a
`link` operation when possible.
I think this is a great idea.
Am I the only person here who immediately associated link with
the POSIX? Also, am I the only one who read when possible as
when on a POSIX system where link is available, in other words,
when not on Windows? One starts to wonder...
However, the HTTPRequest class currently uses the default
implementation
of the cgi module's FieldStorage.
I propose to create a small subclass to override the `make_file`
method
to use `NamedTemporaryFile` instead of `TemporaryFile` to allow
the file
being accessible from a filename so I can apply a `link` operation.
+1
--
http://worldcookery.com -- Professional Zope documentation and
training
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/ct%40gocept.com
--
gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development
signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Proposal for optimized Blob handling
Christian Theune wrote: Hi, [modified slightly from a similar proposal to zope3-dev to match Zope 2's publisher] I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. I think this is a great idea. Am I the only person here who immediately associated link with the POSIX? Also, am I the only one who read when possible as when on a POSIX system where link is available, in other words, when not on Windows? One starts to wonder... However, the HTTPRequest class currently uses the default implementation of the cgi module's FieldStorage. I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. +1 -- http://worldcookery.com -- Professional Zope documentation and training ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Proposal for optimized Blob handling
Note that one micro-optimization for PUT requests is to not use a
FieldStorage at all because the body is never mime-encoded anyway in
practice.
I have a monkey patch to do this now, which I turned into a patch for
the core, but took out because Phillipp whined at a sprint once. ;-)
Here's the monkey patch...
def patch_httprequest_processinputs():
Patch HTTPRequest.processInputs to not do any processing on a
PUT request (it's pointless, and foils our on-the-fly encryption,
as it creates a new tempfile via FieldStorage).
# note that OTF encryption support only works for PUT requests
import re
from ZPublisher.HTTPRequest import HTTPRequest
oldProcessInputs = HTTPRequest.processInputs
def newProcessInputs(
self,
# static variables that we want to be local for speed
SEQUENCE=1,
DEFAULT=2,
RECORD=4,
RECORDS=8,
REC=12, # RECORD|RECORDS
EMPTY=16,
CONVERTED=32,
hasattr=hasattr,
getattr=getattr,
setattr=setattr,
search_type=re.compile('(:[a-zA-Z][-a-zA-Z0-9_]+|\\.[xy])
$').search,
):
Process request inputs
We need to delay input parsing so that it is done under
publisher control for error handling purposes.
method=self.environ.get('REQUEST_METHOD','GET')
if method == 'PUT':
# we don't need to do any real input processing if we
are handling
# a PUT request. This is an optimization especially
because
# FieldStorage creates an additional tempfile if we
allow it to
# parse the body, and PUT uploads can tend to be large.
self._file = self.stdin
return
return oldProcessInputs(self)
HTTPRequest.processInputs = newProcessInputs
- C
On Mar 7, 2007, at 9:57 PM, Philipp von Weitershausen wrote:
Christian Theune wrote:
Hi,
[modified slightly from a similar proposal to zope3-dev to match Zope
2's publisher]
I'm writing up a proposal for the ZODB to make even more efficient
Blob
handling possible.
This includes not copying the data from an uploaded file, but using a
`link` operation when possible.
I think this is a great idea.
Am I the only person here who immediately associated link with
the POSIX? Also, am I the only one who read when possible as
when on a POSIX system where link is available, in other words,
when not on Windows? One starts to wonder...
However, the HTTPRequest class currently uses the default
implementation
of the cgi module's FieldStorage.
I propose to create a small subclass to override the `make_file`
method
to use `NamedTemporaryFile` instead of `TemporaryFile` to allow
the file
being accessible from a filename so I can apply a `link` operation.
+1
--
http://worldcookery.com -- Professional Zope documentation and
training
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
