Re: [Zope3-dev] Proposal for optimized Blob handling
Am Donnerstag, den 08.03.2007, 01:30 -0300 schrieb Sidnei da Silva: > On 3/7/07, Christian Theune <[EMAIL PROTECTED]> wrote: > > I propose to create a small subclass to override the `make_file` method > > to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file > > being accessible from a filename so I can apply a `link` operation. > > > > Notice: The FieldStorage explicitly provides the `make_file` method to > > allow overriding in this sense. > > Since you're proposing to replace TemporaryFile anyway, and your > 'os.link()' proposal requires that both the file and the hard link are > on the same 'drive' or 'partition' or whatever, why not create the > temporary file on a temp directory that is close (hopefully a sibling) > of the final destination blob directory, then you can be sure that > 'os.link()' will work and will be 'O(1)' without any extra effort. > > Is there any reason not to do that? Admin-Choice. You can already configure this directory to be on the same destination by setting your tempdir environment accordingly. -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
On 3/7/07, Uwe Oestermeier <[EMAIL PROTECTED]> wrote: Ok, but what Dieter means is that the tmp dir as a whole is emptied from time to time. Not a likely scenario other than at reboot time or when a clueless sysadmin clears it by hand. Automated /tmp cleaning takes file age into account to avoid deleting current files, at least on any UNIX system I have encountered. I don't think clueless sysadmins need protecting.. ;) -- Martijn Pieters ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
On 3/7/07, Benji York <[EMAIL PROTECTED]> wrote: Dieter Maurer wrote: > Python's "os.link" creates a hard link which will only work > if source and destination are on the same file system. ...and is also not available on Windows. os.link() isn't, but hard links are as long as you use NTFS [1]. [1] http://msdn2.microsoft.com/en-us/library/aa363860.aspx -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
On 3/7/07, Christian Theune <[EMAIL PROTECTED]> wrote: I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. Notice: The FieldStorage explicitly provides the `make_file` method to allow overriding in this sense. Since you're proposing to replace TemporaryFile anyway, and your 'os.link()' proposal requires that both the file and the hard link are on the same 'drive' or 'partition' or whatever, why not create the temporary file on a temp directory that is close (hopefully a sibling) of the final destination blob directory, then you can be sure that 'os.link()' will work and will be 'O(1)' without any extra effort. Is there any reason not to do that? -- Sidnei da Silva Enfold Systemshttp://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214 ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
On 3/7/07, Christian Theune <[EMAIL PROTECTED]> wrote:
This is how the dance looks like to do the link():
>>> import tempfile, os
>>> d = tempfile.NamedTemporaryFile()
>>> os.path.exists(d.name)
True
>>> d.write('Test')
>>> os.path.exists('/tmp/asdf')
False
>>> os.link(d.name, '/tmp/asdf')
>>> d.close()
>>> os.path.exists(d.name)
False
>>> os.path.exists('/tmp/asdf')
True
>>> open('/tmp/asdf').read()
'Test'
Ok, this is how it looks on Windows. :)
import os
import win32file
f = open('test.txt', 'wb')
f.write('hello world')
f.close()
win32file.CreateHardLink('hello.txt', 'test.txt')
os.listdir(os.getcwd())
['hello.txt', 'test.txt']
f = open('hello.txt', 'wb')
f.write('hello there')
f.close()
print open('hello.txt').read()
hello there
print open('test.txt').read()
hello there
os.remove('test.txt')
print open('hello.txt').read()
hello there
print open('test.txt').read()
Traceback (most recent call last):
File "", line 1, in ?
IOError: [Errno 2] No such file or directory: 'test.txt'
--
Sidnei da Silva
Enfold Systemshttp://enfoldsystems.com
Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Am Donnerstag, den 08.03.2007, 00:32 +0100 schrieb Wichert Akkerman:
> Previously Christian Theune wrote:
> > Am Mittwoch, den 07.03.2007, 21:31 +0100 schrieb Uwe Oestermeier:
> > > Christian Theune <[EMAIL PROTECTED]> schreibt:
> > > >
> > > >Nope. It won't disappear if you link it again. And the link(src, dst)
> > > >does move it to a 'save' location ;)
> > >
> > > Again I'm not convinced because you cannot be sure that no other process
> > > deletes the temp file.
> > > In the following I simulate this with a os.system call:
> > >
> > > >>> import tempfile, os
> > > >>> d = tempfile.NamedTemporaryFile()
> > > >>> os.path.exists(d.name)
> > > True
> > > >>> d.write('Test')
> > > >>> os.path.exists('/tmp/asdf')
> > > False
> > > >>> os.link(d.name, '/tmp/asdf')
> > > >>> d.close()
> > > >>> os.system('rm /tmp/asdf')
> > > 0
> > > >>> os.path.exists('/tmp/asdf')
> > > False
> > > >>> open('/tmp/asdf').read()
> > > Traceback (most recent call last):
> > > File "", line 1, in ?
> > > IOError: [Errno 2] No such file or directory: '/tmp/asdf'
> >
> > Nope this is not the correct simulation. Who except your application
> > knows about /tmp/asdf? You have to simulate deleting d.name and then
> > you'll see that /tmp/asdf does not disappear.
> >
> > The link operation *is* the rename except that now two files link to it.
> > This keeps the semantics of a NamedTemporaryFile in respect to the
> > request processing and the publisher intact.
>
> Two possible things come to mind:
> - if the tempfile is on a different filesystem you can not link the file
> elsehere. Can we guarantee that they will be on the same filesystem?
> - does this work on non-posix systems such as windows?
>
> Wichert.
Please have a look at http://wiki.zope.org/ZODB/BlobsZeroCopy before
continuing this thread.
--
gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development
signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Previously Christian Theune wrote:
> Am Mittwoch, den 07.03.2007, 21:31 +0100 schrieb Uwe Oestermeier:
> > Christian Theune <[EMAIL PROTECTED]> schreibt:
> > >
> > >Nope. It won't disappear if you link it again. And the link(src, dst)
> > >does move it to a 'save' location ;)
> >
> > Again I'm not convinced because you cannot be sure that no other process
> > deletes the temp file.
> > In the following I simulate this with a os.system call:
> >
> > >>> import tempfile, os
> > >>> d = tempfile.NamedTemporaryFile()
> > >>> os.path.exists(d.name)
> > True
> > >>> d.write('Test')
> > >>> os.path.exists('/tmp/asdf')
> > False
> > >>> os.link(d.name, '/tmp/asdf')
> > >>> d.close()
> > >>> os.system('rm /tmp/asdf')
> > 0
> > >>> os.path.exists('/tmp/asdf')
> > False
> > >>> open('/tmp/asdf').read()
> > Traceback (most recent call last):
> > File "", line 1, in ?
> > IOError: [Errno 2] No such file or directory: '/tmp/asdf'
>
> Nope this is not the correct simulation. Who except your application
> knows about /tmp/asdf? You have to simulate deleting d.name and then
> you'll see that /tmp/asdf does not disappear.
>
> The link operation *is* the rename except that now two files link to it.
> This keeps the semantics of a NamedTemporaryFile in respect to the
> request processing and the publisher intact.
Two possible things come to mind:
- if the tempfile is on a different filesystem you can not link the file
elsehere. Can we guarantee that they will be on the same filesystem?
- does this work on non-posix systems such as windows?
Wichert.
--
Wichert Akkerman <[EMAIL PROTECTED]>It is simple to make things.
http://www.wiggy.net/ It is hard to make things simple.
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 22:04 +0100 schrieb Uwe Oestermeier:
> Christian Theune <[EMAIL PROTECTED]> schreibt:
> >Nope this is not the correct simulation. Who except your application
> >knows about /tmp/asdf? You have to simulate deleting d.name and then
> >you'll see that /tmp/asdf does not disappear.
>
> Ok, but what Dieter means is that the tmp dir as a whole is emptied from
> time to time.
> I see your point that closing a temp file removes it whereas the link
> still works,
> but that does not solve the problem that system administrators may sweep
> things out.
It does because we do need tmp partition to be the same as the blob
partition.
> The following solution avoids this problem:
>
> >>> os.mkdir('/Users/uo/blobs')
> >>> d = tempfile.NamedTemporaryFile()
> >>> d.write('Test')
> >>> d.flush()
> >>> os.rename(d.name, '/Users/uo/blobs/asdf')
> >>> open('/Users/uo/blobs/asdf').read()
> 'Test'
> >>> os.path.exists('/tmp/asdf')
> False
This won't work because if it's not the same partition (which you imply
is needed because the tmp-partition might be cleared out by admins) then
rename isn't available.
This also doesn't work on windows as an opened file can not be renamed
on windows.
Again, another pointer to move this dicussion to zodb-dev.
Christian
--
gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development
signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 15:59 -0500 schrieb Benji York: > Dieter Maurer wrote: > > Python's "os.link" creates a hard link which will only work > > if source and destination are on the same file system. > > ...and is also not available on Windows. Hrn. I tried to point you all to the upcoming proposal only checking for the change in the publisher. This was a mistake as I didn't make it clear enough to handle those two issues seperately. Please check the proposal. I'm referring to all those issues there already. Christian -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Hi, Am Mittwoch, den 07.03.2007, 21:37 +0100 schrieb Dieter Maurer: > Christian Theune wrote at 2007-3-7 21:17 +0100: > > ... > >Nope. It won't disappear if you link it again. And the link(src, dst) > >does move it to a 'save' location ;) > > You do not tell us, which "link" you mean. Sorry. I mean os.link(). Again, check the ZODB list for the actual proposal. > Python's "os.link" creates a hard link which will only work > if source and destination are on the same file system. Thats a known restriction. You can only do any O(1) operation to moving data of a file within a single file system. > It is not uncommon that temporary files are on their own > filesystem. In the situation that you want to handle blobs this will not be possible. This is a restriction that I'm willing to deal with. Having a temporary file system that handles blobs in addition to a large filesystem that handles blobs might be required in certain situations, however, than the O(1) operation won't work and we do have a fallback implementation already anyway. Christian -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Christian Theune <[EMAIL PROTECTED]> schreibt:
>Nope this is not the correct simulation. Who except your application
>knows about /tmp/asdf? You have to simulate deleting d.name and then
>you'll see that /tmp/asdf does not disappear.
Ok, but what Dieter means is that the tmp dir as a whole is emptied from
time to time.
I see your point that closing a temp file removes it whereas the link
still works,
but that does not solve the problem that system administrators may sweep
things out.
The following solution avoids this problem:
>>> os.mkdir('/Users/uo/blobs')
>>> d = tempfile.NamedTemporaryFile()
>>> d.write('Test')
>>> d.flush()
>>> os.rename(d.name, '/Users/uo/blobs/asdf')
>>> open('/Users/uo/blobs/asdf').read()
'Test'
>>> os.path.exists('/tmp/asdf')
False
Uwe
Dr. Uwe Oestermeier
Institut für Wissensmedien
Knowledge Media Research Center
Konrad-Adenauer-Str. 40
D-72072 Tuebingen
Germany
[EMAIL PROTECTED]
Tel. +49 7071 979-208
Fax +49 7071 979-100
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Dieter Maurer wrote: Python's "os.link" creates a hard link which will only work if source and destination are on the same file system. ...and is also not available on Windows. -- Benji York Senior Software Engineer Zope Corporation ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Christian Theune wrote at 2007-3-7 21:17 +0100: > ... >Nope. It won't disappear if you link it again. And the link(src, dst) >does move it to a 'save' location ;) You do not tell us, which "link" you mean. Python's "os.link" creates a hard link which will only work if source and destination are on the same file system. It is not uncommon that temporary files are on their own filesystem. -- Dieter ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 21:31 +0100 schrieb Uwe Oestermeier:
> Christian Theune <[EMAIL PROTECTED]> schreibt:
> >
> >Nope. It won't disappear if you link it again. And the link(src, dst)
> >does move it to a 'save' location ;)
>
> Again I'm not convinced because you cannot be sure that no other process
> deletes the temp file.
> In the following I simulate this with a os.system call:
>
> >>> import tempfile, os
> >>> d = tempfile.NamedTemporaryFile()
> >>> os.path.exists(d.name)
> True
> >>> d.write('Test')
> >>> os.path.exists('/tmp/asdf')
> False
> >>> os.link(d.name, '/tmp/asdf')
> >>> d.close()
> >>> os.system('rm /tmp/asdf')
> 0
> >>> os.path.exists('/tmp/asdf')
> False
> >>> open('/tmp/asdf').read()
> Traceback (most recent call last):
> File "", line 1, in ?
> IOError: [Errno 2] No such file or directory: '/tmp/asdf'
Nope this is not the correct simulation. Who except your application
knows about /tmp/asdf? You have to simulate deleting d.name and then
you'll see that /tmp/asdf does not disappear.
The link operation *is* the rename except that now two files link to it.
This keeps the semantics of a NamedTemporaryFile in respect to the
request processing and the publisher intact.
Christian
--
gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development
signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Christian Theune <[EMAIL PROTECTED]> schreibt:
>
>Nope. It won't disappear if you link it again. And the link(src, dst)
>does move it to a 'save' location ;)
Again I'm not convinced because you cannot be sure that no other process
deletes the temp file.
In the following I simulate this with a os.system call:
>>> import tempfile, os
>>> d = tempfile.NamedTemporaryFile()
>>> os.path.exists(d.name)
True
>>> d.write('Test')
>>> os.path.exists('/tmp/asdf')
False
>>> os.link(d.name, '/tmp/asdf')
>>> d.close()
>>> os.system('rm /tmp/asdf')
0
>>> os.path.exists('/tmp/asdf')
False
>>> open('/tmp/asdf').read()
Traceback (most recent call last):
File "", line 1, in ?
IOError: [Errno 2] No such file or directory: '/tmp/asdf'
:)
Why not rename the temp file to a place in the blob directory?
That would also avoid the copy operation.
Uwe
Dr. Uwe Oestermeier
Institut für Wissensmedien
Knowledge Media Research Center
Konrad-Adenauer-Str. 40
D-72072 Tuebingen
Germany
[EMAIL PROTECTED]
Tel. +49 7071 979-208
Fax +49 7071 979-100
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 21:12 +0100 schrieb Uwe Oestermeier:
> Christian Theune <[EMAIL PROTECTED]> schreibt:
> >This is how the dance looks like to do the link():
> >
> > >>> import tempfile, os
> > >>> d = tempfile.NamedTemporaryFile()
> > >>> os.path.exists(d.name)
> > True
> > >>> d.write('Test')
> > >>> os.path.exists('/tmp/asdf')
> > False
> > >>> os.link(d.name, '/tmp/asdf')
> > >>> d.close()
> > >>> os.path.exists(d.name)
> > False
> > >>> os.path.exists('/tmp/asdf')
> > True
> > >>> open('/tmp/asdf').read()
> > 'Test'
>
> Yeah, but as Dieter said the temp file should be better renamed or moved
> to a save location.
> It is likely that a temp file disappears.
Nope. It won't disappear if you link it again. And the link(src, dst)
does move it to a 'save' location ;)
Christian
--
gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development
signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Christian Theune <[EMAIL PROTECTED]> schreibt:
>This is how the dance looks like to do the link():
>
> >>> import tempfile, os
> >>> d = tempfile.NamedTemporaryFile()
> >>> os.path.exists(d.name)
> True
> >>> d.write('Test')
> >>> os.path.exists('/tmp/asdf')
> False
> >>> os.link(d.name, '/tmp/asdf')
> >>> d.close()
> >>> os.path.exists(d.name)
> False
> >>> os.path.exists('/tmp/asdf')
> True
> >>> open('/tmp/asdf').read()
> 'Test'
Yeah, but as Dieter said the temp file should be better renamed or moved
to a save location.
It is likely that a temp file disappears.
Uwe
Dr. Uwe Oestermeier
Institut für Wissensmedien
Knowledge Media Research Center
Konrad-Adenauer-Str. 40
D-72072 Tuebingen
Germany
[EMAIL PROTECTED]
Tel. +49 7071 979-208
Fax +49 7071 979-100
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 21:03 +0100 schrieb Bernd Dorn: > On 07.03.2007, at 17:37, Christian Theune wrote: > > > Hi, > > > > I'm writing up a proposal for the ZODB to make even more efficient > > Blob > > handling possible. > > > > This includes not copying the data from an uploaded file, but using a > > `link` operation when possible. > > > > However, the Zope 3 publisher currently uses the default > > implementation > > of the cgi module's FieldStorage. > > > > I propose to create a small subclass to override the `make_file` > > method > > to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the > > file > > being accessible from a filename so I can apply a `link` operation. > > > > Notice: The FieldStorage explicitly provides the `make_file` method to > > allow overriding in this sense. > > > > Does anybody feel like this would be a bad idea? > > that would be nice, i would prefer to make the whoe FieldStorage > class pluggable via a factory interface > > this is a long outstanding issue for z3c.extfile too, i also wanted > to access the file directly I've started a little different dicussion on the FieldStorage implementation on the Zope 2 list as FieldStorage has some more problems. Christian -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 20:50 +0100 schrieb Dieter Maurer:
> Christian Theune wrote at 2007-3-7 17:37 +0100:
> >I'm writing up a proposal for the ZODB to make even more efficient Blob
> >handling possible.
> >
> >This includes not copying the data from an uploaded file, but using a
> >`link` operation when possible.
>
> Is it possible at all?
>
> Uploaded files end up in a temporary file.
>
> They need to get moved away from this temporary location
> (as they are likely to be deleted at various administrator decided dates).
This is how the dance looks like to do the link():
>>> import tempfile, os
>>> d = tempfile.NamedTemporaryFile()
>>> os.path.exists(d.name)
True
>>> d.write('Test')
>>> os.path.exists('/tmp/asdf')
False
>>> os.link(d.name, '/tmp/asdf')
>>> d.close()
>>> os.path.exists(d.name)
False
>>> os.path.exists('/tmp/asdf')
True
>>> open('/tmp/asdf').read()
'Test'
--
gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development
signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
On 07.03.2007, at 17:37, Christian Theune wrote: Hi, I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. However, the Zope 3 publisher currently uses the default implementation of the cgi module's FieldStorage. I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. Notice: The FieldStorage explicitly provides the `make_file` method to allow overriding in this sense. Does anybody feel like this would be a bad idea? that would be nice, i would prefer to make the whoe FieldStorage class pluggable via a factory interface this is a long outstanding issue for z3c.extfile too, i also wanted to access the file directly greez, bernd Christian -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/zope- mailinglist%40mopa.at ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Christian Theune wrote at 2007-3-7 17:37 +0100: >I'm writing up a proposal for the ZODB to make even more efficient Blob >handling possible. > >This includes not copying the data from an uploaded file, but using a >`link` operation when possible. Is it possible at all? Uploaded files end up in a temporary file. They need to get moved away from this temporary location (as they are likely to be deleted at various administrator decided dates). -- Dieter ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
Am Mittwoch, den 07.03.2007, 11:55 -0500 schrieb Stephan Richter: > On Wednesday 07 March 2007 11:37, Christian Theune wrote: > > Does anybody feel like this would be a bad idea? > > As long as you do not monkey patch the existing code, this is totally fine. *k* No. I was talking about doing it right (tm). ;) Christian -- gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Proposal for optimized Blob handling
On Wednesday 07 March 2007 11:37, Christian Theune wrote: > Does anybody feel like this would be a bad idea? As long as you do not monkey patch the existing code, this is totally fine. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
