Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-08 Thread Christian Theune
Am Donnerstag, den 08.03.2007, 01:30 -0300 schrieb Sidnei da Silva: On 3/7/07, Christian Theune [EMAIL PROTECTED] wrote: I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a

[Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Christian Theune
Hi, I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. However, the Zope 3 publisher currently uses the default implementation of the cgi module's

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Stephan Richter
On Wednesday 07 March 2007 11:37, Christian Theune wrote: Does anybody feel like this would be a bad idea? As long as you do not monkey patch the existing code, this is totally fine. Regards, Stephan -- Stephan Richter CBU Physics Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Christian Theune
Am Mittwoch, den 07.03.2007, 11:55 -0500 schrieb Stephan Richter: On Wednesday 07 March 2007 11:37, Christian Theune wrote: Does anybody feel like this would be a bad idea? As long as you do not monkey patch the existing code, this is totally fine. *k* No. I was talking about doing it

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Dieter Maurer
Christian Theune wrote at 2007-3-7 17:37 +0100: I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. Is it possible at all? Uploaded files end up in a

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Bernd Dorn
On 07.03.2007, at 17:37, Christian Theune wrote: Hi, I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation when possible. However, the Zope 3 publisher currently

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Christian Theune
Am Mittwoch, den 07.03.2007, 20:50 +0100 schrieb Dieter Maurer: Christian Theune wrote at 2007-3-7 17:37 +0100: I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a `link` operation

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Christian Theune
Am Mittwoch, den 07.03.2007, 21:03 +0100 schrieb Bernd Dorn: On 07.03.2007, at 17:37, Christian Theune wrote: Hi, I'm writing up a proposal for the ZODB to make even more efficient Blob handling possible. This includes not copying the data from an uploaded file, but using a

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Uwe Oestermeier
Christian Theune [EMAIL PROTECTED] schreibt: Nope. It won't disappear if you link it again. And the link(src, dst) does move it to a 'save' location ;) Again I'm not convinced because you cannot be sure that no other process deletes the temp file. In the following I simulate this with a

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Christian Theune
Am Mittwoch, den 07.03.2007, 21:31 +0100 schrieb Uwe Oestermeier: Christian Theune [EMAIL PROTECTED] schreibt: Nope. It won't disappear if you link it again. And the link(src, dst) does move it to a 'save' location ;) Again I'm not convinced because you cannot be sure that no other process

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Dieter Maurer
Christian Theune wrote at 2007-3-7 21:17 +0100: ... Nope. It won't disappear if you link it again. And the link(src, dst) does move it to a 'save' location ;) You do not tell us, which link you mean. Python's os.link creates a hard link which will only work if source and destination are on the

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Benji York
Dieter Maurer wrote: Python's os.link creates a hard link which will only work if source and destination are on the same file system. ...and is also not available on Windows. -- Benji York Senior Software Engineer Zope Corporation ___ Zope3-dev

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Uwe Oestermeier
Christian Theune [EMAIL PROTECTED] schreibt: Nope this is not the correct simulation. Who except your application knows about /tmp/asdf? You have to simulate deleting d.name and then you'll see that /tmp/asdf does not disappear. Ok, but what Dieter means is that the tmp dir as a whole is emptied

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Christian Theune
Hi, Am Mittwoch, den 07.03.2007, 21:37 +0100 schrieb Dieter Maurer: Christian Theune wrote at 2007-3-7 21:17 +0100: ... Nope. It won't disappear if you link it again. And the link(src, dst) does move it to a 'save' location ;) You do not tell us, which link you mean. Sorry. I mean

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Christian Theune
Am Mittwoch, den 07.03.2007, 15:59 -0500 schrieb Benji York: Dieter Maurer wrote: Python's os.link creates a hard link which will only work if source and destination are on the same file system. ...and is also not available on Windows. Hrn. I tried to point you all to the upcoming

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Christian Theune
Am Mittwoch, den 07.03.2007, 22:04 +0100 schrieb Uwe Oestermeier: Christian Theune [EMAIL PROTECTED] schreibt: Nope this is not the correct simulation. Who except your application knows about /tmp/asdf? You have to simulate deleting d.name and then you'll see that /tmp/asdf does not disappear.

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Wichert Akkerman
Previously Christian Theune wrote: Am Mittwoch, den 07.03.2007, 21:31 +0100 schrieb Uwe Oestermeier: Christian Theune [EMAIL PROTECTED] schreibt: Nope. It won't disappear if you link it again. And the link(src, dst) does move it to a 'save' location ;) Again I'm not convinced

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Christian Theune
Am Donnerstag, den 08.03.2007, 00:32 +0100 schrieb Wichert Akkerman: Previously Christian Theune wrote: Am Mittwoch, den 07.03.2007, 21:31 +0100 schrieb Uwe Oestermeier: Christian Theune [EMAIL PROTECTED] schreibt: Nope. It won't disappear if you link it again. And the link(src, dst)

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Sidnei da Silva
On 3/7/07, Christian Theune [EMAIL PROTECTED] wrote: This is how the dance looks like to do the link(): import tempfile, os d = tempfile.NamedTemporaryFile() os.path.exists(d.name) True d.write('Test') os.path.exists('/tmp/asdf') False os.link(d.name, '/tmp/asdf') d.close()

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Sidnei da Silva
On 3/7/07, Christian Theune [EMAIL PROTECTED] wrote: I propose to create a small subclass to override the `make_file` method to use `NamedTemporaryFile` instead of `TemporaryFile` to allow the file being accessible from a filename so I can apply a `link` operation. Notice: The FieldStorage

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Sidnei da Silva
On 3/7/07, Benji York [EMAIL PROTECTED] wrote: Dieter Maurer wrote: Python's os.link creates a hard link which will only work if source and destination are on the same file system. ...and is also not available on Windows. os.link() isn't, but hard links are as long as you use NTFS [1]. [1]

Re: [Zope3-dev] Proposal for optimized Blob handling

2007-03-07 Thread Martijn Pieters
On 3/7/07, Uwe Oestermeier [EMAIL PROTECTED] wrote: Ok, but what Dieter means is that the tmp dir as a whole is emptied from time to time. Not a likely scenario other than at reboot time or when a clueless sysadmin clears it by hand. Automated /tmp cleaning takes file age into account to avoid