Shane Hathaway wrote:
Jim Fulton wrote:
You are confusing Zope 2 and Zope 3. Zope 2's response.write
does handle large output effciently, Zope 3's did not.
Last time I worked on it, Zope 2 and Zope 3 used the same strategy for
handling large output. It worked, I promise. :-)
I believe you. Uh, did you test it?
> Yet I hesitate to
call the push-to-temporary-file strategy efficient, since it makes many
extra copies of the data and only helps when a lot of conditions are
Many extra copies? Seems it makes just one.
I'll also note that making file copies is reasonably fast.
> A pull strategy will be efficient for a lot more people.
I don't know what you mean by this.
Making code efficient interests me a lot.
> When I worked on this stuff
earlier, I thought of a way to get Zope to serve files and images out of
the database more efficiently. I'd like to know whether you think it's
a good idea; if it is, we can plan to implement it when it's needed.
The idea is this: run a thread whose only responsibility is to stream
image and file data out of the object database to browsers. The thread
will maintain a permanently open database connection. Other threads
will send open HTTP connections to this thread, asking it to finish the
response. The special thread will get the binary data from the database
using its own connection, which would play well with the ZODB cache,
since we'd only ever have one copy of the image/file data in memory at a
time. Contrast this with Zope 2's image and file serving, which loads
image/file data into arbitrary database connections and immediately
evicts them from the cache. It seems like having a special connection
just for serving images/files would be a lot more efficient, especially
when concurrency is high.
This sounds too complicated to me. Have you followed the ZODB blob work?
I think this will provide great benefits for people who choose to store large
(megabytes, not gigabytes or terabytes) files in the database. I think the
people working on it could use some help finishing it, if you'd like to help.
Blobs are managed as temporary files on a ZEO client, so that they
can be served by the publisher without making copies (except, of course,
for the copy necessary to get them to the client). They are also managed
outside the normal object and ZEO caches.
However, this assumes that most images and files come out of the object
database. I suspect that current Zope 3 software gets most images and
files from the filesystem, but that will change as Zope 3 takes on more
To the degree that Zope applications manage data as files, the new publisher
strategy will server them very well. Of course, I hope that blobs will
make ZODB storage a resonable option in the future, as it provides a number
of advantages, such as simplicity, transactions, and ZEO distribution.
Jim Fulton mailto:[EMAIL PROTECTED] Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
Zope3-dev mailing list