Jim Fulton wrote:
You are confusing Zope 2 and Zope 3. Zope 2's response.write
does handle large output effciently, Zope 3's did not.
Last time I worked on it, Zope 2 and Zope 3 used the same strategy for
handling large output. It worked, I promise. :-) Yet I hesitate to
call the push-to-temporary-file strategy efficient, since it makes many
extra copies of the data and only helps when a lot of conditions are
met. A pull strategy will be efficient for a lot more people.
Making code efficient interests me a lot. When I worked on this stuff
earlier, I thought of a way to get Zope to serve files and images out of
the database more efficiently. I'd like to know whether you think it's
a good idea; if it is, we can plan to implement it when it's needed.
The idea is this: run a thread whose only responsibility is to stream
image and file data out of the object database to browsers. The thread
will maintain a permanently open database connection. Other threads
will send open HTTP connections to this thread, asking it to finish the
response. The special thread will get the binary data from the database
using its own connection, which would play well with the ZODB cache,
since we'd only ever have one copy of the image/file data in memory at a
time. Contrast this with Zope 2's image and file serving, which loads
image/file data into arbitrary database connections and immediately
evicts them from the cache. It seems like having a special connection
just for serving images/files would be a lot more efficient, especially
when concurrency is high.
However, this assumes that most images and files come out of the object
database. I suspect that current Zope 3 software gets most images and
files from the filesystem, but that will change as Zope 3 takes on more
Zope3-dev mailing list