Re: [Zope] Re: Slow File Access With LocalFS

2000-06-28 Thread Dieter Maurer

Jonothan Farr writes:
 > It sounds to me like the Apache server is just writing the data directly to the
 > HTTP connection as it is read. I believe that's how HTTP "streaming" works. I
 > also believe this is possible to do using the Zope HTTPResponse object. Does
 > anyone have any ideas?
We use it to deliver images from a multimedia database.

In Zope 2.1.0, it was not at all easy to use the HTTP streaming.
Many proxies and IE require the Content-Length header
to be correct.
To set a correct "Content-Length" was not a problem.
However, ZPublisher.HTTPResponse reset it afterwards to 0,
leading to very strange effects in various situations (i.e.
with IE and some Proxies).

We finally settled it in the following way:

  RESPONSE.setHeader('Content-Length',length)
  RESPONSE.setHeader('transfer-encoding','binary') # to prevent RESPONSE.__str__ 
to overwrite "Content-Length"
  RESPONSE.headers['Content-Length']= length
  # now do several "RESPONSE.write"

I am not sure, whether 2.1.6 or 2.2 still requires this strange
code sequence.


For Zope 2.1.6, we detected a nasty ZServer bug with the
streaming protocol: missing thread synchronization.
As a result, browsers got a non-deterministic part of the image.
Very nice effects
DC told me, that this bug is fixed in the CVS and for Zope 2.2.


Thus, do not use the HTTP streaming protocol for the standard
Zope 2.1.6 distribution (or be ready to patch ZServer).


Dieter

___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




[Zope] Re: Slow File Access With LocalFS

2000-06-27 Thread Jonothan Farr


It sounds to me like the Apache server is just writing the data directly to the
HTTP connection as it is read. I believe that's how HTTP "streaming" works. I
also believe this is possible to do using the Zope HTTPResponse object. Does
anyone have any ideas?

--jfarr

"Perl is worse than Python because people wanted it worse."
Larry Wall, 14 Oct 1998

- Original Message -
From: Rob Pratt <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Tuesday, June 27, 2000 10:56 AM
Subject: Re: Slow File Access With LocalFS


> Hey, thanks for the quick reply.
>
> > No, it just takes a while to read a very large file and construct the
> ZObject
> > (performance patches accepted ;)). It can also take a while to download
> them
> but
> > that should be obvious. How is it that you're "streaming" the mp3 files?
>
> Sorry. Streaming is not exactly the right word I think. What I have set up
> is that Zope delivers a playlist (*.m3u), which is opened by a player on
> the
> local machine (assuming there's one installed), and the player app in turn
> makes a request for the actual mp3 file, which starts playing as soon as
> it's properly buffered in the player app. That avoids having to download
> each mp3 before playing it.
>
> I had the collection stashed behind an Apache rewrite rule, but when I
> checked out LocalFS, I decided to try to move it into Zope under a LocalFS
> object (which I can use to control access to the files more easily than in
> Apache). Using Apache rewrite, requests for mp3 files just bypassed Zope,
> and "streaming" was immediate. But handled as a LocalFS object, mp3 file
> requests (I guess from your reply) have to wait to create the ZObject
> before
> they start "streaming."
>
> Is there a way to get LocalFS to serve up the file without creating a
> ZObject?
>
> Wish I knew more about Python and programming. (I'm workin' on it, but it's
> a steep learning curve for a newbie like me.) I'd jump in and see if I
> could
> come up with a performance patch.
>
> Anyway, I've just started learning Zope, and I see that it has immense
> possibilities for automating a lot of what I do in my day job (I'm an
> editor
> for an alternative newsweekly in Santa Cruz, Calif., USA). The application
> I
> envision will probably make heavy use of the LocalFS product.
>
> I was hoping that it would be easy to digitize our extensive demo library
> on touring
> bands so that I wouldn't have to lend CDs to writers who never seem to
> return them ;) I could just get an intern to rip CDs into mp3s and post
> them
> on a password-protected mp3 server, and all the writers could have 24-7
> access.
>
> That's just one feature. Many, many others that I can think of (not
> involving mp3s ;)) could take hours and hours of work off of my weekly
> workload.
>


___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )




[Zope] Re: Slow File Access With LocalFS

2000-06-27 Thread Jonothan Farr

> I'm using LocalFS to reference a large collection of mp3 files. (I figured
> it was probably a good idea not to upload the whole shebang to the ZODB, so
> I just added a LocalFS object to reference the top directory of the
> collection.) When I try to download a file by calling the Zope object (for
> instance, http://zopeserver/LocalFSObject/mp3file.mp3) it takes 30 seconds
> for the average mp3 file to start streaming. Other files, i.e., way smaller
> ones, like 4K playlist files, download very quickly.
>
> Is Zope or LocalFS choking on the large files? Is there some work-around
> I'm missing?

No, it just takes a while to read a very large file and construct the ZObject
(performance patches accepted ;)). It can also take a while to download them but
that should be obvious. How is it that you're "streaming" the mp3 files?

--jfarr



___
Zope maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )