Hello all, Sorry, I should have cc:ed this to the mailing lists. It's my reply to eric's message re: flash video streaming through perlbal.
Etienne ---------- Forwarded message ---------- From: EJH Pollard <[EMAIL PROTECTED]> Date: Dec 4, 2007 11:05 PM Subject: Re: request - help with streaming flash video through perlbal To: Eric Lambrecht <[EMAIL PROTECTED]> Cc: Matthew Somerville <[EMAIL PROTECTED]> Hi Eric, Thanks for your email a few days ago - sorry I haven't been in touch earlier. This evening I sat down to try and make it work, but unfortunately I got an error that I don't really know how to fix - details below, after your original suggestion to give it some context. On Nov 25, 2007 6:29 PM, Eric Lambrecht <[EMAIL PROTECTED]> wrote: > > Is it possible to configure perlbal, patched as per > > http://lists.danga.com/pipermail/mogilefs/2007-April/000862.html, to > > behave in the manner described above? If not, do you have any > > suggestions for what I should do to the patch/config to make it work > > like that? We have a modest budget, and if it's a lot of work then I > > Hi Etienne - > I believe the patch I submitted will do what you want, but you're > missing one component. > > To make things work with my patch, you need to first set up perlbal to > reproxy all requests to some webserver you've got running on the same > machine. > > Then let's assume your requests for flash files all look like this: > > GET /play?id=123123&start=5675675 > > Where 'id' is the id of the flash file, and 'start' is the byte offset > for the FLV file. > > The '/play' cgi script on your webserver should map the value of 'id' to > the location of your flash file in S3 (lets say that is > 's3.amazon.com/123123'), and send these headers back to perlbal: > > X-REPROXY-URL: http://s3.amazon.com/123123 > X-REPROXY-FLV-START: 5675675 > > perlbal will then close the connection to the webserver, open a > connection to S3, jump to byte offset 5675675, send the 13 byte magic > header to the client, then send the remainder of the flv file to the client. I've applied the patch, and written the script to produce the headers you described, but unfortunately perlbal is giving me 503 Service Unavailable when I present it with those HTTP headers. I'm sending the following headers to perlbal (have verified this output by debugging these lines with a preceding Content-type: text/plain): X-REPROXY-URL: http://s3.amazonaws.com/parliament-flash-video/2249.flv X-REPROXY-FLV-START: 3212 The second line is only there if I give the script a start param, of course; the base case is just the X-REPROXY_URL line, which gives a perfectly sane URL that works fine when I fetch it directly (http://s3.amazonaws.com/parliament-flash-video/2260.flv). Enabling debug output on perlbal gives a whole bunch of stuff, copied at the bottom of this email, but the strangest thing is the 404 error it seems to be getting from Amazon. Running lynx from the same machine gets a 200 OK, and downloads fine onto the local disk. So I'm a bit stuck! Any ideas? The perlbal config is: # always good to keep an internal management port open: CREATE SERVICE mgmt SET role = management SET listen = 127.0.0.1:60000 ENABLE mgmt CREATE POOL backend POOL backend ADD 127.0.0.1:81 CREATE SERVICE reproxy SET listen = 0.0.0.0:80 SET role = reverse_proxy SET enable_reproxy = true SET pool = backend ENABLE reproxy I'm on perlbal 1.59, patched as per your patch from http://lists.danga.com/pipermail/mogilefs/2007-April/000867.html . Thanks, Etienne Debug output: Got new conn: IO::Socket::INET=GLOB(0x86547f0) (127.0.0.1:49852) for reverse_proxy ClientProxy::event_read no headers. reading. Perlbal::Socket::read_headers(Perlbal::ClientProxy=ARRAY(0x80693e8)) is_res=0 pre-parsed headers: [GET /redirect/2249.flv HTTP/1.0 Host: 127.0.0.1 Accept: text/html, text/plain, audio/mod, image/*, application/msword, application/pdf, application/postscript, text/sgml, */*;q=0.01 Accept-Encoding: gzip, compress Accept-Language: en User-Agent: Lynx/2.8.5rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.8b] got valid headers Backend Perlbal::BackendHTTP=ARRAY(0x806997c) is writeable! Backend requesting client, got normal = 3. write(Perlbal::BackendHTTP=ARRAY(0x806997c), <396>"GET /redirect/224...") from (Perlbal::BackendHTTP, /usr/lib/perl5/vendor_perl/5.8.8/Perlbal/BackendHTTP.pm, 292) write(Perlbal::BackendHTTP=ARRAY(0x806997c), <15>"CODE(0x869457c)") from (Perlbal::BackendHTTP, /usr/lib/perl5/vendor_perl/5.8.8/Perlbal/BackendHTTP.pm, 302) Backend Perlbal::BackendHTTP=ARRAY(0x806997c) is readable! Perlbal::Socket::read_headers(Perlbal::BackendHTTP=ARRAY(0x806997c)) is_res=1 pre-parsed headers: [HTTP/1.0 200 OK Connection: close X-REPROXY-URL: http://s3.amazonaws.com/parliament-flash-video/2249.flv Date: Tue, 04 Dec 2007 22:55:17 GMT Server: lighttpd/1.4.16] pushing back 1 bytes after header got valid headers BackendHTTP: handle_response Backend Perlbal::BackendHTTP=ARRAY(0x869496c) is writeable! write(Perlbal::BackendHTTP=ARRAY(0x869496c), <90>"GET /parliament-f...") from (Perlbal::ClientProxy, /usr/lib/perl5/vendor_perl/5.8.8/Perlbal/ClientProxy.pm, 244) Backend Perlbal::BackendHTTP=ARRAY(0x869496c) is writeable! Use of uninitialized value in concatenation (.) or string at /usr/lib/perl5/vendor_perl/5.8.8/Perlbal/Socket.pm line 74. write(Perlbal::BackendHTTP=ARRAY(0x869496c), <undef>"") from (Perlbal::BackendHTTP, /usr/lib/perl5/vendor_perl/5.8.8/Perlbal/BackendHTTP.pm, 368) Backend Perlbal::BackendHTTP=ARRAY(0x869496c) is readable! Perlbal::Socket::read_headers(Perlbal::BackendHTTP=ARRAY(0x869496c)) is_res=1 pre-parsed headers: [HTTP/1.1 404 Not Found x-amz-request-id: 4B2E0B50A76F8FC4 x-amz-id-2: IEje1k7yXJ5J1jv2X+Glj5izkk0XRXunOJGmNiZZZHyJ8YpSVQNvJexbqINPk65A Content-Type: application/xml Date: Tue, 04 Dec 2007 22:55:17 GMT Connection: close Server: AmazonS3] pushing back 287 bytes after header got valid headers BackendHTTP: handle_response ClientHTTPBase::setup_keepalive(Perlbal::ClientProxy=ARRAY(0x80693e8)) service's persist_client = 0 doing connection: close write(Perlbal::ClientProxy=ARRAY(0x80693e8), <117>"HTTP/1.0 503 Serv...") from (Perlbal::ClientHTTPBase, /usr/lib/perl5/vendor_perl/5.8.8/Perlbal/ClientHTTPBase.pm, 734) write(Perlbal::ClientProxy=ARRAY(0x80693e8), <35>"<h1>503 - Service...") from (Perlbal::ClientHTTPBase, /usr/lib/perl5/vendor_perl/5.8.8/Perlbal/ClientHTTPBase.pm, 735) write(Perlbal::ClientProxy=ARRAY(0x80693e8), <15>"CODE(0x86950c8)") from (Perlbal::ClientHTTPBase, /usr/lib/perl5/vendor_perl/5.8.8/Perlbal/ClientHTTPBase.pm, 741) -- Etienne Pollard [EMAIL PROTECTED] +44 (0) 7946 415 996
