On Sun, Sep 12, 2010 at 10:32:37PM +0200, Sebastien Estienne wrote: > And as dormando said, this is for a usecase really similar to mogilefs > (by the way i discovered the reproxy feature with mogilefs). > It is not to serve content like css/html or php generated content, > it's for serving files like videos and photos. > We don't want to expose the files directly and want a fine control on > their access something that you can't really have with a 302, > especially if your file servers only support basic Http with no > support for rewrite or fancy features like checking a secure url or > access token.
I understand this point, but quite frankly, the *only* reason you try to put that on the load balancer is because it's the only component you have in front of your servers. What you're describing should normally be processed either by passing a token to the static file server (thus modifying an existing server for that purpose) or by putting the static server before the application server, making it proxy requests to the server and sometimes serve static contents on valid responses. This would also require changes. But implementing the token check in a flexible server such as nginx should not be too hard and would also provide the ability to fetch static files from many places, as well as using geolocation to use the server closest to the user. > And even with secure url, you can't control things like the number of > time that a file can be downloaded by an authenticated user. I don't agree, I even think it's the opposite. With your method you can't control because once the server has emitted its redirect, you don't know if the user gets the entire file or not. With a token, either the static server can talk to the application server to check for authorization, or it can store local information about the downloads in progress and their statuses. If you only want to work on a single site, you could even do that with an LDAP or radius auth server. Your app then has to create dynamic accounts and your static servers rely on them. > With reproxy we can implement all these features inside the app server > really easily. I see the ease of use for the developer, but I also see a lot of downsides as I explained in the first mail. I understand that in case of single-site video download service you can relatively easily consider that you don't mind about many headers and forge simple requests, but I also see how that can quickly be misused. I have another question BTW. I know some large download sites running either on NFS farms or with decentralized nodes (one node = storage+app). I still fail to see the advantage of your solution compared to these methods. If your concern is to have a large capacity online, then the NFS servers can provide that. If the bandwidth is the concern, then your solution is still limited to one site (one internet link, and one or even a few LBs). And within a single server, whatever the techno you're using, you don't even need to maintain a thread during the transfer, you can pass the file descriptor to another process which will take care of sending it using sendfile(). Regards, Willy
