Hi,

As long as you don't share more details on how your files are accessed
and what makes each URL unique, I can't help.
As I said, splitting your files by directory path or by Host header may be good.

Concerning example in haproxy, having the following in your frontend
will do the stuff:
  acl dir1 path_beg /dir1/
  usebackend bk_dir1 if dir1
  acl dir1 path_beg /dir2/
  usebackend bk_dir2 if dir2
...

then create the backends:
  bk_dir1
    balance roundrobin
    server srv1
    server srv2
  bk_dir2
    balance roundrobin
    server srv3
    server srv4
...

Hope this helps

On Mon, Nov 21, 2011 at 3:24 PM, Rerngvit Yanggratoke <rerng...@kth.se> wrote:
> Dear Baptiste,
>             Could you please exemplify a criterion that would reduce the
> number of files per backends? And, if possible, how to implement that with
> HAProxy?
>
> On Sat, Nov 19, 2011 at 8:29 PM, Baptiste <bed...@gmail.com> wrote:
>>
>> On Fri, Nov 18, 2011 at 5:48 PM, Rerngvit Yanggratoke <rerng...@kth.se>
>> wrote:
>> > Hello All,
>> >         First of all, pardon me if I'm not communicating very well.
>> > English
>> > is not my native language. We are running a static file distribution
>> > cluster. The cluster consists of many web servers serving static files
>> > over
>> > HTTP.  We have very large number of files such that a single server
>> > simply
>> > can not keep all files (don't have enough disk space). In particular, a
>> > file
>> > can be served only from a subset of servers. Each file is uniquely
>> > identified by a file's URI. I would refer to this URI later as a key.
>> >         I am investigating deploying HAProxy as a front end to this
>> > cluster.
>> > We want HAProxy to provide load balancing and automatic fail over. In
>> > other
>> > words, a request comes first to HAProxy and HAProxy should forward the
>> > request to appropriate backend server. More precisely, for a particular
>> > key,
>> > there should be at least two servers being forwarded to from HAProxy for
>> > the
>> > sake of load balancing. My question is what load
>> > balancing strategy should I
>> > use?
>> >         I could use hashing(based on key) or consistent hashing.
>> > However,
>> > each file would end up being served by a single server on a particular
>> > moment. That means I wouldn't have load balancing and fail over for a
>> > particular key.
>> >        Is there something like a combination of hashing and
>> > roundrobin strategy? In particular, for a particular key, there would be
>> > multiple servers serving the requests and HAProxy selects one of them
>> > according to roundrobin policy. If there isn't such a stretegy, any
>> > suggestions on how to implement this into HAProxy? Any other comments
>> > are
>> > welcome as well.
>> >
>> > --
>> > Best Regards,
>> > Rerngvit Yanggratoke
>> >
>>
>> Hi,
>>
>> You could create several backends and redirect requests based on an
>> arbitrary criteria to reduce the number of files per backends. Using
>> URL path prefix might be a good idea.
>> Then inside a backend, you can use the hash url load-balancing algorithm.
>>
>> cheers
>
>
>
> --
> Best Regards,
> Rerngvit Yanggratoke
>

Reply via email to