Re: hashing + roundrobin algorithm

2011-12-04 Thread Willy Tarreau
On Sun, Dec 04, 2011 at 08:37:31AM +0100, Rerngvit Yanggratoke wrote: John, thanks for your idea. I thought about redirection as well. Nonetheless, for the legacy software, I could not change anything at the moment. They are simply not under my control. Haproxy can do the redirection itself if

Re: hashing + roundrobin algorithm

2011-12-03 Thread Rerngvit Yanggratoke
Thank you for the suggestion. Consistent hashing sounds promising. The number of files I would have to redistributed is limited if some servers failures. On Sun, Nov 27, 2011 at 10:42 PM, Allan Wind allan_w...@lifeintegrity.comwrote: On 2011-11-26 01:30:41, Rerngvit Yanggratoke wrote:

Re: Re: hashing + roundrobin algorithm

2011-12-03 Thread Rerngvit Yanggratoke
-11-29 14:24 *To:* wsq003 wsq...@sina.com *CC:* Rerngvit Yanggratoke rerng...@kth.se; haproxyhaproxy@formilux.org; Baptiste bed...@gmail.com *Subject:* Re: Re: hashing + roundrobin algorithm Hi, On Tue, Nov 29, 2011 at 01:52:31PM +0800, wsq003 wrote: We add a new keyword 'vgroup' under

Re: Re: hashing + roundrobin algorithm

2011-12-03 Thread Rerngvit Yanggratoke
; haproxyhaproxy@formilux.org; Baptiste bed...@gmail.com *Subject:* Re: Re: hashing + roundrobin algorithm On Tue, Nov 29, 2011 at 02:56:49PM +0800, wsq003 wrote: Backend proxies may be multiple layers, then every layer can have its own LB param. Logically this is a tree-like structure, every

Re: hashing + roundrobin algorithm

2011-12-03 Thread Rerngvit Yanggratoke
John, thanks for your idea. I thought about redirection as well. Nonetheless, for the legacy software, I could not change anything at the moment. They are simply not under my control. On Sun, Nov 27, 2011 at 11:01 PM, John Marrett jo...@zioncluster.ca wrote: Rerngvit, However, I couldn't

Re: Re: hashing + roundrobin algorithm

2011-11-30 Thread Baptiste
assigen_server()? From: Willy Tarreau Date: 2011-11-30 01:47 To: wsq003 CC: Rerngvit Yanggratoke; haproxy; Baptiste Subject: Re: Re: hashing + roundrobin algorithm On Tue, Nov 29, 2011 at 02:56:49PM +0800, wsq003 wrote:  Backend proxies may be multiple layers, then every layer can have its

Re: Re: hashing + roundrobin algorithm

2011-11-29 Thread Willy Tarreau
On Tue, Nov 29, 2011 at 02:56:49PM +0800, wsq003 wrote: Backend proxies may be multiple layers, then every layer can have its own LB param. Logically this is a tree-like structure, every real server is a leaf. Every none-leaf node is a backend proxy and may have LB param. I clearly

Re: Re: hashing + roundrobin algorithm

2011-11-29 Thread wsq003
-30 01:47 To: wsq003 CC: Rerngvit Yanggratoke; haproxy; Baptiste Subject: Re: Re: hashing + roundrobin algorithm On Tue, Nov 29, 2011 at 02:56:49PM +0800, wsq003 wrote: Backend proxies may be multiple layers, then every layer can have its own LB param. Logically this is a tree-like structure

Re: Re: hashing + roundrobin algorithm

2011-11-28 Thread wsq003
: Re: Re: hashing + roundrobin algorithm Hello wsq003, That sounds very interesting. It would be great if you could share your patch. If that is not possible, providing guideline on how to implement that would be helpful as well. Thank you! 2011/11/23 wsq003 wsq...@sina.com I've made

Re: Re: hashing + roundrobin algorithm

2011-11-28 Thread Willy Tarreau
Hi, On Tue, Nov 29, 2011 at 01:52:31PM +0800, wsq003 wrote: We add a new keyword 'vgroup' under 'server' key word. server wsqa 0.0.0.0 vgroup subproxy1 weight 32 check inter 4000 rise 3 fall 3 means request assigned to this server will be treated as set backend 'subproxy1'. Then in

Re: hashing + roundrobin algorithm

2011-11-27 Thread Allan Wind
On 2011-11-26 01:30:41, Rerngvit Yanggratoke wrote: We have over three millions of files. Each static file is rather small ( 5MB) and has a unique identifier used as well as an URL. As a result, we are in the second case you mentioned. In particular, we should concern about if

Re: hashing + roundrobin algorithm

2011-11-25 Thread Rerngvit Yanggratoke
Thank you, Baptiste. Let me elaborate more details then. We have over three millions of files. Each static file is rather small ( 5MB) and has a unique identifier used as well as an URL. I referred to this URL as a file key. The performance goal we want from our system is high availability and

Re: hashing + roundrobin algorithm

2011-11-25 Thread Rerngvit Yanggratoke
Dear Willy, Thank you for your help. We have a clear performance goal for our cluster. The goal is high availability and maximizing throughput under a predefined constant latency. However, we don't have a clear idea what architecture or software would allow us to achieve that yet. Let me

Re: Re: hashing + roundrobin algorithm

2011-11-25 Thread Rerngvit Yanggratoke
...@kth.se *CC:* haproxy haproxy@formilux.org; Baptiste bed...@gmail.com *Subject:* Re: hashing + roundrobin algorithm Hi, On Fri, Nov 18, 2011 at 05:48:54PM +0100, Rerngvit Yanggratoke wrote: Hello All, First of all, pardon me if I'm not communicating very well. English is not my

Re: hashing + roundrobin algorithm

2011-11-22 Thread Baptiste
Hi, As long as you don't share more details on how your files are accessed and what makes each URL unique, I can't help. As I said, splitting your files by directory path or by Host header may be good. Concerning example in haproxy, having the following in your frontend will do the stuff: acl

Re: hashing + roundrobin algorithm

2011-11-21 Thread Rerngvit Yanggratoke
Dear Baptiste, Could you please exemplify a criterion that would reduce the number of files per backends? And, if possible, how to implement that with HAProxy? On Sat, Nov 19, 2011 at 8:29 PM, Baptiste bed...@gmail.com wrote: On Fri, Nov 18, 2011 at 5:48 PM, Rerngvit Yanggratoke

Re: hashing + roundrobin algorithm

2011-11-19 Thread Baptiste
On Fri, Nov 18, 2011 at 5:48 PM, Rerngvit Yanggratoke rerng...@kth.se wrote: Hello All,         First of all, pardon me if I'm not communicating very well. English is not my native language. We are running a static file distribution cluster. The cluster consists of many web servers serving