Thanks a lot Willy. Yes, I tried my luck with sticky tables, but could not find a way to store key value mapping for 1000s of host names.
I will move this to testing, thanks for you help as always :) Thanks Sachin On 12/2/14 1:01 PM, "Willy Tarreau" <w...@1wt.eu> wrote: >Hi Sachin, > >On Sat, Nov 29, 2014 at 04:19:54PM +0530, Sachin Shetty wrote: >> Hi, >> >> In our architecture, we have thousands of host names resolving to a >>single >> haproxy, we dynamically decide a sticky backend based on our own custom >> sharding. To determine the shard info, we let the request flow in to a >> default apache proxy that processes the requests and also responds >>with >> the shard info. To be able to serve the consequent requests directly >> bypassing the apache, we want to store the shard info received in the >>first >> request in a map and use it for subsequent request >> >> 1. Store the shard info from apache >> backend apache_l1 >> mode http >> http-response set-map(/opt/haproxy/current/conf/proxy.map) >> %[res.hdr(X-Request-Host)] %[res.hdr(X-Backend-Id)] >> server apache_l1 <IP>:80 >> >> 2. Use the backend directly for subsequent requests: >> acl is_a_v-1 hdr(host),map(/opt/haproxy/current/conf/proxy.map) a_v-1 >> use_backend l2_haproxy if is_a_v-1 >> >> I have tested this config and it works well, but I am not sure about the >> performance. For every request sent to Apache, we will be adding a key, >> value to the map and we will be looking up the key value for every >>requests >> that is coming in to haproxy is that ok considering that this is very >>high >> performance stack? The haproxy servers are pretty powerful and >>dedicated to >> just doing proxy. > >Here you're using string-to-string mapping, it's one of the cheapest one >since there's no conversion of text to patterns. The string lookups are >performed in a few tens of nanoseconds so that does not count. The update >here will require : > - building a new key : log-format + strdup(result) > - building a new value : log-format + strdup(new) > - lookup of the key in the tree > - replacement or insertion of the key in the tree > - free(old_key) > - free(old_value) > >I suspect that below 10-20000 req/s you will not notice a significant >difference. Above it can cost a few percent extra CPU usage. > >It's interesting to see that you have basically reimplemented stickiness >using maps :-) > >Regards, >Willy >
