Backend status changes continuously

2015-04-21 Thread Krishna Kumar (Engineering)
Hi all, While running the command: : ab -n 10 -c 1000 192.168.122.110:80/256, the haproxy stats page shows the 4 different backend servers changing status between Active up, going down, Active or backup down, Down, Backup down, going UP, sometimes all 4 backends are in DOWN state. The result

Re: Backend status changes continuously

2015-04-21 Thread Krishna Kumar (Engineering)
VM's. Sometimes during the test, I also see many nf_conntrack: table full, dropping packet messages on the host system. Thanks. - Krishna On Tue, Apr 21, 2015 at 1:29 PM, Krishna Kumar (Engineering) krishna...@flipkart.com wrote: Hi all, While running the command: : ab -n 10 -c 1000

Re: [haproxy]: Performance of haproxy-to-4-nginx vs direct-to-nginx

2015-04-29 Thread Krishna Kumar (Engineering)
Dear all, Sorry, my lab systems were down for many days and I could not get back on this earlier. After new systems were allocated, I managed to get all the requested information with a fresh ru (Sorry, this is a long mail too!). There are now 4 physical servers, running Debian 3.2.0-4-amd64,

Re: [haproxy]: Performance of haproxy-to-4-nginx vs direct-to-nginx

2015-05-05 Thread Krishna Kumar (Engineering)
Hi Willy, Pavlos, Thank you once again for your advice. Requests per second:19071.55 [#/sec] (mean) Transfer rate: 9461.28 [Kbytes/sec] received These numbers are extremely low and very likely indicate an http close mode combined with an untuned nf_conntrack. Yes, it

Couple of questions on future support

2015-05-06 Thread Krishna Kumar (Engineering)
Hi all, 1. Is there any plan to support HTTP/2? Any estimate on the amount of work/time it would take to implement? 2. Is there any plan to have support for Geolocation (other than what is mentioned in the homepage)? Thanks, - Krishna Kumar

Re: [haproxy]: Performance of haproxy-to-4-nginx vs direct-to-nginx

2015-05-05 Thread Krishna Kumar (Engineering)
Hi Baptiste, On Wed, May 6, 2015 at 1:24 AM, Baptiste bed...@gmail.com wrote: Also, during the test, the status of various backend's change often between OK to DOWN, and then gets back to OK almost immediately: www-backend,nginx-3,0,0,0,10,3,184,23843,96517588,,0,,27,0,0,180,DOWN

Re: [haproxy]: Performance of haproxy-to-4-nginx vs direct-to-nginx

2015-05-05 Thread Krishna Kumar (Engineering)
Hi Pavlos On Wed, May 6, 2015 at 1:24 AM, Pavlos Parissis pavlos.paris...@gmail.com wrote: Shall I assume that you have run the same tests without iptables and got the same results? Yes, I had tried it yesterday and saw no measurable difference. May I suggest to try also httpress and wrk

Re: Couple of questions on future support

2015-05-06 Thread Krishna Kumar (Engineering)
On Wed, May 6, 2015 at 3:26 PM, Baptiste bed...@gmail.com wrote: Hi Baptiste, you can do it natively with maps and conversion of maxmind ip ranges into HAProxy's subnets. Thank you, using this information, I was able to find your article:

Re: HA Proxy

2015-05-07 Thread Krishna Kumar (Engineering)
, - Krishna Kumar On Thu, May 7, 2015 at 11:28 AM, ANISH S IYER anish.subramaniai...@gmail.com wrote: -- Forwarded message -- From: Krishna Kumar (Engineering) krishna...@flipkart.com Date: Thu, May 7, 2015 at 11:21 AM Subject: Re: HA Proxy To: ANISH S IYER anish.subramaniai

Re: [haproxy]: Performance of haproxy-to-4-nginx vs direct-to-nginx

2015-05-06 Thread Krishna Kumar (Engineering)
the various parameters that were set (system level + haproxy + backend) if it will be useful for someone else in future. Thanks, - Krishna Kumar On Thu, May 7, 2015 at 8:31 AM, Baptiste bed...@gmail.com wrote: Le 7 mai 2015 04:24, Krishna Kumar (Engineering) krishna...@flipkart.com a écrit : I

Re: [haproxy]: Performance of haproxy-to-4-nginx vs direct-to-nginx

2015-05-06 Thread Krishna Kumar (Engineering)
configuration without specifying source port is working, no connection flaps now. Thanks, - Krishna Kumar On Wed, May 6, 2015 at 4:53 PM, Willy Tarreau w...@1wt.eu wrote: On Wed, May 06, 2015 at 12:03:12PM +0200, Baptiste wrote: On Wed, May 6, 2015 at 7:15 AM, Krishna Kumar (Engineering) krishna

Re: HA Proxy

2015-05-06 Thread Krishna Kumar (Engineering)
On Thu, May 7, 2015 at 9:44 AM, ANISH S IYER anish.subramaniai...@gmail.com wrote: 1) how ha proxy is know both of his front and backend server is waiting or busy.? I am not sure if I understood this right. Depending on the algo, the backend is picked. It should not care if the backend is

Re: Issue with SSL

2015-05-13 Thread Krishna Kumar (Engineering)
, 2015 at 10:07 AM, Krishna Kumar (Engineering) krishna...@flipkart.com wrote: Hi all, I am having the following problem with SSL + large I/O. Details are: Distribution: Debian 7, Kernel: 3.19.6, ab version: 2.3, haproxy: 1.5.12, nginx: 1.2.1 $ ab -k -n 10 -c 100 http://IP:80/128K

Re: Issue with SSL

2015-05-13 Thread Krishna Kumar (Engineering)
...@gmail.com wrote: On Wed, May 13, 2015 at 2:16 PM, Krishna Kumar (Engineering) krishna...@flipkart.com wrote: Hi Baptiste, Thank you very much for the tips. I have nbproc=8 in my configuration. Made the following changes: Added both bind and tune.bufsize change

Issue with SSL

2015-05-13 Thread Krishna Kumar (Engineering)
Hi all, I am having the following problem with SSL + large I/O. Details are: Distribution: Debian 7, Kernel: 3.19.6, ab version: 2.3, haproxy: 1.5.12, nginx: 1.2.1 $ ab -k -n 10 -c 100 http://IP:80/128K Works correctly. $ ab -k -n 1 -c 10 https://IP:443/4K Works correctly. $ ab -k -n

HAProxy SSL performance issue

2015-05-21 Thread Krishna Kumar (Engineering)
Hi all, I am getting a big performance hit with SSL termination for small I/O, and errors when testing with bigger I/O sizes (ab version is 2.3): 1. Non-SSL vs SSL for small I/O (128 bytes): ab -k -n 100 -c 500 http://HAPROXY/128 RPS: 181763.65 vs 133611.69- 27% drop BW:

Re: LB as a first row of defence against DDoS

2015-06-24 Thread Krishna Kumar (Engineering)
On Wed, Jun 24, 2015 at 11:33 PM, Shawn Heisey hapr...@elyograg.org wrote: I agree - the blog talks of handling multiple attacks individually, but what we are trying to understand is - how can we handle multiple types of attacks in a single configuration. Not the exact configuration file, but

LB as a first row of defence against DDoS

2015-06-17 Thread Krishna Kumar (Engineering)
Referring to Baptiste's excellent blog on Use a lb as a first row of defense against DDoS @ http://blog.haproxy.com/2012/02/27/use-a-load-balancer-as-a-first-row-of-defense-against-ddos/ I am not able to find a follow up, if it was written, on combining configuration examples to improve

Re: Health check of backends without explicit health-check?

2015-06-17 Thread Krishna Kumar (Engineering)
On Tue, Jun 16, 2015 at 4:29 PM, Krishna Kumar (Engineering) krishna...@flipkart.com wrote: I was referring to HAProxy as the LB here. If there is any means to do this, kindly let me know. Thanks, - Krishna Kumar Hi list, Is there any way to log, or report, or notify, or identify any

Re: Health check of backends without explicit health-check?

2015-06-17 Thread Krishna Kumar (Engineering)
On Wed, Jun 17, 2015 at 11:38 AM, Krishna Kumar (Engineering) krishna...@flipkart.com wrote: On Tue, Jun 16, 2015 at 4:29 PM, Krishna Kumar (Engineering) krishna...@flipkart.com wrote: I was referring to HAProxy as the LB here. If there is any means to do this, kindly let me know

Health check of backends without explicit health-check?

2015-06-16 Thread Krishna Kumar (Engineering)
Hi list, Is there any way to log, or report, or notify, or identify any backend that is not responding, without using explicit health-checks? The reason for this is that we are planning a big deployment of LB/servers, something along the lines of: LB1, LB2, LB100 or more ^

Re: HAProxy SSL performance issue

2015-05-22 Thread Krishna Kumar (Engineering)
On Thu, May 21, 2015 at 5:58 PM, Willy Tarreau w...@1wt.eu wrote: Hi Willy, Thank you for your reply. I suspect the BW unit is bytes per second above though I could be That's correct, and the BW is as you had stated: 8gpbs vs 2.8 gbps. Hmmm, would you be running from multiple load

Re: Strange system behaviour of during haproxy run

2015-07-07 Thread Krishna Kumar (Engineering)
this is the issue, which I am not sure of, do you have any ideas how to get around this, or any other suggestions? Thanks, - Krishna Kumar On Tue, Jul 7, 2015 at 1:38 PM, Willy Tarreau w...@1wt.eu wrote: Hi, On Tue, Jul 07, 2015 at 01:24:28PM +0530, Krishna Kumar (Engineering) wrote: Hi all

Strange system behaviour of during haproxy run

2015-07-07 Thread Krishna Kumar (Engineering)
Hi all, This is not related to haproxy, but I am having a performance issue with number of packets processed. I am running haproxy on a 48 core system (we have 64 such servers at present, which is going to increase for production tessting), where cpus 0,2,4,6,..46 are part of NUMA node 1, and

Multi-part message failure during http mode (haproxy 1.5.12)

2015-08-04 Thread Krishna Kumar (Engineering)
Hi all, We are getting either ECONNRESET, or sometimes ETIMEDOUT errors when the backend sends a large multi-part message via haproxy. It works for small file sizes of 4K and 8K, but fails for 2 Mb files. Is there any option or setup that will help fix this? Thanks, - Krishna Kumar --

Re: Multi-part message failure during http mode (haproxy 1.5.12)

2015-08-06 Thread Krishna Kumar (Engineering)
Not to spam again, but a request to anyone who has faced this, and know how to get around or fix it. I checked the source a bit, there is a reference to multipart message in the compression code only (do not compress multi-part). Thank you. On Wed, Aug 5, 2015 at 9:26 AM, Krishna Kumar

How to disable backend servers without health check

2015-07-16 Thread Krishna Kumar (Engineering)
Hi all, We have a large set of machines running haproxy (1.5.12), and each of them have hundreds of backends, many of which are the same across systems. nbproc is set to 12 at present for our 48 core systems. We are planning a centralized health check, and disable the same in haproxy, to avoid

Re: Strange system behaviour of during haproxy run

2015-07-16 Thread Krishna Kumar (Engineering)
On Tue, Jul 7, 2015 at 2:31 PM, Willy Tarreau w...@1wt.eu wrote: Hi Willy, Thank you once again for the quick response, and apologize for my tardiness. Unless I'm wrong, for me ixgbe does its own RSS and uses the Rx queues. Each Rx queue is bound to an IRQ, and each IRQ may be delivered to a

Re: How to disable backend servers without health check

2015-07-16 Thread Krishna Kumar (Engineering)
of accomplishing this though. With our 20 proc configuration we've decided to deal with the pain of 20 health checks which has caused us some issues, but nothing being a show stopper. On Thu, Jul 16, 2015 at 5:53 AM, Krishna Kumar (Engineering) krishna...@flipkart.com wrote: Hi all, We have

Re: How to disable backend servers without health check

2015-07-16 Thread Krishna Kumar (Engineering)
Thanks Pavlos, this looks very promising, I will take a look on how we can use this. Regards, - Krishna Kumar On Thu, Jul 16, 2015 at 8:36 PM, Pavlos Parissis pavlos.paris...@gmail.com wrote: On 16/07/2015 04:02 μμ, Krishna Kumar (Engineering) wrote: Hi John, Your suggestion works very

Re: [1.6.1] Utilizing http-reuse

2015-11-11 Thread Krishna Kumar (Engineering)
Hi Willy, >> B. Run 8 wgets in parallel. Each opens a new connection to get a 128 byte >> file. >> Again, 8 separate connections are opened to the backend server. > > But are they *really* processed in parallel ? If the file is only 128 bytes, > I can easily imagine that the connections are

[1.6.1] Utilizing http-reuse

2015-11-10 Thread Krishna Kumar (Engineering)
Dear all, I am comparing 1.6.1 with 1.5.12. Following are the relevant snippets from the configuration file: global maxconn 100 defaults option http-keep-alive option clitcpka option srvtcpka frontend private-frontend maxconn 100

Re: [1.6.1] Utilizing http-reuse

2015-11-10 Thread Krishna Kumar (Engineering)
bed...@gmail.com> wrote: > On Tue, Nov 10, 2015 at 11:44 AM, Krishna Kumar (Engineering) > <krishna...@flipkart.com> wrote: >> Dear all, >> >> I am comparing 1.6.1 with 1.5.12. Following are the relevant snippets from >> the >> configuration file: >>

Re: [blog] What's new in HAProxy 1.6

2015-10-15 Thread Krishna Kumar (Engineering)
Extremely useful, thanks a lot. On Thu, Oct 15, 2015 at 5:13 AM, Igor Cicimov wrote: > > On 14/10/2015 9:41 PM, "Baptiste" wrote: >> >> Hey, >> >> I summarized what's new in HAProxy 1.6 with some configuration >> examples in a blog post to help

Re: Unexpected error messages

2015-10-16 Thread Krishna Kumar (Engineering)
, Oct 16, 2015 at 3:49 PM, Baptiste <bed...@gmail.com> wrote: > Is your problem fixed? > > We may emit a warning for such configuration. > > Baptiste > > Le 15 oct. 2015 07:34, "Krishna Kumar (Engineering)" > <krishna...@flipkart.com> a écrit : >

Unexpected error messages

2015-10-14 Thread Krishna Kumar (Engineering)
Hi all, We are occasionally getting these messages (about 25 errors/per occurrence, 1 occurrence per hour) in the *error* log: 10.xx.xxx.xx:60086 [14/Oct/2015:04:21:25.048] Alert-FE Alert-BE/10.xx.xx.xx 0/5000/1/32/+5033 200 +149 - - --NN 370/4/1/0/+1 0/0 {10.xx.x.xxx||367||} {|||432} "POST

Re: Unexpected error messages

2015-10-14 Thread Krishna Kumar (Engineering)
Hi Baptiste, Thank you for the advise and solution, I didn't realize retries had to be >1. Regards, - Krishna Kumar On Wed, Oct 14, 2015 at 7:51 PM, Baptiste <bed...@gmail.com> wrote: > On Wed, Oct 14, 2015 at 3:03 PM, Krishna Kumar (Engineering) > <krishna...@flipkart.com

Re: [1.6.1] Utilizing http-reuse

2015-12-08 Thread Krishna Kumar (Engineering)
arreau <w...@1wt.eu> wrote: > On Tue, Dec 08, 2015 at 07:44:45AM +0530, Krishna Kumar (Engineering) > wrote: > > Great, will be glad to test and report on the finding. Thanks! > > Sorry I forgot to post the patch after committing it. Here it comes. > Regarding the second

Re: [1.6.1] Utilizing http-reuse

2015-12-06 Thread Krishna Kumar (Engineering)
ailable polling systems : epoll : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use epoll. Thanks, - Krishna Kumar On Thu, Nov 12, 2015 at 12:50 PM, Willy Tarreau <w...@1wt.eu> wrote: > Hi Kris

Re: [1.6.1] Utilizing http-reuse

2015-12-06 Thread Krishna Kumar (Engineering)
Thanks a lot, Willy. Regards, - Krishna On Mon, Dec 7, 2015 at 11:59 AM, Willy Tarreau <w...@1wt.eu> wrote: > Hi Krishna, > > On Mon, Dec 07, 2015 at 08:31:19AM +0530, Krishna Kumar (Engineering) > wrote: > > Hi Willy, Baptiste, > > > > Apologies for

Re: [1.6.1] Utilizing http-reuse

2015-12-07 Thread Krishna Kumar (Engineering)
Great, will be glad to test and report on the finding. Thanks! Regards, - Krishna On Mon, Dec 7, 2015 at 9:07 PM, Willy Tarreau wrote: > Hi Krishna, > > I found a bug explaining your observations and noticed a second one I have > not yet troubleshooted. > > The bug causing your

Setting TOS field dynamically in 1.5.12 (or later)

2016-01-12 Thread Krishna Kumar (Engineering)
Hi all, Is there any way to set the TOS field dynamically (any mechanism, acl or others) for response back to the client? We require to be able to dynamically select one of the ISP links when a client makes a new connection. One option was to set the TOS field for responses from HAProxy.

Re: haproxy hangs on old linux kernels (2.6.24)

2016-04-12 Thread Krishna Kumar (Engineering)
This is actually a kernel Oops as it is accessing an invalid memory location: fff4 (bug in code). Only kernel upgrade can fix that. On Tue, Apr 12, 2016 at 12:43 PM, Alexey Vlasov wrote: > Hi, > > I have some linux boxes with very old kernels. Unfortunately, I

Re: [PATCH] MEDIUM/RFC: Implement time-based server latency metrics

2017-01-31 Thread Krishna Kumar (Engineering)
Hi Willy, Thanks for your detailed mail. I will get back to you very soon on this. Regards, - Krishna On Tue, Jan 31, 2017 at 12:54 AM, Willy Tarreau <w...@1wt.eu> wrote: > Hi Krishna, > > back on earth ;-) > > On Tue, Jan 03, 2017 at 03:07:26PM +0530, Krishna Kumar

Re: [PATCH] MEDIUM/RFC: Implement time-based server latency metrics

2017-01-23 Thread Krishna Kumar (Engineering)
Hi Willy, Sorry to bother you again, but a quick note in case you have forgotten this patch/email-thread. Regards, - Krishna On Thu, Jan 5, 2017 at 12:53 PM, Willy Tarreau <w...@1wt.eu> wrote: > Hi Krishna, > > On Thu, Jan 05, 2017 at 11:15:46AM +0530, Krishna Kumar (Engin

RFC: HAProxy shared health-check for nbproc > 1

2017-02-13 Thread Krishna Kumar (Engineering)
Hi Willy, Some time back, I had worked on making health checks being done by only one HAProxy process, and to share this information on a UP/DOWN event to other processes (tested for 64 processes). Before I finish it completely, I wanted to check with you if this feature is useful. At that time,

Re: RFC: HAProxy shared health-check for nbproc > 1

2017-02-14 Thread Krishna Kumar (Engineering)
.eu> wrote: > Hi Krishna, > > On Tue, Feb 14, 2017 at 12:45:31PM +0530, Krishna Kumar (Engineering) > wrote: > > Hi Willy, > > > > Some time back, I had worked on making health checks being done by only > > one HAProxy process, and to share this information

Re: [PATCH] MEDIUM/RFC: Implement time-based server latency metrics

2017-01-03 Thread Krishna Kumar (Engineering)
ase let me know if this sounds reasonable. Thanks, - Krishna On Thu, Dec 22, 2016 at 4:23 PM, Willy Tarreau <w...@1wt.eu> wrote: > Hi Krishna, > > On Thu, Dec 22, 2016 at 09:41:49AM +0530, Krishna Kumar (Engineering) > wrote: > > We have found that the current mechanism o

Re: [PATCH] MEDIUM/RFC: Implement time-based server latency metrics

2017-01-04 Thread Krishna Kumar (Engineering)
this topic, please let me know your opinion. Regards, - Krishna On Tue, Jan 3, 2017 at 3:07 PM, Krishna Kumar (Engineering) < krishna...@flipkart.com> wrote: > Hi Willy, > > Sorry for the late response as I was out during the year end, and thanks > once again for your review c

[PATCH] MEDIUM/RFC: Implement time-based server latency metrics

2016-12-21 Thread Krishna Kumar (Engineering)
We have found that the current mechanism of qtime, ctime, rtime, and ttime based on last 1024 requests is not the most suitable to debug/visualize latency issues with servers, especially if they happen to last a very short time. For live dashboards showing server timings, we found an additional

Re: [PATCH] MEDIUM/RFC: Implement time-based server latency metrics

2016-12-22 Thread Krishna Kumar (Engineering)
6 at 09:41:49AM +0530, Krishna Kumar (Engineering) > wrote: > > We have found that the current mechanism of qtime, ctime, rtime, and > ttime > > based on last 1024 requests is not the most suitable to debug/visualize > > latency issues with servers, especially if they happen to

Re: [PATCH] [MEDIUM] Improve "no free ports" error case

2017-03-15 Thread Krishna Kumar (Engineering)
ar 9, 2017 at 2:22 PM, Krishna Kumar (Engineering) < krishna...@flipkart.com> wrote: > Hi Willy, > > Excellent, I will try this idea, it should definitely help! > Thanks for the explanations. > > Regards, > - Krishna > > > On Thu, Mar 9, 2017 at 1:37 PM, Willy Ta

Re: Restricting RPS to a service

2017-04-19 Thread Krishna Kumar (Engineering)
-er.de> wrote: > Hi Krishna, > > Krishna Kumar (Engineering) wrote: > > Thanks for your response. However, I want to restrict the requests > > per second either at the frontend or backend, not session rate. I > > may have only 10 connections from clients, but the backends

Restricting RPS to a service

2017-04-19 Thread Krishna Kumar (Engineering)
Hi Willy, others, I have seen documents that describe how to rate limit from a single client. What is the way to rate limit on the entire service, without caring about which client is hitting it? Something like "All RPS should be < 1000/sec"? Thanks, - Krishna

[PATCH] [MEDIUM] Improve "no free ports" error case

2017-03-08 Thread Krishna Kumar (Engineering)
Hi Willy, We use HAProxy as a Forward Proxy (I know this is not the intended application for HAProxy) to access outside world from within the DC, and this requires setting a source port range for return traffic to reach the correct box from which a connection was established. On our production

Re: [PATCH] [MEDIUM] Improve "no free ports" error case

2017-03-09 Thread Krishna Kumar (Engineering)
Hi Willy, Excellent, I will try this idea, it should definitely help! Thanks for the explanations. Regards, - Krishna On Thu, Mar 9, 2017 at 1:37 PM, Willy Tarreau <w...@1wt.eu> wrote: > On Thu, Mar 09, 2017 at 12:50:16PM +0530, Krishna Kumar (Engineering) > wrote: > > 1.

Re: [PATCH] [MEDIUM] Improve "no free ports" error case

2017-03-08 Thread Krishna Kumar (Engineering)
ttempting to connect to the same remote server. Hope I explained that clearly. Thanks, - Krishna On Thu, Mar 9, 2017 at 12:19 PM, Willy Tarreau <w...@1wt.eu> wrote: > Hi Krishna, > > On Thu, Mar 09, 2017 at 12:03:19PM +0530, Krishna Kumar (Engineering) > wrote: > > Hi Willy,

Using http_proxy in HAProxy 1.6.3

2017-07-16 Thread Krishna Kumar (Engineering)
I have configured a backend as follows: backend be-testing mode http option httpclose option http_proxy and hit this using Google's IP as: wget --header="Host: http://216.58.197.46:80; but this fails with 503 error, debug shows:

HAProxy 1.6.3: 100% cpu utilization for >17 days with 1 connection

2017-05-18 Thread Krishna Kumar (Engineering)
Hi, First of all, thanks for a great product that is working extremely well for Flipkart! I saw many similar issues posted earlier by others, but could not find a thread where this is resolved or fixed in a newer release. We are using Ubuntu 16.04 with distro HAProxy (1.6.3), and see that

Re: HAProxy 1.6.3: 100% cpu utilization for >17 days with 1 connection

2017-05-18 Thread Krishna Kumar (Engineering)
ll upgrade after internal processes are sorted out. Thanks once again for this quick information on the source of the problem. Regards, - Krishna On Fri, May 19, 2017 at 10:34 AM, Willy Tarreau <w...@1wt.eu> wrote: > Hi Krishna, > > On Fri, May 19, 2017 at 09:47:52AM +0530, Krishna Kum

Re: Long Running TCP Connections and Reloads

2017-09-14 Thread Krishna Kumar (Engineering)
Regarding #1, I think this was fixed sometime back. Maybe you are running a old version of haproxy? commit e39683c4d4c527d1b561c3ba3983d26cc3e7f42d Author: Hongbo Long Date: Fri Mar 10 18:41:51 2017 +0100 BUG/MEDIUM: stream: fix client-fin/server-fin handling

Re: Throughput issue after moving between kernels.

2017-11-03 Thread Krishna Kumar (Engineering)
Though it would not cause your problem, the reason for this is: In 3.10.18: https://elixir.free-electrons.com/linux/v3.10.18/source/net/ipv4/tcp.c void tcp_init_mem (struct net *net){ unsigned long limit

Request rate limiting on the backend section

2017-11-07 Thread Krishna Kumar (Engineering)
Hi all, I am trying to implement request rate limiting to protect our servers from too many requests. While we were able to get this working correctly in the frontend section, it is required to implement the same in the backend section due to the configuration we use in our data center for

Re: Request rate limiting on the backend section

2017-11-07 Thread Krishna Kumar (Engineering)
On Tue, Nov 7, 2017 at 11:57 PM, Lukas Tribus wrote: Hi Lukas, > Yes, in 1.7 you can change server maxconn values in real time using > the admin socket: > https://cbonte.github.io/haproxy-dconv/1.7/management. html#9.3-set%20maxconn%20server Thanks, will take a look at if we can

Re: [Working update]: Request rate limiting on the backend section

2017-11-08 Thread Krishna Kumar (Engineering)
To remove the reported "margin of error", the config needed a fix: acl within_limit sc2_gpc0_rate() lt 1000 since the first request was at rate==0, and last one is at 999. On Wed, Nov 8, 2017 at 3:11 PM, Krishna Kumar (Engineering) < krishna...@flipkart.com> wrote:

[Working update]: Request rate limiting on the backend section

2017-11-08 Thread Krishna Kumar (Engineering)
% Thanks, - Krishna On Wed, Nov 8, 2017 at 10:02 AM, Krishna Kumar (Engineering) < krishna...@flipkart.com> wrote: > On Tue, Nov 7, 2017 at 11:57 PM, Lukas Tribus <lu...@ltri.eu> wrote: > > Hi Lukas, > > > Yes, in 1.7 you can change server maxconn values in real t

Re: Request rate limiting on the backend section

2017-11-07 Thread Krishna Kumar (Engineering)
Hi Lukas, On Tue, Nov 7, 2017 at 6:46 PM, Lukas Tribus wrote: > I'd suggest to use maxconn. This limits the amount of connections opened > to a single server, and is therefor equivalent to in-flight requests. That's is > a more appropriate limit than RPS because it doesn't matter

Idle HAProxy 1.8 spins at 100% in user space

2018-03-12 Thread Krishna Kumar (Engineering)
As an aside, could someone also post a simple configuration file to enable 40 listeners (thread)? I get 100% cpu util when running high number (>30, on a 48 core system) of threads, I have tried both these versions: HA-Proxy version 1.8.4-1ppa1~xenial 2018/02/10: Installed via .deb file HA-Proxy

Re: Idle HAProxy 1.8 spins at 100% in user space

2018-03-12 Thread Krishna Kumar (Engineering)
Hi Cyril, Thanks, this patch fixes it, it is now back to 0%. Confirmed it a few times, and undid the patch, back to 100%, and re-added the patch, back to 0%. Fixes perfectly. Thanks, - Krishna On Mon, Mar 12, 2018 at 5:23 PM, Willy Tarreau wrote: > On Mon, Mar 12, 2018 at

Re: Few problems seen in haproxy? (threads, connections).

2018-10-16 Thread Krishna Kumar (Engineering)
Hi Willy, My systems were out of rotation for some other tests so did not get to this till now. I have pulled the latest bits just now and tested. Regarding maxconn, I simply kept maxconn in global/defaults to 1 million and have this line in the backend section: default-server maxconn 100

Re: Few problems seen in haproxy? (threads, connections).

2018-10-05 Thread Krishna Kumar (Engineering)
Sorry for repeating once again, but this is my last unsolicited mail on this topic. Any directions for what to look out for? Thanks, - Krishna On Thu, Oct 4, 2018 at 8:42 AM Krishna Kumar (Engineering) < krishna...@flipkart.com> wrote: > Re-sending in case this mail was missed. To

Few problems seen in haproxy? (threads, connections).

2018-10-02 Thread Krishna Kumar (Engineering)
Hi Willy, and community developers, I am not sure if I am doing something wrong, but wanted to report some issues that I am seeing. Please let me know if this is a problem. 1. HAProxy system: Kernel: 4.17.13, CPU: 48 core E5-2670 v3 Memory: 128GB memory NIC: Mellanox 40g with IRQ pinning 2.

Re: Few problems seen in haproxy? (threads, connections).

2018-10-11 Thread Krishna Kumar (Engineering)
(fd, task_rq, task_wq, proxy, server, lbprm and buf_wq increased significantly). Thanks! On Thu, Oct 11, 2018 at 8:53 AM Krishna Kumar (Engineering) < krishna...@flipkart.com> wrote: > Thanks, will do that. > > On Thu, Oct 11, 2018 at 8:37 AM Willy Tarreau wrote: > >> On

Re: Few problems seen in haproxy? (threads, connections).

2018-10-10 Thread Krishna Kumar (Engineering)
and the results. Regards, - Krishna On Wed, Oct 10, 2018 at 6:39 PM Willy Tarreau wrote: > Hi Krishna, > > On Tue, Oct 02, 2018 at 09:18:19PM +0530, Krishna Kumar (Engineering) > wrote: > (...) > > 1. HAProxy system: > > Kernel: 4.17.13, > > CPU: 48 core E5-2670 v3

Re: Few problems seen in haproxy? (threads, connections).

2018-10-04 Thread Krishna Kumar (Engineering)
m youself if you have time/wish :) > > > чт, 4 окт. 2018 г. в 11:53, Krishna Kumar (Engineering) < > krishna...@flipkart.com>: > >> 1. haproxy config: Same as given above (both processes and threads were >> given in the mail) >> 2. nginx: default, no changes. &

Re: Few problems seen in haproxy? (threads, connections).

2018-10-03 Thread Krishna Kumar (Engineering)
, - Krishna On Tue, Oct 2, 2018 at 9:18 PM Krishna Kumar (Engineering) < krishna...@flipkart.com> wrote: > Hi Willy, and community developers, > > I am not sure if I am doing something wrong, but wanted to report > some issues that I am seeing. Please let me know if this is a probl

Re: Few problems seen in haproxy? (threads, connections).

2018-10-03 Thread Krishna Kumar (Engineering)
M Илья Шипицин wrote: > load testing is somewhat good. > can you describe an overall setup ? (I want to reproduce and play with it) > > чт, 4 окт. 2018 г. в 8:16, Krishna Kumar (Engineering) < > krishna...@flipkart.com>: > >> Re-sending in case this mail was missed. To

Re: Few problems seen in haproxy? (threads, connections).

2018-10-04 Thread Krishna Kumar (Engineering)
utput ? do you have nf > conntrack enabled ? what are its limits ? > > чт, 4 окт. 2018 г. в 9:59, Krishna Kumar (Engineering) < > krishna...@flipkart.com>: > >> Sure. >> >> 1. Client: Use one of the following two setup's: >> - a single baremetal (48