set weight bug?

2013-11-05 Thread Igor
Using newest snapshot, when I do

echo set weight s1/p1 100| socat stdio /tmp/haproxy

to a server already has weight 100, then fresh haproxy's stat page, it
requires password, and it doesn't accept the right password set in
stats auth until I reload the haproxy.

I have a script to set servers weight, I found sometimes set weight to
servers rapidly, like multi echo set weight s(*)/p(*) 100| socat
stdio /tmp/haproxy, will crash haproxy daemon.


Bests,
-Igor



[PATCH v7 0/5] Auxiliary Agent Check Preparation

2013-11-05 Thread Simon Horman
Hi,

This series is a revised subset of
[PATCH v6 00/22] Agent Check Enhancements and External Check.
I have broken it out as a smaller set of patches to allow
review to be done in smaller chunks.

Although these patches aren't particularly useful in their own right
they also should not break anything and were positively reviewed
when last posted.


This series is based on 0bb166b
(MINOR: tcp: don't use tick_add_ifset() when timeout is known to be set)
which is the head of the current master branch on haproxy.org.

This series depends on  [PATCH v7] CLEANUP: Make parameters of srv_downtime
and srv_getinter const to apply cleanly.

To aid review this series is available in git at
https://github.com/horms/haproxy.git/agent-check-prep-v7

To further aid review, my working version of the
fuller patch-set, based on this patchset, is available at
https://github.com/horms/haproxy.git/agent-check-v7.snapshot.20131105

Simon Horman (5):
  MEDIUM: Move result element to struct check
  MEDIUM: Paramatise functions over the check of a server
  MEDIUM: cfgparse: Factor out check initialisation
  MEDIUM: Add state to struct check
  MEDIUM: Move health element to struct check

 include/proto/checks.h |   4 +-
 include/proto/server.h |   2 +-
 include/types/server.h |  11 +-
 src/cfgparse.c |  70 ---
 src/checks.c   | 543 +
 src/dumpstats.c|  22 +-
 src/proto_http.c   |   6 +-
 src/server.c   |  14 +-
 8 files changed, 349 insertions(+), 323 deletions(-)

-- 
1.8.4




[PATCH v7 4/5] MEDIUM: Add state to struct check

2013-11-05 Thread Simon Horman
Add state to struct check. This is currently used to store one bit,
CHK_RUNNING, which is set if a check is running and clear otherwise.
This bit was previously SRV_CHK_RUNNING of the state element of struct
server.

This is in preparation for associating a agent check
with a server which runs as well as the server's existing check.

Signed-off-by: Simon Horman horms+rene...@verge.net.au

---

v8
* Use CHK_STATE_RUNNING instead of CHK_RUNNING

Notes by Willy on v6:
Please use CHK_STATE_RUNNING instead of CHK_RUNNING so that it's
easier to know it is related to the state attribute of the check.

Also, we have many HCHK_* and now CHK_* as well, so I wonder how/when
this will become confusing.
---
 include/types/server.h | 5 -
 src/checks.c   | 8 
 src/dumpstats.c| 2 +-
 3 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/include/types/server.h b/include/types/server.h
index 0e4c33e..bcde01b 100644
--- a/include/types/server.h
+++ b/include/types/server.h
@@ -55,7 +55,6 @@
 /* unused: 0x0100, 0x0200, 0x0400 */
 #define SRV_SEND_PROXY 0x0800  /* this server talks the PROXY protocol */
 #define SRV_NON_STICK  0x1000  /* never add connections allocated to this 
server to a stick table */
-#define SRV_CHK_RUNNING 0x2000  /* a check is currently running on this server 
*/
 
 /* function which act on servers need to return various errors */
 #define SRV_STATUS_OK   0   /* everything is OK. */
@@ -70,6 +69,9 @@
 #define SRV_CHK_PASSED  0x0002   /* server check succeeded unless FAILED is 
also set */
 #define SRV_CHK_DISABLE 0x0004   /* server returned a disable code */
 
+/* check flags */
+#define CHK_STATE_RUNNING  0x0001  /* this check is currently running */
+
 /* various constants */
 #define SRV_UWGHT_RANGE 256
 #define SRV_UWGHT_MAX   (SRV_UWGHT_RANGE)
@@ -117,6 +119,7 @@ struct check {
int send_proxy; /* send a PROXY protocol header 
with checks */
int inter, fastinter, downinter;/* checks: time in milliseconds 
*/
int result; /* health-check result : 
SRV_CHK_* */
+   int state;  /* health-check result : CHK_* 
*/
int type;   /* Check type, one of 
PR_O2_*_CHK */
struct server *server;  /* back-pointer to server */
 };
diff --git a/src/checks.c b/src/checks.c
index a2ff237..c37afbd 100644
--- a/src/checks.c
+++ b/src/checks.c
@@ -1315,7 +1315,7 @@ static struct task *process_chk(struct task *t)
int ret;
int expired = tick_is_expired(t-expire, now_ms);
 
-   if (!(s-state  SRV_CHK_RUNNING)) {
+   if (!(check-state  CHK_STATE_RUNNING)) {
/* no check currently running */
if (!expired) /* woke up too early */
return t;
@@ -1329,7 +1329,7 @@ static struct task *process_chk(struct task *t)
/* we'll initiate a new check */
set_server_check_status(check, HCHK_STATUS_START, NULL);
 
-   s-state |= SRV_CHK_RUNNING;
+   check-state |= CHK_STATE_RUNNING;
check-bi-p = check-bi-data;
check-bi-i = 0;
check-bo-p = check-bo-data;
@@ -1420,7 +1420,7 @@ static struct task *process_chk(struct task *t)
 
/* here, we have seen a synchronous error, no fd was allocated 
*/
 
-   s-state = ~SRV_CHK_RUNNING;
+   check-state = ~CHK_STATE_RUNNING;
if (s-health  s-rise) {
s-health--; /* still good */
s-counters.failed_checks++;
@@ -1516,7 +1516,7 @@ static struct task *process_chk(struct task *t)
set_server_up(check);
}
}
-   s-state = ~SRV_CHK_RUNNING;
+   check-state = ~CHK_STATE_RUNNING;
 
rv = 0;
if (global.spread_checks  0) {
diff --git a/src/dumpstats.c b/src/dumpstats.c
index 9a422c1..be0aac6 100644
--- a/src/dumpstats.c
+++ b/src/dumpstats.c
@@ -2273,7 +2273,7 @@ static int stats_dump_sv_stats(struct stream_interface 
*si, struct proxy *px, in
if (sv-state  SRV_CHECKED) {
chunk_appendf(trash,
  /tdtd class=acu %s%s,
- (sv-state  SRV_CHK_RUNNING) ? *  : ,
+ (sv-check.state  CHK_STATE_RUNNING) ? 
*  : ,
  get_check_status_info(sv-check.status));
 
if (sv-check.status = HCHK_STATUS_L57DATA)
-- 
1.8.4




[PATCH v7 2/5] MEDIUM: Paramatise functions over the check of a server

2013-11-05 Thread Simon Horman
Paramatise the following functions over the check of a server

* set_server_down
* set_server_up
* srv_getinter
* server_status_printf
* set_server_check_status
* set_server_disabled
* set_server_enabled

Generally the server parameter of these functions has been removed.
Where it is still needed it is obtained using check-server.

This is in preparation for associating a agent check
with a server which runs as well as the server's existing check.
By paramatising these functions they may act on each of the checks
without further significant modification.

Explanation of the SSP_O_HCHK portion of this change:

* Prior to this patch SSP_O_HCHK serves a single purpose which
  is to tell server_status_printf() weather it should print
  the details of the check of a server or not.

  With the paramatisation that this patch adds there are two cases.
  1) Printing the details of the check in which case a
 valid check parameter is needed.
  2) Not printing the details of the check in which case
 the contents check parameter are unused.

  In case 1) we could pass SSP_O_HCHK and a valid check and;
  In case 2) we could pass !SSP_O_HCHK and any value for check
  including NULL.

  If NULL is used for case 2) then SSP_O_HCHK becomes supurfulous
  and as NULL is used for case 2) SSP_O_HCHK has been removed.

Signed-off-by: Simon Horman ho...@verge.net.au

---

This is a rather large patch and I would be happy to split it up
on request.

---

v7
* Paramatise set_server_enabled() over the check of a server
* Update changelog to explain SSP_O_HCHK removal
* Update set_server_disabled()
  to use its check parameter rather than check-server-check which will
  not be the same thing once there is more than one check associated with a
  server.
* Always check-type instead of s-proxy-options2  PR_O2_CHK_ANY.
  This should always be correct and being consistent makes things
  less confusing.

Notes by Willy on v6:
- still need to understand the SSP_O_HCHK - check change
- set_server_disabled() : takes a check arg, converts it to server then
  uses s-check, seems suspicious or at least confusing
  (why not simply check ?)
- also :
for(srv = s-tracknext; srv; srv = srv-tracknext)
-   set_server_disabled(srv);
+   set_server_disabled(s-check);

   isn't it srv-check instead ?

- it's nice to have a check being the owner of a connection instead of the
  server, looks much cleaner than what we used to have.
- it should probably be made it more obvious what case is covered by
   !check-type (if relevant at this point)

v4
* Use check-type in process_check()

  Use check-type instead of s-proxy-options2  PR_O2_CHK_ANY
  as the former is specific to the check being processed whereas
  the latter relates only to the primary check of a server.

* Add type to struct check

  This is used to indicate the type of a check independent of
  its server's proxy's check type.

v2 - v3
* No Change
---
 include/proto/checks.h |   4 +-
 include/proto/server.h |   2 +-
 include/types/server.h |   1 +
 src/cfgparse.c |   1 +
 src/checks.c   | 487 +
 src/dumpstats.c|   8 +-
 src/proto_http.c   |   4 +-
 src/server.c   |  10 +-
 8 files changed, 263 insertions(+), 254 deletions(-)

diff --git a/include/proto/checks.h b/include/proto/checks.h
index 042b854..f4f9c49 100644
--- a/include/proto/checks.h
+++ b/include/proto/checks.h
@@ -27,8 +27,8 @@
 
 const char *get_check_status_description(short check_status);
 const char *get_check_status_info(short check_status);
-void set_server_down(struct server *s);
-void set_server_up(struct server *s);
+void set_server_down(struct check *check);
+void set_server_up(struct check *check);
 int start_checks();
 void health_adjust(struct server *s, short status);
 
diff --git a/include/proto/server.h b/include/proto/server.h
index b2df045..1151b3c 100644
--- a/include/proto/server.h
+++ b/include/proto/server.h
@@ -33,7 +33,7 @@
 #include proto/freq_ctr.h
 
 int srv_downtime(const struct server *s);
-int srv_getinter(const struct server *s);
+int srv_getinter(const struct check *check);
 
 /* increase the number of cumulated connections on the designated server */
 static void inline srv_inc_sess_ctr(struct server *s)
diff --git a/include/types/server.h b/include/types/server.h
index be11605..0e4c33e 100644
--- a/include/types/server.h
+++ b/include/types/server.h
@@ -117,6 +117,7 @@ struct check {
int send_proxy; /* send a PROXY protocol header 
with checks */
int inter, fastinter, downinter;/* checks: time in milliseconds 
*/
int result; /* health-check result : 
SRV_CHK_* */
+   int type;   /* Check type, one of 
PR_O2_*_CHK */
struct server *server;  /* back-pointer to server */
 };
 
diff --git a/src/cfgparse.c 

[PATCH v7 3/5] MEDIUM: cfgparse: Factor out check initialisation

2013-11-05 Thread Simon Horman
This is in preparation for struct server having two elements
of type struct check.

Signed-off-by: Simon Horman ho...@verge.net.au

---

v7
* No change

Notes by Willy on v6: OK

v5
* Remove server argument from init_check. It is not used.
* Set type in init_check

  This allows a zero type to be used to indicate that
  a task for a secondary agent check should not be stated.
---
 src/cfgparse.c | 62 +++---
 1 file changed, 38 insertions(+), 24 deletions(-)

diff --git a/src/cfgparse.c b/src/cfgparse.c
index efbdad8..c70700d 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -1620,6 +1620,34 @@ out:
return err_code;
 }
 
+static int init_check(struct check *check, int type, const char * file, int 
linenum)
+{
+   check-type = type;
+
+   /* Allocate buffer for requests... */
+   if ((check-bi = calloc(sizeof(struct buffer) + global.tune.chksize, 
sizeof(char))) == NULL) {
+   Alert(parsing [%s:%d] : out of memory while allocating check 
buffer.\n, file, linenum);
+   return ERR_ALERT | ERR_ABORT;
+   }
+   check-bi-size = global.tune.chksize;
+
+   /* Allocate buffer for responses... */
+   if ((check-bo = calloc(sizeof(struct buffer) + global.tune.chksize, 
sizeof(char))) == NULL) {
+   Alert(parsing [%s:%d] : out of memory while allocating check 
buffer.\n, file, linenum);
+   return ERR_ALERT | ERR_ABORT;
+   }
+   check-bo-size = global.tune.chksize;
+
+   /* Allocate buffer for partial results... */
+   if ((check-conn = calloc(1, sizeof(struct connection))) == NULL) {
+   Alert(parsing [%s:%d] : out of memory while allocating check 
connection.\n, file, linenum);
+   return ERR_ALERT | ERR_ABORT;
+   }
+
+   check-conn-t.sock.fd = -1; /* no agent in progress yet */
+
+   return 0;
+}
 
 int cfg_parse_listen(const char *file, int linenum, char **args, int kwm)
 {
@@ -4231,6 +4259,9 @@ stats_error_parsing:
 
newsrv-health = newsrv-rise;  /* up, but will fall 
down at first failure */
 
+   newsrv-check.status= HCHK_STATUS_INI;
+   newsrv-check.server= newsrv;
+
cur_arg = 3;
} else {
newsrv = curproxy-defsrv;
@@ -4790,6 +4821,8 @@ stats_error_parsing:
}
 
if (do_check) {
+   int ret;
+
if (newsrv-trackit) {
Alert(parsing [%s:%d]: unable to enable checks 
and tracking at the same time!\n,
file, linenum);
@@ -4835,33 +4868,14 @@ stats_error_parsing:
goto out;
}
 
-   /* Allocate buffer for check requests... */
-   if ((newsrv-check.bi = calloc(sizeof(struct buffer) + 
global.tune.chksize, sizeof(char))) == NULL) {
-   Alert(parsing [%s:%d] : out of memory while 
allocating check buffer.\n, file, linenum);
-   err_code |= ERR_ALERT | ERR_ABORT;
-   goto out;
-   }
-   newsrv-check.bi-size = global.tune.chksize;
-
-   /* Allocate buffer for check responses... */
-   if ((newsrv-check.bo = calloc(sizeof(struct buffer) + 
global.tune.chksize, sizeof(char))) == NULL) {
-   Alert(parsing [%s:%d] : out of memory while 
allocating check buffer.\n, file, linenum);
-   err_code |= ERR_ALERT | ERR_ABORT;
-   goto out;
-   }
-   newsrv-check.bo-size = global.tune.chksize;
-
-   /* Allocate buffer for partial check results... */
-   if ((newsrv-check.conn = calloc(1, sizeof(struct 
connection))) == NULL) {
-   Alert(parsing [%s:%d] : out of memory while 
allocating check connection.\n, file, linenum);
-   err_code |= ERR_ALERT | ERR_ABORT;
+   ret = init_check(newsrv-check,
+curproxy-options2  PR_O2_CHK_ANY,
+file, linenum);
+   if (ret) {
+   err_code |= ret;
goto out;
}
 
-   newsrv-check.conn-t.sock.fd = -1; /* no check in 
progress yet */
-   newsrv-check.status = HCHK_STATUS_INI;
-   newsrv-check.type = curproxy-options2  PR_O2_CHK_ANY;
-   newsrv-check.server = newsrv;
newsrv-state |= SRV_CHECKED;
}
 
-- 
1.8.4




[PATCH v7 5/5] MEDIUM: Move health element to struct check

2013-11-05 Thread Simon Horman
This is in preparation for associating a agent check
with a server which runs as well as the server's existing check.

Signed-off-by: Simon Horman ho...@verge.net.au

---

v7
* Merge line-reordering update from a subsequent patch
* Rebase as name field is no longer added to struct check
  by a previous patch

Notes by Willy on v6: OK
---
 include/types/server.h |  3 ++-
 src/cfgparse.c | 11 +--
 src/checks.c   | 50 +-
 src/dumpstats.c| 12 ++--
 src/proto_http.c   |  2 +-
 src/server.c   |  4 ++--
 6 files changed, 41 insertions(+), 41 deletions(-)

diff --git a/include/types/server.h b/include/types/server.h
index bcde01b..1df56e9 100644
--- a/include/types/server.h
+++ b/include/types/server.h
@@ -120,6 +120,8 @@ struct check {
int inter, fastinter, downinter;/* checks: time in milliseconds 
*/
int result; /* health-check result : 
SRV_CHK_* */
int state;  /* health-check result : CHK_* 
*/
+   int health; /* 0 to server-rise-1 = bad;
+* rise to 
server-rise+server-fall-1 = good */
int type;   /* Check type, one of 
PR_O2_*_CHK */
struct server *server;  /* back-pointer to server */
 };
@@ -151,7 +153,6 @@ struct server {
 
struct server *tracknext, *track;   /* next server in a tracking 
list, tracked server */
char *trackit;  /* temporary variable to make 
assignment deferrable */
-   int health; /* 0-rise-1 = bad; 
rise-rise+fall-1 = good */
int consecutive_errors; /* current number of 
consecutive errors */
int rise, fall; /* time in iterations */
int consecutive_errors_limit;   /* number of consecutive errors 
that triggers an event */
diff --git a/src/cfgparse.c b/src/cfgparse.c
index c70700d..a1c9cdf 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -4257,9 +4257,8 @@ stats_error_parsing:
newsrv-uweight = newsrv-iweight
= curproxy-defsrv.iweight;
 
-   newsrv-health = newsrv-rise;  /* up, but will fall 
down at first failure */
-
newsrv-check.status= HCHK_STATUS_INI;
+   newsrv-check.health= newsrv-rise; /* up, but will 
fall down at first failure */
newsrv-check.server= newsrv;
 
cur_arg = 3;
@@ -4295,8 +4294,8 @@ stats_error_parsing:
goto out;
}
 
-   if (newsrv-health)
-   newsrv-health = newsrv-rise;
+   if (newsrv-check.health)
+   newsrv-check.health = newsrv-rise;
cur_arg += 2;
}
else if (!strcmp(args[cur_arg], fall)) {
@@ -4477,7 +4476,7 @@ stats_error_parsing:
else if (!defsrv  !strcmp(args[cur_arg], disabled)) 
{
newsrv-state |= SRV_MAINTAIN;
newsrv-state = ~SRV_RUNNING;
-   newsrv-health = 0;
+   newsrv-check.health = 0;
cur_arg += 1;
}
else if (!defsrv  !strcmp(args[cur_arg], observe)) {
@@ -6767,7 +6766,7 @@ out_uri_auth_compat:
if (srv-state  SRV_MAINTAIN) {
newsrv-state |= SRV_MAINTAIN;
newsrv-state = ~SRV_RUNNING;
-   newsrv-health = 0;
+   newsrv-check.health = 0;
}
 
newsrv-track = srv;
diff --git a/src/checks.c b/src/checks.c
index c37afbd..44ad4b7 100644
--- a/src/checks.c
+++ b/src/checks.c
@@ -229,8 +229,8 @@ static void set_server_check_status(struct check *check, 
short status, const cha
}
 
if (s-proxy-options2  PR_O2_LOGHCHKS 
-   (((s-health != 0)  (check-result  SRV_CHK_FAILED)) ||
-   ((s-health != s-rise + s-fall - 1)  (check-result  
SRV_CHK_PASSED)) ||
+   (((check-health != 0)  (check-result  SRV_CHK_FAILED)) ||
+   ((check-health != s-rise + s-fall - 1)  (check-result  
SRV_CHK_PASSED)) ||
((s-state  SRV_GOINGDOWN)  !(check-result  SRV_CHK_DISABLE)) 
||
(!(s-state  SRV_GOINGDOWN)  (check-result  
SRV_CHK_DISABLE {
 
@@ -239,7 +239,7 @@ static void set_server_check_status(struct check *check, 
short 

Re: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-05 Thread Willy Tarreau
Hello Fred,

[ first, please avoid top-posting, this is very cumbersome for replying
  in context afterwards, and tends to pollute subscribers mailboxes with
  overly large emails ]

  Also, can you confirm that this is a real machine and that we're not
  troubleshooting a VM ?

 Yes, this is a 'real machine', running FreeBSD 9 x64.
 
 It is a Xeon E5-2650 Dual (So we have 16 physical cores to use here and 32
 threads).

OK. Do you know if you have a single or multiple interrupts on your NICs,
and if they're delivered to a single core, multiple cores, or floating
around more or less randomly ?

  That said, assuming you're dealing with 300 Mbps (about 40 MB/s) and say
  500 bytes per message, this turns into 80k messages per second, which
  require :
- 2 recvfrom()
- 1 getsockopt()  (we can remove this one, 1.5 doesn't have it)
- 1 sendto()
  
  So 4 syscalls per message, resulting in 320k syscalls per second. It can
  start to represent some CPU usage. But there's more. Such small messages are
  transferred using TCP_NODELAY meaning that a TCP PUSH is set on each
  outgoing packet and that each of them is immediately ACKed. So you get
  80kpps per side in each direction, resulting in 320kpps as well. If you have
  a firewall running on the system, it might take its share of load as well,
  which is possibly attributed to the sending process on outgoing messages.
 
  That said, even with that in mind, I still consider that the system load is
  high for the workload. Could you please share the output of vmstat 1
  (just take the first 10 lines) ?

 Here is the vmstat 1 result :
 
 procs  memory  pagedisks faults cpu
  r b w avmfre   flt  re  pi  pofr  sr da0 pa0   in   sy   cs us 
 sy id
  7 0 0   4818M35G   643   0   0   0   714   0   0   0 4977 1364 5996  8 
 25 67
  3 0 0   4818M35G   224   0   0   0   174   0   0   0 42698 355001 170303 
  8 22 71
  3 0 0   4818M35G   177   0   0   0   174   0   0   0 28715 383061 138108 
  7 23 69
  4 0 0   4818M35G   173   0   0   0   174   0   0   0 28342 375281 138067 
  8 24 69
  5 0 0   4818M35G   185   0   0   0   174   0   0   0 32900 372294 148576 
  7 21 71
  5 0 0   4818M35G   372   0   0   0   174   0   0   0 29112 364030 138826 
  7 25 68

It seems that your numbers below tend to confirm this model.

I still don't know why you have that high a context switch rate. Are you
running with more processes than CPUs ? Also it looks like the system is
mostly spending its time idling. Is it that haproxy is on the same CPU as
the network's interrupts ? Then maybe it could make sense to start multiple
processes and pin them to specific CPU cores, and do the same with the
interrupts. Delivering 500-bytes large messages between two NICs via
userspace experiences a high overhead and everything which could be saved
must be saved (including CPU cache misses).

 We are speaking about 100Kpps (input) and 140Kpps (output) 'approximately'.

OK, so probably about 30k msg/s in each direction with their respective ACKs.
That just makes me think it could possibly do better since we can do better
with HTTP messages.

Do you have enough concurrent connections to fill the wire and ensure that
the system never waits for either a client or a server ? I'm assuming that
OK given the values assigned to the file descriptors in your latest email,
which were up to 1428. With such numbers and that small messages, it can
make sense to use multiple processes if that's not the case yet.

Best regards,
Willy




Re: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-05 Thread Jonathan Matthews
On 5 November 2013 11:16, Willy Tarreau w...@1wt.eu wrote:
 It is a Xeon E5-2650 Dual (So we have 16 physical cores to use here and 32
 threads).

 OK. Do you know if you have a single or multiple interrupts on your NICs,
 and if they're delivered to a single core, multiple cores, or floating
 around more or less randomly ?
[snip]

 I still don't know why you have that high a context switch rate. Are you
 running with more processes than CPUs ?

Fred is running with at least 30 separate haproxy processes (as per
his top output in message-id
col129-ds31e074947100ad71da09cb0...@phx.gbl) and 16 real (32 H/T)
cores.

I haven't seen a mail in this thread where Fred's shown that his
problems persist after moving to a single haproxy instance.

/wood-for-the-trees :-)

Jonathan



RES: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-05 Thread Fred Pedrisa
 OK. Do you know if you have a single or multiple interrupts on your NICs,
and if they're delivered to a single core, multiple cores, or floating
around more or less randomly ?

This is managed by FreeBSD, it currently have multiple queues and irq
balance with msix.

 It seems that your numbers below tend to confirm this model.

 I still don't know why you have that high a context switch rate. Are you
running with more processes than CPUs ? Also it looks like the system is
mostly spending its time idling. Is it that haproxy is on the same CPU as
the network's interrupts ? Then maybe it could make sense to start multiple
processes and pin them to specific CPU cores, and do the same with the
interrupts. Delivering 500-bytes large messages between two NICs via
userspace experiences a high overhead and everything which could be saved
must be saved (including CPU cache misses).

Yes, if we have 40 processes running and 16 physical cores, I suppose this
is more than the number of physical cores available right ?

However, in FreeBSD we can't do that IRQ Assigning, like we can on linux.
(As far I know).

 We are speaking about 100Kpps (input) and 140Kpps (output)
'approximately'.

 OK, so probably about 30k msg/s in each direction with their respective
ACKs.
 That just makes me think it could possibly do better since we can do
better with HTTP messages.

 Do you have enough concurrent connections to fill the wire and ensure
that the system never waits for either a client or a server ? I'm assuming
that OK given the values assigned to the file descriptors in your latest
email, which were up to 1428. With such numbers and that small messages, it
can make  sense to use multiple processes if that's not the case yet.

In theory yes, the connections are quick, because they are pure tcp
applications and in other cases, http websites, but behind the pure tcp mode
instead of http mode (not in all cases tho).

Fred




Re: RES: RES: RES: RES: RES: RES: RES: RES: High CPU Usage (HaProxy)

2013-11-05 Thread Dmitry Sivachenko
On 05 нояб. 2013 г., at 19:33, Fred Pedrisa fredhp...@hotmail.com wrote:

 
 However, in FreeBSD we can't do that IRQ Assigning, like we can on linux.
 (As far I know).
 


JFYI: you can assign IRQs to CPUs via cpuset -x irq
(I can’t tell you if it is “like on linux” or not though).




BOLSAS PARA ANCHETAS NAVIDEÑAS y REGALOS CORPORATIVOS

2013-11-05 Thread BIO BOLSA SAS

Esta newsletter por email le fué enviada en formato gráfico HTML.
Si usted está viendo esta versión, su programa de email prefiere mensajes de 
texto sin formato.
Usted puede leer la versión original online:
http://ymlp333.net/z4B3ij



Haga click aquí SI NO puede ver las imágenes de éste E-mail (
http://ymlp333.net/z4B3ij )

Para ésta temporada Navideña, contamos con bolsas ideales para
anchetas navideñas de todo tipo, material y tamaño, así mismo para
empacar esos regalos que desean obsequiar a clientes, proveedores y
familiares.

Las bolsas van con el logo que deseen, de su empresa o alusivo a la
temporada.

LLÁMENOS y CON GUSTO LE COTIZAREMOS!!!

BOGOTÁ D.C - COLOMBIA
Tel:  (57) (1) 713 87 31
(57) (1) 713 75 16
Cels:  (57) 300 275 45 51
(57) 300 555 89 48

MEDELLIN - COLOMBIA
Tel:   (57) (4) 268 42 03

Cels:  (57) 300 885 35 19

Distribuimos a todas las Ciudades del País y Fabricamos en Tiempo
Récord !!

_
Cancelar Suscripción / Cambiar Perfil: 
http://ymlp333.net/ugmbmjhbgsgjqsmjsgymwggesewhh
Desarrollado por YourMailingListProvider



RE: set weight bug?

2013-11-05 Thread Lukas Tribus
Hi Igor,


 Using newest snapshot, when I do

 echo set weight s1/p1 100| socat stdio /tmp/haproxy

 to a server already has weight 100, then fresh haproxy's stat page, it
 requires password, and it doesn't accept the right password set in
 stats auth until I reload the haproxy.

 I have a script to set servers weight, I found sometimes set weight to
 servers rapidly, like multi echo set weight s(*)/p(*) 100| socat
 stdio /tmp/haproxy, will crash haproxy daemon.


I can't reproduce neither problem. Can you post your configurations so we
can try to reproduce?

If you know a certain snapshot/release where this worked fine for you,
please tell, this will help reducing the regression range (if its a bug).



Regards,

Lukas 


Re: [PATCH v7 4/5] MEDIUM: Add state to struct check

2013-11-05 Thread Simon Horman
On Tue, Nov 05, 2013 at 06:04:35PM +0900, Simon Horman wrote:
 Add state to struct check. This is currently used to store one bit,
 CHK_RUNNING, which is set if a check is running and clear otherwise.
 This bit was previously SRV_CHK_RUNNING of the state element of struct
 server.
 
 This is in preparation for associating a agent check
 with a server which runs as well as the server's existing check.
 
 Signed-off-by: Simon Horman horms+rene...@verge.net.au

s/[+]renesas//

 
 ---
 
 v8
 * Use CHK_STATE_RUNNING instead of CHK_RUNNING
 
 Notes by Willy on v6:
 Please use CHK_STATE_RUNNING instead of CHK_RUNNING so that it's
 easier to know it is related to the state attribute of the check.
 
 Also, we have many HCHK_* and now CHK_* as well, so I wonder how/when
 this will become confusing.
 ---
  include/types/server.h | 5 -
  src/checks.c   | 8 
  src/dumpstats.c| 2 +-
  3 files changed, 9 insertions(+), 6 deletions(-)
 
 diff --git a/include/types/server.h b/include/types/server.h
 index 0e4c33e..bcde01b 100644
 --- a/include/types/server.h
 +++ b/include/types/server.h
 @@ -55,7 +55,6 @@
  /* unused: 0x0100, 0x0200, 0x0400 */
  #define SRV_SEND_PROXY   0x0800  /* this server talks the PROXY protocol 
 */
  #define SRV_NON_STICK0x1000  /* never add connections allocated to 
 this server to a stick table */
 -#define SRV_CHK_RUNNING 0x2000  /* a check is currently running on this 
 server */
  
  /* function which act on servers need to return various errors */
  #define SRV_STATUS_OK   0   /* everything is OK. */
 @@ -70,6 +69,9 @@
  #define SRV_CHK_PASSED  0x0002   /* server check succeeded unless FAILED is 
 also set */
  #define SRV_CHK_DISABLE 0x0004   /* server returned a disable code */
  
 +/* check flags */
 +#define CHK_STATE_RUNNING0x0001  /* this check is currently running */
 +
  /* various constants */
  #define SRV_UWGHT_RANGE 256
  #define SRV_UWGHT_MAX   (SRV_UWGHT_RANGE)
 @@ -117,6 +119,7 @@ struct check {
   int send_proxy; /* send a PROXY protocol header 
 with checks */
   int inter, fastinter, downinter;/* checks: time in milliseconds 
 */
   int result; /* health-check result : 
 SRV_CHK_* */
 + int state;  /* health-check result : CHK_* 
 */
   int type;   /* Check type, one of 
 PR_O2_*_CHK */
   struct server *server;  /* back-pointer to server */
  };
 diff --git a/src/checks.c b/src/checks.c
 index a2ff237..c37afbd 100644
 --- a/src/checks.c
 +++ b/src/checks.c
 @@ -1315,7 +1315,7 @@ static struct task *process_chk(struct task *t)
   int ret;
   int expired = tick_is_expired(t-expire, now_ms);
  
 - if (!(s-state  SRV_CHK_RUNNING)) {
 + if (!(check-state  CHK_STATE_RUNNING)) {
   /* no check currently running */
   if (!expired) /* woke up too early */
   return t;
 @@ -1329,7 +1329,7 @@ static struct task *process_chk(struct task *t)
   /* we'll initiate a new check */
   set_server_check_status(check, HCHK_STATUS_START, NULL);
  
 - s-state |= SRV_CHK_RUNNING;
 + check-state |= CHK_STATE_RUNNING;
   check-bi-p = check-bi-data;
   check-bi-i = 0;
   check-bo-p = check-bo-data;
 @@ -1420,7 +1420,7 @@ static struct task *process_chk(struct task *t)
  
   /* here, we have seen a synchronous error, no fd was allocated 
 */
  
 - s-state = ~SRV_CHK_RUNNING;
 + check-state = ~CHK_STATE_RUNNING;
   if (s-health  s-rise) {
   s-health--; /* still good */
   s-counters.failed_checks++;
 @@ -1516,7 +1516,7 @@ static struct task *process_chk(struct task *t)
   set_server_up(check);
   }
   }
 - s-state = ~SRV_CHK_RUNNING;
 + check-state = ~CHK_STATE_RUNNING;
  
   rv = 0;
   if (global.spread_checks  0) {
 diff --git a/src/dumpstats.c b/src/dumpstats.c
 index 9a422c1..be0aac6 100644
 --- a/src/dumpstats.c
 +++ b/src/dumpstats.c
 @@ -2273,7 +2273,7 @@ static int stats_dump_sv_stats(struct stream_interface 
 *si, struct proxy *px, in
   if (sv-state  SRV_CHECKED) {
   chunk_appendf(trash,
 /tdtd class=acu %s%s,
 -   (sv-state  SRV_CHK_RUNNING) ? *  : ,
 +   (sv-check.state  CHK_STATE_RUNNING) ? 
 *  : ,
 get_check_status_info(sv-check.status));
  
   if (sv-check.status = HCHK_STATUS_L57DATA)
 -- 
 1.8.4
 



Re: set weight bug?

2013-11-05 Thread Igor
Here is my config http://pastie.org/private/wf0dv30krqpasgmhtdnahw
(Deleted some servers and two backends for clear config)

I used script to handle servers weight since haproxy-ss-20131031, so I
never tried previous versions.

Bests,
-Igor


On Wed, Nov 6, 2013 at 5:55 AM, Lukas Tribus luky...@hotmail.com wrote:
 Hi Igor,


 Using newest snapshot, when I do

 echo set weight s1/p1 100| socat stdio /tmp/haproxy

 to a server already has weight 100, then fresh haproxy's stat page, it
 requires password, and it doesn't accept the right password set in
 stats auth until I reload the haproxy.

 I have a script to set servers weight, I found sometimes set weight to
 servers rapidly, like multi echo set weight s(*)/p(*) 100| socat
 stdio /tmp/haproxy, will crash haproxy daemon.


 I can't reproduce neither problem. Can you post your configurations so we
 can try to reproduce?

 If you know a certain snapshot/release where this worked fine for you,
 please tell, this will help reducing the regression range (if its a bug).



 Regards,

 Lukas