Re: Rewrite image path based on HTTP_REQUEST

[Aleksandar Lazic]

On 17/05/2018, Lotic Lists wrote:
> Hi experts
> 
> How can I rewrite a image path based on URL?
> 
> Example, users request the url www.example.com/images/logo.png, haproxy just
> balance to backend servers normally.
> 
> Now users request www.newdomain.com, I need rewrite URI to
> /images/new-logo.png

Well what have you already tried?

I would try, untested.

https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4.2-reqrep

acl old_dom hdr(host) -i www.example.com
acl old_path path_beg -i /images/logo.png

reqrep "^([^ :]*) /images/logo.png /images/new-logo.png" if old_dom && old_path
reqirep "^Host: www.example.com   "Host: www.newdomain.com" if old_dom && 
old_path

> Tks.
> 
> Marcelo

Regards
Aleks

Re: 1.8.8 & 1.9dev, lua, xref_get_peer_and_lock hang / 100% cpu usage after restarting haproxy a few times

[PiBa-NL]

Hi Thierry,

Op 18-5-2018 om 20:00 schreef Thierry FOURNIER:

Hi Pieter,

Could you test the attached patch ? It seems to fix the problem, but I
have some doubts about the reliability of the patch.

Thierry
The crash seems 'fixed' indeed.. but the lua scipt below now takes 
5seconds instead of 150ms.


Regards,
PiBa-NL (Pieter)

con = core.tcp()
con:connect_ssl("216.58.212.132",443) --google: 216.58.212.132
request = [[GET / HTTP/1.0

]]
con:send(request)
res = con:receive("*a")
con:close()

Re: [ANNOUNCE] haproxy-1.8.9

[Aleksandar Lazic]

Hi.

The image is also updated.

https://hub.docker.com/r/me2digital/haproxy18/

Regards
Aleks


 Ursprüngliche Nachricht 
Von: William Lallemand 
Gesendet: 18. Mai 2018 16:02:23 MESZ
An: haproxy@formilux.org
Betreff: [ANNOUNCE] haproxy-1.8.9

Hi,

HAProxy 1.8.9 was released on 2018/05/18. It added 20 new commits after version
1.8.8.

This version fixes a crash which can occurs when haproxy is built using certain
version of GCC (<= 3.x or >= 5.x). It also fixes the way fd were shared by
mutiple threads which was leading to missed event or spurious wakeups.

This version also implements the missing support for chunked encoded uploads
when using the h2 frontend, which was a problem for some applications.

Important for the users of the SPOP protocol: Thierry found an inconsistency in
the SPOP reference implementation and haproxy. Indeed, the SPOP flags are
directly memcpy() instead of encoded in network order which means that on
little endian architecture, the bytes are not in the order specified by the
documentation. This will be fixed in the next version of the 1.8 branch. In
the meantime you are encouraged to fix your SPOP agents if necessary. The next
update might break them if you don't. See commits c4dcaff, 48d02d0, 633f3bf in
the master.


Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Sources  : http://www.haproxy.org/download/1.8/src/
   Git repository   : http://git.haproxy.org/git/haproxy-1.8.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git
   Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Aurélien Nephtali (1):
  BUG/MINOR: pattern: Add a missing HA_SPIN_INIT() in pat_ref_newid()

Christopher Faulet (3):
  BUG/MINOR: lua/threads: Make lua's tasks sticky to the current thread
  BUG/MINOR: checks: Fix check->health computation for flapping servers
  BUG/MEDIUM: threads: Fix the sync point for more than 32 threads

Dragan Dosen (1):
  BUG/MINOR: map: correctly track reference to the last ref_elt being dumped

Olivier Houchard (2):
  BUG/MEDIUM: task: Don't free a task that is about to be run.
  BUG/MEDIUM: pollers: Use a global list for fd shared between threads.

Patrick Hemmer (1):
  DOC/MINOR: clean up LUA documentation re: servers & array/table.

PiBa-NL (2):
  BUG/MINOR: lua: Put tasks to sleep when waiting for data
  BUG/MINOR: lua: schedule socket task upon lua connect()

Rian McGuire (1):
  BUG/MINOR: log: t_idle (%Ti) is not set for some requests

Thierry FOURNIER (1):
  BUG/MINOR: spoe: Mistake in error message about SPOE configuration

Tim Duesterhus (2):
  BUG/MAJOR: channel: Fix crash when trying to read from a closed socket
  BUG/MEDIUM: lua: Fix segmentation fault if a Lua task exits

Willy Tarreau (6):
  MINOR: h2: detect presence of CONNECT and/or content-length
  BUG/MEDIUM: h2: implement missing support for chunked encoded uploads
  BUG/MINOR: config: disable http-reuse on TCP proxies
  BUG/MINOR: lua: ensure large proxy IDs can be represented
  BUG/MEDIUM: http: don't always abort transfers on CF_SHUTR
  BUG/MEDIUM: ssl: properly protect SSL cert generation

Re: 1.8.8 & 1.9dev, lua, xref_get_peer_and_lock hang / 100% cpu usage after restarting haproxy a few times

[Thierry FOURNIER]

Hi Pieter,

Could you test the attached patch ? It seems to fix the problem, but I
have some doubts about the reliability of the patch.

Thierry




On Fri, 11 May 2018 20:13:25 +0200
PiBa-NL  wrote:

> Hi Thierry,
> 
> Okay found a simple reproduction with tcploop with a 6 second delay in 
> there and a short sleep before calling kqueue.
> 
> ./tcploop 81 L W N20 A R S:"response1\r\n" R P6000 S:"response2\r\n" R [ 
> F K ]
> 
>   gettimeofday(_poll, NULL);
> +    usleep(100);
>   status = kevent(kqueue_fd[tid], // int kq
> 
> Together with the attached config the issue is reproduced every time the 
> /myapplet url is requested.
> 
> Output as below:
> :stats.clihdr[0007:]: Accept-Language: 
> nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
> [info] 130/195936 (76770) : Wait for it..
> [info] 130/195937 (76770) : Wait response 2..
>    xref_get_peer_and_lock xref->peer == 1
> 
> Hope this helps to come up with a solution..
> 
> Thanks in advance,
> PiBa-NL (Pieter)
> 
> Op 9-5-2018 om 19:47 schreef PiBa-NL:
> > Hi Thierry,
> >
> > Op 9-5-2018 om 18:30 schreef Thierry Fournier:
> >> It seems a dead lock, but you observe a loop. 
> > Effectively it is a deadlock, it keeps looping over these few lines of 
> > code below from xref.h 
> > ..
> >  
> > The XCHG just swaps the 2 values (both are '1') and continues on, then 
> > the local==BUSY check is true it loops and swaps 1 and 1 again, and 
> > the circle continues..
> >
> > Thanks for looking into it :) Ill try and get 'simpler' reproduction 
> > with some well placed sleep() as you suggest.
> > Regards,
> > PiBa-NL
> >
> > http://git.haproxy.org/?p=haproxy.git;a=blob;f=include/common/xref.h;h=6dfa7b62758dfaebe12d25f66aaa858dc873a060;hb=29d698040d6bb56b29c036aeba05f0d52d8ce94b
> >  
> >
> 
>From ea4ad78120759351f203797ca263b8247edd88a5 Mon Sep 17 00:00:00 2001
From: Thierry FOURNIER 
Date: Fri, 18 May 2018 16:22:20 +0200
Subject: [PATCH 1/2] BUG/MEDIUM: lua: Dead-lock with socket

In some cases, ... fill later
---
 src/hlua.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/src/hlua.c b/src/hlua.c
index 8cc305138..15a5397e1 100644
--- a/src/hlua.c
+++ b/src/hlua.c
@@ -1781,7 +1781,6 @@ __LJMP static int hlua_socket_receive_yield(struct lua_State *L, int status, lua
 	co_skip(oc, len + skip_at_end);
 
 	/* Don't wait anything. */
-	stream_int_notify(>si[0]);
 	stream_int_update_applet(>si[0]);
 
 	/* If the pattern reclaim to read all the data
@@ -2004,7 +2003,6 @@ static int hlua_socket_write_yield(struct lua_State *L,int status, lua_KContext
 	}
 
 	/* update buffers. */
-	stream_int_notify(>si[0]);
 	stream_int_update_applet(>si[0]);
 
 	s->req.rex = TICK_ETERNITY;
-- 
2.16.3

[RFC PATCH] MINOR: ssl: set SSL_OP_PRIORITIZE_CHACHA

[Lukas Tribus]

Sets OpenSSL 1.1.1's SSL_OP_PRIORITIZE_CHACHA unconditionally, as per [1]:

When SSL_OP_CIPHER_SERVER_PREFERENCE is set, temporarily reprioritize
ChaCha20-Poly1305 ciphers to the top of the server cipher list if a
ChaCha20-Poly1305 cipher is at the top of the client cipher list. This
helps those clients (e.g. mobile) use ChaCha20-Poly1305 if that cipher
is anywhere in the server cipher list; but still allows other clients to
use AES and other ciphers. Requires SSL_OP_CIPHER_SERVER_PREFERENCE.

[1] https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_clear_options.html
---

RFC because we need to confirm we want to set this option unconditionally.

We could also add another configuration option, however I don't see the
real benefit honestly:

Symmetric crypto performance of ChaCha20-Poly1305 should not be a real
world concern at all as it performs very good. If it is a concern, it
should not be in the allowed cipher list in the first place.

I'm not aware of any clients not supporting a recent AES-GCM (AEAD)
cipher while they do support ChaCha20-Poly1305, it certainly doesn't
make much sense.

That is why I'm proposing to set this option unconditionally.

---
 doc/configuration.txt | 3 +++
 src/ssl_sock.c| 4 
 2 files changed, 7 insertions(+)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index cbea330..d59a446 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -10961,6 +10961,9 @@ prefer-client-ciphers
   Use the client's preference when selecting the cipher suite, by default
   the server's preference is enforced. This option is also available on
   global statement "ssl-default-bind-options".
+  Note that with OpenSSL >= 1.1.1 ChaCha20-Poly1305 is reprioritized anyway
+  (without setting this option), if a ChaCha20-Poly1305 cipher is at the top of
+  the client cipher list.
 
 process [/]
   This restricts the list of processes and/or threads on which this listener is
diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index 7a602ad..5a003dc 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -1957,6 +1957,9 @@ ssl_sock_generate_certificate_from_conn(struct bind_conf 
*bind_conf, SSL *ssl)
 #ifndef SSL_MODE_SMALL_BUFFERS  /* needs 
small_records.patch */
 #define SSL_MODE_SMALL_BUFFERS 0
 #endif
+#ifndef SSL_OP_PRIORITIZE_CHACHA/* needs OpenSSL >= 
1.1.1 */
+#define SSL_OP_PRIORITIZE_CHACHA 0
+#endif
 
 #if (OPENSSL_VERSION_NUMBER < 0x101fL)
 typedef enum { SET_CLIENT, SET_SERVER } set_context_func;
@@ -3711,6 +3714,7 @@ ssl_sock_initial_ctx(struct bind_conf *bind_conf)
SSL_OP_SINGLE_DH_USE |
SSL_OP_SINGLE_ECDH_USE |
SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION |
+   SSL_OP_PRIORITIZE_CHACHA |
SSL_OP_CIPHER_SERVER_PREFERENCE;
long mode =
SSL_MODE_ENABLE_PARTIAL_WRITE |
-- 
2.7.4

Re: Show: h-app-proxy – Application server inside haproxy

[Thierry FOURNIER]

On Fri, 18 May 2018 16:30:46 +0200
Tim Düsterhus  wrote:

> Thierry,
> 
> Am 18.05.2018 um 12:47 schrieb Thierry FOURNIER:
> > Hi,
> > 
> > This is a great usage of Lua. I add a link on my own blog:
> > 
> >http://blog.arpalert.org/p/interesting-links.html
> 
> Thank you! Don't forget to fix the typo in your redis connection pool I
> mentioned in my mail.


Thanks, I missed it. Done.

> 
> Best regards
> Tim Düsterhus

Re: Show: h-app-proxy – Application server inside haproxy

[Tim Düsterhus]

Thierry,

Am 18.05.2018 um 12:47 schrieb Thierry FOURNIER:
> Hi,
> 
> This is a great usage of Lua. I add a link on my own blog:
> 
>http://blog.arpalert.org/p/interesting-links.html

Thank you! Don't forget to fix the typo in your redis connection pool I
mentioned in my mail.

Best regards
Tim Düsterhus

[ANNOUNCE] haproxy-1.8.9

[William Lallemand]

Hi,

HAProxy 1.8.9 was released on 2018/05/18. It added 20 new commits after version
1.8.8.

This version fixes a crash which can occurs when haproxy is built using certain
version of GCC (<= 3.x or >= 5.x). It also fixes the way fd were shared by
mutiple threads which was leading to missed event or spurious wakeups.

This version also implements the missing support for chunked encoded uploads
when using the h2 frontend, which was a problem for some applications.

Important for the users of the SPOP protocol: Thierry found an inconsistency in
the SPOP reference implementation and haproxy. Indeed, the SPOP flags are
directly memcpy() instead of encoded in network order which means that on
little endian architecture, the bytes are not in the order specified by the
documentation. This will be fixed in the next version of the 1.8 branch. In
the meantime you are encouraged to fix your SPOP agents if necessary. The next
update might break them if you don't. See commits c4dcaff, 48d02d0, 633f3bf in
the master.


Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Sources  : http://www.haproxy.org/download/1.8/src/
   Git repository   : http://git.haproxy.org/git/haproxy-1.8.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git
   Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Aurélien Nephtali (1):
  BUG/MINOR: pattern: Add a missing HA_SPIN_INIT() in pat_ref_newid()

Christopher Faulet (3):
  BUG/MINOR: lua/threads: Make lua's tasks sticky to the current thread
  BUG/MINOR: checks: Fix check->health computation for flapping servers
  BUG/MEDIUM: threads: Fix the sync point for more than 32 threads

Dragan Dosen (1):
  BUG/MINOR: map: correctly track reference to the last ref_elt being dumped

Olivier Houchard (2):
  BUG/MEDIUM: task: Don't free a task that is about to be run.
  BUG/MEDIUM: pollers: Use a global list for fd shared between threads.

Patrick Hemmer (1):
  DOC/MINOR: clean up LUA documentation re: servers & array/table.

PiBa-NL (2):
  BUG/MINOR: lua: Put tasks to sleep when waiting for data
  BUG/MINOR: lua: schedule socket task upon lua connect()

Rian McGuire (1):
  BUG/MINOR: log: t_idle (%Ti) is not set for some requests

Thierry FOURNIER (1):
  BUG/MINOR: spoe: Mistake in error message about SPOE configuration

Tim Duesterhus (2):
  BUG/MAJOR: channel: Fix crash when trying to read from a closed socket
  BUG/MEDIUM: lua: Fix segmentation fault if a Lua task exits

Willy Tarreau (6):
  MINOR: h2: detect presence of CONNECT and/or content-length
  BUG/MEDIUM: h2: implement missing support for chunked encoded uploads
  BUG/MINOR: config: disable http-reuse on TCP proxies
  BUG/MINOR: lua: ensure large proxy IDs can be represented
  BUG/MEDIUM: http: don't always abort transfers on CF_SHUTR
  BUG/MEDIUM: ssl: properly protect SSL cert generation

-- 
William Lallemand

Re: SPOE patchs

[Willy Tarreau]

On Fri, May 18, 2018 at 02:58:44PM +0200, Christopher Faulet wrote:
> And because this is Friday and the sun shines, 2 more patches to do some
> cleanup in the SPOE.

Because it's Friday and the sun shines, they were merged.

thanks,
Willy

Re: SPOE patchs

[Christopher Faulet]

Le 18/05/2018 à 14:53, Christopher Faulet a écrit :

Le 18/05/2018 à 12:38, Thierry FOURNIER a écrit :

Hi,

In attachment two patches for SPOE.
The first fix an error message, and the second fix a mistake in the protocol.



Thanks Thierry for these patches. Here are fixes for the SPOAs
modsecurity and mod_defender.



And because this is Friday and the sun shines, 2 more patches to do some 
cleanup in the SPOE.


Thanks
--
Christopher Faulet
>From 78bb0889c45e5ffecfd693787823f32b9a5e90cf Mon Sep 17 00:00:00 2001
From: Christopher Faulet 
Date: Fri, 6 Apr 2018 11:34:12 +0200
Subject: [PATCH 1/2] CLEANUP: spoe: Remove unused variables the agent
 structure

applets_act and applets_idle were used for debugging purpose. Now, these values
are part of the agent's counters.
---
 include/types/spoe.h |  4 
 src/flt_spoe.c   | 12 ++--
 2 files changed, 2 insertions(+), 14 deletions(-)

diff --git a/include/types/spoe.h b/include/types/spoe.h
index c01a03ef..2f13d375 100644
--- a/include/types/spoe.h
+++ b/include/types/spoe.h
@@ -265,10 +265,6 @@ struct spoe_agent {
 	/* running info */
 	struct {
 		unsigned intframe_size; /* current maximum frame size, only used to encode messages */
-#if defined(DEBUG_SPOE) || defined(DEBUG_FULL)
-		unsigned intapplets_act;/* # of applets alive at a time */
-		unsigned intapplets_idle;   /* # of applets in the state SPOE_APPCTX_ST_IDLE */
-#endif
 		unsigned intprocessing;
 		struct freq_ctr processing_per_sec;
 
diff --git a/src/flt_spoe.c b/src/flt_spoe.c
index 4e27c63b..f96a94f3 100644
--- a/src/flt_spoe.c
+++ b/src/flt_spoe.c
@@ -1239,7 +1239,6 @@ spoe_release_appctx(struct appctx *appctx)
 		__FUNCTION__, appctx);
 
 	/* Remove applet from the list of running applets */
-	SPOE_DEBUG_STMT(agent->rt[tid].applets_act--);
 	HA_ATOMIC_SUB(>counters.applets, 1);
 	HA_SPIN_LOCK(SPOE_APPLET_LOCK, >rt[tid].lock);
 	if (!LIST_ISEMPTY(_appctx->list)) {
@@ -1252,7 +1251,6 @@ spoe_release_appctx(struct appctx *appctx)
 	if (appctx->st0 != SPOE_APPCTX_ST_END) {
 		if (appctx->st0 == SPOE_APPCTX_ST_IDLE) {
 			eb32_delete(_appctx->node);
-			SPOE_DEBUG_STMT(agent->rt[tid].applets_idle--);
 			HA_ATOMIC_SUB(>counters.idles, 1);
 		}
 
@@ -1437,7 +1435,6 @@ spoe_handle_connecting_appctx(struct appctx *appctx)
 		default:
 			/* HELLO handshake is finished, set the idle timeout and
 			 * add the applet in the list of running applets. */
-			SPOE_DEBUG_STMT(agent->rt[tid].applets_idle++);
 			HA_ATOMIC_ADD(>counters.idles, 1);
 			appctx->st0 = SPOE_APPCTX_ST_IDLE;
 			SPOE_APPCTX(appctx)->node.key = 0;
@@ -1725,7 +1722,6 @@ spoe_handle_processing_appctx(struct appctx *appctx)
 	}
 
 	if (appctx->st0 == SPOE_APPCTX_ST_PROCESSING && SPOE_APPCTX(appctx)->cur_fpa < agent->max_fpa) {
-		SPOE_DEBUG_STMT(agent->rt[tid].applets_idle++);
 		HA_ATOMIC_ADD(>counters.idles, 1);
 		appctx->st0 = SPOE_APPCTX_ST_IDLE;
 		eb32_insert(>rt[tid].idle_applets, _APPCTX(appctx)->node);
@@ -1889,7 +1885,6 @@ spoe_handle_appctx(struct appctx *appctx)
 			goto switchstate;
 
 		case SPOE_APPCTX_ST_IDLE:
-			SPOE_DEBUG_STMT(agent->rt[tid].applets_idle--);
 			HA_ATOMIC_SUB(>counters.idles, 1);
 			eb32_delete(_APPCTX(appctx)->node);
 			if (stopping &&
@@ -2007,7 +2002,6 @@ spoe_create_appctx(struct spoe_config *conf)
 	HA_SPIN_LOCK(SPOE_APPLET_LOCK, >agent->rt[tid].lock);
 	LIST_ADDQ(>agent->rt[tid].applets, _APPCTX(appctx)->list);
 	HA_SPIN_UNLOCK(SPOE_APPLET_LOCK, >agent->rt[tid].lock);
-	SPOE_DEBUG_STMT(conf->agent->rt[tid].applets_act++);
 	HA_ATOMIC_ADD(>agent->counters.applets, 1);
 
 	task_wakeup(SPOE_APPCTX(appctx)->task, TASK_WOKEN_INIT);
@@ -2099,9 +2093,9 @@ spoe_queue_context(struct spoe_context *ctx)
 
 	SPOE_PRINTF(stderr, "%d.%06d [SPOE/%-15s] %s: stream=%p"
 		" - Add stream in sending queue"
-		" - applets_act=%u - applets_idle=%u - processing=%u\n",
+		" - applets=%u - idles=%u - processing=%u\n",
 		(int)now.tv_sec, (int)now.tv_usec, agent->id, __FUNCTION__,
-		ctx->strm, agent->rt[tid].applets_act, agent->rt[tid].applets_idle,
+		ctx->strm, agent->counters.applets, agent->counters.idles,
 		agent->rt[tid].processing);
 
 	/* Finally try to wakeup an IDLE applet. */
@@ -3358,8 +3352,6 @@ cfg_parse_spoe_agent(const char *file, int linenum, char **args, int kwm)
 		}
 		for (i = 0; i < global.nbthread; ++i) {
 			curagent->rt[i].frame_size   = curagent->max_frame_size;
-			SPOE_DEBUG_STMT(curagent->rt[i].applets_act  = 0);
-			SPOE_DEBUG_STMT(curagent->rt[i].applets_idle = 0);
 			curagent->rt[i].processing   = 0;
 			LIST_INIT(>rt[i].applets);
 			LIST_INIT(>rt[i].sending_queue);
-- 
2.14.3

>From 89b43747600962ed2225b3ba69549fa4156ccfcd Mon Sep 17 00:00:00 2001
From: Christopher Faulet 
Date: Thu, 26 Apr 2018 14:25:43 +0200
Subject: [PATCH 2/2] DOC: spoe: fix a typo

---
 doc/SPOE.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/SPOE.txt b/doc/SPOE.txt

Re: SPOE patchs

[Christopher Faulet]

Le 18/05/2018 à 12:38, Thierry FOURNIER a écrit :

Hi,

In attachment two patches for SPOE.
The first fix an error message, and the second fix a mistake in the protocol.



Thanks Thierry for these patches. Here are fixes for the SPOAs 
modsecurity and mod_defender.


--
Christopher Faulet
>From 444e432df47b77a635296d85fb6673f60b92bef7 Mon Sep 17 00:00:00 2001
From: Christopher Faulet 
Date: Fri, 18 May 2018 14:38:56 +0200
Subject: [PATCH 1/2] BUG/MEDIUM: contrib/mod_defender: Use network order to
 encode/decode flags

A recent fix on the SPOE revealed a mismatch between the SPOE specification and
the mod_defender implementation on the way flags are encoded or decoded. They
must be exchanged using the network bytes order and not the host one.

Be careful though, this patch breaks the compatiblity with HAProxy SPOE before
commit c4dcaff3 ("BUG/MEDIUM: spoe: Flags are not encoded in network order").
---
 contrib/mod_defender/spoa.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/contrib/mod_defender/spoa.c b/contrib/mod_defender/spoa.c
index c4e15bb4..1191260a 100644
--- a/contrib/mod_defender/spoa.c
+++ b/contrib/mod_defender/spoa.c
@@ -460,6 +460,7 @@ handle_hahello(struct spoe_frame *frame)
 
 	/* Retrieve flags */
 	memcpy((char *)&(frame->flags), p, 4);
+	frame->flags = ntohl(frame->flags);
 	p += 4;
 
 	/* Fragmentation is not supported for HELLO frame */
@@ -567,6 +568,7 @@ handle_hadiscon(struct spoe_frame *frame)
 
 	/* Retrieve flags */
 	memcpy((char *)&(frame->flags), p, 4);
+	frame->flags = ntohl(frame->flags);
 	p += 4;
 
 	/* Fragmentation is not supported for DISCONNECT frame */
@@ -648,6 +650,7 @@ handle_hanotify(struct spoe_frame *frame)
 
 	/* Retrieve flags */
 	memcpy((char *)&(frame->flags), p, 4);
+	frame->flags = ntohl(frame->flags);
 	p += 4;
 
 	/* Fragmentation is not supported */
@@ -710,6 +713,7 @@ handle_hafrag(struct spoe_frame *frame)
 
 	/* Retrieve flags */
 	memcpy((char *)&(frame->flags), p, 4);
+	frame->flags = ntohl(frame->flags);
 	p+= 4;
 
 	/* Read the stream-id and frame-id */
@@ -772,6 +776,7 @@ prepare_agenthello(struct spoe_frame *frame)
 	*p++ = SPOE_FRM_T_AGENT_HELLO;
 
 	/* Set flags */
+	flags = htonl(flags);
 	memcpy(p, (char *), 4);
 	p += 4;
 
@@ -853,6 +858,7 @@ prepare_agentdicon(struct spoe_frame *frame)
 	*p++ = SPOE_FRM_T_AGENT_DISCON;
 
 	/* Set flags */
+	flags = htonl(flags);
 	memcpy(p, (char *), 4);
 	p += 4;
 
@@ -900,6 +906,7 @@ prepare_agentack(struct spoe_frame *frame)
 	*p++ = SPOE_FRM_T_AGENT_ACK;
 
 	/* Set flags */
+	flags = htonl(flags);
 	memcpy(p, (char *), 4);
 	p += 4;
 
-- 
2.14.3

>From d424c222e690cf93075ba2f94911cb557218b521 Mon Sep 17 00:00:00 2001
From: Christopher Faulet 
Date: Fri, 18 May 2018 14:46:32 +0200
Subject: [PATCH 2/2] BUG/MEDIUM: contrib/modsecurity: Use network order to
 encode/decode flags

A recent fix on the SPOE revealed a mismatch between the SPOE specification and
the modsecurity implementation on the way flags are encoded or decoded. They
must be exchanged using the network bytes order and not the host one.

Be careful though, this patch breaks the compatiblity with HAProxy SPOE before
commit c4dcaff3 ("BUG/MEDIUM: spoe: Flags are not encoded in network order").
---
 contrib/modsecurity/spoa.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/contrib/modsecurity/spoa.c b/contrib/modsecurity/spoa.c
index 506ff824..ab3e8b2b 100644
--- a/contrib/modsecurity/spoa.c
+++ b/contrib/modsecurity/spoa.c
@@ -465,6 +465,7 @@ handle_hahello(struct spoe_frame *frame)
 
 	/* Retrieve flags */
 	memcpy((char *)&(frame->flags), p, 4);
+	frame->flags = ntohl(frame->flags);
 	p += 4;
 
 	/* Fragmentation is not supported for HELLO frame */
@@ -572,6 +573,7 @@ handle_hadiscon(struct spoe_frame *frame)
 
 	/* Retrieve flags */
 	memcpy((char *)&(frame->flags), p, 4);
+	frame->flags = ntohl(frame->flags);
 	p += 4;
 
 	/* Fragmentation is not supported for DISCONNECT frame */
@@ -653,6 +655,7 @@ handle_hanotify(struct spoe_frame *frame)
 
 	/* Retrieve flags */
 	memcpy((char *)&(frame->flags), p, 4);
+	frame->flags = ntohl(frame->flags);
 	p += 4;
 
 	/* Fragmentation is not supported */
@@ -715,6 +718,7 @@ handle_hafrag(struct spoe_frame *frame)
 
 	/* Retrieve flags */
 	memcpy((char *)&(frame->flags), p, 4);
+	frame->flags = ntohl(frame->flags);
 	p+= 4;
 
 	/* Read the stream-id and frame-id */
@@ -777,6 +781,7 @@ prepare_agenthello(struct spoe_frame *frame)
 	*p++ = SPOE_FRM_T_AGENT_HELLO;
 
 	/* Set flags */
+	flags = htonl(flags);
 	memcpy(p, (char *), 4);
 	p += 4;
 
@@ -858,6 +863,7 @@ prepare_agentdicon(struct spoe_frame *frame)
 	*p++ = SPOE_FRM_T_AGENT_DISCON;
 
 	/* Set flags */
+	flags = htonl(flags);
 	memcpy(p, (char *), 4);
 	p += 4;
 
@@ -905,6 +911,7 @@ prepare_agentack(struct spoe_frame *frame)
 	*p++ = SPOE_FRM_T_AGENT_ACK;
 
 	/* Set flags */
+	flags = htonl(flags);
 	memcpy(p, (char *), 4);
 	p += 4;
 
-- 
2.14.3

Re: Modifying TCP payload via LUA action & yield issues

[thierry . fournier]

Hi,

I can reproduce the bug, forgot my previous workaround proposition and just add 
an inspect delay:

   
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-tcp-request%20inspect-delay
   tcp-request inspect-delay 10s

If some actions can't be executed because it have no data, haproxy wait
XX seconds for more data. In this case, the yield is allowed during at
mean 10s.

Thierry



On Fri, 18 May 2018 14:22:20 +0200
thierry.fourn...@arpalert.org wrote:

> Could you change your configuration to test a workaround ?
> 
> After the line:
> 
>tcp-request content lua.enrich_query if !has_client_version
> 
> Add a fake action:
> 
>tcp-request set-var(req.wa) int(0)
> 
> Tell me if this work around fix the issue.
> 
> Thierry
> 
> 
> On Tue, 15 May 2018 01:11:46 +
> Grant Byers  wrote:
> 
> > Hi all,
> > 
> > Is it at all feasible to modify a TCP payload via a custom LUA action
> > in a stable manner? ie, something like this;
> > 
> > 
> > function enrich_query(txn)
> > --- Duplicate request. :set() will replace the request, if possible
> > local req = txn.req:dup()
> > 
> > -- if error or client closes connection, quit
> > if req == nil then return end
> > 
> > --- Add an identifier for haproxy and include client's IP address.
> > if txn.req:set("-Vha1.8," .. txn.f:src() .. " " .. req) == -1 then
> > txn.Error(txn, "Failed to enrich query")
> > end
> > end
> > 
> > core.register_action('enrich_query', {'tcp-req'}, enrich_query)
> > 
> > 
> > Then called as an action in the frontend;
> > 
> > # Don't enrich if query already contains client identification
> > acl has_client_version req.payload(0,0) -m sub -- -V --client
> > 
> > # Enrich our query
> > tcp-request content lua.enrich_query if !has_client_version
> > WAIT_END
> > 
> > 
> > The reason I ask is that I randomly get yield errors, I assume during
> > the txn.req:set() ;
> > 
> > Lua function 'enrich_query': yield not allowed.
> > 
> > 
> > I have experimented with different values of tune.lua.forced-yield with
> > little success. Sometimes this occurs very infrequently, other times,
> > very frequently.
> > 
> > FYI - I have experimented with PROXY, but our backend application is
> > using a java API that doesn't have native PROXY support (although
> > there's been a patch pending for about 5 years). Have also experimented
> > with PROXY and Cloudflare's "mmproxy". That works well, but i've been
> > asked to use LUA to extend haproxy if possible.
> > 
> > 
> > Thanks,
> > Grant
> 

Re: Modifying TCP payload via LUA action & yield issues

[thierry . fournier]

Could you change your configuration to test a workaround ?

After the line:

   tcp-request content lua.enrich_query if !has_client_version

Add a fake action:

   tcp-request set-var(req.wa) int(0)

Tell me if this work around fix the issue.

Thierry


On Tue, 15 May 2018 01:11:46 +
Grant Byers  wrote:

> Hi all,
> 
> Is it at all feasible to modify a TCP payload via a custom LUA action
> in a stable manner? ie, something like this;
> 
> 
> function enrich_query(txn)
> --- Duplicate request. :set() will replace the request, if possible
> local req = txn.req:dup()
> 
> -- if error or client closes connection, quit
> if req == nil then return end
> 
> --- Add an identifier for haproxy and include client's IP address.
> if txn.req:set("-Vha1.8," .. txn.f:src() .. " " .. req) == -1 then
> txn.Error(txn, "Failed to enrich query")
> end
> end
> 
> core.register_action('enrich_query', {'tcp-req'}, enrich_query)
> 
> 
> Then called as an action in the frontend;
> 
> # Don't enrich if query already contains client identification
> acl has_client_version req.payload(0,0) -m sub -- -V --client
> 
> # Enrich our query
> tcp-request content lua.enrich_query if !has_client_version
> WAIT_END
> 
> 
> The reason I ask is that I randomly get yield errors, I assume during
> the txn.req:set() ;
> 
> Lua function 'enrich_query': yield not allowed.
> 
> 
> I have experimented with different values of tune.lua.forced-yield with
> little success. Sometimes this occurs very infrequently, other times,
> very frequently.
> 
> FYI - I have experimented with PROXY, but our backend application is
> using a java API that doesn't have native PROXY support (although
> there's been a patch pending for about 5 years). Have also experimented
> with PROXY and Cloudflare's "mmproxy". That works well, but i've been
> asked to use LUA to extend haproxy if possible.
> 
> 
> Thanks,
> Grant

Re: Modifying TCP payload via LUA action & yield issues

[Thierry FOURNIER]

Hi Grant,

It seems to be a bug. The action should allow yield.

Could you have a method for reproducing the bug ?

Thierry


On Tue, 15 May 2018 01:11:46 +
Grant Byers  wrote:

> Hi all,
> 
> Is it at all feasible to modify a TCP payload via a custom LUA action
> in a stable manner? ie, something like this;
> 
> 
> function enrich_query(txn)
> --- Duplicate request. :set() will replace the request, if possible
> local req = txn.req:dup()
> 
> -- if error or client closes connection, quit
> if req == nil then return end
> 
> --- Add an identifier for haproxy and include client's IP address.
> if txn.req:set("-Vha1.8," .. txn.f:src() .. " " .. req) == -1 then
> txn.Error(txn, "Failed to enrich query")
> end
> end
> 
> core.register_action('enrich_query', {'tcp-req'}, enrich_query)
> 
> 
> Then called as an action in the frontend;
> 
> # Don't enrich if query already contains client identification
> acl has_client_version req.payload(0,0) -m sub -- -V --client
> 
> # Enrich our query
> tcp-request content lua.enrich_query if !has_client_version
> WAIT_END
> 
> 
> The reason I ask is that I randomly get yield errors, I assume during
> the txn.req:set() ;
> 
> Lua function 'enrich_query': yield not allowed.
> 
> 
> I have experimented with different values of tune.lua.forced-yield with
> little success. Sometimes this occurs very infrequently, other times,
> very frequently.
> 
> FYI - I have experimented with PROXY, but our backend application is
> using a java API that doesn't have native PROXY support (although
> there's been a patch pending for about 5 years). Have also experimented
> with PROXY and Cloudflare's "mmproxy". That works well, but i've been
> asked to use LUA to extend haproxy if possible.
> 
> 
> Thanks,
> Grant

Re: SPOE patchs

[Willy Tarreau]

On Fri, May 18, 2018 at 12:38:07PM +0200, Thierry FOURNIER wrote:
> Hi,
> 
> In attachment two patches for SPOE.
> The first fix an error message, and the second fix a mistake in the protocol.

Thank you, now applied,

Willy

Re: [PATCH] BUG/MINOR: lua: Socket.send threw runtime error: 'close' needs 1 arguments.

[Willy Tarreau]

On Fri, May 18, 2018 at 12:46:07PM +0200, thierry.fourn...@arpalert.org wrote:
> Hi Sada,
> 
> Thanks for the patch,
> Willy, can you apply ?

Now done, thanks!

Willy

Re: Show: h-app-proxy – Application server inside haproxy

[Thierry FOURNIER]

Hi,

This is a great usage of Lua. I add a link on my own blog:

   http://blog.arpalert.org/p/interesting-links.html

Thierry


On Sat, 12 May 2018 11:23:31 +0200
Aleksandar Lazic  wrote:

> Hi Tim.
> 
> Am 11.05.2018 um 20:57 schrieb Tim Düsterhus:
> > Hi list,
> > 
> > I recently experimented with the Lua API to check out it's capabilities
> > and wanted to show off the results:
> > 
> > I implemented a very simple short URL service entirely in haproxy with
> > Redis as it's backend. No backend service needed :-)
> 
> Cool stuff ;-)
> Thanks for sharing.
> 
> > Thanks to Thierry for his Redis Connection Pool implementation:
> > http://blog.arpalert.org/2018/02/haproxy-lua-redis-connection-pool.html
> > 
> > Thierry, note that you made a small typo in your pool: r.release(conn)
> > in renew should read r:release(conn).
> > 
> > Blog post  : https://bl.duesterhus.eu/20180511/
> > GitHub : https://github.com/TimWolla/h-app-roxy
> > Live Demo  : https://bl.duesterhus.eu/20180511/demo/DWhxJf2Gpt
> > Hacker News: https://news.ycombinator.com/item?id=17049715
> >
> > Best regards
> > Tim Düsterhus
> > 
> > PS: Don't use this at home or at work even :-)
> 
> ;-)
> 
> Best regards
> Aleks
> 

Re: [PATCH] BUG/MINOR: lua: Socket.send threw runtime error: 'close' needs 1 arguments.

[thierry . fournier]

Hi Sada,

Thanks for the patch,
Willy, can you apply ?

BR,
Thierry


On Fri, 11 May 2018 11:48:18 -0700
sada  wrote:

> Function `hlua_socke_close` expected exactly one argument on the Lua stack.
> But when `hlua_socket_close` was called from `hlua_socket_write_yield`,
> Lua stack had 3 arguments. So `hlua_socket_close` threw the exception with
> message "'close' needs 1 arguments".
> 
> Introduced new helper function `hlua_socket_close_helper`, which removed the
> Lua stack argument count check and only checked if the first argument was
> a socket.
> 
> This fix should be backported to 1.8, 1.7 and 1.6.
> ---
>  src/hlua.c | 14 ++
>  1 file changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/src/hlua.c b/src/hlua.c
> index d07e8d67..8cc30513 100644
> --- a/src/hlua.c
> +++ b/src/hlua.c
> @@ -1629,14 +1629,12 @@ __LJMP static int hlua_socket_gc(lua_State *L)
>  /* The close function send shutdown signal and break the
>   * links between the stream and the object.
>   */
> -__LJMP static int hlua_socket_close(lua_State *L)
> +__LJMP static int hlua_socket_close_helper(lua_State *L)
>  {
>   struct hlua_socket *socket;
>   struct appctx *appctx;
>   struct xref *peer;
>  
> - MAY_LJMP(check_args(L, 1, "close"));
> -
>   socket = MAY_LJMP(hlua_checksocket(L, 1));
>  
>   /* Check if we run on the same thread than the xreator thread.
> @@ -1659,6 +1657,14 @@ __LJMP static int hlua_socket_close(lua_State *L)
>   return 0;
>  }
>  
> +/* The close function calls close_helper.
> + */
> +__LJMP static int hlua_socket_close(lua_State *L)
> +{
> + MAY_LJMP(check_args(L, 1, "close"));
> + return hlua_socket_close_helper(L);
> +}
> +
>  /* This Lua function assumes that the stack contain three parameters.
>   *  1 - USERDATA containing a struct socket
>   *  2 - INTEGER with values of the macro defined below
> @@ -1990,7 +1996,7 @@ static int hlua_socket_write_yield(struct lua_State 
> *L,int status, lua_KContext
>   if (len == -1)
>   s->req.flags |= CF_WAKE_WRITE;
>  
> - MAY_LJMP(hlua_socket_close(L));
> + MAY_LJMP(hlua_socket_close_helper(L));
>   lua_pop(L, 1);
>   lua_pushinteger(L, -1);
>   xref_unlock(>xref, peer);
> -- 
> 2.17.0
> 
> 
>From 7034eef0ed9afefde6a2ff94ce6acfa63caa1832 Mon Sep 17 00:00:00 2001
From: sada 
Date: Fri, 11 May 2018 11:48:18 -0700
Subject: [PATCH] BUG/MINOR: lua: Socket.send threw runtime error: 'close'
 needs 1 arguments.

Function `hlua_socke_close` expected exactly one argument on the Lua stack.
But when `hlua_socket_close` was called from `hlua_socket_write_yield`,
Lua stack had 3 arguments. So `hlua_socket_close` threw the exception with
message "'close' needs 1 arguments".

Introduced new helper function `hlua_socket_close_helper`, which removed the
Lua stack argument count check and only checked if the first argument was
a socket.

This fix should be backported to 1.8, 1.7 and 1.6.
---
 src/hlua.c | 14 ++
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/src/hlua.c b/src/hlua.c
index 3845a11bb..cb683fa19 100644
--- a/src/hlua.c
+++ b/src/hlua.c
@@ -1629,14 +1629,12 @@ __LJMP static int hlua_socket_gc(lua_State *L)
 /* The close function send shutdown signal and break the
  * links between the stream and the object.
  */
-__LJMP static int hlua_socket_close(lua_State *L)
+__LJMP static int hlua_socket_close_helper(lua_State *L)
 {
 	struct hlua_socket *socket;
 	struct appctx *appctx;
 	struct xref *peer;
 
-	MAY_LJMP(check_args(L, 1, "close"));
-
 	socket = MAY_LJMP(hlua_checksocket(L, 1));
 
 	/* Check if we run on the same thread than the xreator thread.
@@ -1659,6 +1657,14 @@ __LJMP static int hlua_socket_close(lua_State *L)
 	return 0;
 }
 
+/* The close function calls close_helper.
+ */
+__LJMP static int hlua_socket_close(lua_State *L)
+{
+	MAY_LJMP(check_args(L, 1, "close"));
+	return hlua_socket_close_helper(L);
+}
+
 /* This Lua function assumes that the stack contain three parameters.
  *  1 - USERDATA containing a struct socket
  *  2 - INTEGER with values of the macro defined below
@@ -1992,7 +1998,7 @@ static int hlua_socket_write_yield(struct lua_State *L,int status, lua_KContext
 		if (len == -1)
 			s->req.flags |= CF_WAKE_WRITE;
 
-		MAY_LJMP(hlua_socket_close(L));
+		MAY_LJMP(hlua_socket_close_helper(L));
 		lua_pop(L, 1);
 		lua_pushinteger(L, -1);
 		xref_unlock(>xref, peer);
-- 
2.16.3

SPOE patchs

[Thierry FOURNIER]

Hi,

In attachment two patches for SPOE.
The first fix an error message, and the second fix a mistake in the protocol.

BR,
Thierry
>From c0274215ed91e90e127bda790b785b698fb149e7 Mon Sep 17 00:00:00 2001
From: Thierry FOURNIER 
Date: Thu, 10 May 2018 16:41:26 +0200
Subject: [PATCH 1/3] BUG/MINOR: spoe: Mistake in error message about SPOE
 configuration

The announced accepted chars are "[a-zA-Z_-.]", but
the real accepted alphabet is "[a-zA-Z0-9_.]".

Numbers are supported and "-" is not supported.

This patch should be backported to 1.8 and 1.7
---
 src/flt_spoe.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/flt_spoe.c b/src/flt_spoe.c
index 6ac48fc7d..cc6c55e29 100644
--- a/src/flt_spoe.c
+++ b/src/flt_spoe.c
@@ -3535,7 +3535,7 @@ cfg_parse_spoe_agent(const char *file, int linenum, char **args, int kwm)
 			tmp = args[2];
 			while (*tmp) {
 if (!isalnum(*tmp) && *tmp != '_' && *tmp != '.') {
-	ha_alert("parsing [%s:%d]: '%s %s' only supports [a-zA-Z_-.] chars.\n",
+	ha_alert("parsing [%s:%d]: '%s %s' only supports [a-zA-Z0-9_.] chars.\n",
 		 file, linenum, args[0], args[1]);
 	err_code |= ERR_ALERT | ERR_FATAL;
 	goto out;
@@ -3569,7 +3569,7 @@ cfg_parse_spoe_agent(const char *file, int linenum, char **args, int kwm)
 			tmp = args[2];
 			while (*tmp) {
 if (!isalnum(*tmp) && *tmp != '_' && *tmp != '.') {
-	ha_alert("parsing [%s:%d]: '%s %s' only supports [a-zA-Z_-.] chars.\n",
+	ha_alert("parsing [%s:%d]: '%s %s' only supports [a-zA-Z0-9_.] chars.\n",
 		 file, linenum, args[0], args[1]);
 	err_code |= ERR_ALERT | ERR_FATAL;
 	goto out;
@@ -3593,7 +3593,7 @@ cfg_parse_spoe_agent(const char *file, int linenum, char **args, int kwm)
 			tmp = args[2];
 			while (*tmp) {
 if (!isalnum(*tmp) && *tmp != '_' && *tmp != '.') {
-	ha_alert("parsing [%s:%d]: '%s %s' only supports [a-zA-Z_-.] chars.\n",
+	ha_alert("parsing [%s:%d]: '%s %s' only supports [a-zA-Z0-9_.] chars.\n",
 		 file, linenum, args[0], args[1]);
 	err_code |= ERR_ALERT | ERR_FATAL;
 	goto out;
@@ -3617,7 +3617,7 @@ cfg_parse_spoe_agent(const char *file, int linenum, char **args, int kwm)
 			tmp = args[2];
 			while (*tmp) {
 if (!isalnum(*tmp) && *tmp != '_' && *tmp != '.') {
-	ha_alert("parsing [%s:%d]: '%s %s' only supports [a-zA-Z_-.] chars.\n",
+	ha_alert("parsing [%s:%d]: '%s %s' only supports [a-zA-Z0-9_.] chars.\n",
 		 file, linenum, args[0], args[1]);
 	err_code |= ERR_ALERT | ERR_FATAL;
 	goto out;
-- 
2.16.3

>From 67d525fec42cd9739a093d0e53415b5209d1ee4e Mon Sep 17 00:00:00 2001
From: Thierry FOURNIER 
Date: Fri, 18 May 2018 12:25:39 +0200
Subject: [PATCH 3/3] BUG/MEDIUM: spoe: Flags are not encoded in network order

The flags are direct copy of the "unsigned int" in the network stream,
so the stream contains a 32 bits field encoded with the host endian.
 - This is not reliable for stream betwen different architecture host
 - For x86, the bits doesn't correspond to the documentation.

This patch add some precision in the documentation and put the bitfield
in the stream usig network butes order.

Warning: this patch can break compatibility with existing agents.

This patch should be backported in all version supporing SPOE

Original network capture:

   12:28:16.181343 IP 127.0.0.1.46782 > 127.0.0.1.12345: Flags [P.], seq 134:168, ack 59, win 342, options [nop,nop,TS val 2855241281 ecr 2855241281], length 34
   0x:  4500 0056 6b94 4000 4006 d10b 7f00 0001  E..Vk.@.@...
   0x0010:  7f00 0001 b6be 3039 a3d1 ee54 7d61 d6f7  ..09...T}a..
   0x0020:  8018 0156 fe4a  0101 080a aa2f 8641  ...V.J.../.A
   0x0030:  aa2f 8641  001e 0301   010f  ./.A
  ^^
   0x0040:  6368 6563 6b2d 636c 6965 6e74 2d69 7001  check-client-ip.
   0x0050:  0006 7f00 0001   ..

Fixed network capture:

   12:24:26.948165 IP 127.0.0.1.46706 > 127.0.0.1.12345: Flags [P.], seq 4066280627:4066280661, ack 3148908096, win 342, options [nop,nop,TS val 2855183972 ecr 2855177690], length 34
   0x:  4500 0056 0538 4000 4006 3768 7f00 0001  E..V.8@.@.7h
   0x0010:  7f00 0001 b672 3039 f25e 84b3 bbb0 8640  .r09.^.@
   0x0020:  8018 0156 fe4a  0101 080a aa2e a664  ...V.J.d
   0x0030:  aa2e 8dda  001e 0300  0114 010f  
  ^^
   0x0040:  6368 6563 6b2d 636c 6965 6e74 2d69 7001  check-client-ip.
   0x0050:  0006 7f00 0001   ..
---
 contrib/spoa_example/spoa.c | 7 +++
 doc/SPOE.txt| 7 ++-
 src/flt_spoe.c  | 7 +++
 3 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/contrib/spoa_example/spoa.c 

Re: Dynamically adding/deleting SSL certificates

[Aurélien Nephtali]

Hello,

On Wed, Apr 18, 2018 at 9:34 PM, Aurélien Nephtali
 wrote:
> Hello,
>
> I have some patches to support dynamically loading and unloading PEM
> certificates through the CLI. It is mainly a big refactoring of some
> part of the SSL code (thanks Thierry for your patches, we came to the
> same conclusion :) !).
>

Here is an updated version of this feature. The changes are:
- Use a payload in the CLI to pass the certificate
- Change the way to specify on which listener the certificate is
to be added/removed: using the "bind name".
  If the listeners are not named, the only way to update their
certificates is to do a global operation (using just the frontend name
in the command).

One thing that should be discussed is what will be the command syntax
when it will support more certificate options (OCSP, SCTL) ?
I thought about sending something like an .ini file:

[certificate]
a===

[ocsp]
b===

etc...

but one needs to prepare these files: it may not be very handy for a
one shot operation ?
Plus, without streaming we're quickly limited by the payload size with
the default value.

-- 
Aurélien Nephtali
From bfdd0e89ee79b996cd4a4b05afbde79e5919d8bc Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Aur=C3=A9lien=20Nephtali?= 
Date: Wed, 18 Apr 2018 15:34:33 +0200
Subject: [PATCH 1/2] MINOR/WIP: ssl: Refactor ssl_sock_load_cert_file() into
 ssl_sock_load_cert2()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

- ssl_sock_get_dh_from_file() -> ssl_sock_get_dh()
- ssl_sock_load_dh_params() takes a BIO * instead of a char *
- ssl_sock_load_cert_chain_file() -> ssl_sock_load_cert_chain() + takes a
  BIO * instead of a char *

Signed-off-by: Aurélien Nephtali 
---
 src/ssl_sock.c | 210 +
 1 file changed, 136 insertions(+), 74 deletions(-)

diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index 7a602ad57..eb0d43ded 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -2583,43 +2583,39 @@ static DH *ssl_get_tmp_dh(SSL *ssl, int export, int keylen)
 	return dh;
 }
 
-static DH * ssl_sock_get_dh_from_file(const char *filename)
+static DH * ssl_sock_get_dh(BIO *in)
 {
-	DH *dh = NULL;
-	BIO *in = BIO_new(BIO_s_file());
+	return PEM_read_bio_DHparams(in, NULL, NULL, NULL);
+}
 
-	if (in == NULL)
-		goto end;
+int ssl_sock_load_global_dh_param_from_file(const char *filename)
+{
+	BIO *in;
+	int ret = -1;
+
+	in = BIO_new(BIO_s_file());
+	if (!in)
+		return -1;
 
 	if (BIO_read_filename(in, filename) <= 0)
 		goto end;
 
-	dh = PEM_read_bio_DHparams(in, NULL, NULL, NULL);
+	global_dh = ssl_sock_get_dh(in);
+	if (global_dh)
+		ret = 0;
 
 end:
-if (in)
-BIO_free(in);
+	BIO_free(in);
 
-	return dh;
-}
-
-int ssl_sock_load_global_dh_param_from_file(const char *filename)
-{
-	global_dh = ssl_sock_get_dh_from_file(filename);
-
-	if (global_dh) {
-		return 0;
-	}
-
-	return -1;
+	return ret;
 }
 
 /* Loads Diffie-Hellman parameter from a file. Returns 1 if loaded, else -1
if an error occured, and 0 if parameter not found. */
-int ssl_sock_load_dh_params(SSL_CTX *ctx, const char *file)
+int ssl_sock_load_dh_params(SSL_CTX *ctx, BIO *in)
 {
 	int ret = -1;
-	DH *dh = ssl_sock_get_dh_from_file(file);
+	DH *dh = ssl_sock_get_dh(in);
 
 	if (dh) {
 		ret = 1;
@@ -3192,10 +3188,9 @@ static int ssl_sock_load_multi_cert(const char *path, struct bind_conf *bind_con
 /* Loads a certificate key and CA chain from a file. Returns 0 on error, -1 if
  * an early error happens and the caller must call SSL_CTX_free() by itelf.
  */
-static int ssl_sock_load_cert_chain_file(SSL_CTX *ctx, const char *file, struct bind_conf *s,
-	 struct ssl_bind_conf *ssl_conf, char **sni_filter, int fcount)
+static int ssl_sock_load_cert_chain(SSL_CTX *ctx, BIO *in, struct bind_conf *s,
+struct ssl_bind_conf *ssl_conf, char **sni_filter, int fcount)
 {
-	BIO *in;
 	X509 *x = NULL, *ca;
 	int i, err;
 	int ret = -1;
@@ -3211,14 +3206,6 @@ static int ssl_sock_load_cert_chain_file(SSL_CTX *ctx, const char *file, struct
 	STACK_OF(GENERAL_NAME) *names;
 #endif
 
-	in = BIO_new(BIO_s_file());
-	if (in == NULL)
-		goto end;
-
-	if (BIO_read_filename(in, file) <= 0)
-		goto end;
-
-
 	passwd_cb = SSL_CTX_get_default_passwd_cb(ctx);
 	passwd_cb_userdata = SSL_CTX_get_default_passwd_cb_userdata(ctx);
 
@@ -3308,44 +3295,110 @@ end:
 	if (x)
 		X509_free(x);
 
-	if (in)
-		BIO_free(in);
-
 	return ret;
 }
 
-static int ssl_sock_load_cert_file(const char *path, struct bind_conf *bind_conf, struct ssl_bind_conf *ssl_conf,
-   char **sni_filter, int fcount, char **err)
+struct ssl_input {
+const char *cert_path;
+struct chunk *cert_buf;
+};
+
+static int ssl_sock_load_cert2(struct ssl_input *si, struct bind_conf *bind_conf, struct ssl_bind_conf *ssl_conf,
+   char **sni_filter, int