date:20170606

Re: OpenSSL engine and async support

2017-06-06 Thread Emeric Brun

Hi Grant, Willy,
On 05/27/2017 09:03 PM, Grant Zhang wrote:
> 
>> On May 26, 2017, at 22:21, Willy Tarreau  wrote:
>>
>> Hi Emeric, Grant,
>>
>> patch set now merged! Thank you both for this great work!
>>
>> Willy
> 
> Bravo! Really appreciate your and Emeric's help in this effort.
> 
> Grant
> 

Here 3 new fixes:

I noticed a segfault sometime at connection closure (first patch)

I noticed buffer overflows using the cipher offloading in async:

The moving or reuse of buffer addresses passed to SSL_read/write in haproxy
is not compliant with the ASYNC API. I had a discussion about that on the
openssl-dev mailing list with Matt Caswell.

So the second patch disables the async mode for the symmetric stuff.

The last one to not call directly the conn_fd_handler from the async_fd_handler.

R,
Emeric


>From 535ef040f7c6ee31f8c8943d1db5236f66cb6e43 Mon Sep 17 00:00:00 2001
From: root 
Date: Fri, 2 Jun 2017 15:54:06 +
Subject: [PATCH 3/3] BUG/MINOR: ssl: do not call directly the conn_fd_handler
 from async_fd_handler

This patch modifies the way to re-enable the connection from the async fd
handler calling conn_update_sock_polling instead of the conn_fd_handler.

It also ensures that the polling is really stopped on the async fd.
---
 src/ssl_sock.c | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index a009803..04c4cbb 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -363,17 +363,19 @@ fail_get:
 static void ssl_async_fd_handler(int fd)
 {
 	struct connection *conn = fdtab[fd].owner;
-	int conn_fd = conn->t.sock.fd;
 
 	/* fd is an async enfine fd, we must stop
 	 * to poll this fd until it is requested
 	 */
+fd_stop_recv(fd);
 fd_cant_recv(fd);
 
 	/* crypto engine is available, let's notify the associated
 	 * connection that it can pursue its processing.
 	 */
-	conn_fd_handler(conn_fd);
+	__conn_sock_want_recv(conn);
+	__conn_sock_want_send(conn);
+	conn_update_sock_polling(conn);
 }
 
 /*
-- 
2.1.4

>From 79b70ede0baed17e52c17ae2f6c93437fa68b824 Mon Sep 17 00:00:00 2001
From: root 
Date: Tue, 6 Jun 2017 12:35:14 +
Subject: [PATCH 2/3] BUG/MAJOR: ssl: buffer overflow using offloaded ciphering
 on async engine

The Openssl's ASYNC API does'nt support moving buffers on SSL_read/write
This patch disables the ASYNC mode dynamically when the handshake
is left and re-enables it on reneg.
---
 doc/configuration.txt |  6 +-
 src/ssl_sock.c| 39 ---
 2 files changed, 33 insertions(+), 12 deletions(-)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 75f9961..836c8b9 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -1285,7 +1285,11 @@ ssl-engine  [algo ]
 ssl-mode-async
   Adds SSL_MODE_ASYNC mode to the SSL context. This enables asynchronous TLS
   I/O operations if asynchronous capable SSL engines are used. The current
-  implementation supports a maximum of 32 engines.
+  implementation supports a maximum of 32 engines. The Openssl ASYNC API
+  doesn't support moving read/write buffers and is not compliant with
+  haproxy's buffer management. So the asynchronous mode is disabled on
+  read/write  operations (it is only enabled during initial and reneg
+  handshakes).
 
 tune.buffers.limit 
   Sets a hard limit on the number of buffers which may be allocated per process.
diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index a04deb6..a009803 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -4632,6 +4632,15 @@ int ssl_sock_handshake(struct connection *conn, unsigned int flag)
 	}
 
 reneg_ok:
+
+#if OPENSSL_VERSION_NUMBER >= 0x101fL
+	/* ASYNC engine API doesn't support moving read/write
+	 * buffers. So we disable ASYNC mode right after
+	 * the handshake to avoid buffer oveflows.
+	 */
+	if (global_ssl.async)
+		SSL_clear_mode(conn->xprt_ctx, SSL_MODE_ASYNC);
+#endif
 	/* Handshake succeeded */
 	if (!SSL_session_reused(conn->xprt_ctx)) {
 		if (objt_server(conn->target)) {
@@ -4750,6 +4759,11 @@ static int ssl_sock_to_buf(struct connection *conn, struct buffer *buf, int coun
 /* handshake is running, and it needs to enable write */
 conn->flags |= CO_FL_SSL_WAIT_HS;
 __conn_sock_want_send(conn);
+#if OPENSSL_VERSION_NUMBER >= 0x101fL
+/* Async mode can be re-enabled, because we're leaving data state.*/
+if (global_ssl.async)
+	SSL_set_mode(conn->xprt_ctx, SSL_MODE_ASYNC);
+#endif
 break;
 			}
 			else if (ret == SSL_ERROR_WANT_READ) {
@@ -4757,18 +4771,17 @@ static int ssl_sock_to_buf(struct connection *conn, struct buffer *buf, int coun
 	/* handshake is running, and it may need to re-enable read */
 	conn->flags |= CO_FL_SSL_WAIT_HS;
 	__conn_sock_want_recv(conn);
+#if OPENSSL_VERSION_NUMBER >= 0x101fL
+	/* Async mode can be re-enabled, because we're leaving data state.*/
+	if (global_ssl.async)
+		SSL_set_mode(conn->xprt_ctx, SSL_MODE_ASYNC);

Re: [PATCH] BUILD: ssl: add Makefile parameter to help build with boringssl

2017-06-06 Thread Emmanuel Hocdet

up
Willy, i think this patch could be helpful.

> Le 28 mars 2017 à 16:01, Emmanuel Hocdet  a écrit :
> 
> This patch can be helpful to build easier with boringssl.
> 
> Manu
> 
> <0001-BUILD-ssl-add-Makefile-parameter-to-help-build-with-.patch>
> 
>

Re: [PATCH] bis contrib mod security

2017-06-06 Thread Thierry Fournier

It seems good for me. Willy can you integrate this patch ?

Thanks,
Thierry

—
Thierry Fournier
Web Performance & Security Expert
m: +33 6 68 69 21 85  | e: thierry.fourn...@ozon.io
w: http://www.ozon.io/| b: http://blog.ozon.io/

> On 6 Jun 2017, at 11:22, David CARLIER  wrote:
> 
> <0001-BUG-MINOR-contrib-modsecurity-BSD-build-fix.patch>

Re: [PATCH] bis contrib mod security

2017-06-06 Thread David CARLIER

Hi and thanks. Here a little change. Regards.

On 6 June 2017 at 09:58, Thierry Fournier  wrote:

> Thanks for the ping, I have a lot of work and I dont saw your message.
>
> Your patch seems good except this replacement:
>
>-LDFLAGS += -lpthread  -levent -levent_pthreads -lcurl -lapr-1
> -laprutil-1 -lxml2 -lpcre -lyajl
>+LDFLAGS += -lpthread  -levent_core -levent_pthreads -lcurl -lapr-1
> -laprutil-1 -lxml2 -lpcre -lyajl
>
> I’m afraid that the replacement of -levent by -levent_core will break the
> compilation for Linux systems.
> Maybe it will be better to add some variables like EVENT_INC and EVENT_LIB
>
> Thierry
>
> On 6 Jun 2017, at 09:44, David CARLIER  wrote:
>
> ping
>
> On 4 May 2017 at 21:50, David CARLIER  wrote:
>
>> Hi apologies the patch sent yesterday was not the correct one.
>>
>> Kind regards.
>>
>
>
>
From 6b2d8a973a8a969bc913dfed0145a46a294b19ec Mon Sep 17 00:00:00 2001
From: David Carlier 
Date: Tue, 6 Jun 2017 10:20:51 +0100
Subject: [PATCH] BUG/MINOR: contrib/modsecurity: BSD build fix

previous version introduced in the last commit was not the correct one.
---
 contrib/modsecurity/Makefile | 16 
 contrib/modsecurity/spoa.h   |  2 +-
 2 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/contrib/modsecurity/Makefile b/contrib/modsecurity/Makefile
index 7853397..bb918c3 100644
--- a/contrib/modsecurity/Makefile
+++ b/contrib/modsecurity/Makefile
@@ -2,7 +2,7 @@ DESTDIR=
 PREFIX = /usr/local
 BINDIR = $(PREFIX)/bin
 
-CC = gcc
+CC ?= gcc
 LD = $(CC)
 
 ifeq ($(MODSEC_INC),)
@@ -25,9 +25,17 @@ ifeq ($(LIBXML_INC),)
 LIBXML_INC := /usr/include/libxml2
 endif
 
-CFLAGS  = -g -Wall -pthread
-LDFLAGS += -lpthread  -levent -levent_pthreads -lcurl -lapr-1 -laprutil-1 -lxml2 -lpcre -lyajl
-INCS += -I../../include -I../../ebtree -I$(MODSEC_INC) -I$(APACHE2_INC) -I$(APR_INC) -I$(LIBXML_INC)
+ifeq ($(EVENT_LIB),)
+EVENT_LIB := -levent
+endif
+
+ifeq ($(EVENT_INC),)
+EVENT_INC := /usr/include
+endif
+
+CFLAGS  += -g -Wall -pthread
+LDFLAGS += -lpthread  $(EVENT_LIB) -levent_pthreads -lcurl -lapr-1 -laprutil-1 -lxml2 -lpcre -lyajl
+INCS += -I../../include -I../../ebtree -I$(MODSEC_INC) -I$(APACHE2_INC) -I$(APR_INC) -I$(LIBXML_INC) -I$(EVENT_INC)
 LIBS =
 
 OBJS = spoa.o modsec_wrapper.o
diff --git a/contrib/modsecurity/spoa.h b/contrib/modsecurity/spoa.h
index d618f9b..ea7a94e 100644
--- a/contrib/modsecurity/spoa.h
+++ b/contrib/modsecurity/spoa.h
@@ -20,7 +20,7 @@
 #ifndef __SPOA_H__
 #define __SPOA_H__
 
-#include 
+#undef LIST_HEAD
 
 #include 
 #include 
-- 
2.7.4

Re: [PATCH] bis contrib mod security

2017-06-06 Thread Thierry Fournier

Thanks for the ping, I have a lot of work and I dont saw your message.

Your patch seems good except this replacement:

   -LDFLAGS += -lpthread  -levent -levent_pthreads -lcurl -lapr-1 -laprutil-1 
-lxml2 -lpcre -lyajl
   +LDFLAGS += -lpthread  -levent_core -levent_pthreads -lcurl -lapr-1 
-laprutil-1 -lxml2 -lpcre -lyajl

I’m afraid that the replacement of -levent by -levent_core will break the 
compilation for Linux systems.
Maybe it will be better to add some variables like EVENT_INC and EVENT_LIB

Thierry

> On 6 Jun 2017, at 09:44, David CARLIER  wrote:
> 
> ping
> 
> On 4 May 2017 at 21:50, David CARLIER  > wrote:
> Hi apologies the patch sent yesterday was not the correct one.
> 
> Kind regards.
>

Re:Re: Re: haproxy does not capture the complete request header host sometimes

2017-06-06 Thread siclesang

strace.log:


15:48:48.826818 setsockopt(12, SOL_SOCKET, SO_LINGER, {onoff=1, linger=0}, 8) = 
0 <0.23>
15:48:48.826881 close(12)   = 0 <0.37>
15:48:48.826968 epoll_wait(3, {{EPOLLIN, {u32=8, u64=8}}}, 500, 68) = 1 
<0.048443>
15:48:48.875473 accept4(8, {sa_family=AF_INET, sin_port=htons(36509), 
sin_addr=inet_addr("111.206.221.97")}, [16], SOCK_NONBLOCK) = 10 <0.40>
15:48:48.875580 setsockopt(10, SOL_TCP, TCP_NODELAY, [1], 4) = 0 <0.28>
15:48:48.875665 write(1, "3c5e:website.accept(0008)=000a from 
[111.206.221.97:36509]\n", 62) = 62 <0.32>
15:48:48.875746 accept4(8, 0x7fff80cf31e0, [128], SOCK_NONBLOCK) = -1 EAGAIN 
(Resource temporarily unavailable) <0.29>
15:48:48.875843 recvfrom(10, "POST /token/GetToken/t505 HTTP/1.1\r\nHost: 
preview-member-fnapp.website.com\r\nContent-Length: 296\r\nUser-Agent: 
Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 
(KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; 
Baiduspider-render/2.0; 
+http://www.baidu.com/search/spider.html)\r\nProxy-Connection: 
keep-alive\r\nOrigin: http://preview.hd.website.com\r\nAccept-Language: 
zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3\r\nContent-Type: 
application/x-www-form-urlencoded\r\nAccept:"..., 47616, 0, NULL, NULL) = 1331 
<0.26>
15:48:48.876269 write(1, "3c5e:website.clireq[000a:]: POST 
/token/GetToken/t505 HTTP/1.1\n", 74) = 74 <0.30>
15:48:48.876357 write(1, "3c5e:website.clihdr[000a:]: Host: 
preview\n", 53) = 53 <0.26>
15:48:48.876431 write(1, "3c5e:website.clihdr[000a:]: 
ember-fnapp.website.com\n", 62) = 62 <0.37>
15:48:48.876516 write(1, "3c5e:website.clihdr[000a:]: h: 296\n", 
46) = 46 <0.26>
15:48:48.876590 write(1, "3c5e:website.clihdr[000a:]: S 9_1 like 
Mac OS X\n", 59) = 59 <0.25>
15:48:48.876663 write(1, "3c5e:website.clihdr[000a:]: 
AppleWebKit/601.1.46 (KHTML, like Gecko) Ver\n", 84) = 84 <0.25>
15:48:48.876737 write(1, "3c5e:website.clihdr[000a:]: on/9.0 
Mobile/13B\n", 57) = 57 <0.26>
15:48:48.876810 write(1, "3c5e:website.clihdr[000a:]: 3 
Safari/601.1 (comp\n", 60) = 60 <0.25>
15:48:48.876883 write(1, "3c5e:website.clihdr[000a:]: ible; 
Baiduspider-render/2.0;\n", 69) = 69 <0.25>
15:48:48.876999 write(1, "3c5e:website.clihdr[000a:]: 
http://www.baidu.com\n;, 60) = 60 <0.16>
15:48:48.877261 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 12 <0.29>
15:48:48.877334 fcntl(12, F_SETFL, O_RDONLY|O_NONBLOCK) = 0 <0.24>
15:48:48.877407 setsockopt(12, SOL_TCP, TCP_NODELAY, [1], 4) = 0 <0.26>
15:48:48.877482 connect(12, {sa_family=AF_INET, sin_port=htons(), 
sin_addr=inet_addr("10.201.167.125")}, 16) = -1 EINPROGRESS (Operation now in 
progress) <0.55>
15:48:48.877593 epoll_wait(3, {}, 500, 0) = 0 <0.34>
15:48:48.877678 sendto(12, "POST /token/GetToken/t505 HTTP/1.1\r\nHost: 
preview-member-fnapp.website.com\r\nContent-Length: 296\r\nUser-Agent: 
Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 
(KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; 
Baiduspider-render/2.0; 
+http://www.baidu.com/search/spider.html)\r\nProxy-Connection: 
keep-alive\r\nOrigin: http://preview.hd.website.com\r\nAccept-Language: 
zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3\r\nContent-Type: 
application/x-www-form-urlencoded\r\nAccept:"..., 1087, 
MSG_DONTWAIT|MSG_NOSIGNAL, NULL, 0) = 1087 <0.43>
15:48:48.877813 recvfrom(10, 0x2722a6b, 47320, 0, 0, 0) = -1 EAGAIN (Resource 
temporarily unavailable) <0.28>
15:48:48.877921 epoll_ctl(3, EPOLL_CTL_ADD, 10, {EPOLLIN|0x2000, {u32=10, 
u64=10}}) = 0 <0.67>
15:48:48.878043 epoll_wait(3, {}, 500, 0) = 0 <0.24>
15:48:48.878112 recvfrom(12, "HTTP/1.1 404 Not Found\r\nServer: nginx\r\nDate: 
Tue, 06 Jun 2017 07:48:48 GMT\r\nContent-Type: text/html\r\nContent-Length: 
162\r\nConnection: close\r\n\r\n\r\n404 Not 
Found\r\n\r\n404 Not 
Found\r\nnginx\r\n\r\n\r\n", 
47616, 0, NULL, NULL) = 305 <0.25>
15:48:48.878321 write(1, "3c5e:no_server.srvrep[000a:000c]: HTTP/1.1 404 
Not Found\n", 61) = 61 <0.35>
15:48:48.878405 write(1, "3c5e:no_server.srvhdr[000a:000c]: Server: 
nginx\n", 52) = 52 <0.25>
15:48:48.878477 write(1, "3c5e:no_server.srvhdr[000a:000c]: 2017 07:48:48 
GMT\n", 56) = 56 <0.24>
15:48:48.878547 write(1, "3c5e:no_server.srvhdr[000a:000c]: text/html\n", 
48) = 48 <0.25>
15:48:48.878617 write(1, "3c5e:no_server.srvhdr[000a:000c]:  162\n", 43) = 
43 <0.24>
15:48:48.878687 write(1, "3c5e:no_server.srvhdr[000a:000c]: >\n", 40) = 40 
<0.24>
15:48:48.878785 epoll_wait(3, {}, 500, 0) = 0 <0.24>
15:48:48.878851 sendto(10, "HTTP/1.1 404 Not Found\r\nServer: nginx\r\nDate: 
Tue, 06 Jun 2017 07:48:48 GMT\r\nContent-Type: text/html\r\nContent-Length: 
162\r\nConnection: close\r\nConnection: close\r\n\r\n\r\n404 
Not Found\r\n\r\n404 Not

Re: [PATCH] bis contrib mod security

2017-06-06 Thread David CARLIER

ping

On 4 May 2017 at 21:50, David CARLIER  wrote:

> Hi apologies the patch sent yesterday was not the correct one.
>
> Kind regards.
>

HOW TO MAKE MORE MONEY

2017-06-06 Thread l...@vpc-pneumatic.com

Hey Sir/Madam

Good day!QUALITY IS OUR CULTURE!

We accpet paypment by Paypal that most convenient for American customer .

This is Lina from VPC Pneumatic Group that one of white hair for pneumatic 
business.

Yeah,we knew that you buy fiting from EMC Pneumatic Co.,ltd ,so how about their 
quality and price ?

Would you like to try our products ?We can offer competitive price advantage 
products.
If needed ,free samples will be send.

We hope we can have the chance to serve you .

Look forward to hearing from you soon.

Yours Sincerely
Lina

Re: OpenSSL engine and async support

Re: [PATCH] BUILD: ssl: add Makefile parameter to help build with boringssl

Re: [PATCH] bis contrib mod security

Re: [PATCH] bis contrib mod security

Re: [PATCH] bis contrib mod security

Re:Re: Re: haproxy does not capture the complete request header host sometimes

Re: [PATCH] bis contrib mod security

HOW TO MAKE MORE MONEY

8 matches

Site Navigation

Mail list logo

Footer information