Re: OpenSSL engine and async support

Hi Emeric, On Tue, Jun 06, 2017 at 03:16:39PM +0200, Emeric Brun wrote: > Here 3 new fixes: > > I noticed a segfault sometime at connection closure (first patch) > > I noticed buffer overflows using the cipher offloading in async: > > The moving or reuse of buffer addresses passed to

Re: OpenSSL engine and async support

Hi Grant, Willy, On 05/27/2017 09:03 PM, Grant Zhang wrote: > >> On May 26, 2017, at 22:21, Willy Tarreau wrote: >> >> Hi Emeric, Grant, >> >> patch set now merged! Thank you both for this great work! >> >> Willy > > Bravo! Really appreciate your and Emeric's help in this effort. >

Re: OpenSSL engine and async support

> On May 26, 2017, at 22:21, Willy Tarreau wrote: > > Hi Emeric, Grant, > > patch set now merged! Thank you both for this great work! > > Willy Bravo! Really appreciate your and Emeric's help in this effort. Grant

Re: OpenSSL engine and async support

Hi Emeric, Grant, patch set now merged! Thank you both for this great work! Willy

Re: OpenSSL engine and async support

Hi Willy, On 05/17/2017 10:10 PM, Willy Tarreau wrote: > Hi Emeric, > > On Wed, May 17, 2017 at 09:49:32PM +0200, Emeric Brun wrote: >> More fixes, it appears stable now, even if session are closed during >> handshake. >> >> I also added the support of multiple async engines (latest patch: it

Re: OpenSSL engine and async support

Hi Emeric, On Wed, May 17, 2017 at 09:49:32PM +0200, Emeric Brun wrote: > More fixes, it appears stable now, even if session are closed during > handshake. > > I also added the support of multiple async engines (latest patch: it is > limited to 32 engines) > > I did some tests using my

Re: OpenSSL engine and async support

Hi Grant, On 05/16/2017 12:05 PM, Emeric Brun wrote: > Hi Grant, > > On 05/15/2017 08:11 PM, Grant Zhang wrote: >> >>> On May 15, 2017, at 03:14, Emeric Brun wrote: >>> >>> What does it look like? >> New patches attached. >> >>> >>> The issue is very similar: >>>

Re: OpenSSL engine and async support

Hi Grant, On 05/15/2017 08:11 PM, Grant Zhang wrote: > >> On May 15, 2017, at 03:14, Emeric Brun wrote: >> >> What does it look like? > New patches attached. > >> >> The issue is very similar: >> https://mta.openssl.org/pipermail/openssl-dev/2016-March/005909.html >

Re: OpenSSL engine and async support

> On May 15, 2017, at 03:14, Emeric Brun wrote: > > What does it look like? New patches attached. > > The issue is very similar: > https://mta.openssl.org/pipermail/openssl-dev/2016-March/005909.html Interesting. yeah, it looks similar. Regards, Grant

Re: OpenSSL engine and async support

Hi Grant, On 05/15/2017 12:14 PM, Emeric Brun wrote: > On 05/13/2017 01:14 AM, Grant Zhang wrote: >> >>> On May 10, 2017, at 04:51, Emeric Brun wrote: >>> It looks like the main process stalls at DH_free(local_dh_1024) (part of __ssl_sock_deinit). Not sure why but I

Re: OpenSSL engine and async support

On 05/13/2017 01:14 AM, Grant Zhang wrote: > >> On May 10, 2017, at 04:51, Emeric Brun wrote: >> >>> It looks like the main process stalls at DH_free(local_dh_1024) (part of >>> __ssl_sock_deinit). Not sure why but I will debug and report back. >>> >>> Thanks, >> >> I

Re: OpenSSL engine and async support

> On May 10, 2017, at 04:51, Emeric Brun wrote: > >> It looks like the main process stalls at DH_free(local_dh_1024) (part of >> __ssl_sock_deinit). Not sure why but I will debug and report back. >> >> Thanks, > > I experienced the same issue (stalled on a futex) if i run

Re: OpenSSL engine and async support

Hi Grant, On 05/09/2017 10:38 PM, Grant Zhang wrote: > >> On May 9, 2017, at 02:38, Emeric Brun wrote: >> >> Hi Grant, >> >> On 05/06/2017 12:41 AM, Grant Zhang wrote: >>> Hi Emeric, >>> >>> Thanks for your review! Please see the updated patches and let me know if >>> your

Re: OpenSSL engine and async support

> On May 9, 2017, at 02:38, Emeric Brun wrote: > > Hi Grant, > > On 05/06/2017 12:41 AM, Grant Zhang wrote: >> Hi Emeric, >> >> Thanks for your review! Please see the updated patches and let me know if >> your comments have been properly addressed. >> >> Thanks, >> >>

Re: OpenSSL engine and async support

Hi Grant, On 05/06/2017 12:41 AM, Grant Zhang wrote: > Hi Emeric, > > Thanks for your review! Please see the updated patches and let me know if > your comments have been properly addressed. > > Thanks, > > Grant > > > > > > > >> On May 2, 2017, at 04:49, Emeric Brun

Re: OpenSSL engine and async support

Hi Emeric, Thanks for your review! Please see the updated patches and let me know if your comments have been properly addressed. Thanks, Grant 0001-ssl-add-basic-support-for-OpenSSL-crypto-engine.patch Description: Binary data 0002-ssl-add-openssl-async-mode-support.patch Description:

Re: OpenSSL engine and async support

Hi Grant, An other issue: static void ssl_sock_close(struct connection *conn) { if (conn->xprt_ctx) { if (global_ssl.async) { /* the async fd is created and owned by the SSL engine, which is * responsible for fd closure.

Re: OpenSSL engine and async support

Hi Grant, >>> >> >> I've made a POC of a soft async engine. Based on dasync engine it launchs a >> thread on priv_rsa_enc to spread the load on multiple cores. >> >> Regarding openssl s_server it is efficient and scale very well depending the >> number of core (1700 rsa2048/s on one core, 7400

Re: OpenSSL engine and async support

Hi Emeric, > On Apr 25, 2017, at 04:03, Emeric Brun wrote: > > Hi Grant, > > On 04/10/2017 05:16 PM, Grant Zhang wrote: >> >>> On Apr 10, 2017, at 07:42, Emeric Brun wrote: >>> >> * openssl version (1.1.0b-e?) >>> compiled 1.1.0e >>>

Re: OpenSSL engine and async support

Hi Grant, On 04/10/2017 05:16 PM, Grant Zhang wrote: > >> On Apr 10, 2017, at 07:42, Emeric Brun wrote: >> > >>> * openssl version (1.1.0b-e?) >> compiled 1.1.0e >>> >>> >> Could you provide patches rebased on current dev master branch? > I am kinda busy with other project

Re: OpenSSL engine and async support

> On Apr 10, 2017, at 07:42, Emeric Brun wrote: > >> * openssl version (1.1.0b-e?) > compiled 1.1.0e >> >> > Could you provide patches rebased on current dev master branch? I am kinda busy with other project but will try to provide rebased patches this week. Thanks,

Re: OpenSSL engine and async support

Hi Grant, On 04/01/2017 02:01 AM, Grant Zhang wrote: > Hi Emeric, > > Sorry for my delayed reply. > > > On 03/28/2017 01:47 AM, Emeric Brun wrote: >> >>> This is an atom C2518 and it seems that --disable-prf has cut the >>> performance >>> in half. We should receive a 8920 soon. >>> >

Re: OpenSSL engine and async support

Hi Emeric, Sorry for my delayed reply. On 03/28/2017 01:47 AM, Emeric Brun wrote: This is an atom C2518 and it seems that --disable-prf has cut the performance in half. We should receive a 8920 soon. Stopping the injection, the haproxy process continue to steal cpu doing nothing (top

Re: OpenSSL engine and async support

Hi Grant, On 03/28/2017 07:38 AM, Willy Tarreau wrote: > Hi guys, > > On Mon, Mar 27, 2017 at 02:57:38PM -0700, Grant Zhang wrote: >>> signverifysign/s verify/s >>> rsa 2048 bits 0.000745s 0.34s 1342.9 29050.3 >> >> Hmm, the numbers are less than what I've seen: >

Re: OpenSSL engine and async support

Hi guys, On Mon, Mar 27, 2017 at 02:57:38PM -0700, Grant Zhang wrote: > > signverifysign/s verify/s > > rsa 2048 bits 0.000745s 0.34s 1342.9 29050.3 > > Hmm, the numbers are less than what I've seen: (...) This is an atom C2518 and it seems that --disable-prf has

Re: OpenSSL engine and async support

> On Mar 27, 2017, at 02:21, Emeric Brun wrote: > Intel's guys told me that the bug is related to prf and asked me to recompile > the engine using '--disable_qat_prf'. Doing that i can do some tests iwth the > qat engine but i'm facing stability issues: > > [root@centos

Re: OpenSSL engine and async support

Hi Grant, > Hey Emeric, > > Thank you very much for the information. Hopefully the s_server + qat issue > could be addressed soon. > > Regards, > > Grant > > > Intel's guys told me that the bug is related to prf and asked me to recompile the engine using '--disable_qat_prf'. Doing that

Re: OpenSSL engine and async support

> On Mar 21, 2017, at 06:56, Emeric Brun wrote: > > Hi Grant, > >>> >>> I'm not sure that the issue is related to your patch, i may reach an issue >>> int QAT engine >>> >>> I've made some test using openssl s_server. >>> >>> Doing a curl request shows this error: >>>

Re: OpenSSL engine and async support

Hi Grant, >> >> I'm not sure that the issue is related to your patch, i may reach an issue >> int QAT engine >> >> I've made some test using openssl s_server. >> >> Doing a curl request shows this error: >> [root@centos bin]# ./openssl s_server -accept 9443 -engine qat -cert >> /root/2048.pem

Re: OpenSSL engine and async support

Hi Grant, On 03/15/2017 06:20 PM, Grant Zhang wrote: > Hi Emeric >> On Mar 15, 2017, at 10:05, Emeric Brun wrote: >> >> Hi John, >> There is some inconsistencies between the engine and the used client: here the conf: global

Re: OpenSSL engine and async support

Hi Grant, On Wed, Mar 15, 2017 at 10:20:01AM -0700, Grant Zhang wrote: > Maybe you run into the openssl 1.1 SNI issue. Does your test branch have the > following patch: > http://git.haproxy.org/?p=haproxy.git;a=commit;h=d3850603933c9319528375088a9b28b9b345246b > I think not because Emeric had

Re: OpenSSL engine and async support

Hi Emeric > On Mar 15, 2017, at 10:05, Emeric Brun wrote: > > Hi John, > >>> >>> There is some inconsistencies between the engine and the used client: >>> >>> here the conf: >>> global >>> tune.ssl.default-dh-param 2048 >>> ssl-engine qat >>> ssl-async >>>

Re: OpenSSL engine and async support

Hi John, >> >> There is some inconsistencies between the engine and the used client: >> >> here the conf: >> global >>tune.ssl.default-dh-param 2048 >>ssl-engine qat >>ssl-async >> >> listen gg >>mode http >>bind 0.0.0.0:8443 ssl crt /root/2048.pem >>

Re: OpenSSL engine and async support

Hi Emeric, Thanks for testing. I will try repro the issues locally and report back. Regards, Grant > On Mar 15, 2017, at 07:41, Emeric Brun wrote: > > Hi Grant, > > On 03/15/2017 12:46 PM, Emeric Brun wrote: >> Hi Grant, >> >> On 03/15/2017 12:05 PM, Emeric Brun wrote:

Re: OpenSSL engine and async support

Hi Grant, On 03/15/2017 12:05 PM, Emeric Brun wrote: > Hi Grant, > > On 02/04/2017 12:55 AM, Grant Zhang wrote: >> This patch set adds the basic support for OpenSSL crypto engine and >> async mode. >> >> Changes since V2: >> - support keyword "algo" >> - ensure SSL engines are initialized

OpenSSL engine and async support

This patch set adds the basic support for OpenSSL crypto engine and async mode. Changes since V2: - support keyword "algo" - ensure SSL engines are initialized before loading certs. - limit one async fd per SSL connection - better integrate with event cache Changes since V1: - add multiple