Hi, I've slightly updated my patch to improve it and to fix some inconsistencies.
First of all, now "ssl-upgrade" and "no-ssl-upgrade" actions can be used on "tcp-request content" rules _AND_ "tcp-request connection" rules, in a frontend _OR_ a backend definition. Then, these actions are now custom actions. I think this is cleaner this way. And finally, by default, no SSL upgrade is done when "defer-ssl-upgrade" option is used. So you need to use explicitly a "ssl-upgrade" rule to perform it. For a lack of finding the right place to do SSL upgrades when no "tcp-request" rule is defined, I've decided to change the default behavior. I've kept the "defer-ssl-upgrade" keyword, but now, "skip-ssl-upgrade" could be more appropriate. If you prefer, i can do the change. -- Christopher
>From 05c439bfcb39ae297c1d1652f8e4fd72850ff775 Mon Sep 17 00:00:00 2001 From: Christopher Faulet <[email protected]> Date: Thu, 3 Mar 2016 16:15:28 +0100 Subject: [PATCH] MAJOR: ssl: Allow the postpone of the SSL upgrade and a way to enable/disable it the bind option "defer-ssl-upgrade" has been added. It can be used iff "ssl" option is set. It postpone the SSL upgrade when evaluation of "tcp-request" rules. 2 actions have been added in "tcp-request connection/content" rules to conditionally enable/disable this upgrade: * ssl-upgrade * no-ssl-upgrade The "ssl-upgrade" is used to upgrade a TCP connection to a SSL connection and "no-ssl-upgrade" is used to disable it. This is only available for connections opened on a SSL listener configured with the "defer-ssl-upgrade" option and ignored for all others. When a "ssl-upgrade" rule matches, it invalidates "no-ssl-upgrade" rules that follow, and conversely. if neither the "ssl-upgrade" rules nor the "no-ssl-upgrade" rules matchn, no SSL upgrade is performed (this is the default behavior). Here are two examples: # Do the SSL connection upgrade only if the SNI field is set to "example.com" frontend l mode tcp bind *:1234 ssl crt /path/to/srv.pem defer-ssl-upgrade tcp-request inspect-delay 5s tcp-request content ssl-upgrade if { req.ssl_sni -i example.com } # by default no SSL upgrade is performed use_backend offloaded-ssl if ssl_fc default_backend raw-ssl # Accept SSL and non-SSL connctions on the same listener frontend l mode tcp bind *:1234 ssl crt /path/to/srv.pem defer-ssl-upgrade tcp-request inspect-delay 5s tcp-request content ssl-upgrade { req.ssl_ver gt 0 } use_backend ssl if ssl_fc default_backend tcp This feature is based on the work of PiBa-NL <[email protected]> --- doc/configuration.txt | 36 ++++++++++++++++- include/proto/ssl_sock.h | 1 + include/types/connection.h | 4 +- include/types/listener.h | 25 ++++++------ src/listener.c | 7 +++- src/proto_tcp.c | 99 ++++++++++++++++++++++++++++++++++++++++++++-- src/session.c | 6 +++ src/ssl_sock.c | 71 +++++++++++++++++++++++++++++++++ 8 files changed, 230 insertions(+), 19 deletions(-) diff --git a/doc/configuration.txt b/doc/configuration.txt index 279d076..d490afa 100644 --- a/doc/configuration.txt +++ b/doc/configuration.txt @@ -8534,7 +8534,7 @@ tcp-request connection <action> [{if | unless} <condition>] accept the incoming connection. There is no specific limit to the number of rules which may be inserted. - Four types of actions are supported : + Following actions are supported : - accept : accepts the connection if the condition is true (when used with "if") or false (when used with "unless"). The first such rule executed ends @@ -8646,6 +8646,19 @@ tcp-request connection <action> [{if | unless} <condition>] TCP reset doesn't pass the first router, though it's still delivered to local networks. Do not use it unless you fully understand how it works. + - "ssl-upgrade" : + When SSL upgrade of a TCP connection was postponed, this action + performs it. This is only available for connections opened on a SSL + listener configured with the "defer-ssl-upgrade" option and ignored for + all others. The next "ssl-upgrade" and "no-ssl-upgrade" rules will have + no effect. See also the bind option "defer-ssl-upgrade". + + - "no-ssl-upgrade" : + This does the opposite of the previous action. It explicitly disables + any postponed SSL upgrade. The next "ssl-upgrade" and "no-ssl-upgrade" + rules will have no effect. See also the bind option "defer-ssl-upgrade". + + Note that the "if/unless" condition is optional. If no condition is set on the action, it is simply performed unconditionally. That can be useful for "track-sc*" actions as well as for changing the default action to a reject. @@ -8718,6 +8731,8 @@ tcp-request content <action> [{if | unless} <condition>] - set-gpt0(<sc-id>) <int> - set-var(<var-name>) <expr> - silent-drop + - ssl-upgrade + - no-ssl-upgrade They have the same meaning as their counter-parts in "tcp-request connection" so please refer to that section for a complete description. @@ -8820,6 +8835,18 @@ tcp-request content <action> [{if | unless} <condition>] tcp-request content track-sc1 src tcp-request content reject if click_too_fast mark_as_abuser + Example: + # Do the SSL connection upgrade only if the SNI field is set + # to "example.com" + tcp-request inspect-delay 5s + tcp-request content ssl-upgrade if { req.ssl_sni -i example.com } + # no SSL upgrade is done if SNI field does not match + + Example: + # Handle SSL and non-SSL connctions on the same listener + tcp-request inspect-delay 5s + tcp-request content ssl-upgrade if { req.ssl_ver gt 0 } + See section 7 about ACL usage. See also : "tcp-request connection", "tcp-request inspect-delay" @@ -9797,6 +9824,13 @@ defer-accept an established connection while the proxy will only see it in SYN_RECV. This option is only supported on TCPv4/TCPv6 sockets and ignored by other ones. +defer-ssl-upgrade + This setting is only available when support for OpenSSL was built in. It + postpones the SSL upgrade of TCP connexions when evaluation of "tcp-request" + rules. SSL upgrade is performed only if a "ssl-upgrade" rule matches. So by + default, there is no SSL upgrade. + See also "tcp-request connection" and"tcp-request content". + force-sslv3 This option enforces use of SSLv3 only on SSL connections instantiated from this listener. SSLv3 is generally less expensive than the TLS counterparts diff --git a/include/proto/ssl_sock.h b/include/proto/ssl_sock.h index cb9a1e9..152aa6e 100644 --- a/include/proto/ssl_sock.h +++ b/include/proto/ssl_sock.h @@ -43,6 +43,7 @@ int ssl_sock_is_ssl(struct connection *conn) } int ssl_sock_handshake(struct connection *conn, unsigned int flag); +int ssl_sock_upgrade(struct connection *conn, struct buffer *buf); int ssl_sock_prepare_ctx(struct bind_conf *bind_conf, SSL_CTX *ctx, struct proxy *proxy); int ssl_sock_prepare_all_ctx(struct bind_conf *bind_conf, struct proxy *px); int ssl_sock_prepare_srv_ctx(struct server *srv, struct proxy *px); diff --git a/include/types/connection.h b/include/types/connection.h index dfbff6a..308abb7 100644 --- a/include/types/connection.h +++ b/include/types/connection.h @@ -107,7 +107,8 @@ enum { CO_FL_SEND_PROXY = 0x01000000, /* send a valid PROXY protocol header */ CO_FL_SSL_WAIT_HS = 0x02000000, /* wait for an SSL handshake to complete */ CO_FL_ACCEPT_PROXY = 0x04000000, /* receive a valid PROXY protocol header */ - /* unused : 0x08000000 */ + + CO_FL_DEF_SSL_UPG = 0x08000000, /* postpone the ssl upgrade of the connection */ /* below we have all handshake flags grouped into one */ CO_FL_HANDSHAKE = CO_FL_SEND_PROXY | CO_FL_SSL_WAIT_HS | CO_FL_ACCEPT_PROXY, @@ -263,6 +264,7 @@ struct connection { int fd; /* file descriptor for a stream driver when known */ } sock; } t; + enum obj_type *target; /* the target to connect to (server, proxy, applet, ...) */ struct list list; /* attach point to various connection lists (idle, ...) */ const struct netns_entry *proxy_netns; diff --git a/include/types/listener.h b/include/types/listener.h index 4da6cac..5249bd9 100644 --- a/include/types/listener.h +++ b/include/types/listener.h @@ -80,18 +80,19 @@ enum li_state { */ /* listener socket options */ -#define LI_O_NONE 0x0000 -#define LI_O_NOLINGER 0x0001 /* disable linger on this socket */ -#define LI_O_FOREIGN 0x0002 /* permit listening on foreing addresses */ -#define LI_O_NOQUICKACK 0x0004 /* disable quick ack of immediate data (linux) */ -#define LI_O_DEF_ACCEPT 0x0008 /* wait up to 1 second for data before accepting */ -#define LI_O_TCP_RULES 0x0010 /* run TCP rules checks on the incoming connection */ -#define LI_O_CHK_MONNET 0x0020 /* check the source against a monitor-net rule */ -#define LI_O_ACC_PROXY 0x0040 /* find the proxied address in the first request line */ -#define LI_O_UNLIMITED 0x0080 /* listener not subject to global limits (peers & stats socket) */ -#define LI_O_TCP_FO 0x0100 /* enable TCP Fast Open (linux >= 3.7) */ -#define LI_O_V6ONLY 0x0200 /* bind to IPv6 only on Linux >= 2.4.21 */ -#define LI_O_V4V6 0x0400 /* bind to IPv4/IPv6 on Linux >= 2.4.21 */ +#define LI_O_NONE 0x0000 +#define LI_O_NOLINGER 0x0001 /* disable linger on this socket */ +#define LI_O_FOREIGN 0x0002 /* permit listening on foreing addresses */ +#define LI_O_NOQUICKACK 0x0004 /* disable quick ack of immediate data (linux) */ +#define LI_O_DEF_ACCEPT 0x0008 /* wait up to 1 second for data before accepting */ +#define LI_O_TCP_RULES 0x0010 /* run TCP rules checks on the incoming connection */ +#define LI_O_CHK_MONNET 0x0020 /* check the source against a monitor-net rule */ +#define LI_O_ACC_PROXY 0x0040 /* find the proxied address in the first request line */ +#define LI_O_UNLIMITED 0x0080 /* listener not subject to global limits (peers & stats socket) */ +#define LI_O_TCP_FO 0x0100 /* enable TCP Fast Open (linux >= 3.7) */ +#define LI_O_V6ONLY 0x0200 /* bind to IPv6 only on Linux >= 2.4.21 */ +#define LI_O_V4V6 0x0400 /* bind to IPv4/IPv6 on Linux >= 2.4.21 */ +#define LI_O_DEF_SSL_UPG 0x0800 /* postpone SSL upgrade of the connection */ /* Note: if a listener uses LI_O_UNLIMITED, it is highly recommended that it adds its own * maxconn setting to the global.maxsock value so that its resources are reserved. diff --git a/src/listener.c b/src/listener.c index 3759c78..07594a9 100644 --- a/src/listener.c +++ b/src/listener.c @@ -495,8 +495,11 @@ void listener_accept(int fd) global.sps_max = global.sess_per_sec.curr_ctr; } #ifdef USE_OPENSSL - if (!(l->options & LI_O_UNLIMITED) && l->bind_conf && l->bind_conf->is_ssl) { - + /* if SSL upgrade is postponed, we do no update ssl/sec + * frequency now */ + if (!(l->options & LI_O_DEF_SSL_UPG) && + !(l->options & LI_O_UNLIMITED) && + l->bind_conf && l->bind_conf->is_ssl) { update_freq_ctr(&global.ssl_per_sec, 1); if (global.ssl_per_sec.curr_ctr > global.ssl_max) global.ssl_max = global.ssl_per_sec.curr_ctr; diff --git a/src/proto_tcp.c b/src/proto_tcp.c index cce0acb..fae5d29 100644 --- a/src/proto_tcp.c +++ b/src/proto_tcp.c @@ -61,6 +61,11 @@ #include <proto/stream_interface.h> #include <proto/task.h> +#ifdef USE_OPENSSL +#include <proto/ssl_sock.h> +#endif /*USE_OPENSSL */ + + static int tcp_bind_listeners(struct protocol *proto, char *errmsg, int errlen); static int tcp_bind_listener(struct listener *listener, char *errmsg, int errlen); @@ -1099,7 +1104,6 @@ int tcp_inspect_request(struct stream *s, struct channel *req, int an_bit) if (rule->cond->pol == ACL_COND_UNLESS) ret = !ret; } - if (ret) { act_flags |= ACT_FLAG_FIRST; resume_execution: @@ -1491,6 +1495,45 @@ static enum act_return tcp_exec_action_silent_drop(struct act_rule *rule, struct return ACT_RET_STOP; } +/* Executes the "ssl-upgrade" action. May be called from tcp-request */ +static enum act_return tcp_exec_action_ssl_upgrade(struct act_rule *rule, struct proxy *px, + struct session *sess, struct stream *strm, int flags) +{ + struct connection *conn = objt_conn(sess->origin); + enum act_return ret = ACT_RET_CONT; + +#ifdef USE_OPENSSL + if (!conn) + goto out; + if (!(conn->flags & CO_FL_DEF_SSL_UPG)) + goto out; + + ret = ACT_RET_ERR; + if (strm) { + if (ssl_sock_upgrade(conn, strm->req.buf)) + ret = ACT_RET_YIELD; + } + else { + if (ssl_sock_upgrade(conn, NULL)) + ret = ACT_RET_CONT; + } + out: +#endif /* USE_OPENSSL */ + return ret; +} + +/* Executes the "no-ssl-upgrade" action. May be called from tcp-request */ +static enum act_return tcp_exec_action_no_ssl_upgrade(struct act_rule *rule, struct proxy *px, + struct session *sess, struct stream *strm, int flags) +{ + struct connection *conn = objt_conn(sess->origin); + + if (conn) + conn->flags &= ~CO_FL_DEF_SSL_UPG; + return ACT_RET_CONT; +} + + /* Parse a tcp-response rule. Return a negative value in case of failure */ static int tcp_parse_response_rule(char **args, int arg, int section_type, struct proxy *curpx, struct proxy *defpx, @@ -2044,6 +2087,52 @@ static enum act_parse_ret tcp_parse_silent_drop(const char **args, int *orig_arg return ACT_RET_PRS_OK; } +/* Parse a "ssl-upgrade" action. It takes no argument. It returns ACT_RET_PRS_OK + * on success, ACT_RET_PRS_ERR on error. + */ +static enum act_parse_ret tcp_parse_ssl_upgrade(const char **args, int *orig_arg, struct proxy *px, + struct act_rule *rule, char **err) +{ +#ifdef USE_OPENSSL + if (px->mode == PR_MODE_HTTP) { + memprintf(err, + "'%s %s %s' : invalid rule as it requires TCP mode", + args[0], args[1], args[2]); + return ACT_RET_PRS_ERR; + } + + rule->action = ACT_CUSTOM; + rule->action_ptr = tcp_exec_action_ssl_upgrade; + return ACT_RET_PRS_OK; +#else + memprintf(err, "'%s %s %s' : not implemented", args[0], args[1], args[2]); + return ACT_RET_PRS_ERR; +#endif +} + +/* Parse a "no-ssl-upgrade" action. It takes no argument. It returns + * ACT_RET_PRS_OK on success, ACT_RET_PRS_ERR on error. + */ +static enum act_parse_ret tcp_parse_no_ssl_upgrade(const char **args, int *orig_arg, struct proxy *px, + struct act_rule *rule, char **err) +{ +#ifdef USE_OPENSSL + if (px->mode == PR_MODE_HTTP) { + memprintf(err, + "'%s %s %s' : invalid rule as it requires TCP mode", + args[0], args[1], args[2]); + return ACT_RET_PRS_ERR; + } + + rule->action = ACT_CUSTOM; + rule->action_ptr = tcp_exec_action_no_ssl_upgrade; + return ACT_RET_PRS_OK; +#else + memprintf(err, "'%s %s %s' : not implemented", args[0], args[1], args[2]); + return ACT_RET_PRS_ERR; +#endif +} + /************************************************************************/ /* All supported sample fetch functions must be declared here */ @@ -2423,12 +2512,16 @@ static struct srv_kw_list srv_kws = { "TCP", { }, { }}; static struct action_kw_list tcp_req_conn_actions = {ILH, { - { "silent-drop", tcp_parse_silent_drop }, + { "silent-drop", tcp_parse_silent_drop }, + { "ssl-upgrade", tcp_parse_ssl_upgrade }, + { "no-ssl-upgrade", tcp_parse_no_ssl_upgrade }, { /* END */ } }}; static struct action_kw_list tcp_req_cont_actions = {ILH, { - { "silent-drop", tcp_parse_silent_drop }, + { "silent-drop", tcp_parse_silent_drop }, + { "ssl-upgrade", tcp_parse_ssl_upgrade }, + { "no-ssl-upgrade", tcp_parse_no_ssl_upgrade }, { /* END */ } }}; diff --git a/src/session.c b/src/session.c index fdb2404..f80a0f9 100644 --- a/src/session.c +++ b/src/session.c @@ -136,6 +136,12 @@ int session_accept_fd(struct listener *l, int cfd, struct sockaddr_storage *addr conn_ctrl_init(cli_conn); + /* postpone ssl upgrade of the connection */ + if (l->options & LI_O_DEF_SSL_UPG) { + cli_conn->flags &= ~CO_FL_SSL_WAIT_HS; + cli_conn->flags |= CO_FL_DEF_SSL_UPG; + } + /* wait for a PROXY protocol header */ if (l->options & LI_O_ACC_PROXY) { cli_conn->flags |= CO_FL_ACCEPT_PROXY; diff --git a/src/ssl_sock.c b/src/ssl_sock.c index 1017388..11bfdf2 100644 --- a/src/ssl_sock.c +++ b/src/ssl_sock.c @@ -83,9 +83,11 @@ #include <proto/server.h> #include <proto/log.h> #include <proto/proxy.h> +#include <proto/raw_sock.h> #include <proto/shctx.h> #include <proto/ssl_sock.h> #include <proto/stream.h> +#include <proto/stream_interface.h> #include <proto/task.h> /* Warning, these are bits, not integers! */ @@ -3547,6 +3549,56 @@ reneg_ok: return 0; } +static long ssl_sock_upgrade_callback(BIO *bio, int cmd, const char *argp, + int argi, long argl, long ret) +{ + if ((cmd & (BIO_CB_READ|BIO_CB_RETURN)) == (BIO_CB_READ|BIO_CB_RETURN)) { + struct connection *conn = (struct connection *)BIO_get_callback_arg(bio); + if (ret >= 0) { + struct buffer *buf = si_ic(conn->owner)->buf; + bi_fast_delete(buf, ret); + if (buf->i == 0) + SSL_set_fd(conn->xprt_ctx, conn->t.sock.fd); + } + } + return ret; +} + +int ssl_sock_upgrade(struct connection *conn, struct buffer *buf) +{ + BIO *mbio; + + conn->xprt = &ssl_sock; + conn->flags &= ~CO_FL_XPRT_READY; + conn_sock_want_recv(conn); + conn_data_want_recv(conn); + if (conn_xprt_init(conn) < 0) { + conn->flags |= CO_FL_ERROR; + return 0; + } + + if (objt_listener(conn->target) && + !(objt_listener(conn->target)->options & LI_O_UNLIMITED)) { + update_freq_ctr(&global.ssl_per_sec, 1); + if (global.ssl_per_sec.curr_ctr > global.ssl_max) + global.ssl_max = global.ssl_per_sec.curr_ctr; + } + + conn->flags |= (CO_FL_SSL_WAIT_HS | CO_FL_WAIT_L6_CONN); + conn->flags &= ~(CO_FL_CONNECTED|CO_FL_DEF_SSL_UPG); + + if (buf && buf->i) { + mbio = BIO_new_mem_buf(buf->p, buf->i); + SSL_set_bio(conn->xprt_ctx, mbio, NULL); + SSL_set_wfd(conn->xprt_ctx, conn->t.sock.fd); + BIO_set_callback(mbio, ssl_sock_upgrade_callback); + BIO_set_callback_arg(mbio, (char *)conn); + } + fd_may_recv(conn->t.sock.fd); + + return 1; +} + /* Receive up to <count> bytes from connection <conn>'s socket and store them * into buffer <buf>. Only one call to recv() is performed, unless the * buffer wraps, in which case a second call may be performed. The connection's @@ -5446,6 +5498,24 @@ static int bind_parse_verify(char **args, int cur_arg, struct proxy *px, struct return 0; } +/* parse the "defer-ssl-upgrade" bind keyword */ +static int bind_parse_def_ssl_upg(char **args, int cur_arg, struct proxy *px, struct bind_conf *conf, char **err) +{ + struct listener *l; + + if (!conf->is_ssl) { + if (err) + memprintf(err, "'%s' : must be declared after 'ssl' keyword", args[cur_arg]); + return ERR_ALERT | ERR_FATAL; + } + + list_for_each_entry(l, &conf->listeners, by_bind) { + l->xprt = &raw_sock; + l->options |= LI_O_DEF_SSL_UPG; + } + return 0; +} + /************** "server" keywords ****************/ /* parse the "ca-file" server keyword */ @@ -5892,6 +5962,7 @@ static struct bind_kw_list bind_kws = { "SSL", { }, { { "crt", bind_parse_crt, 1 }, /* load SSL certificates from this location */ { "crt-ignore-err", bind_parse_ignore_err, 1 }, /* set error IDs to ingore on verify depth == 0 */ { "crt-list", bind_parse_crt_list, 1 }, /* load a list of crt from this location */ + { "defer-ssl-upgrade", bind_parse_def_ssl_upg, 0 }, /* Defer the SSL connection upgrade */ { "ecdhe", bind_parse_ecdhe, 1 }, /* defines named curve for elliptic curve Diffie-Hellman */ { "force-sslv3", bind_parse_force_sslv3, 0 }, /* force SSLv3 */ { "force-tlsv10", bind_parse_force_tlsv10, 0 }, /* force TLSv10 */ -- 2.5.0
