Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp

[Graham Leggett]

On 12 Mar 2016, at 5:46 PM, Graham Leggett  wrote:

> The following patch provides support for TCP proxying to httpd.

Here is an updated patch with changes to the ap_mpm_register_poll* functions.

Regards,
Graham
--



httpd-tcp-proxy2.patch
Description: Binary data

Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp

[Graham Leggett]

On 14 Mar 2016, at 11:01 AM, Yann Ylavic  wrote:

>> The following patch provides support for TCP proxying to httpd.
>> 
>> It consists of the following three parts:
>> 
>> - mod_tcp: Allows the frontend to receive pure TCP connections
> 
> It looks like this module is only needed to remove HTTP filters from the 
> chain.
> Is the goal to have this core module instead of mod_http and make the
> latter dynamic?

Hmmm - good point.

What we need next is a proper protocol handling mechanism to efficiently 
determine the protocol in use on the connection, the same way we can 
efficiently determine the HTTP method.

Once we have that the core can be free of HTTP modules and we can just use the 
mod_tcp process_connection() handler.

>> - mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a 
>> backend
> 
> Thanks, this will be very useful.
> 
>> - mod_ssl_tcp: Allows the proxy to route incoming connections based on the 
>> SNI header (tlsext)
> 
> Hmm, isn't mod_ssl (underlying-)protocol agnostic?
> Why couldn't it be used as-is (or adapted), and avoid code duplication?

It was like that to start with, but I split it all out so it could stand alone.

I see the value of mod_ssl just having this as an extra input filter, will 
simplify this.

>> In the following example config, incoming TCP connections are routed based 
>> on their SNI (the tlsext protocol) to given backend servers, which then 
>> complete the SSL connections as raw tunnels.
>> 
>> This allows you to use client certificates through the httpd proxy balancer 
>> all the way to the backend server without the proxy terminating any SSL 
>> along the way.
>> 
>> 
>>  Protocol tlsext
> 
> Maybe "tcps"? I agree that SNI extension is needed, but "tlsext" looks
> confusing.

The “tlsext” refers to the TLS extentions which are parsed to determine what 
the client is trying to talk to. These extensions are SNI and APLN (not yet 
supported but would be great if we could).

“tcps” implies “tcp over ssl”, which we already can do - just turn on SSLEnable.

Regards,
Graham
—

Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp

[Yann Ylavic]

On Sat, Mar 12, 2016 at 4:46 PM, Graham Leggett  wrote:
>
> The following patch provides support for TCP proxying to httpd.
>
> It consists of the following three parts:
>
> - mod_tcp: Allows the frontend to receive pure TCP connections

It looks like this module is only needed to remove HTTP filters from the chain.
Is the goal to have this core module instead of mod_http and make the
latter dynamic?

> - mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a 
> backend

Thanks, this will be very useful.

> - mod_ssl_tcp: Allows the proxy to route incoming connections based on the 
> SNI header (tlsext)

Hmm, isn't mod_ssl (underlying-)protocol agnostic?
Why couldn't it be used as-is (or adapted), and avoid code duplication?

>
> In the following example config, incoming TCP connections are routed based on 
> their SNI (the tlsext protocol) to given backend servers, which then complete 
> the SSL connections as raw tunnels.
>
> This allows you to use client certificates through the httpd proxy balancer 
> all the way to the backend server without the proxy terminating any SSL along 
> the way.
>
> 
>   Protocol tlsext

Maybe "tcps"? I agree that SNI extension is needed, but "tlsext" looks
confusing.

I'll look at the patch in more details, I may have missed things...

Regards,
Yann.

Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp

[Daniel Ruggeri]

+1
Really nice work

-- 
Daniel Ruggeri

On 3/13/2016 10:45 AM, Jim Jagielski wrote:
> I've given it a quick look-thru and I. Am. Impressed.
>
> This is more Super Cool Mojo!

Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp

[Jim Jagielski]

I've given it a quick look-thru and I. Am. Impressed.

This is more Super Cool Mojo!

> On Mar 12, 2016, at 10:46 AM, Graham Leggett  wrote:
> 
> Hi all,
> 
> The following patch provides support for TCP proxying to httpd.
> 
> It consists of the following three parts:
> 
> - mod_tcp: Allows the frontend to receive pure TCP connections
> - mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a 
> backend
> - mod_ssl_tcp: Allows the proxy to route incoming connections based on the 
> SNI header (tlsext)
> 
> In the following example config, incoming TCP connections are routed based on 
> their SNI (the tlsext protocol) to given backend servers, which then complete 
> the SSL connections as raw tunnels.
> 
> This allows you to use client certificates through the httpd proxy balancer 
> all the way to the backend server without the proxy terminating any SSL along 
> the way.
> 
> 
>  Protocol tlsext
> 
>  ServerName jira.example.com
> 
>  ProxyPass / tcp://john.example.com:443
> 
> 
> 
>  Protocol tlsext
> 
>  ServerName www.example.com
> 
>  ProxyPass / tcp://erica.example.com:443
> 
> 
> In order for mod_ssl_tcp to work, it needs to read ahead to see if any client 
> hello message is present, and then set aside any extra data so it could be 
> read again. This is fundamentally incompatible with c->data_in_input_filters 
> which only allows the core filter to set aside unread data. For this reason 
> the ability to set aside data was rolled out to all filters.
> 
> mod_ssl_tcp just cares about SNI for now, but could conceivably support APLN 
> too, making a configuration something like this:
> 
> 
>  Protocol tlsext
>  ServerName secure.example.com
>  
>ProxyPass / tcp://imap.example.com:993
>  
>  
>ProxyPass / tcp://pop3.example.com:995
>  
> 
> 
> Regards,
> Graham
> --
> 
> 

Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp

[Graham Leggett]

On 12 Mar 2016, at 6:26 PM, Eric Covener  wrote:

> Very cool stuff. Only looked on the surface so far, but one thing it
> reminded me of in the async case is that the MPM knows the "suspended"
> count but we never gave an API to really "unsuspend" when using a
> timed or socket callback.  Longer term we really need to find a way to
> share that raw forwarding code as it's now probably in three modules.

I am slowly working through this, teasing out each problem that prevents async 
behaviour.

A key part of mod_tcp is the “switch ladder”, which allows 
tcp_process_connection() to exit at any time and be called again, at which 
point it will continue where it left off. We need corresponding ladders deeper 
into the core to make each layer down able to continue where it left off if it 
got EAGAIN. We also need to fix some of the function signatures - 
ap_process_request() is void, and so cannot signal to us elegantly that it 
wants to be called again, or whether it is done and we can move on.

Another problem that has become apparent is that the MPMs are too limiting with 
respect to the events the core is willing to wait on. Right now the only event 
the core will handle for us is to wait for the underlying socket to be 
readable/writable. What I have in mind is a more general mechanism, where 
instead of “one socket” we have “a stack of sockets/pipes/files[1]”, and the 
event we wait for is the socket/pipe/file on the top of the stack. This gives 
us AND behaviour. If browser socket is writable AND backend socket is readable 
then [do stuff].

To state it another way, we wait for the browser socket to be writable, this 
triggers the normal pattern of events until we need to block on something else, 
for example we might block waiting for the backend proxy socket to be readable. 
That socket is added to the stack and the MPM now waits for that event. When 
that event triggers the top of the stack is popped off and we’re back to 
waiting on the underlying event (the socket).

I just need to find an elegant way to trigger all of this. It would be really 
nice if we had SOCKET, PIPE, FILE (etc) buckets that pushed themselves onto 
this stack as soon as they returned EAGAIN via some kind of callback.

I need to think this through to work out how to incorporate OR into this. 
Perhaps a list of stacks. This would allow a set of read events to run 
concurrently with write events. Then the sense of the connection has to be 
worked into it too so mod_ssl works non blocking.

[1] files are typically not pollable, but we shouldn't restrict our API. It’s 
pipes we want to block on and in our world that’s apr_file_t.

Regards,
Graham
—

Re: [Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp

[Eric Covener]

On Sat, Mar 12, 2016 at 10:46 AM, Graham Leggett  wrote:
> The following patch provides support for TCP proxying to httpd.
>
> It consists of the following three parts:
>
> - mod_tcp: Allows the frontend to receive pure TCP connections
> - mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a 
> backend
> - mod_ssl_tcp: Allows the proxy to route incoming connections based on the 
> SNI header (tlsext)


Very cool stuff. Only looked on the surface so far, but one thing it
reminded me of in the async case is that the MPM knows the "suspended"
count but we never gave an API to really "unsuspend" when using a
timed or socket callback.  Longer term we really need to find a way to
share that raw forwarding code as it's now probably in three modules.

[Patch] mod_tcp / mod_proxy_tcp / mod_ssl_tcp

[Graham Leggett]

Hi all,

The following patch provides support for TCP proxying to httpd.

It consists of the following three parts:

- mod_tcp: Allows the frontend to receive pure TCP connections
- mod_proxy_tcp: Allows the proxy to make pure tcp or tls connections to a 
backend
- mod_ssl_tcp: Allows the proxy to route incoming connections based on the SNI 
header (tlsext)

In the following example config, incoming TCP connections are routed based on 
their SNI (the tlsext protocol) to given backend servers, which then complete 
the SSL connections as raw tunnels.

This allows you to use client certificates through the httpd proxy balancer all 
the way to the backend server without the proxy terminating any SSL along the 
way.


  Protocol tlsext

  ServerName jira.example.com

  ProxyPass / tcp://john.example.com:443



  Protocol tlsext

  ServerName www.example.com

  ProxyPass / tcp://erica.example.com:443


In order for mod_ssl_tcp to work, it needs to read ahead to see if any client 
hello message is present, and then set aside any extra data so it could be read 
again. This is fundamentally incompatible with c->data_in_input_filters which 
only allows the core filter to set aside unread data. For this reason the 
ability to set aside data was rolled out to all filters.

mod_ssl_tcp just cares about SNI for now, but could conceivably support APLN 
too, making a configuration something like this:


  Protocol tlsext
  ServerName secure.example.com
  
ProxyPass / tcp://imap.example.com:993
  
  
ProxyPass / tcp://pop3.example.com:995
  


Regards,
Graham
--



httpd-tcp-proxy.patch
Description: Binary data