Re: TLS communication director -> backend with X.509 cert checks?

[Timo Sirainen]

> On 15 Oct 2015, at 00:28, Heiko Schlittermann  wrote:
> 
> Hi Timo
> 
> Heiko Schlittermann  (Mi 14 Okt 2015 01:10:20 CEST):
> …
>> Ah, the information comes from the other director running. The other one
>> is using an unpatched version of dovecot.
> 
> Your patch for backend-certificate verification works. Thank you for the
> good and fast work. Is there any chance that this will make it into
> Dovecot's next release?

Implemented also support for sending the hostname within director ring:

http://hg.dovecot.org/dovecot-2.2/rev/8e9cada0c8fc
http://hg.dovecot.org/dovecot-2.2/rev/7f718c840aff
http://hg.dovecot.org/dovecot-2.2/rev/5876ca2d63fb

Although it's not possible right now to add hostname using "doveadm director 
add", so that probably needs to be implemented at some point.

> BTW: The ambiguity of 2001:db8::9090 remains. Shouldn't you allow
> [2001:db8::]¹ resp [2001:db8::9090]¹ resp. [2001:db8::]:9090² for such
> cases? (In case one want's to use IPv6 addresses instead of hostnames in
> the director_servers option. (And probably in other places too.))

http://hg.dovecot.org/dovecot-2.2/rev/c5c34c02fda3

Re: TLS communication director -> backend with X.509 cert checks?

[Heiko Schlittermann]

Hi Timo

Heiko Schlittermann  (Mi 14 Okt 2015 01:10:20 CEST):
…
> Ah, the information comes from the other director running. The other one
> is using an unpatched version of dovecot.

Your patch for backend-certificate verification works. Thank you for the
good and fast work. Is there any chance that this will make it into
Dovecot's next release?

BTW: The ambiguity of 2001:db8::9090 remains. Shouldn't you allow
[2001:db8::]¹ resp [2001:db8::9090]¹ resp. [2001:db8::]:9090² for such
cases? (In case one want's to use IPv6 addresses instead of hostnames in
the director_servers option. (And probably in other places too.))

¹) Address
²) Address:Port

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
-- 
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -
 ! key id 7CBF764A and 972EAC9F are revoked since 2015-01  -


signature.asc
Description: Digital signature

Re: TLS communication director -> backend with X.509 cert checks?

[Timo Sirainen]

On 13 Oct 2015, at 22:18, Heiko Schlittermann  wrote:
> 
> Timo Sirainen  (Di 13 Okt 2015 21:02:59 CEST):
> …
>>> On connection setup from a client the director connects to the
>>> selected backend. But it seems (not checked in the source yet),
>>> that for SSL certificate verification the director doesn't know the
>>> original host name anymore. The certificate's CN gets compared to
>>> the IP address the director connects to.
>> 
>> Right. The hostnames are lost immediately at director startup. I've never 
>> really thought about needing this functionality for director, since they're 
>> usually in the same trusted network with backends..
>> 
> 
> That's it… "ususally". And additionally local policy says that we should use
> secured connections whenever credentials are transported … And since the
> director uses either a master password or the credentials obtained from
> the client, we want to use secured connections. And using TLS w/o
> verified certs is better than nothing, but it's far from being perfect.

I've been planning to add support for non-plaintext SASL for Dovecot proxy. 
Probably SCRAM-SHA1. That would avoid sending credentials in plaintext, 
although it wouldn't prevent other kind of MITM.

> I see:
> 
>a) pass the host *names* to the director too, for CN verification
>   purpose
> 
>   May be in struct mail_host could be a field for the original
>   hostname we used to obtain the adress(es)?

Does the attached patch work? Compiles, but untested.



director-host.diff
Description: Binary data

Re: TLS communication director -> backend with X.509 cert checks?

[Heiko Schlittermann]

Timo Sirainen  (Di 13 Okt 2015 21:36:40 CEST):
…
> > I see:
> > 
> >a) pass the host *names* to the director too, for CN verification
> >   purpose
> > 
> >   May be in struct mail_host could be a field for the original
> >   hostname we used to obtain the adress(es)?
> 
> Does the attached patch work? Compiles, but untested.

I'm about to test it.

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
-- 
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -
 ! key id 7CBF764A and 972EAC9F are revoked since 2015-01  -


signature.asc
Description: Digital signature

Re: TLS communication director -> backend with X.509 cert checks?

[Heiko Schlittermann]

Timo Sirainen  (Di 13 Okt 2015 21:02:59 CEST):
> > the IP address the director connects to.
> 
> Right. The hostnames are lost immediately at director startup. I've never 
> really thought about needing this functionality for director, since they're 
> usually in the same trusted network with backends..
> 

Ooo.
What if 

director_mail_servers = backends.

and the DNS entry for backends. gets updated? Does the director
catch up the change automatically w/o restart?

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
-- 
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -
 ! key id 7CBF764A and 972EAC9F are revoked since 2015-01  -


signature.asc
Description: Digital signature

Re: TLS communication director -> backend with X.509 cert checks?

[Heiko Schlittermann]

Timo Sirainen  (Di 13 Okt 2015 21:02:59 CEST):
…
> > On connection setup from a client the director connects to the
> > selected backend. But it seems (not checked in the source yet),
> > that for SSL certificate verification the director doesn't know the
> > original host name anymore. The certificate's CN gets compared to
> > the IP address the director connects to.
> 
> Right. The hostnames are lost immediately at director startup. I've never 
> really thought about needing this functionality for director, since they're 
> usually in the same trusted network with backends..
> 

That's it… "ususally". And additionally local policy says that we should use
secured connections whenever credentials are transported … And since the
director uses either a master password or the credentials obtained from
the client, we want to use secured connections. And using TLS w/o
verified certs is better than nothing, but it's far from being perfect.

I see:

a) pass the host *names* to the director too, for CN verification
   purpose

   May be in struct mail_host could be a field for the original
   hostname we used to obtain the adress(es)?

or
b) allow some kind of certificate pinning, that is loose the implied
   relation CN <=> hostname

> > Should I create certificates with IP address in SAN? (Any hint about the
> > correct syntax for the openssl.conf is welcome). Or is there any chance
> > that this is fixed already or will be fixed in the near future or even
> > better, that it's my fault?
> 
> I guess that could work for now. No idea about how to do such certificates.

I'll try that, but I think it's not a solution as soon as we reach out
for "official" certs. And because it puts more details about the
infrastructure into the configuration than would be necessary.

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
-- 
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -
 ! key id 7CBF764A and 972EAC9F are revoked since 2015-01  -


signature.asc
Description: Digital signature

Re: TLS communication director -> backend with X.509 cert checks?

[Timo Sirainen]

> On 13 Oct 2015, at 22:21, Heiko Schlittermann  wrote:
> 
> Timo Sirainen  (Di 13 Okt 2015 21:02:59 CEST):
>>> the IP address the director connects to.
>> 
>> Right. The hostnames are lost immediately at director startup. I've never 
>> really thought about needing this functionality for director, since they're 
>> usually in the same trusted network with backends..
>> 
> 
> Ooo.
> What if 
> 
>director_mail_servers = backends.
> 
> and the DNS entry for backends. gets updated? Does the director
> catch up the change automatically w/o restart?

No, and I'm not sure it even should. Use "doveadm director ring remove" to get 
rid of unwanted directors.

Re: TLS communication director -> backend with X.509 cert checks?

[Timo Sirainen]

On 13 Oct 2015, at 21:44, Heiko Schlittermann  wrote:
> 
> Hello,
> 
> using Dovecot 2.2.9 and a setup with directors and backends.
> The communication between directors and backends needs to be TLS
> secured.
> 
> The director config contains a list of hostnames for the backends.
> (implicit list because of multiple A/ records for a single hostname
> or explicit list of several host names)
> 
> On connection setup from a client the director connects to the
> selected backend. But it seems (not checked in the source yet),
> that for SSL certificate verification the director doesn't know the
> original host name anymore. The certificate's CN gets compared to
> the IP address the director connects to.

Right. The hostnames are lost immediately at director startup. I've never 
really thought about needing this functionality for director, since they're 
usually in the same trusted network with backends..

> Should I create certificates with IP address in SAN? (Any hint about the
> correct syntax for the openssl.conf is welcome). Or is there any chance
> that this is fixed already or will be fixed in the near future or even
> better, that it's my fault?

I guess that could work for now. No idea about how to do such certificates.

TLS communication director -> backend with X.509 cert checks?

[Heiko Schlittermann]

Hello,

using Dovecot 2.2.9 and a setup with directors and backends.
The communication between directors and backends needs to be TLS
secured.

The director config contains a list of hostnames for the backends.
(implicit list because of multiple A/ records for a single hostname
or explicit list of several host names)

On connection setup from a client the director connects to the
selected backend. But it seems (not checked in the source yet),
that for SSL certificate verification the director doesn't know the
original host name anymore. The certificate's CN gets compared to
the IP address the director connects to.

Oct 12 23:56:51 director2 dovecot: director: Error: 
director(2001:683:921:f33::5:1:9090/out): connect() failed: Connection reset by 
peer
Oct 12 23:57:53 director2 dovecot: imap-login: Error: proxy: hostname doesn't 
match SSL certificate at 2001:683:921:f33::5:fe:993: user=, method=PLAIN, 
rip=2001:638:912:f33::1:1, lip=2001:638:912:f33::5:2, TLS, 
session=

In 10-directors.conf I've:
director_mail_servers = backend1. backend2.

Should I create certificates with IP address in SAN? (Any hint about the
correct syntax for the openssl.conf is welcome). Or is there any chance
that this is fixed already or will be fixed in the near future or even
better, that it's my fault?

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
-- 
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -
 ! key id 7CBF764A and 972EAC9F are revoked since 2015-01  -


signature.asc
Description: Digital signature

Re: TLS communication director -> backend with X.509 cert checks?

[Heiko Schlittermann]

Timo Sirainen  (Di 13 Okt 2015 23:49:20 CEST):
…
> 
> Proxying in general does check that hostname matches the SSL certificate, 
> because both the hostname and IP address are sent to login process. So it 
> should work in a way that host= and hostip= is sent. I thought 
> my patch did that.. Normally auth_debug=yes would be enough to debug this, 
> but this happens between director and login process so I don't think it's 
> going to be of much use. login process's client_auth_parse_args() is what 
> should see these two parameters correctly.
> 
> I can check this further tomorrow.

I've put an i_warning("*** %s: ...", __FUNCTION__, ...) into several places.

Oct 14 00:02:33 director1 dovecot: director: Warning: *** login_host_callback: 
OK#0112#011user=foo#011proxy#011ssl=yes#011nopassword=y#011lip=2001:x.y:f33::5:1#011lport=993#011pass=x#011proxy_refresh=450#011host=2001:x.y:f33::5:fe

Here it seems that the director doesn't send it's knowledge about the
hostname.

Here some other output, to show that the host list contains names and addresses:

Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added 
backends. [2001:x.y:f33::5:fe]
Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added 
backends. [2001:x.y:f33::5:ff]
Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added 
backends. [149.x.y.103]
Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added 
backends. [149.x.y.102]


-- 
Heiko


signature.asc
Description: Digital signature

Re: TLS communication director -> backend with X.509 cert checks?

[Heiko Schlittermann]

Hi Timo,

Heiko Schlittermann  (Di 13 Okt 2015 22:33:23 CEST):
> > Does the attached patch work? Compiles, but untested.
> I'm about to test it.

It seems to update the struct mail_host, but it looks as if the data 
in mail_host do not propagate down to login_proxy_new(). 

In other words, in login_proxy_new() set->host contains the IP address,
correctly, because the director choose it, but where can I find the
hostname there? And we need a way to pass the host*name* further, to the
SSL verifcation step, don't we?

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
-- 
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -
 ! key id 7CBF764A and 972EAC9F are revoked since 2015-01  -


signature.asc
Description: Digital signature

Re: TLS communication director -> backend with X.509 cert checks?

[Timo Sirainen]

On 14 Oct 2015, at 00:34, Heiko Schlittermann  wrote:
> 
> Hi Timo,
> 
> Heiko Schlittermann  (Di 13 Okt 2015 22:33:23 CEST):
>>> Does the attached patch work? Compiles, but untested.
>> I'm about to test it.
> 
> It seems to update the struct mail_host, but it looks as if the data 
> in mail_host do not propagate down to login_proxy_new(). 
> 
> In other words, in login_proxy_new() set->host contains the IP address,
> correctly, because the director choose it, but where can I find the
> hostname there? And we need a way to pass the host*name* further, to the
> SSL verifcation step, don't we?

Proxying in general does check that hostname matches the SSL certificate, 
because both the hostname and IP address are sent to login process. So it 
should work in a way that host= and hostip= is sent. I thought my 
patch did that.. Normally auth_debug=yes would be enough to debug this, but 
this happens between director and login process so I don't think it's going to 
be of much use. login process's client_auth_parse_args() is what should see 
these two parameters correctly.

I can check this further tomorrow.

Re: TLS communication director -> backend with X.509 cert checks?

[Heiko Schlittermann]

Heiko Schlittermann  (Mi 14 Okt 2015 00:10:50 CEST):
> Timo Sirainen  (Di 13 Okt 2015 23:49:20 CEST):
> …
> > 
> > Proxying in general does check that hostname matches the SSL certificate, 
> > because both the hostname and IP address are sent to login process. So it 
> > should work in a way that host= and hostip= is sent. I 
> > thought my patch did that.. Normally auth_debug=yes would be enough to 
> > debug this, but this happens between director and login process so I don't 
> > think it's going to be of much use. login process's 
> > client_auth_parse_args() is what should see these two parameters correctly.
> > 
> > I can check this further tomorrow.
> 
> I've put an i_warning("*** %s: ...", __FUNCTION__, ...) into several places.
> 
> Oct 14 00:02:33 director1 dovecot: director: Warning: *** 
> login_host_callback: 
> OK#0112#011user=foo#011proxy#011ssl=yes#011nopassword=y#011lip=2001:x.y:f33::5:1#011lport=993#011pass=x#011proxy_refresh=450#011host=2001:x.y:f33::5:fe
> 
> Here it seems that the director doesn't send it's knowledge about the
> hostname.
> 
> Here some other output, to show that the host list contains names and 
> addresses:
> 
> Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added 
> backends. [2001:x.y:f33::5:fe]
> Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added 
> backends. [2001:x.y:f33::5:ff]
> Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added 
> backends. [149.x.y.103]
> Oct 14 00:02:32 director1 dovecot: director: Warning: ** mail_host_add: added 
> backends. [149.x.y.102]

And if I add -D to the director service, I can see "Debug: request  
refreshed timeout to …",
but never I see "Debug: request  added".  And from what I
understand this would be the place where the mail_host info comes into
the game. 

But probably I do not understand how director_request_continue() is
supposed to work.

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
-- 
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -
 ! key id 7CBF764A and 972EAC9F are revoked since 2015-01  -


signature.asc
Description: Digital signature

Re: TLS communication director -> backend with X.509 cert checks?

[Heiko Schlittermann]

Heiko Schlittermann  (Mi 14 Okt 2015 00:46:11 CEST):
…
> 
> And if I add -D to the director service, I can see "Debug: request  
> refreshed timeout to …",
> but never I see "Debug: request  added".  And from what I
> understand this would be the place where the mail_host info comes into
> the game. 
> 
> But probably I do not understand how director_request_continue() is
> supposed to work.

Ah, the information comes from the other director running. The other one
is using an unpatched version of dovecot.

If I shutdown the other director instance, it seems to work.
Tomorrow I'll do more testing. Good work, thank you.

BTW: I've put there an IPv6 address into the director_servers list (not
an DNS name). 

director_servers = 2001:x:y:f33::5:1
…
inet_listener {
address = ::
port = 9090
}

it doesn't recognize itself: 
Oct 14 01:06:13 director1 dovecot: director: Fatal: director_servers doesn't 
list ourself

director_servers = 2001:x:y:f33::5:1:9090
…
inet_listener {
address = ::
port = 9090
}

works, but is ambigous, isn't it? Shouldn't we use [2001:x:y:f33::5:1]:9090
in such a case? But: *Unknown director host: [2001:x:y:f33::5:1]*

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
-- 
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -
 ! key id 7CBF764A and 972EAC9F are revoked since 2015-01  -


signature.asc
Description: Digital signature