[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

Jeremy Harris  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|ASSIGNED|RESOLVED

--- Comment #12 from Jeremy Harris  ---
Lacking any info, closing on the assumtion it's fixed.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

Git Commit  changed:

   What|Removed |Added

 CC||g...@exim.org

--- Comment #11 from Git Commit  ---
Git commit:
https://git.exim.org/exim.git/commitdiff/b7d3afcfad94edf99a8dbc50ab670ded417e6bea

commit b7d3afcfad94edf99a8dbc50ab670ded417e6bea
Author: Jeremy Harris 
AuthorDate: Sun Apr 30 22:11:27 2017 +0100
Commit: Jeremy Harris 
CommitDate: Sun Apr 30 22:57:21 2017 +0100

fix continue_more on tls connection.  bug 2104

 doc/doc-txt/ChangeLog  |  8 ++
 src/src/deliver.c  |  5 +++-
 src/src/transport.c|  8 --
 src/src/transports/smtp.c  | 37 +++-
 test/confs/2038| 63 +
 test/confs/2138| 64 ++
 test/log/2038  | 41 +++
 test/log/2138  | 41 +++
 test/mail/2038.userx   | 17 +++
 test/mail/2038.userx1  | 17 +++
 test/mail/2038.usery   | 17 +++
 test/mail/2038.usery1  | 17 +++
 test/mail/2038.userz   | 17 +++
 test/mail/2038.userz1  | 17 +++
 test/mail/2138.userx   | 17 +++
 test/mail/2138.userx1  | 17 +++
 test/mail/2138.usery   | 17 +++
 test/mail/2138.usery1  | 17 +++
 test/mail/2138.userz   | 17 +++
 test/mail/2138.userz1  | 17 +++
 test/scripts/2000-GnuTLS/2038  | 18 
 test/scripts/2100-OpenSSL/2138 | 17 +++
 22 files changed, 489 insertions(+), 17 deletions(-)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

--- Comment #10 from Jeremy Harris  ---
continue_more is set when we know some later messages will be sent to the same
host.  Given that, we need to close down TLS if we'll be using a new process.
I think we do, but a careful check of the debug output (which I've not seen
yet)
should tell.  If we do, this fix is about right (except that the devel HEAD has
introduced some complexity in that area of code).

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

--- Comment #9 from Wolfgang Breyha  ---
Created attachment 1015
  --> https://bugs.exim.org/attachment.cgi?id=1015=edit
is it that simple?

Assuming that even in this case (continue_more == 1, first_addr == NULL) TLS
has to be shut down I think the fix is as simple as moving the return down a
bit. But I have to admit that I'm not sure if this triggers unwanted
sideeffects.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

--- Comment #8 from Wolfgang Breyha  ---
More precisely it is the case
continue_more = 1 && first_addr == NULL

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

--- Comment #7 from Wolfgang Breyha  ---
Currently I think that at least 4.87 is affected as well. It looks like it
fails if continue_more=1 is set in smtp_deliver(). All 5 debug logs I have
written show this pattern as well.

If it is true smtp_deliver() returns just before shutting down TLS and
reissuing the EHLO (exim-4.89/src/transports/smtp.c:3212). But
smtp_setup_conn() (or the older version in smtp_deliver() in 4.87)
unconditionally sends the STARTTLS if TLS is requested.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

--- Comment #6 from Wolfgang Breyha  ---
right. I already noticed that while sending it to you.

Meanwhile "hosts_nopass_tls" cleared my queue. I removed it again. Will check
disabled CHUNKING as soon as possible.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

--- Comment #5 from Jeremy Harris  ---
Looking closer, that logging does not show that the STARTTLS wasn't done by a
new process.  The process doing the logging isn't always the one taking an
action.  Check the PIDs in the debug trace for better info.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

Jeremy Harris  changed:

   What|Removed |Added

 Status|NEW |ASSIGNED

--- Comment #4 from Jeremy Harris  ---
Hmpf.  Perhaps I broke something in 4.89.  Did you get a debug trace of this
run?  I'd like to see it.

I note from the server response for the first message that CHUNKING was used,
so you could also check to see if disabling that affects the symptom.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

--- Comment #3 from Wolfgang Breyha  ---
It happend again today on one host and I was able to capture a full session of
"exim -d -M " with tcpdump. 

To simplify things I limited max_remote_parallel to 1 (usually 3) and cipher to
a non PFS one.

In the end I see the SSL decoded 
"S: 250  14047 byte chunk, total 15292 | 250 OK id=1d2byJ-0006GB-5y"
an the next packet is the cleartext 
"C: STARTTLS" on the still encrypted channel.

what's interesting ... this connection was *not* handed over to another
process. It was reused by the same pid... which failed:
Apr 24 13:15:59 jarvis exim[19975]: 1d2bxU-0004L3-Fz => xxx...@.de
<...@univie.ac.at> R=smart_host T=remote_smtp M8D=8/8
H=zidrelay.univie.ac.at [131.130.3.115]* X=TLSv1.2:AES128-GCM-SHA256:128 CV=yes
DN="/C=AT/ST=Vienna/L=Wien/O=Universit\xC3\xA4t
Wien/OU=ZID/CN=grace.univie.ac.at" K C="250- 14047 byte chunk, total
15292\\n250 OK id=1d2byJ-0006GB-5y" QT=51s DT=0s
Apr 24 13:15:59 jarvis exim[19975]: 1d2bxU-0004L3-Fz == xxx...@.de
R=smart_host T=remote_smtp defer (-18) H=zidrelay.univie.ac.at [131.130.3.115]:
Remote host closed connection in response to STARTTLS

So, IMO this happens if an email
*) has multiple recipients
*) which are (mostly) redirected
*) and "X-Resent-For" is added

So every redirected e-mail becomes a new message. It seems that this process
tries to issue another STARTTLS on the already encrypted socket.

Surprisingly
hosts_nopass_tls 
seems to influence this case as well, since even the same sending exim shuts
down it's connection completely then and reconnects succeeding with STARTTLS as
well.

I can provide the full packet dump if needed on a more private channel.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

Re: [exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[Jeremy Harris]

On 23/04/17 10:52, Phil Pennock wrote:
> Jeremy is doing the hard work of trying to change this, partially based
> on a crazy idea of mine; this is not yet released, it's part of Exim
> 4.90 (I think J slightly mis-spoke in his phrasing around 4.89).

I should have said "Up to and including 4.89".
-- 
Cheers,
  Jeremy

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

Re: [exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[Phil Pennock]

On 2017-04-23 at 00:55 -0400, Viktor Dukhovni wrote:
> I find that rather perplexing.  Over a single TCP connection it is not valid
> to issue a second STARTTLS.  I am misreading the above?

If you tear down TLS, the standards are silent about what happens next.

Exim's current model, which Jeremy is working on changing, has all TLS
state in the process which delivers one message.  Delivering multiple
messages in one connection requires passing an open file descriptor to
another Unix process, but TLS libraries generally don't make their
internal state serializable for passing around in such a manner, so Exim
has no choice but to tear down TLS and see if the remote server is happy
to have TLS re-established again.  For some remote MTAs it works, for
others it doesn't.

Jeremy is doing the hard work of trying to change this, partially based
on a crazy idea of mine; this is not yet released, it's part of Exim
4.90 (I think J slightly mis-spoke in his phrasing around 4.89).  In the
new model, the TLS client is capable of being a process proxy for other
delivery processes, via a Unix socket, so TLS never needs to be torn
down.  Performance should be significantly better, despite the extra
copies and extra process locally.

There's some bug-fixing and cleanup required though, around things as
basic as "what gets logged", since this isn't how Exim was designed.

-Phil

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

Re: [exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[Viktor Dukhovni]

> On Apr 23, 2017, at 12:18 AM, ad...@bugs.exim.org wrote:
> 
> https://bugs.exim.org/show_bug.cgi?id=2104

Looking at the bug history I see:

> Up until 4.89 using a continued-TCP-connection for TLS involves dropping
> the TLS (in the old transport process) and starting up a new TLS connection
> on the same TCP connection in the new transport process  (you should be able
> to see that sequence in the debug trace).
> 
> I'm not sure whether or not the 1.0.2 OpenSSL version is relevant, but it
> does appear to be the new TLS startup that's the problem.
> 
> Testcase 2113 ("TLS client: multiple messages over one connection")
> in the testsuite looks relevant - and that appears to be passing on your
> buildfarm machine, which has 1.0.2k.   Are you in a position to compare debug
> output of that vs. your failing case?
> 
> Possibly another direction of investigation would be a packet capture of
> a failing case, looking specifically at the TLS startup sequence after that
> second STARTTLS.

I find that rather perplexing.  Over a single TCP connection it is not valid
to issue a second STARTTLS.  I am misreading the above?

-- 
Viktor.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2104] TLS wrong version number on connection reuse

[admin]

https://bugs.exim.org/show_bug.cgi?id=2104

--- Comment #1 from Jeremy Harris  ---
Up until 4.89 using a continued-TCP-connection for TLS involves dropping
the TLS (in the old transport process) and starting up a new TLS connection
on the same TCP connection in the new transport process  (you should be able
to see that sequence in the debug trace).

I'm not sure whether or not the 1.0.2 OpenSSL version is relevant, but it
does appear to be the new TLS startup that's the problem.

Testcase 2113 ("TLS client: multiple messages over one connection")
in the testsuite looks relevant - and that appears to be passing on your
buildfarm machine, which has 1.0.2k.   Are you in a position to compare debug
output of that vs. your failing case?

Possibly another direction of investigation would be a packet capture of
a failing case, looking specifically at the TLS startup sequence after that
second STARTTLS.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##