[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

--- Comment #11 from Leonhard Knauff  ---
(In reply to Heiko Schlittermann from comment #10)
> (In reply to Leonhard Knauff from comment #9)
> > Please,
> > for  very good old Solaris ;-) :
> > 
> >
> > 
> > -- diff  macros.h.489-2016  macros.h.489-2016-LK
> > 21a22,25
> > > /* Minimum of two items */
> > > #ifndef MIN
> > > # define MIN(a,b) ((a) < (b) ? (a) : (b))
> > > #endif
> 
> src/OS/os.h-SunOS5 contains it, since 0d28287c (Jan, 24th).
> Which OS header do you include?

Sorry:  solved: you are right.

I adapted the changes manually to the orig exim-4.88.tar.gz (to a copy tree 
of "exim-4.88/"), and there was the old "src/OS/os.h-SunOS5" from Dec.2016
.  :-

"test/exim-4.89_RC1.tar.gz" is right.  :-))

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

--- Comment #10 from Heiko Schlittermann  ---
(In reply to Leonhard Knauff from comment #9)
> Please,
> for  very good old Solaris ;-) :
> 
>
> 
> -- diff  macros.h.489-2016  macros.h.489-2016-LK
> 21a22,25
> > /* Minimum of two items */
> > #ifndef MIN
> > # define MIN(a,b) ((a) < (b) ? (a) : (b))
> > #endif

src/OS/os.h-SunOS5 contains it, since 0d28287c (Jan, 24th).
Which OS header do you include?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

Leonhard Knauff  changed:

   What|Removed |Added

 CC||l...@mlucom21.urz.uni-halle.d
   ||e

--- Comment #9 from Leonhard Knauff  ---
Please,
for  very good old Solaris ;-) :

can you add in "macros.h" like "MAX(a,b)" a definition for "MIN(a,b)":

18  /* Maximum of two items */
19  #ifndef MAX
20  # define MAX(a,b) ((a) > (b) ? (a) : (b))
21  #endif

 +  /* Minimum of two items */
 +  #ifndef MIN
 +  # define MIN(a,b) ((a) < (b) ? (a) : (b))
 +  #endif

+

That means

-- diff  macros.h.489-2016  macros.h.489-2016-LK
21a22,25
> /* Minimum of two items */
> #ifndef MIN
> # define MIN(a,b) ((a) < (b) ? (a) : (b))
> #endif


Regards

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

Jeremy Harris  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|ASSIGNED|RESOLVED

--- Comment #8 from Jeremy Harris  ---
Nobody commented

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

Git Commit  changed:

   What|Removed |Added

 CC||g...@exim.org

--- Comment #7 from Git Commit  ---
Git commit:
https://git.exim.org/exim.git/commitdiff/bd8fbe3606d80e5a3fc02fe71b521146c6938448

commit bd8fbe3606d80e5a3fc02fe71b521146c6938448
Author: Jeremy Harris 
AuthorDate: Thu Jan 19 15:37:16 2017 +
Commit: Jeremy Harris 
CommitDate: Sat Jan 21 22:34:17 2017 +

fix dkim verify when used with chunking.  bug 2016

 doc/doc-txt/ChangeLog | 14 ++--
 src/src/auths/get_data.c  |  2 +-
 src/src/auths/get_no64_data.c |  2 +-
 src/src/dkim.c|  9 +++--
 src/src/functions.h   |  8 ++---
 src/src/globals.c |  4 +--
 src/src/globals.h |  4 +--
 src/src/macros.h  |  4 +++
 src/src/pdkim/pdkim.c |  5 +++
 src/src/receive.c | 22 ++--
 src/src/smtp_in.c | 80 ++-
 src/src/tls-gnu.c |  8 ++---
 src/src/tls-openssl.c |  9 ++---
 test/log/0901 |  6 ++--
 test/scripts/-Basic/0901  | 48 +-
 test/stderr/0021  |  4 +++
 test/stderr/0022  | 12 +++
 test/stderr/0303  |  6 
 test/stderr/0371  |  5 +++
 test/stderr/0386  | 11 ++
 test/stderr/0465  |  6 
 test/stderr/0487  |  1 +
 test/stderr/0575  |  4 +++
 test/stderr/5410  |  9 +
 test/stderr/5420  |  9 +
 test/stdout/0901  | 64 --
 26 files changed, 283 insertions(+), 73 deletions(-)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

Jeremy Harris  changed:

   What|Removed |Added

 Status|NEW |ASSIGNED
   Priority|medium  |high
   Assignee|t...@duncanthrax.net |jgh146...@wizmail.org
 OS|FreeBSD |All
   Hardware|x86 |All

--- Comment #6 from Jeremy Harris  ---
Oops.  Any BDAT commands after the first were being fed into pdkim for
processing
so they (usually) mess up the body hash.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

Andreas Metzler  changed:

   What|Removed |Added

 CC||eximus...@bebt.de

--- Comment #5 from Andreas Metzler  ---
(In reply to Jeremy Harris from comment #4)
> Possibly a duplicate of bug 2004?

No, it still seems to be reproducible even with
f4630439f888df191a151ac935eacf04517fb2ee (CHUNKING: fix non-pipelined synch
checks.  Bug 2004) applied.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

Heiko Schlittermann  changed:

   What|Removed |Added

 CC||h...@schlittermann.de

--- Comment #3 from Heiko Schlittermann  ---
As a short term work-around you can set the main config option
chunking_advertise_hosts to an empty string.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

--- Comment #4 from Jeremy Harris  ---
Possibly a duplicate of bug 2004?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

--- Comment #2 from Ivo Truxa  ---
I found the problem 

When debugging with "exim -d-all+acl -bh ", I did not find a way to get rid
of the SMTP data terminating dot, which Exim always includes into the body hash
in -bh mode. In result, DKIM body hash never matches in -bh mode - both at
saved messages that arrived with good and failed DKIM. However, when
crosschecked with shell sha256 and with Perl Mail:DKIM:Verify, after appedning
the dot to the data too, I could confirm that the hashes are identical to those
of Exim. That means that both the hashing algorithm and the canonalization are
working properly in Exim.

Therefore, I decided to debug in daemon mode, and there it is where I could see
the problem. It is caused by BDAT (https://tools.ietf.org/html/rfc1830). 

Gmail uses CHUNKING, and the BDAT command sent after each chunk of data (around
63kB in Goggle's case), the command gets inserted into the canonalized string.
The data stream remains clean, so the attachments remain intact. It only
influences the canonalizing buffer string.

I think you do not need my configuration or logs. You can easily reproduce the
problem by sending yourself an email from Gmail, containing a binary file over
65kB. When you run Exim in daemon mode with debugging output redirected to a
file, you'll see the string "BDAT 34182 LAST" after some 63kB of data (the
number may differ). You'll spot it easily, since it mostly breaks down the
otherwise uniform data column of the base64 data. An example from my debug file
is below:

voFq5zaCir2h86qtRTxOujxMvOuRqDGUaQpgDYEwF5nU1fFOXrrMYpdl21qXgH05XPqZ8S5lK1fM^M
hhkmDssWITGWTtqIY0bEHhVmGSJZK3pOeRpbsB1KtTStnu+deo1jx9As9FG6JnZ3x7K+AuxfO9jV^M
LCads43D9v5gkfjC9Nwa1mvmns8h9IL/AC2pp8vb69M8IICAgICAgICAgICAgICAgICAgICAgICA^M
gIMKPittVME7BBKgSFIkjdRgSAmBITsJkyFAgydkkhLCBtBDAT 34182 LAST^M
81B1SCSdkyJBMbiEFplBY7gBBIO6Jh^M
fnZRhG6QSVVaFxHVJAAAkjZR2FoBInooEVaradNzydgJKRGR0/nXFRd31QNfIadET9VNU7slMNWD^M
BI4IB6qq8LPaNMAb8jsolaHzDvMLhpOkbxChMI8kVfdIEkyES6z8c8J8/LltdeX7lCq6k7fjUNvx^M
aufxCjmt5dbhNzluTT5eWbqk61vnsMgA7Ly96l661VlkbYhzgXbwJhcy5S6FqqXN009Y9O60bmYb^M

And this is the true data (not canonalized):

voFq5zaCir2h86qtRTxOujxMvOuRqDGUaQpgDYEwF5nU1fFOXrrMYpdl21qXgH05XPqZ8S5lK1fM^M
hhkmDssWITGWTtqIY0bEHhVmGSJZK3pOeRpbsB1KtTStnu+deo1jx9As9FG6JnZ3x7K+AuxfO9jV^M
LCads43D9v5gkfjC9Nwa1mvmns8h9IL/AC2pp8vb69M8IICAgICAgICAgICAgICAgICAgICAgICA^M
gIMKPittVME7BBKgSFIkjdRgSAmBITsJkyFAgydkkhLCBt81B1SCSdkyJBMbiEFplBY7gBBIO6Jh^M
fnZRhG6QSVVaFxHVJAAAkjZR2FoBInooEVaradNzydgJKRGR0/nXFRd31QNfIadET9VNU7slMNWD^M
BI4IB6qq8LPaNMAb8jsolaHzDvMLhpOkbxChMI8kVfdIEkyES6z8c8J8/LltdeX7lCq6k7fjUNvx^M
aufxCjmt5dbhNzluTT5eWbqk61vnsMgA7Ly96l661VlkbYhzgXbwJhcy5S6FqqXN009Y9O60bmYb^M

I verified with sha256() that Exim indeed calculates the body hash including
the BDAT command, which it definitely should not.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##

[exim-dev] [Bug 2016] DKIM bodyhash mismatch on Gmail > 64kB

[admin]

https://bugs.exim.org/show_bug.cgi?id=2016

--- Comment #1 from Jeremy Harris  ---
Could you give "exim -d-all+acl -bV" output from the system, and a sample
acceptance (<=) log line for an affected message?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##