[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-16 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

Jeremy Harris  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|ASSIGNED|RESOLVED

--- Comment #11 from Jeremy Harris  ---
Nobody commented further

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


Re: [exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-06 Thread Simon Arlott via Exim-dev
On 06/12/2018 00:02, Jasen Betts via Exim-dev wrote:
> NDRs go to the envelope sender, so they will bounce back to a single
> address (per message), but there can be several "delayed" messages and
> a single bounce (retry timeout exceeded) for each input, so that
> provides small-scale amplification, until the timeout is reached,
> after that no amplification.
> 
> where available SPF is one mitigation for this. It prevents the attacker 
> from forging the sender address.

Alternatively, set delay_warning_condition to only send warnings if the
email was sent by one of your own users.

-- 
Simon Arlott

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


Re: [exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-05 Thread Jasen Betts via Exim-dev
On 2018-12-05, Andrew C Aitchison via Exim-dev  wrote:
> On Wed, 5 Dec 2018, admin--- via Exim-dev wrote:
>
>> https://bugs.exim.org/show_bug.cgi?id=2341
>>
>> --- Comment #8 from Jeremy Harris  ---
>> Thanks.  It seems I'm on the right track; now we need to decide if that is
>> now too much notification... there's no per-sender tracking on this, it's
>> per-deferred message, so (as in your test) one sender will get a warning
>> for every message they send, repeated per the delay_warning setting
>> as modified by the queue run occasions.
>>
>> Leaving it like that will be simplest.  I'll commit that for now, but 
>> opinions
>> on any need for additional control are welcom.
>
> Like David, I'd expect and want a message for every delayed message.
>
> However, if someone malicious finds a full mailbox, could they use it
> to amplify attacks on a third party ?
> On the face of it, sending an almost null message to the full mailbox
> could result in larger messages being sent to an innocent third-party.
> What happens when a message with multiple senders (is that reply-to or from)
> is delayed; does each one get a delay warning ?
>
> [ Am I giving too many hints to spammers by asking in a public list ? ]

NDRs go to the envelope sender, so they will bounce back to a single
address (per message), but there can be several "delayed" messages and
a single bounce (retry timeout exceeded) for each input, so that
provides small-scale amplification, until the timeout is reached,
after that no amplification.

where available SPF is one mitigation for this. It prevents the attacker 
from forging the sender address.

-- 
  When I tried casting out nines I made a hash of it.

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-05 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

Git Commit  changed:

   What|Removed |Added

 CC||g...@exim.org

--- Comment #10 from Git Commit  ---
Git commit:
https://git.exim.org/exim.git/commitdiff/de6f74f297d040a34746bc8e1829ad4b867441c9

commit de6f74f297d040a34746bc8e1829ad4b867441c9
Author: Jeremy Harris 
AuthorDate: Wed Dec 5 16:09:01 2018 +
Commit: Jeremy Harris 
CommitDate: Wed Dec 5 17:09:00 2018 +

send delay-mdn for any queurun past delay_warning, even if not retry time
yet.  bug 2341

 doc/doc-txt/ChangeLog|8 +-
 src/src/deliver.c|   54 +-
 src/src/macros.h |2 +
 src/src/retry.c  |   21 +-
 test/confs/0098  |   16 +
 test/log/0098|   16 +
 test/mail/0098.CALLER|   63 ++
 test/rejectlog/0098  |3 +
 test/runtest |2 +-
 test/scripts/-Basic/0098 |   19 +
 test/stderr/0275 |4 +-
 test/stderr/0278 |4 +-
 test/stderr/0357 |   15 +-
 test/stderr/0358 |   14 +-
 test/stderr/0361 |   12 +-
 test/stderr/0386 |8 +-
 test/stderr/0388 |8 +-
 test/stderr/0402 |   20 +-
 test/stderr/0403 |4 +-
 test/stderr/0404 | 2436 +-
 test/stderr/0408 |4 +-
 test/stderr/0487 |4 +-
 test/stderr/0529 |   10 +-
 test/stderr/0554 |7 +-
 test/stderr/2600 |4 +-
 test/stderr/2610 |4 +-
 test/stderr/2620 |4 +-
 test/stderr/5004 |4 +-
 test/stderr/5005 |   16 +-
 test/stderr/5006 |4 +-
 test/stdout/0098 |2 +
 31 files changed, 1479 insertions(+), 1313 deletions(-)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


Re: [exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-05 Thread Andrew C Aitchison via Exim-dev

On Wed, 5 Dec 2018, admin--- via Exim-dev wrote:


https://bugs.exim.org/show_bug.cgi?id=2341

--- Comment #8 from Jeremy Harris  ---
Thanks.  It seems I'm on the right track; now we need to decide if that is
now too much notification... there's no per-sender tracking on this, it's
per-deferred message, so (as in your test) one sender will get a warning
for every message they send, repeated per the delay_warning setting
as modified by the queue run occasions.

Leaving it like that will be simplest.  I'll commit that for now, but opinions
on any need for additional control are welcom.


Like David, I'd expect and want a message for every delayed message.

However, if someone malicious finds a full mailbox, could they use it
to amplify attacks on a third party ?
On the face of it, sending an almost null message to the full mailbox
could result in larger messages being sent to an innocent third-party.
What happens when a message with multiple senders (is that reply-to or from)
is delayed; does each one get a delay warning ?

[ Am I giving too many hints to spammers by asking in a public list ? ]

--
Andrew C. Aitchison Cambridge, UK
and...@aitchison.me.uk

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-05 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

--- Comment #9 from David Carter  ---
I think that the new behaviour is fine.

Each warning message contains summary information:

 The message identifier is: 1gUXhL-0007kL-sO
 The date of the message is:Wed, 05 Dec 2018 13:58:43 +
 The subject of the message is: Test message

and full headers about the specific message in question.

I imagine that senders would get confused if they only received notifications
for a subset of the messages that they sent. For example in an "Over quota"
situation small messages may still be delivered even when large ones get stuck.

20+ messages from a single sender in a short time is a pathological test case.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-05 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

--- Comment #8 from Jeremy Harris  ---
Thanks.  It seems I'm on the right track; now we need to decide if that is
now too much notification... there's no per-sender tracking on this, it's
per-deferred message, so (as in your test) one sender will get a warning
for every message they send, repeated per the delay_warning setting
as modified by the queue run occasions.

Leaving it like that will be simplest.  I'll commit that for now, but opinions
on any need for additional control are welcom.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-05 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

--- Comment #7 from David Carter  ---
Okay, I installed a patched version of Exim 4.91 onto a spare mailhub and
tweaked the delay_warning configuration:

  delay_warning = 1h:4h:8h:24h:30d

I then sent 26 test messages to a local account which has run out of quota.

An hour later I received precisely 26 warning messages, almost all of which
immediately followed a "retry time not reached" warning:

2018-12-05 14:59:00 + 1gUXhU-0007lX-pk == x...@hermes.cam.ac.uk routing
defer (-51): retry time not reached

Only a single actual delivery attempt on that particular queue run.

So that definitely looks like an improvement.

Is there anything else that you would like me to test?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-05 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

--- Comment #6 from Jeremy Harris  ---
(In reply to David Carter from comment #5)
> The deliver.c part of your patch appears to apply cleanly against a vanilla
> Exim 4.91 tarball:

> Would that be a useful test?

Yes; please do.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-05 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

--- Comment #5 from David Carter  ---
I'm a bit nervous about running current master (and I would have to rejig my
build scripts).

The deliver.c part of your patch appears to apply cleanly against a vanilla
Exim 4.91 tarball:

patching file src/deliver.c
Hunk #1 succeeded at 5840 with fuzz 1 (offset -45 lines).
Hunk #2 succeeded at 6557 (offset -51 lines).
Hunk #3 succeeded at 6580 (offset -51 lines).
Hunk #4 succeeded at 6592 (offset -51 lines).
Hunk #5 succeeded at 6617 (offset -51 lines).
Hunk #6 succeeded at 6671 (offset -51 lines).
Hunk #7 succeeded at 7006 (offset -51 lines).
Hunk #8 succeeded at 7975 (offset -55 lines).
Hunk #9 succeeded at 8002 (offset -55 lines).
Hunk #10 succeeded at 8125 (offset -55 lines).

Would that be a useful test?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-03 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

--- Comment #4 from Jeremy Harris  ---
Created attachment 1151
  --> https://bugs.exim.org/attachment.cgi?id=1151=edit
proposed patch

David - are you in a position to test a patch?  This, on top of current master,
tries to pull from the retry DB to build a warning message on a "retry time
not reached".  Not all possible causes result in a reasonable error message
in the mail (roughly, only errors from actual SMTP conversations do), but
all should get a mail generated.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-03 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

--- Comment #3 from Simon Arlott  ---
(In reply to Jeremy Harris from comment #2)
> Bug -> Wishlist, as current operation is as documented.

The current behaviour not documented.

The documentation only tells you to run retries more frequently than the
warnings should be sent. It doesn't tell you that Exim will not send warnings
for each message when there is more than one message in the queue for the same
destination.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-03 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

Jeremy Harris  changed:

   What|Removed |Added

   Severity|bug |wishlist

--- Comment #2 from Jeremy Harris  ---
Bug -> Wishlist, as current operation is as documented.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##


[exim-dev] [Bug 2341] delay_warning failing to send messages

2018-12-03 Thread admin--- via Exim-dev
https://bugs.exim.org/show_bug.cgi?id=2341

Jeremy Harris  changed:

   What|Removed |Added

 Status|NEW |ASSIGNED
   Assignee|ni...@exim.org  |jgh146...@wizmail.org

--- Comment #1 from Jeremy Harris  ---
> It looks like the delay_warning stuff only kicks in when there is an actual 
> delivery attempt

Yup.  The docs say:

  Note that the option is only evaluated at the time a delivery attempt fails,  
  which depends on retry and queue-runner configuration. Typically retries will
  be configured more frequently than warning messages.

I don't think your suggested workaround would do anything but get another
"retry time not reached" for most of the attempts.

Possibly you could divert old messages to an alternate queue using the
EXPERIMENTAL_QUEUEFILE facility, and hack something with that - maybe an
alternate config for the queue-runner for that - but "hack" really is the word.


I'll have a look at the possibility of hooking the "retry time not reached"
path to the warning message generator.  It looks like the latter keeps a count
of messages sent in the message spool file, presumably modifying it when one
is sent - and compares that with a "how many should have been sent by now"
evaluation.  This should be safe against being called more often, at the
cost of extra processing; the cost would then relate to the queue-runner repeat
rate rather than the retry-rule applying to the relevant message.
Perhaps it should be configurable as to which domains this is applied.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim 
details at http://www.exim.org/ ##