Re: [exim] exim-4.94.2+taintwarn - when will it be EOL?
>> To rephrase it: > > - Exim 4.95 will contain "taintwarn" > - It is meant as support for upgrading your config, w/o breaking your > setup instantly. > - With a future¹ release of Exim we will drop the "taintwarn" support. > - If you failed to upgrade your config, your setup will be broken with a > future¹ release of Exim. > > ¹) It is not decided yet, what "future" means. It may or may not be 4.96. Although I understand that the Exim project may not want to wait that long, from my perspective it would be ideal if the taintwarn feature lasted long enough to make it into LTS Linux releases. Otherwise, from the perspective of LTS people who use distribution packages, the feature basically won't exist; they will jump straight into a version that breaks their setup (if they haven't already[*]). Based on past timing, Ubuntu 22.04 will freeze the Debian Exim package sometime early next year. I don't know if Exim is in the base version of Red Hat Enterprise, but RHEL EPEL appears to be following upstream Exim versions instead of freezing on one (EPEL currently has 4.94.2 for both 7 and 8). Other LTS Linuxes I don't know about. (Debian might be considered a 'LTS' enough distribution, and they're going at a pace of roughly a release every two years, with one probably due soon and so another likely in roughly mid 2023.) - cks [*: We're skipping Ubuntu 20.04 on our Exim machines because the 20.04 Exim version is broken. Other people may be in the same situation with Ubuntu or other distributions. ] -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
Jeremy Harris: > > Semi-radical: provide an ACL, router, and transport modifier that > > checks some variable or content for dangerous contents > > We have that. All data provided by an untrusted source, described > as "tainted" for a shorthand. Tainted variables contain potentially dangerous contents, not actually dangerous contents. Most of the time, the contents of tainted variables are not dangerous, but sometimes they are. I think that it would be useful for Exim to provide assistance in telling the two apart. I say this because I strongly believe that people are going to write Exim configuration code that de-taints variables in brute force ways (and the more that Exim doesn't provide mechanisms to do relatively arbitrary 'safe' de-tainting, the more that people are going to do so). I think it's relatively important to let people guard these de-taintings with safety checks, such as 'is there dangerous content here'. Also, even with relatively safe de-tainting, sometimes I would rather reject funny content immediately. This is actually a relatively popular thing to do today in ad-hoc ways; for example, the Debian 'split' Exim configuration has for years shipped with a set of checks for dangerous characters in local parts. Sysadmins can maybe write these checks in Exim configurations themselves, but in ad-hoc ways and sysadmins probably don't know as much about what things are dangerous (or valid) in various conditions as other people do. - cks -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
> Yes, but its a positive match only - meaning you have to explicitly > specify allowed characters.The function should NOT be able to > specify forbidden characters - as then it would ve easy to miss bad > characters.If an sysadmin then writes a filter which is too broad - > its his own fault. History has vividly demonstrated that providing a toolkit and asking people to build their own secure implementation from it does not create effective security (although it does let you blame a different set of people for the resulting insecurity). Many people will either make mistakes or not know all of the surprises and traps that are lurking, and so won't know to counter them. Effective security is almost always created by a few people who know the area very well spending a bunch of time to thoroughly explore all of the dark corners, then building something with as few options as possible (ideally none). Or to put it another way: I feel that people should not need to be experts in knowing what are safe and dangerous characters and character sequences in order to create safe Exim configurations. If they have to be such experts, there will be a lot of unsafe Exim configurations created. I think it's potentially sensible to provide an escape hatch, but I don't think that it should be the only way to do it; I think it should only be something that you resort to if you have highly unusual needs. And I share Jeremy Harris's worry that it will be cargo-culted and copied widely, even then. (Possibly unlike Jeremy Harris, I feel that if Exim provides no such safe-ish general un-tainting, people will still find ways to do it and then cargo-cult it. People are going to cargo-cult answers for this; the only question is whether Exim can help make that as safe as possible.) - cks -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
> We understand that this introduces an additional level of complexity
> for the configuration. And we're seeking for better ways, to balance
> between a secure and a simple configuration.
>
> We're open for suggestions. And intentionally we do not provide
> suggestions from our side here and now (this doesn't mean that we do
> not have ideas ;)) My thoughts I'll present here later.
Here are some suggestions, ranging from moderate to radical.
Moderate: there should be a full chapter in the Exim documentation on
tainting and how to deal with it. This should cover the security risks
that it's there to deal with, common configuration snippets that are now
a problem, and how to deal with tainting and de-tainting data in common
configuration needs. Right now there is not even an explicit subsection
for tainting in the Security Considerations chapter.
(If in the process of writing this chapter things are difficult to
explain or the configuration changes seem awkward, well, that is a
sign of where improvements are needed.)
Semi-radical: there should be a configuration file option that turns
taint errors into warnings that do not stop processing but that are
merely logged to the main log or perhaps also the paniclog. Many system
administrators have existing Exim configurations that likely have
tainting problems, and we have no assurance that we can find them all
in advance of a live deployment. Live deployment with tainting creating
hard failures is a serious land mine and problem.
(It would be nice if all system administrators had automated test
suites that completely cover their entire Exim configuration. If people
think that having such a test suite should be required for reliable
Exim version upgrades, perhaps someone should write a chapter for the
Exim documentation on how to create and operate it, and also how to
measure the configuration file coverage so that you know you have 100%
coverage.)
An optional but nice extra would be a third choice for this option,
which generates 'this would have been rejected' warnings if the contents
of the tainted variable being used seem harmless, but force hard errors
if it appears to have dangerous contents (whatever the Exim developers
consider that to be). That would be potentially somewhat safer than
running with the 'just always produce warnings and no more' option, but
might invite people to use this option permanently instead of upgrading
their configuration.
Semi-radical: provide an ACL, router, and transport modifier that
checks some variable or content for dangerous contents (possibly in
several variations), and either passes it or triggers a condition
failure or deferral. Possibly this should be implemented as a function
that can be used in 'condition = ...' and '${if ...}' clauses, but if so
care should be taken to allow for a 'defer' result, not just pass/fail.
This would provide more assurances to sysadmins who are writing more
dynamic configurations that they haven't let something slip through in
their process for de-tainting variables through database queries or
whatever.
People will inevitably wind up wanting to check various pieces of data
for 'bad stuff' and then de-taint them if there isn't any (or reject
them if there is), so the only question is whether Exim will provide
an official best effort at this (one that can be evolved over time
and perhaps have options for what sort of dangerous thing should be
accepted), or whether it will leave people writing configuration files
on their own to write hundreds of different and likely inferior and
flawed versions.
(Frankly, we would probably use it early in our ACLs too even though
we have hard-coded lists of valid recipients and so on, just so that
we can log messages if some attacker is trying to feed us dangerous
stuff.)
Radical: $local_part, $domain, and similar tainted variables should
be automatically un-tainted once they pass a check that would generate
the untainted version of the variable. Forcing system administrators
to switch which variable they use has two problems. First, it is
make-work, since the two variables are essentially bolted together
(and if they are not, it is likely to surprise people, for example if
you somehow get $local_part changed but $local_part_data stays with
an old value). Second, it is part of what makes it impossible to have
a configuration file that works in both an old un-tainted Exim and
a new tainting Exim, since $local_part_data does not exist in older
Exims. Forcing configuration file updates at the same time as Exim
version updates complicates sysadmin life.
Radical: in ACLs, verifying the receiver or sender address through
'verify = ' should optionally un-taint $local_part, $domain,
and so on. This should be controlled through a new option that has to
be turned on. In many configurations, verifying at least recipient
addresses will automatically restrict them to a list of known good
domains, local parts, and so on. Forcing ACLs to additionally
[exim] Trying to understand when $local_part_data is set in ACLs
I thought I understood how to deal with the new rules for tainted data
in Exim, but evidently I don't, and I can't figure out what is going on
from the documentation and from Internet searches.
My starting situation is that we have some simple uses of $local_part
in SMTP RCPT ACLs. We also have a big list of all valid local users in
a file. My initial understanding of the documentation was that I could
add a ACL condition of 'local_parts = /our/userlist/file' and have
$local_part_data defined in the rest of that ACL statement, because we
had successfully done a lookup on $local_part. However, this doesn't
seem to be the case. I've also tried 'lsearch;/our/userlist/file',
following some apparent recipes on the Internet, but it also resulted
in $local_part_data not being defined.
I can definitely see that the condition passes, as I'm using a test
ACL like this:
warn
domains = +local_domains
local_parts =
log_message = local_parts test: $local_part -- $local_part_data
-- ...
When I use a straight file or a lsearch; in the 'local_parts = ...' bit,
I get log messages, and $local_part expands to what I expect it to,
but $local_part_data is blank in the log message (and elsewhere).
What am I supposed to do here? Does this require an explicit use of
${lookup}, not just a lookup in general? What ${lookup} should I
substitute here that has the same matching rules as listing a plain
file?
In this particular situation I can substitute a dsearch condition, and
both
local_parts = ${lookup{${lc:$local_part}} dsearch {/a/directory}}
and
local_parts = dsearch;/a/directory
work to set $local_part_data (for local parts that are already in the
proper case in the second one). This leaves me even more confused
about why 'lsearch;/our/userlist/file' succeeds but doesn't set
$local_part_data.
This is all on Ubuntu 20.04 with Ubuntu's version of 4.93.
Thanks.
- cks
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] SPF and DKIM error processing when receiving emails
> Thanks Jeremy, fair enough comment. > > For now, I'd like to accept anything - just have the Data about > whether an incoming email was compliant or not - with the option of > then moving to become more strict. I guess I'd like to be more like > gmail - which actually seems to be quite lenient. > > I'll then do my best to see if other mail admins can rectify their > problems (or me - when its my fault). One thing to be aware of when writing DKIM related rules is that it's quite possible (and in some environments routine) for legitimate incoming email to have multiple DKIM signatures, some of which fail to validate and some of which do validate. One can be unhappy about this, but places like Microsoft Outlook365 don't care about our feelings. (We have actually seen this happen on inbound messages from Microsoft Teams that transited through hosted Office365 email before reaching us; the Teams DKIM signature was invalid, the hosted O365 DKIM signature was valid. Since Microsoft Teams falls under the microsoft.com domain and microsoft.com advertises a strong DMARC policy, this caused a certain amount of heartburn.) - cks -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Sieve filters broken due to tainted expansions?
> > What's the suggested way to do that for virtual domains, that is many
> > mailboxes that all belong to the same local user, and which are not
> > obtained through a lookup, but through the filesystem itself?
>
> The result of a lookup is untainted, and will likely remain so
> (even if the key for the lookup is tainted, eg. $local_part).
> So whatever you're doing now should still work, so long as you
> don't name the DB for the lookup using tainted data.
Is there a way of un-tainting things from $local_part? We have a
long-standing little feature where we implement per-address settings
for things like blocklists using databases read from directories named
after the local addresses, more or less:
UBDIR = CFDIR/${lc:$local_part}
deny
domains = +local_domains
senders = ${if exists {UBDIR/senders} {UBDIR/senders}}
message = mail from ...
We have not yet updated to a version of Exim that uses tainting and so
I don't know if this usage will be immediately affected, but it seems
that this is now being considered questionable in at least this form.
(We already check and sanitize $local_part, so this is safe in our
usage. People do not get to create valid local addresses with eg '/../'
in them, and if they did manage it somehow, explicit address content
checks would reject them earlier. Also, these per-user files must be
updated by root, so we consider their contents safe.)
- cks
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Monitoring and Reporting Eximstats
> I'd like to hear about what techniques you use to monitor your setups,
> especially with setups beyond one or two mail servers.
Our current approach for Prometheus-based monitoring is to process
mailq output to get a snapshot of queue-based information and to use
Linux's 'ss' to get information on SMTP connection counts (we looked
at using exiwhat and processing its output, but exiwhat is fairly
heavyweight; see the manpage).
I hadn't considered the use of running commands or sending messages
in the ACLs in order to keep a running count of messages delivered or
the like; it's a neat approach. The drawback is that you'll definitely
be building relatively custom tools for this. There are apparently some
general programs to basically 'tail' logs and convert the information
there into running counters and so on; the ones for Prometheus I know of
are:
https://github.com/google/mtail
https://github.com/fstab/grok_exporter
It occurs to me that one obvious and very simple thing to do in ACLs
and so on is simply to update a timestamp of 'I most recently accepted
a message at ...' or 'I most recently delivered a message at ...'. This
could be used to drive very basic but useful health checks, where if
you see these fall too out of date something may be wrong.
(You can also use ${run} and so on outside of ACLs, but I'm not sure if
there's a good place to put them so that they happen after a transport
succeeds. If you work really hard you can probably detect bounces by
detecting the expansion of the bounce text, through embedding a ${run}
or the like in it, but perhaps there's a better way[*].)
- cks
[*: We sort of care about a spike in the bounce rate because it's often
an indicator of a problem, such as yet another compromised user account
being used to send out spam.
]
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Moving a queue to another server
> I am wondering if there is a good way to move all pending mails from > one exim server to another. > It is probably not the best idea to move the files from one spool > directory to another (idea 1). For what it's worth, we've historically done exactly this for mail server migrations (eg to a host with a newer OS release). We'll shut down Exim on both hosts, then move over pretty much the entire mail spool. I think the only problems we've sometimes had was needing to reset the retry database(s). This does require a downtime on both sides, though, and we've always been moving to a destination host where the spool was empty (and in fact was disposable). - cks -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Choosing the outbound IP address according to a Database query.
> In Transports - I have something like:-
>
> begin transports
>
> # This transport is used for delivering messages over SMTP connections.
> remote_smtp:
> driver = smtp
> dnssec_request_domains = *
> hosts_try_dane = *
> return_path = ${address:$reply_address}
> interface = <; 192.111.222.1 ; 2001:1234:abcd:5678::1
>
> So could the "interface =" part be changed to receive the result of a
> MySQL query?
> Can I pop a value into a variable - and use that? - otherwise I'll be
> doing two lookups, one for IPv4 and one for IPv6...
The transport 'interface =' is string-expanded, so you can use anything
here that you can use as a normal string expansion, including MySQL
queries. We have some Exim servers with a relatively complex set of
conditions here (although we don't use MySQL lookups) and it works fine.
You may also find it more convenient to set this information up
earlier, for example in an ACL (as a message ACL variable) or during
routing (eg in $address_data), and then simply use it in the transport,
rather than looking it up every time in the transport.
- cks
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
