Re: [exim] Exim 4.94.0.4 works where Exim 4.94 fails with tainted path 'not permitted'

[Jeremy Harris via Exim-users]

On 08/11/2020 22:15, Michael Haardt via Exim-users wrote:

If the documentation stated that somehow, it would ease migrations.

Suggestions for (wording and where to put it) welcome.


The generic discussion how a router works sounds like a good place
to me for giving a hint on the intended usage.  Here is an idea:

Chapter 3, section 12: Router preconditions

[...]

Thanks.   I've pushed a massaged version of that.
--
Cheers,
  Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Sebastian Nielsen via Exim-users]

The thing is that I think that my idea with custom checker is better, since, it 
provides more flexibility, to configure it even stricter, for different use 
cases.

For what I know, the current risks that exist in a exim configuration when 
using user-supplied data is:

- Email redirection (fooling Exim into sending email as a open SMTP) - requires 
a "safe" localpart - and in some cases a safe "remote part"
- Writing outside intended locations
- Reading outside intended locations
- SQL injection
- Command injection in system commands
- Export to Web browser/HTML

Etc.
By making a custom possibility, with a select predefined "guranteed safe string 
sets", it will make exim more secure.
I think we should avoid creating lots of filter functions, when the filter 
could be specified using one of the predetemited filters, or a custom charset.


One example that would be useful for me would be like
${untaint(${localpart},"0123456789")
For my Email-to-SMS gateway.


Note that today its pretty complicated to filter data as "you want" - you need 
to write a regexp and do search/replace. It would be much better if a general 
filter/untaint function could be implemented, so it become easy-as-ass to 
filter data - as I said, even in situations where you don't need untainted 
data, but still want to filter to make data safe.

-Ursprungligt meddelande-
Från: Heiko Schlittermann via Exim-users  
Skickat: den 10 november 2020 22:44
Till: exim-users@exim.org
Ämne: Re: [exim] tainted data issues

Hi,

I welcome the suggestions, especially the idea about gradually enabling 
taintchecks, to allow a smooth transition, as suggested by Mike Tubby.

  taint_mode = yes | no | warn

Another idea from my side (it's similar to Sebastian N's idea)

>   begin transports
> smtp:
>   driver = smtp
>   dkim_domain = $sender_address_domain
>   dkim_selector = 2020-08-25
>   dkim_private_key = 
> /etc/exim/dkim/$dkim_selector.$dkim_domain.pem

We could provide taint checks for different situations, as the risk of using 
tainted data depends on the usage of the data (filename, log message, lookup 
key, …)

Provide a new set of functions:

${XXX{}{}{}}
${XXX{}{}fail}
${XXX{}{}}

With XXX as
- file  (no "/")
- path  (no "..")
- line  (no "\r", "\n")
...

dkim_private_key = 
/etc/exim/dkim/${file{$dkim_selector.$dkim_domain.pem}}
or
dkim_private_key = 
${path{/etc/exim/dkim/$dkim_selector.$dkim_domain.pem}}

This can give us flexibility where the current lookup based way of untainting 
doesn't work.


Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
 SCHLITTERMANN.de  internet & unix support -  Heiko 
Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -  gnupg 
encrypted messages are welcome --- key ID: F69376CE -



smime.p7s
Description: S/MIME Cryptographic Signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Heiko Schlittermann via Exim-users]

Hi,

I welcome the suggestions, especially the idea about gradually enabling
taintchecks, to allow a smooth transition, as suggested by Mike Tubby.

  taint_mode = yes | no | warn

Another idea from my side (it's similar to Sebastian N's idea)

>   begin transports
> smtp:
>   driver = smtp
>   dkim_domain = $sender_address_domain
>   dkim_selector = 2020-08-25
>   dkim_private_key = /etc/exim/dkim/$dkim_selector.$dkim_domain.pem

We could provide taint checks for different situations, as the risk of
using tainted data depends on the usage of the data (filename, log
message, lookup key, …)

Provide a new set of functions:

${XXX{}{}{}}
${XXX{}{}fail}
${XXX{}{}}

With XXX as
- file  (no "/")
- path  (no "..")
- line  (no "\r", "\n")
...

dkim_private_key = 
/etc/exim/dkim/${file{$dkim_selector.$dkim_domain.pem}}
or
dkim_private_key = 
${path{/etc/exim/dkim/$dkim_selector.$dkim_domain.pem}}

This can give us flexibility where the current lookup based way of
untainting doesn't work.


Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Sebastian Nielsen via Exim-users]

I think as I said, provide a untaint tool, that allows custom data to verify
against.

Like:
${untaint(${var},
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")}

Regardless of if ${var} is tained or not, the result of ${untaint} is always
untainted.

The character string, is then what to filter against - all characters not on
that particular list, will be removed from ${var} silently before passing it
on.

This allows system administrators, to do entirely custom checks against
dangerous data, since in all cases where tainted data becomes a security
issue, is when delimiter characters is inserted in the text to escape out of
the enviroment that the command or whatever is executed in, and be able to
execute arbitary commands or write in arbitary locations.

Its very important that the function itself is "safe" - meaning it should
not be possible in ANY way for tainted data to "leak through" the filter,
ergo it must not be possible for any character not on administrator's list,
to end up in the return of the untaint function.

Also the filter list itself, must also be done in such a way variable
expansion never happens in the filter list - only way to specify a filter
list, is to manually specify it in config file.


This gives most flexibility, and also encourages security by making it easy
to filter data before passing it on, making it a "habit" for system
administrators.
Then it will be a habit to always ${untaint} even in cases where untainted
data is not required, incresing security even further.

In the above example, if ${var} contains
"Sebastian/Users/f...@somewhere.com" the result of the untaint operation
with that particular filter list would be "SebastianUsersfilesomewherecom"
which is a completely safe string to pass on to filesystem.

To make it even easier for system administrators, make premade filter lists
like:
%%FILESYSTEM%% = Contains a list of characters that is safe for use in
functions that read or write to filesystem.
%%SYSTEM%% = Containst a list of charactes that is safe to pass on to a live
terminal, like when running system commands or similiar.
%%SQL%% = Contains a list of characters that is safe to pass on to a raw SQL
prompt without rsik of injection.
%%HTML%% = Contains a list of characters that is safe to output to a web
browser without security risks.
%%LOCALPART%% = Contains a list of characters safe in localpart of a email
adress.
%%HOSTPART%% = Contains a list of characters safe to use in a DNS name or
host part of a adress including IP literals.
%%FULLEMAIL%% = Contains a list of characters that is allowed and safe to
use in a full email adress.

So writing:
${untaint(${var}, "%%SQL%%")}

Would filter ${var} in such a way that its safe to pass on to a raw SQL
string without injection risk.

Note that these %% strings is NOT string expansions - instead a customized
system for specifying pre-made filters, preventing variables from being
accidentially expanded and used in a filter list, which in itself is a big
security risk, since then filter lists could be modifyed from the outside.

However, \xXX should be expanded of course, so people can write characters
that don't exist on the keyboard, in some cases they are useful.


Such a untaint function should be easy to implement.
And would make exim much more secure, as its a very easy function, making it
a habit to use the function by system administrators.

YES - it is possible to "shoot yourself in the foot" with the function by
specofying characters that is unsafe for that particular usage, but then the
responsibility is on the system administrator.


Best regards, Sebastian Nielsen

-Ursprungligt meddelande-
Från: Michael Haardt via Exim-users  
Skickat: den 10 november 2020 20:16
Till: Jeremy Harris via Exim-users 
Ämne: Re: [exim] tainted data issues

Jeremy Harris via Exim-users  wrote:
> The one major hole I know of is for the creation of a mailbox file, 
> first time, for an account.

After having reviewed a number of configurations, I am sure there is more.

While I am not pleased with the design of verifying tainted data, or
introducing it in such an invasive manner without a new major version, the
need of doing so absolutely exists.  That said, the current design is usable
and it solves the problem.  Using it may either convince us of being the
best solution, or show which specific alternative is better.

The ongoing configuration reviews certainly uncovered potential problems, so
rolling back is not an option without a replacement for the current
verification.

Michael

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/



smime.p7s
Description: S/MIME Cryptographic Signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Gregory Edigarov via Exim-users]

On 11/10/20 10:37 AM, Julian Bradfield via Exim-users wrote:

I thought it was standard practice in introducing a new feature that
causes major breakage to existing installations, to take a three step
approach. First you provide the feature, and give it an enabling
switch with three levels "off", "warn but don't error", "on".
Then in successive releases you change the default value of the
enabling switch, and ultimately you remove the enabling switch.

I understand that taint protection is considered a security feature,
but it's a feature exim users have done without for decades, so I
can't really see that there was a particularly urgent need to
introduce it in a big bang.

not to say that it was done in incompatible manner,
breaking all configs that was working for years,
and with no way to switch to an old behaviour.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Jeremy Harris via Exim-users]

On 10/11/2020 06:44, Kai Bojens via Exim-users wrote:
The only problem I have with tainting is the lack of documentation. Why 
is there no single page with just "Hey, external data is now considered 
tainted.


To help the discussion along I've put up a copy of the current
(git HEAD) documentation.  It has moved on a little from the
4.94 release - in particular there is more indexing of
de-tainting methods.


https://people.exim.org/jeremy.harris/docs/HEAD/exim-html-4.94-RC999/exim-html-current/doc/html/spec_html/


The bit corresponding to Kai's request is about three sentences, near
the top of the String Expansions chapter.  Perhaps I'm too close to
the subject, but I can't see it taking a whole page.

--
Cheers,
  Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Chris Siebenmann via Exim-users]

> We understand that this introduces an additional level of complexity
> for the configuration. And we're seeking for better ways, to balance
> between a secure and a simple configuration.
>
> We're open for suggestions. And intentionally we do not provide
> suggestions from our side here and now (this doesn't mean that we do
> not have ideas ;)) My thoughts I'll present here later.

 Here are some suggestions, ranging from moderate to radical.

 Moderate: there should be a full chapter in the Exim documentation on
tainting and how to deal with it. This should cover the security risks
that it's there to deal with, common configuration snippets that are now
a problem, and how to deal with tainting and de-tainting data in common
configuration needs. Right now there is not even an explicit subsection
for tainting in the Security Considerations chapter.

(If in the process of writing this chapter things are difficult to
explain or the configuration changes seem awkward, well, that is a
sign of where improvements are needed.)

 Semi-radical: there should be a configuration file option that turns
taint errors into warnings that do not stop processing but that are
merely logged to the main log or perhaps also the paniclog. Many system
administrators have existing Exim configurations that likely have
tainting problems, and we have no assurance that we can find them all
in advance of a live deployment. Live deployment with tainting creating
hard failures is a serious land mine and problem.

(It would be nice if all system administrators had automated test
suites that completely cover their entire Exim configuration. If people
think that having such a test suite should be required for reliable
Exim version upgrades, perhaps someone should write a chapter for the
Exim documentation on how to create and operate it, and also how to
measure the configuration file coverage so that you know you have 100%
coverage.)

 An optional but nice extra would be a third choice for this option,
which generates 'this would have been rejected' warnings if the contents
of the tainted variable being used seem harmless, but force hard errors
if it appears to have dangerous contents (whatever the Exim developers
consider that to be). That would be potentially somewhat safer than
running with the 'just always produce warnings and no more' option, but
might invite people to use this option permanently instead of upgrading
their configuration.

 Semi-radical: provide an ACL, router, and transport modifier that
checks some variable or content for dangerous contents (possibly in
several variations), and either passes it or triggers a condition
failure or deferral. Possibly this should be implemented as a function
that can be used in 'condition = ...' and '${if ...}' clauses, but if so
care should be taken to allow for a 'defer' result, not just pass/fail.
This would provide more assurances to sysadmins who are writing more
dynamic configurations that they haven't let something slip through in
their process for de-tainting variables through database queries or
whatever.

 People will inevitably wind up wanting to check various pieces of data
for 'bad stuff' and then de-taint them if there isn't any (or reject
them if there is), so the only question is whether Exim will provide
an official best effort at this (one that can be evolved over time
and perhaps have options for what sort of dangerous thing should be
accepted), or whether it will leave people writing configuration files
on their own to write hundreds of different and likely inferior and
flawed versions.

(Frankly, we would probably use it early in our ACLs too even though
we have hard-coded lists of valid recipients and so on, just so that
we can log messages if some attacker is trying to feed us dangerous
stuff.)

 Radical: $local_part, $domain, and similar tainted variables should
be automatically un-tainted once they pass a check that would generate
the untainted version of the variable. Forcing system administrators
to switch which variable they use has two problems. First, it is
make-work, since the two variables are essentially bolted together
(and if they are not, it is likely to surprise people, for example if
you somehow get $local_part changed but $local_part_data stays with
an old value). Second, it is part of what makes it impossible to have
a configuration file that works in both an old un-tainted Exim and
a new tainting Exim, since $local_part_data does not exist in older
Exims. Forcing configuration file updates at the same time as Exim
version updates complicates sysadmin life.

 Radical: in ACLs, verifying the receiver or sender address through
'verify = ' should optionally un-taint $local_part, $domain,
and so on. This should be controlled through a new option that has to
be turned on. In many configurations, verifying at least recipient
addresses will automatically restrict them to a list of known good
domains, local parts, and so on. Forcing ACLs to additionally 

Re: [exim] tainted data issues

[Michael Haardt via Exim-users]

Jeremy Harris via Exim-users  wrote:
> The one major hole I know of is for the creation of a
> mailbox file, first time, for an account.

After having reviewed a number of configurations, I am sure there is more.

While I am not pleased with the design of verifying tainted data, or
introducing it in such an invasive manner without a new major version,
the need of doing so absolutely exists.  That said, the current design
is usable and it solves the problem.  Using it may either convince us
of being the best solution, or show which specific alternative is better.

The ongoing configuration reviews certainly uncovered potential problems,
so rolling back is not an option without a replacement for the current
verification.

Michael

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Jeremy Harris via Exim-users]

On 09/11/2020 22:27, Heiko Schlittermann via Exim-users wrote:

We're open for suggestions.


The one major hole I know of is for the creation of a
mailbox file, first time, for an account.

To that end I'm intending an enhancement of the "create_file"
option on the appendfile transport.  The implementation of
this is ready and waiting on the outcome of this discussion.
--
Cheers,
  Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] Trying to understand when $local_part_data is set in ACLs

[Andreas Metzler via Exim-users]

On 2020-11-09 Chris Siebenmann via Exim-users  wrote:
[...]

>  This is all on Ubuntu 20.04 with Ubuntu's version of 4.93.

I would suggest to upgrade to 4.94 + fixes and would keep away from
anything > 4.92 and < 4.94+fixes.

A quick tests on Debian testing works as expected:

  warn local_parts = ametzler
   log_message = local_parts test: $local_part -- $local_part_data

2020-11-10 14:28:21 H=localhost () [::1] Warning: local_parts test: 
ametzler -- ametzler

cu Andreas

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Andreas via Exim-users]

Am 10.11.2020 um 10:33 schrieb Mike Tubby via Exim-users:
> 
> 
> On 10/11/2020 08:37, Julian Bradfield via Exim-users wrote:
>> I thought it was standard practice in introducing a new feature that
>> causes major breakage to existing installations, to take a three step
>> approach. First you provide the feature, and give it an enabling
>> switch with three levels "off", "warn but don't error", "on".
>> Then in successive releases you change the default value of the
>> enabling switch, and ultimately you remove the enabling switch.
>>
>> I understand that taint protection is considered a security feature,
>> but it's a feature exim users have done without for decades, so I
>> can't really see that there was a particularly urgent need to
>> introduce it in a big bang.
>>
> 
> In one word "upvote".
> 
> I am all for improved security but a single "step change" that breaks
> existing configurations is IMHO going too far.
> 
>     taint_mode = off | warn | enforce
> 
> Would have been nice ;-)
> 
> 
> Mike
> 
> 
> 
Or in two words "upvote too".

I think "do or die" never is best practice.

If the next dist-upgrades breaks all of our exim installations all
updgrades had to be done by some "specialists" and and our normal
technicians can't do it during the normal update process without
breaking the whole service.

An exim switch like "test config for security" to check working
configurations for security issues would be nice. Like the "warn" from
above. And if all is good enforce as above. Well, above is nice! :)

And for the files, I really appreciate all your hard and good work and I
really love exim for the freedom to configure it in the way we need it ;)

Andreas


-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Mike Tubby via Exim-users]

On 10/11/2020 08:37, Julian Bradfield via Exim-users wrote:

I thought it was standard practice in introducing a new feature that
causes major breakage to existing installations, to take a three step
approach. First you provide the feature, and give it an enabling
switch with three levels "off", "warn but don't error", "on".
Then in successive releases you change the default value of the
enabling switch, and ultimately you remove the enabling switch.

I understand that taint protection is considered a security feature,
but it's a feature exim users have done without for decades, so I
can't really see that there was a particularly urgent need to
introduce it in a big bang.



In one word "upvote".

I am all for improved security but a single "step change" that breaks 
existing configurations is IMHO going too far.


    taint_mode = off | warn | enforce

Would have been nice ;-)


Mike



--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Kai Bojens via Exim-users]

Am 10.11.20 um 09:37 schrieb Julian Bradfield via Exim-users:


I understand that taint protection is considered a security feature,
but it's a feature exim users have done without for decades, so I
can't really see that there was a particularly urgent need to
introduce it in a big bang.



I'd like to disagree. In light of the recent 2019 RCE CVE's I really 
welcome these changes and think they are long overdue. There's just the 
point of a clear documentation which is missing IMHO.


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Julian Bradfield via Exim-users]

I thought it was standard practice in introducing a new feature that
causes major breakage to existing installations, to take a three step
approach. First you provide the feature, and give it an enabling
switch with three levels "off", "warn but don't error", "on".
Then in successive releases you change the default value of the
enabling switch, and ultimately you remove the enabling switch.

I understand that taint protection is considered a security feature,
but it's a feature exim users have done without for decades, so I
can't really see that there was a particularly urgent need to
introduce it in a big bang.

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Re: [exim] tainted data issues

[Mark Elkins via Exim-users]

On 2020/11/10 08:44, Kai Bojens via Exim-users wrote:

Am 09.11.20 um 23:27 schrieb Heiko Schlittermann via Exim-users:



We're open for suggestions. And intentionally we do not provide
suggestions from our side here and now (this doesn't mean that we do 
not have

ideas ;)) My thoughts I'll present here later.


The only problem I have with tainting is the lack of documentation. 
Why is there no single page with just "Hey, external data is now 
considered tainted. This is how you handle this new stuff:"?


Right now the information about tainting is spread all over the 
documentation so that admins who upgrade have to go through all of this.


...and because of this, I have kept to older versions of EXIM - because 
my configs rely on the fact that all my users are in a MySQL Database.


Some more general "this is how you do it" examples would be greatly 
appreciated.

Thank you Heiko for raising this discussion.

I personally run some 1000 domains with perhaps 4000 e-mail users. Not 
big but not insignificant. I understand that when an email arrives, the 
recipient may not exist - but then the first thing I think I do is see 
if the address exists - and has not been suspended - etc. Surely this 
would cover 'tainted' data checks? Same for mail submission senders, 
they only 'get in' if their username (full email address) and password 
is a valid combination - so what is left to check?


As an aside, I also discovered my MySQL database was running on very old 
software - so there are other issues at hand too - just to confuse my 
particular issues. The old MySQL has just been sorted - so 'tainted' 
data is next.


Running an email service used to be reasonably easy... now people do 
dumb thinks like double SPF records or double sign DKIM (with one always 
broken).


So a suggestion, if its the incoming email that has tainted data - then 
an immediate lookup (give various examples) that then set some globally 
useable variables for everything else - could be an ideal way forward.


--

Mark James ELKINS  -  Posix Systems - (South) Africa
m...@posix.co.za   Tel: +27.826010496 
For fast, reliable, low cost Internet in ZA: https://ftth.posix.co.za 




--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/