Re: [exim] tainted data issues
One could do this for a punycode version of the domain name but the
address part before an '@' can be UTF8 - such as "café". Please don't
break any internationalised addresses (Universal Acceptance and all
that). I'm wondering if an inverse check could be done, as in look for
anything bad? (e.g. Double dots and slashes)
On 2020/11/10 22:45, Sebastian Nielsen via Exim-users wrote:
I think as I said, provide a untaint tool, that allows custom data to verify
against.
Like:
${untaint(${var},
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")}
Regardless of if ${var} is tained or not, the result of ${untaint} is always
untainted.
The character string, is then what to filter against - all characters not on
that particular list, will be removed from ${var} silently before passing it
on.
This allows system administrators, to do entirely custom checks against
dangerous data, since in all cases where tainted data becomes a security
issue, is when delimiter characters is inserted in the text to escape out of
the enviroment that the command or whatever is executed in, and be able to
execute arbitary commands or write in arbitary locations.
Its very important that the function itself is "safe" - meaning it should
not be possible in ANY way for tainted data to "leak through" the filter,
ergo it must not be possible for any character not on administrator's list,
to end up in the return of the untaint function.
Also the filter list itself, must also be done in such a way variable
expansion never happens in the filter list - only way to specify a filter
list, is to manually specify it in config file.
This gives most flexibility, and also encourages security by making it easy
to filter data before passing it on, making it a "habit" for system
administrators.
Then it will be a habit to always ${untaint} even in cases where untainted
data is not required, incresing security even further.
In the above example, if ${var} contains
"Sebastian/Users/f...@somewhere.com" the result of the untaint operation
with that particular filter list would be "SebastianUsersfilesomewherecom"
which is a completely safe string to pass on to filesystem.
To make it even easier for system administrators, make premade filter lists
like:
%%FILESYSTEM%% = Contains a list of characters that is safe for use in
functions that read or write to filesystem.
%%SYSTEM%% = Containst a list of charactes that is safe to pass on to a live
terminal, like when running system commands or similiar.
%%SQL%% = Contains a list of characters that is safe to pass on to a raw SQL
prompt without rsik of injection.
%%HTML%% = Contains a list of characters that is safe to output to a web
browser without security risks.
%%LOCALPART%% = Contains a list of characters safe in localpart of a email
adress.
%%HOSTPART%% = Contains a list of characters safe to use in a DNS name or
host part of a adress including IP literals.
%%FULLEMAIL%% = Contains a list of characters that is allowed and safe to
use in a full email adress.
So writing:
${untaint(${var}, "%%SQL%%")}
Would filter ${var} in such a way that its safe to pass on to a raw SQL
string without injection risk.
Note that these %% strings is NOT string expansions - instead a customized
system for specifying pre-made filters, preventing variables from being
accidentially expanded and used in a filter list, which in itself is a big
security risk, since then filter lists could be modifyed from the outside.
However, \xXX should be expanded of course, so people can write characters
that don't exist on the keyboard, in some cases they are useful.
Such a untaint function should be easy to implement.
And would make exim much more secure, as its a very easy function, making it
a habit to use the function by system administrators.
YES - it is possible to "shoot yourself in the foot" with the function by
specofying characters that is unsafe for that particular usage, but then the
responsibility is on the system administrator.
Best regards, Sebastian Nielsen
-Ursprungligt meddelande-
Från: Michael Haardt via Exim-users
Skickat: den 10 november 2020 20:16
Till: Jeremy Harris via Exim-users
Ämne: Re: [exim] tainted data issues
Jeremy Harris via Exim-users wrote:
The one major hole I know of is for the creation of a mailbox file,
first time, for an account.
After having reviewed a number of configurations, I am sure there is more.
While I am not pleased with the design of verifying tainted data, or
introducing it in such an invasive manner without a new major version, the
need of doing so absolutely exists. That said, the current design is usable
and it solves the problem. Using it may either convince us of being the
best solution, or show which specific alternative is better.
The ongoing configuration reviews certainly uncovered potential problems, so
rolling back is not an option without a replacement for the current
verification.
Michael
--
## List details
Re: [exim] tainted data issues
>>I think it's relatively important to let people guard these de-taintings with safety checks, such as 'is there dangerous content here'. Agree, thats why I propose a simple character filter that also de-taints variables. >> I feel that people should not need to be experts in knowing what are safe and dangerous characters and character sequences in order to create safe Exim configurations. Agreed, thats why I also propose the "standard sets" like %%SQL%%, %%FILESYSTEM%% etc that give safe character sets to use with a particular use case. So theres both "standard" proven ways to do it, but also custom ways to do it if you have special use cases. smime.p7s Description: S/MIME Cryptographic Signature -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
Jeremy Harris: > > Semi-radical: provide an ACL, router, and transport modifier that > > checks some variable or content for dangerous contents > > We have that. All data provided by an untrusted source, described > as "tainted" for a shorthand. Tainted variables contain potentially dangerous contents, not actually dangerous contents. Most of the time, the contents of tainted variables are not dangerous, but sometimes they are. I think that it would be useful for Exim to provide assistance in telling the two apart. I say this because I strongly believe that people are going to write Exim configuration code that de-taints variables in brute force ways (and the more that Exim doesn't provide mechanisms to do relatively arbitrary 'safe' de-tainting, the more that people are going to do so). I think it's relatively important to let people guard these de-taintings with safety checks, such as 'is there dangerous content here'. Also, even with relatively safe de-tainting, sometimes I would rather reject funny content immediately. This is actually a relatively popular thing to do today in ad-hoc ways; for example, the Debian 'split' Exim configuration has for years shipped with a set of checks for dangerous characters in local parts. Sysadmins can maybe write these checks in Exim configurations themselves, but in ad-hoc ways and sysadmins probably don't know as much about what things are dangerous (or valid) in various conditions as other people do. - cks -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
On 11/11/2020 16:29, Ian Zimmerman via Exim-users wrote:
On 2020-11-11 13:16, Jeremy Harris wrote:
Semi-radical: provide an ACL, router, and transport modifier that
checks some variable or content for dangerous contents
We have that. All data provided by an untrusted source, described
as "tainted" for a shorthand.
I will not argue with the rest of your post, but it is not a _modifier_
if it is always on.
Ah. Would an expansion condition be sufficient? So you could write
${if tainted{my_suspect_expansion} {expand_this} {expand_that}}
That would be simple to code and test.
--
Cheers,
Jeremy
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
Jeremy Harris via Exim-users wrote:
> > Radical: $local_part, $domain, and similar tainted variables should
> > be automatically un-tainted once they pass a check that would generate
> > the untainted version of the variable.
> On the other side of the coin, re-using the same variables and
> changing the taint-status of the content makes problem-discovery
> harder. When you do run into a taint issue, under the "same variables"
> approach, it is not immediately obvious from the config file source
> that you forgot to de-taint the value.
The problem is that right now it is not obvious which variables are
tainted, there are no *_data variables for all tainted variables and
there is no generic mechanism to verify a tainted variable.
> > Radical: in ACLs, verifying the receiver or sender address through
> > 'verify = ' should optionally un-taint $local_part, $domain,
> > and so on.
>
> Considerably more complex in its security aspects; more than I want
> to think about at this time. Maybe one for the future.
>From the little experience so far it shows that verification of tainted
variables is logically tied to routers. A central ACL does not
match that structure.
Domains and local_parts are checked very early in the router and I
believe verifying variables early in the router is the right place.
The problem is that the variable verification side effect is neither
obvious nor universal usable for all variables, and I am not
sure we need copies of all variables that are tainted.
> > Radical: provide an ACL and/or router modifier that immediately
> > un-taints $local_part,
>
> No. As responded to elsewhere: providing an overly-obvious way
> to game the security means no security.
To me, half way there would be right: Add a new option for routers that
is checked before all other options and verifies variables, detainting
them instead of copying them. The actual verification using a lookup
is fine to me. It was already shown how to use lookups against regular
expressions, no need to make it easier than that.
To avoid confusion, as verification has its own meaning in Exim, it
could be called validation:
validate_local_part_suffix = ${lookup {$local_part_suffix} }
That way it is obvious which variables are valid and which are not,
validation stays pretty much where it is now and it is clear that in
order to use a variable at a critical place, it must be validated first.
No hidden side effect.
Like local_parts/domains right now, only generic, without creating a
copy of the variable and applicable to all tainted variables.
It would be cool if we had something that validates all variables used
as the key of the lookup for query lookups, again allowing single query
lookups that make use of local part, domain and possibly other variables.
The current need to duplicate lookups makes configurations less clear
than they could be.
Michael
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
On 2020-11-11 13:16, Jeremy Harris wrote: > > Semi-radical: provide an ACL, router, and transport modifier that > > checks some variable or content for dangerous contents > We have that. All data provided by an untrusted source, described > as "tainted" for a shorthand. I will not argue with the rest of your post, but it is not a _modifier_ if it is always on. -- Ian -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
> Yes, but its a positive match only - meaning you have to explicitly > specify allowed characters.The function should NOT be able to > specify forbidden characters - as then it would ve easy to miss bad > characters.If an sysadmin then writes a filter which is too broad - > its his own fault. History has vividly demonstrated that providing a toolkit and asking people to build their own secure implementation from it does not create effective security (although it does let you blame a different set of people for the resulting insecurity). Many people will either make mistakes or not know all of the surprises and traps that are lurking, and so won't know to counter them. Effective security is almost always created by a few people who know the area very well spending a bunch of time to thoroughly explore all of the dark corners, then building something with as few options as possible (ideally none). Or to put it another way: I feel that people should not need to be experts in knowing what are safe and dangerous characters and character sequences in order to create safe Exim configurations. If they have to be such experts, there will be a lot of unsafe Exim configurations created. I think it's potentially sensible to provide an escape hatch, but I don't think that it should be the only way to do it; I think it should only be something that you resort to if you have highly unusual needs. And I share Jeremy Harris's worry that it will be cargo-culted and copied widely, even then. (Possibly unlike Jeremy Harris, I feel that if Exim provides no such safe-ish general un-tainting, people will still find ways to do it and then cargo-cult it. People are going to cargo-cult answers for this; the only question is whether Exim can help make that as safe as possible.) - cks -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
On Wed, 11 Nov 2020, Sebastian Nielsen via Exim-users wrote: Yes, but its a positive match only - meaning you have to explicitly specify allowed characters.The function should NOT be able to specify forbidden characters - as then it would ve easy to miss bad characters.If an sysadmin then writes a filter which is too broad - its his own fault. Even that has pitfalls once you add non-ascii characters. I mean - I have a Email-to-sms gateway which pipes data to a system script.@sebbe.eu is interpreted as outgoing SMS.With the current structure, you need to add every number you want to SMS as whitelist - as you need to do a successful lookup to untaint.Its much better to be able to specify that localpart can only contain numbers to be permitted to be piped to the script - no security risk as nobody can escape out of a shell command with only 0-9 to their disposal. I wonder whether a specific "telephone number" option would make sense ? Do we allow the international code "+", or the pause (which can be used in fax numbers) https://www.dummies.com/consumer-electronics/smartphones/droid/how-to-add-pauses-when-dialing-a-number-on-your-android-phone/ ? -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
Yes, but its a positive match only - meaning you have to explicitly specify
allowed characters.The function should NOT be able to specify forbidden
characters - as then it would ve easy to miss bad characters.If an sysadmin
then writes a filter which is too broad - its his own fault.I mean - I have a
Email-to-sms gateway which pipes data to a system script.@sebbe.eu is
interpreted as outgoing SMS.With the current structure, you need to add every
number you want to SMS as whitelist - as you need to do a successful lookup to
untaint.Its much better to be able to specify that localpart can only contain
numbers to be permitted to be piped to the script - no security risk as nobody
can escape out of a shell command with only 0-9 to their disposal.
Originalmeddelande Från: Jeremy Harris via Exim-users
Datum: 2020-11-11 14:00 (GMT+01:00) Till:
exim-users@exim.org Ämne: Re: [exim] tainted data issues On 10/11/2020 20:45,
Sebastian Nielsen via Exim-users wrote:> I think as I said, provide a untaint
tool, that allows custom data to verify> against.> > Like:> ${untaint(${var},>
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")}No; this is a
bad idea.It is far to easy for someone to write a matcher which justuntaints
everything, disabling the security. Three peoplewould do that, and one would
post it on serverfault. Thenit would be cargo-culted forever.-- Cheers,
Jeremy-- ## List details at
https://lists.exim.org/mailman/listinfo/exim-users## Exim details at
http://www.exim.org/## Please use the Wiki with this list -
http://wiki.exim.org/
smime.p7s
Description: S/MIME Cryptographic Signature
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
On 10/11/2020 19:45, Chris Siebenmann via Exim-users wrote: Moderate: there should be a full chapter in the Exim documentation on tainting and how to deal with it. This should cover the security risks that it's there to deal with, common configuration snippets that are now a problem, and how to deal with tainting and de-tainting data in common configuration needs. Right now there is not even an explicit subsection for tainting in the Security Considerations chapter. (If in the process of writing this chapter things are difficult to explain or the configuration changes seem awkward, well, that is a sign of where improvements are needed.) The problem I had in writing documentation was in the opposite direction: Exim's taint-tracking is a very simple concept and does not take many words to describe. Nor does the ethos of de-taint methods. It didn't take anything near a chapter. Please see the current (post-4.94 release) state of the documentation, posted at https://people.exim.org/jeremy.harris/docs/HEAD/exim-html-4.94-RC999/exim-html-current/doc/html/spec_html/ Semi-radical: there should be a configuration file option that turns taint errors into warnings that do not stop processing but that are merely logged to the main log or perhaps also the paniclog. As noted elsewhere, possible though expensive in developer time. (It would be nice if all system administrators had automated test suites that completely cover their entire Exim configuration. If people think that having such a test suite should be required for reliable Exim version upgrades, perhaps someone should write a chapter for the Exim documentation on how to create and operate it, and also how to measure the configuration file coverage so that you know you have 100% coverage.) "Someone" needs to volunteer, obviously. An optional but nice extra would be a third choice for this option, which generates 'this would have been rejected' warnings if the contents of the tainted variable being used seem harmless, but force hard errors if it appears to have dangerous contents (whatever the Exim developers consider that to be). More complexity is bad. Asking developers to think harder probably also isn't good. Semi-radical: provide an ACL, router, and transport modifier that checks some variable or content for dangerous contents We have that. All data provided by an untrusted source, described as "tainted" for a shorthand. (possibly in several variations), No, complexity again. Radical: $local_part, $domain, and similar tainted variables should be automatically un-tainted once they pass a check that would generate the untainted version of the variable. Forcing system administrators to switch which variable they use has two problems. First, it is make-work, since the two variables are essentially bolted together (and if they are not, it is likely to surprise people, for example if you somehow get $local_part changed but $local_part_data stays with an old value). Second, it is part of what makes it impossible to have a configuration file that works in both an old un-tainted Exim and a new tainting Exim, since $local_part_data does not exist in older Exims. Forcing configuration file updates at the same time as Exim version updates complicates sysadmin life. On the other side of the coin, re-using the same variables and changing the taint-status of the content makes problem-discovery harder. When you do run into a taint issue, under the "same variables" approach, it is not immediately obvious from the config file source that you forgot to de-taint the value. This one is open to more opinions. I chose the "different variables" approach Radical: in ACLs, verifying the receiver or sender address through 'verify = ' should optionally un-taint $local_part, $domain, and so on. Considerably more complex in its security aspects; more than I want to think about at this time. Maybe one for the future. Radical: provide an ACL and/or router modifier that immediately un-taints $local_part, No. As responded to elsewhere: providing an overly-obvious way to game the security means no security. -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
On 10/11/2020 20:45, Sebastian Nielsen via Exim-users wrote:
I think as I said, provide a untaint tool, that allows custom data to verify
against.
Like:
${untaint(${var},
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789")}
No; this is a bad idea.
It is far to easy for someone to write a matcher which just
untaints everything, disabling the security. Three people
would do that, and one would post it on serverfault. Then
it would be cargo-culted forever.
--
Cheers,
Jeremy
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
On 10/11/2020 09:33, Mike Tubby via Exim-users wrote: I am all for improved security but a single "step change" that breaks existing configurations is IMHO going too far. taint_mode = off | warn | enforce Warn and enforce, I could see as an interim measure. But only interim - to be remove at some future release. I do not think it should be possible to turn off. This would make it pointless. The coding to implement this would not be too hard. The regression-testing impact is somewhat higher, and I am not particularly incentivised to spend the effort, personally. Do I hear volunteers? There is also the forward-committed work of removing it all later (and handling the same set of complaints yet again). Otherwise it remains as a bolted-on excrescence, hampering future maintenance. -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] tainted data issues
On 11/10/20 11:36 PM, Heiko Schlittermann via Exim-users wrote:
Hi,
I welcome the suggestions, especially the idea about gradually enabling
taintchecks, to allow a smooth transition, as suggested by Mike Tubby.
taint_mode = yes | no | warn
Another idea from my side (it's similar to Sebastian N's idea)
begin transports
smtp:
driver = smtp
dkim_domain = $sender_address_domain
dkim_selector = 2020-08-25
dkim_private_key = /etc/exim/dkim/$dkim_selector.$dkim_domain.pem
We could provide taint checks for different situations, as the risk of
using tainted data depends on the usage of the data (filename, log
message, lookup key, …)
Provide a new set of functions:
${XXX{}{}{}}
${XXX{}{}fail}
${XXX{}{}}
With XXX as
- file (no "/")
- path (no "..")
- line (no "\r", "\n")
...
dkim_private_key =
/etc/exim/dkim/${file{$dkim_selector.$dkim_domain.pem}}
or
dkim_private_key =
${path{/etc/exim/dkim/$dkim_selector.$dkim_domain.pem}}
This can give us flexibility where the current lookup based way of
untainting doesn't work.
I like the functions idea the best, as tainting is _already_ here, but
really either way could do.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
