(OT)Ham Radio + SMTP (was Re: How to restrict encrypted email)

[jdebert]

On Sat, 16 Jul 2016 11:42:44 -0400
Yuval Levy  wrote:

> It is indeed a matter of interpretation, and I would like to see the
> FCC rules text.  Questions:
> (1) how do they define "encrypted"?

The rules and regulations are very clear on what is permitted. They do
not need to define anything else.

> (2) on who is the obligation imposed?

On all licensed amateur radio operators.

> 
> Imposing the onus on the SMTP server operator is like imposing the
> onus on gas stations for fueling vehicles used in criminal
> endeavors.  It does not fly because the gas station can't possibly
> know what the user will use the vehicle for, other than (probably)
> driving.
> 
> By the definition of encryption, an SMTP server operator can't
> possibly know that a message is encrypted unless the end-user is kind
> enough to say so, e.g. in the MIME headers.
> 
> 
> > Don't let them push you down this slippery slope.  If you are
> > really worried about it, call the FCC or a private attorney and get
> > a solid interpretation.
> 
> If I was the SMTP server operator and they came to me, I'd tell them
> to take a walk.

The encryption ban dates almost from the earliest days of ham radio. It
has included unencrypted digital communications formats as well. It has
been extremely restrictive until recently. The use of ASCII was
prohibited until recently, for example. Violation of the regulations
can result in severe fines and forfeiture of license and equipment.

These are regulations, not laws. There is no due process as there
may be in criminal cases. It's a completely different legal universe.
Enforcement of regulations is administrative and not dealt with in the
courts, until criminal enforcement is necessary.

Please review part 97 of the FCC regulations, which pertains to amateur
radio operation. For the FCC's authority, that would be in Title 47 of
the United States Code.

RE: How to restrict encrypted email

[Michael Fox]

> You could use iptables to look for:
> "--BEGIN"
> "--END"
> "/signed"
> "/encrypted"
> "/pkcs7"
> "/pgp"


Thanks to all.  I've got enough to get me started with my homework.  Lots to
learn.

Regards,
Michael

SV: How to restrict encrypted email

[Sebastian Nielsen]

The problem you got, is that the encrypted content has already travelled the
amateur frequencies even if you block/reject the mail.
Thus the rules are already broken, thus you should deal with those users in
a "AUP" way even if the mail gets blocked. Better might be to block this in
firewall then.
I guess you intend to operate a outgoing mail relay now.

You could use iptables to look for:
"--BEGIN"
"--END"
"/signed"
"/encrypted"
"/pkcs7"
"/pgp"

Anywhere in the packet. In that case, you drop the connection, send  a RST
to source and target, and then you could ban the user account involved.

You can look here how to do with IP-tables, both for blocking encrypted
content, but also triggering some sort of ban.
http://blog.nintechnet.com/how-to-block-w00tw00t-at-isc-sans-dfind-and-other
-web-vulnerability-scanners/
You can also use Fail2Ban for this, and trigger ban based on user account.

(NOTE: Signed content is technically encrypted content too)

-

If you instead operate a incoming mail relay (so internet users can send
messages to amateur radio operators), you said this would only apply to
certain users.
Then its better to block this on IMAP/POP3 level, as the users in question
might connect over public internet. Eg, encrypted mail is received and
stored, but it will never
Be delivered if the user is doing the fetch over radio frequencies (you
could instead send some auto message like "You have 1 new encrypted
message", but if the user happens to fetch over the public internet, then
they get any encrypted content.



smime.p7s
Description: S/MIME Cryptographic Signature

Re: How to restrict encrypted email

[Larry Stone]

> On Jul 16, 2016, at 11:11, Erwan David  wrote:
> 
>> Le 16/07/2016 à 19:04, Jan Ceuleers a écrit :
>>> On 16/07/16 17:42, Yuval Levy wrote:
>>> Imposing the onus on the SMTP server operator is like imposing the onus
>>> on gas stations for fueling vehicles used in criminal endeavors.  It
>>> does not fly because the gas station can't possibly know what the user
>>> will use the vehicle for, other than (probably) driving.
>> You have ignored the OP's statement that he is a radio amateur, and that
>> the FCC prohibits the use of encryption by radio amateurs. This is about
>> ensuring that the spectrum that radio amateurs are licensed to use (a
>> public resource) is not subverted for private purposes. Hams are
>> supposed to be a largely self-policing community; encryption prevents that.
>> 
>> As an individual radio amateur, the OP needs to ensure that he complies
>> with the FCC rules if he wants to keep his license. If he cannot
>> configure his SMTP server in a compliant manner he should not offer an
>> SMTP-based service that transports messages across the ham frequencies.
> Wouldn't this mean that data transport on those frequencies is forbidden ?
> 

I agree. Encryption does not imply TLS. A message could be encrypted while 
still being plain text. For a sufficiently low level definition of encryption, 
it could even be encrypted while appearing to be unencrypted. For instance, if 
two people agree that certain words means something different than their 
commonly accepted meanings, they could communicate in a form that appears to be 
plain language yet have a different meaning to them than to someone who 
intercepts it. But the latter example would also apply to voice communications 
in the amateur bands so since you can't be sure that even voice is unencrypted, 
I guess they aren't legal either.

-- Larry Stone
   lston...@stonejongleux.com

RE: How to restrict encrypted email

[Michael Fox]

Michael Fox:
> > So, are there other obvious ways to recognize encrypted contents, other
> than
> > "Content-Type: multipart/encrypted"?

Albrecht:
> Basically, you need to check for
> - OpenPGP/Inline (inspect every body, see rfc 2440, sect. 6.2)
> - OpenPGP/Mime (multipart/encrypted, see rfc 3156, sect. 4)
> - S/MIME (application/pkcs7-mime with a proper protocol type, see rfc
> 2633, sect. 3.8)

Ah, OK.  That will give me a good start.  Thanks.


Rob0:
> you can use an alternate cleanup service on an alternate 
> smtpd instance, see postconf.5.html#cleanup_service_name and also 
> cleanup.8.html .

Interesting.  I hadn't thought of that.  I'll check into that.


And thanks to everyone for the rest of the comments, legal or otherwise.

I don't want to hijack this list with amateur radio stuff.  But to briefly
respond to those who took the time to respond to me:

-- Encryption should be encouraged:  Generally agreed.  But it's not allowed
on US amateur radio frequencies.

-- All comments about the FCC rules:  I have talked to the FCC and am
confident that I understand the requirements.  

-- You can never block it all:  Understood.  I'm looking to block the
obvious stuff to avoid inadvertent mistakes.  Those who resort to subterfuge
(which hasn't been a problem so far) can be dealt with on an individual
basis as a violation of the acceptable use policy.

Thanks again to all,
Michael

Re: How to restrict encrypted email

[Erwan David]

Le 16/07/2016 à 19:04, Jan Ceuleers a écrit :
> On 16/07/16 17:42, Yuval Levy wrote:
>> Imposing the onus on the SMTP server operator is like imposing the onus
>> on gas stations for fueling vehicles used in criminal endeavors.  It
>> does not fly because the gas station can't possibly know what the user
>> will use the vehicle for, other than (probably) driving.
> You have ignored the OP's statement that he is a radio amateur, and that
> the FCC prohibits the use of encryption by radio amateurs. This is about
> ensuring that the spectrum that radio amateurs are licensed to use (a
> public resource) is not subverted for private purposes. Hams are
> supposed to be a largely self-policing community; encryption prevents that.
>
> As an individual radio amateur, the OP needs to ensure that he complies
> with the FCC rules if he wants to keep his license. If he cannot
> configure his SMTP server in a compliant manner he should not offer an
> SMTP-based service that transports messages across the ham frequencies.
>
Wouldn't this mean that data transport on those frequencies is forbidden ?

Re: How to restrict encrypted email

[Jan Ceuleers]

On 16/07/16 17:42, Yuval Levy wrote:
> Imposing the onus on the SMTP server operator is like imposing the onus
> on gas stations for fueling vehicles used in criminal endeavors.  It
> does not fly because the gas station can't possibly know what the user
> will use the vehicle for, other than (probably) driving.

You have ignored the OP's statement that he is a radio amateur, and that
the FCC prohibits the use of encryption by radio amateurs. This is about
ensuring that the spectrum that radio amateurs are licensed to use (a
public resource) is not subverted for private purposes. Hams are
supposed to be a largely self-policing community; encryption prevents that.

As an individual radio amateur, the OP needs to ensure that he complies
with the FCC rules if he wants to keep his license. If he cannot
configure his SMTP server in a compliant manner he should not offer an
SMTP-based service that transports messages across the ham frequencies.

Re: How to restrict encrypted email

[Yuval Levy]

(Non-US) lawyer here, chiming in after the itch became to strong.
Initially I wanted to stay out of this debate, the solution of which is
obviously non-technical and probably OT.

DISCLAIMER: THE FOLLOWING IS NOT LEGAL ADVICE.

On 16-07-16 11:04 AM, /dev/rob0 wrote:
> You have already discarded STARTTLS from your EHLO reply for packet 
> radio clients, and I think even that is going a bit too far.

In my view, that's the maximum extent to which an SMTP server operator
can be held liable.  I'd disable STARTTLS and forget about the rest.

Principle of the rule of law: if a law can't be possibly followed, no
court in a rule of law country will enforce it.  There is enough
indication in this thread that it is technically impossible to prevent
users from concealing the content of their communication while using
your SMTP server.


> If your interpretation of these FCC rules is accurate, you really 
> can't offer any kind of connection to the Internet in any way, even 
> indirect.  You can't possibly anticipate all kinds of cryptography 
> and steganography.

It is indeed a matter of interpretation, and I would like to see the FCC
rules text.  Questions:
(1) how do they define "encrypted"?
(2) on who is the obligation imposed?

Imposing the onus on the SMTP server operator is like imposing the onus
on gas stations for fueling vehicles used in criminal endeavors.  It
does not fly because the gas station can't possibly know what the user
will use the vehicle for, other than (probably) driving.

By the definition of encryption, an SMTP server operator can't possibly
know that a message is encrypted unless the end-user is kind enough to
say so, e.g. in the MIME headers.


> Don't let them push you down this slippery slope.  If you are really 
> worried about it, call the FCC or a private attorney and get a solid 
> interpretation.

If I was the SMTP server operator and they came to me, I'd tell them to
take a walk.

If they came back and fined me I would not pay and tell them to take a
longer walk.

If they came back and sued me, I would defend in court and argue that
the rule is not enforceable against the SMTP server operator because
technically impossible to comply with.  I would tell them to take an
even longer walk until they may guess that the rule may be enforceable
against the end-user.

If they came to me and ask for the end-user information, I would tell
them to get a warrant.

If they showed me a valid warrant, I would dump on them my log files (or
a justification why I do not keep log files) and tell them good luck.

Yuv

Re: How to restrict encrypted email

[Erwan David]

Le 16/07/2016 à 16:49, Jan Ceuleers a écrit :
> On 16/07/16 15:59, Michael Fox wrote:
>> So, are there other obvious ways to recognize encrypted contents, other than
>> "Content-Type: multipart/encrypted"?
> Theoretical (and therefore possibly entirely impractical) answer:
>
> Encrypted data contains a high amount of entropy, meaning that it does
> not compress well. So that might be a way to detect encryption.
>
> Trouble is, this method would also flag already-compressed data, even if
> the compression has not been password-protected (and so arguably is not
> encrypted).
>
> So you could try a 2-step process: check whether the data is
> compressible. If it is, assume that it is not encrypted. If it is not,
> then check whether you can use the well-known MIME types or file
> extensions to determine whether it contains compressed attachments. If
> you can automatically decompress the attachment (i.e. without a
> password), run the second check again, just to ensure that the
> decompressed file does not again contain encrypted data.
>
> This is expensive and recursive, so possibly open to a DoS attack
> (unless you can put an upper bound on resource usage of the milter that
> does this). Unless you also disallow compressed content even if not
> password-protected, in which case you can omit the second step and
> therefore the recursion.
>
> Furthermore, you would not find steganographically hidden encrypted
> content this way.
>
> Jan
>
I think that pursuing this way you will arrive to allowing only
text/plain messages.

And use some natural language testing on it because any base64 encoded
content can be announced as text/plain

Re: How to restrict encrypted email

[/dev/rob0]

On Fri, Jul 15, 2016 at 10:25:43PM -0700, Michael Fox wrote:
> I'd like to be able to reject mail that contains encrypted content.  
> This is to satisfy US FCC rules against encrypted content on 
> amateur radio frequencies.  Some of our clients may connect via 
> amateur radio.

I think you're taking it too far.  Consider that the regulation was 
written by someone, and then ratified by a group, not one of whom had 
ever heard of GPG.  For that matter, the regulation probably did not 
anticipate packet radio use for IP traffic.  Surely they were only 
thinking about scrambled radio voice traffic.

You have already discarded STARTTLS from your EHLO reply for packet 
radio clients, and I think even that is going a bit too far.

If your interpretation of these FCC rules is accurate, you really 
can't offer any kind of connection to the Internet in any way, even 
indirect.  You can't possibly anticipate all kinds of cryptography 
and steganography.

Israel's then-future-king David, and his friend, then Prince 
Jonathan, cooked up and employed an encryption scheme in the Old 
Testament.  If your interpretation stands, you cannot allow anyone to 
say in email, "... is not the arrow beyond thee?" (KJV, 1 Samuel 
20:37.)  That could contain a hidden message.

Don't let them push you down this slippery slope.  If you are really 
worried about it, call the FCC or a private attorney and get a solid 
interpretation.

> I'd like to be able to restrict it only for certain clients.  But, 
> as I understand it, header checks can only be applied globally, to 
> all mail.

[OOTC: obligatory on-topic content]

Yes, but you can use an alternate cleanup service on an alternate 
smtpd instance, see postconf.5.html#cleanup_service_name and also 
cleanup.8.html .

> Sorry if this is a dumb question.  But, unfortunately, I don't
> have any experience with encrypted mail.  From what I've read,
> I'm thinking I need:
> 
> main.cf:
> 
>   mime_header_checks = pcre:${config_directory}/mime_header_checks.pcre
> 
> 
> mime_header_checks.pcre:
> 
>   # Block encrypted mail
>   /^Content-Type\:.*multipart\/encrypted/ REJECT Encrypted
> content not allowed
> 
> 
> 
> Is that sufficient?

No.  What about inline PGP/GPG messages?  What about encrypted 
messages hidden inside image or audio files?

[/OOTC]

> Any better ideas or other issues to consider?

Don't do it.  See above.
-- 
  http://rob0.nodns4.us/
  Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:

Re: How to restrict encrypted email

[Jan Ceuleers]

On 16/07/16 15:59, Michael Fox wrote:
> So, are there other obvious ways to recognize encrypted contents, other than
> "Content-Type: multipart/encrypted"?

Theoretical (and therefore possibly entirely impractical) answer:

Encrypted data contains a high amount of entropy, meaning that it does
not compress well. So that might be a way to detect encryption.

Trouble is, this method would also flag already-compressed data, even if
the compression has not been password-protected (and so arguably is not
encrypted).

So you could try a 2-step process: check whether the data is
compressible. If it is, assume that it is not encrypted. If it is not,
then check whether you can use the well-known MIME types or file
extensions to determine whether it contains compressed attachments. If
you can automatically decompress the attachment (i.e. without a
password), run the second check again, just to ensure that the
decompressed file does not again contain encrypted data.

This is expensive and recursive, so possibly open to a DoS attack
(unless you can put an upper bound on resource usage of the milter that
does this). Unless you also disallow compressed content even if not
password-protected, in which case you can omit the second step and
therefore the recursion.

Furthermore, you would not find steganographically hidden encrypted
content this way.

Jan

Re: How to restrict encrypted email

[Erwan David]

Le 16/07/2016 à 16:39, Phil Stracchino a écrit :
> On 07/16/16 10:32, Albrecht Dreß wrote:
>> Am 16.07.16 15:59 schrieb(en) Michael Fox:
>>> So, are there other obvious ways to recognize encrypted contents, other than
>>> "Content-Type: multipart/encrypted"?
>> Basically, you need to check for
>> - OpenPGP/Inline (inspect every body, see rfc 2440, sect. 6.2)
>> - OpenPGP/Mime (multipart/encrypted, see rfc 3156, sect. 4)
>> - S/MIME (application/pkcs7-mime with a proper protocol type, see rfc 2633, 
>> sect. 3.8)
>>
>> BTW, IMHO this is the *completely* wrong approach - people should be 
>> *encouraged* to use encryption, at least since we know for sure that their 
>> privacy is threatened everywhere...
> Anyone who knows Theresa May, former UK Home Secretary, just selected as
> the new head of the Conservative party, will be unsurprised to know that
> almost her first public position pronunciation was that she sees no
> problem with passing laws in the UK to prohibit end-to-end encryption
> and require ISPs et al to provide backdoors into all encrypted traffic.
> Because, you know, that idea has no flaws *whatsoever* and has *never*
> ended badly anywhere else...
>
>
And the fact that UK policemen where caught using snooping provision for
personal usage and financial gain won't stop her.




signature.asc
Description: OpenPGP digital signature

Re: How to restrict encrypted email

[Phil Stracchino]

On 07/16/16 10:32, Albrecht Dreß wrote:
> Am 16.07.16 15:59 schrieb(en) Michael Fox:
>> So, are there other obvious ways to recognize encrypted contents, other than
>> "Content-Type: multipart/encrypted"?
> 
> Basically, you need to check for
> - OpenPGP/Inline (inspect every body, see rfc 2440, sect. 6.2)
> - OpenPGP/Mime (multipart/encrypted, see rfc 3156, sect. 4)
> - S/MIME (application/pkcs7-mime with a proper protocol type, see rfc 2633, 
> sect. 3.8)
> 
> BTW, IMHO this is the *completely* wrong approach - people should be 
> *encouraged* to use encryption, at least since we know for sure that their 
> privacy is threatened everywhere...

Anyone who knows Theresa May, former UK Home Secretary, just selected as
the new head of the Conservative party, will be unsurprised to know that
almost her first public position pronunciation was that she sees no
problem with passing laws in the UK to prohibit end-to-end encryption
and require ISPs et al to provide backdoors into all encrypted traffic.
Because, you know, that idea has no flaws *whatsoever* and has *never*
ended badly anywhere else...


-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: 603.293.8485



signature.asc
Description: OpenPGP digital signature

Re: How to restrict encrypted email

[Albrecht Dreß]

Am 16.07.16 15:59 schrieb(en) Michael Fox:

So, are there other obvious ways to recognize encrypted contents, other than
"Content-Type: multipart/encrypted"?


Basically, you need to check for
- OpenPGP/Inline (inspect every body, see rfc 2440, sect. 6.2)
- OpenPGP/Mime (multipart/encrypted, see rfc 3156, sect. 4)
- S/MIME (application/pkcs7-mime with a proper protocol type, see rfc 2633, 
sect. 3.8)

BTW, IMHO this is the *completely* wrong approach - people should be 
*encouraged* to use encryption, at least since we know for sure that their 
privacy is threatened everywhere...

Hth,
Albrecht.

pgpEa5y6YqCoZ.pgp
Description: PGP signature

RE: How to restrict encrypted email

[Michael Fox]

> minimize it with some filtering for the obvious cases
> as you propose.

Thanks Marco.  I hadn't thought of some of those cases.  
But I would still like to block the obvious cases, as you say.

So, are there other obvious ways to recognize encrypted contents, other than
"Content-Type: multipart/encrypted"?

Thanks,
Michael

Re: How to restrict encrypted email

[Marco]

Hello Michael.

It is a near impossible task, as i.e. encrypted data can easily be
transferred as uuencode text in a plain text message, and you will not
notice this in the headers.
And also trying to actively detected whether or not the body or the
attachments of a message are encrypted is quite tricky, and also
potentially prone to a number of positive false detections.
If this is not enough steganography and other methodologies can be used
to hide encrypted data, making the task really complex even for high-end
organizations

More, you could maybe have some form of binding agreement with the
ham-radio operators, prohibiting the usage of encryption. However you
will be not able to have a such legal protection with all other Internet
users sending mails to the ham-radio users.

This means you have to decide whether or not you want to take the risk,
and if yes, maybe, minimize it with some filtering for the obvious cases
as you propose.

Have a nice day

Marco

Il 16. 07. 16 07:25, Michael Fox ha scritto:
> I'd like to be able to reject mail that contains encrypted content.  This is
> to satisfy US FCC rules against encrypted content on amateur radio
> frequencies.  Some of our clients may connect via amateur radio.
>
> I'd like to be able to restrict it only for certain clients.  But, as I
> understand it, header checks can only be applied globally, to all mail.
>
> Sorry if this is a dumb question.  But, unfortunately, I don't have any
> experience with encrypted mail.  From what I've read, I'm thinking I need:
>
> main.cf:
>
>   mime_header_checks = pcre:${config_directory}/mime_header_checks.pcre
>
>
> mime_header_checks.pcre:
>
>   # Block encrypted mail
>   /^Content-Type\:.*multipart\/encrypted/ REJECT Encrypted
> content not allowed
>
>
>
> Is that sufficient?
> Any better ideas or other issues to consider?
>
> Thanks,
> Michael
>
>

Re: Postscreen white listing based on MX, SPF

[Erwan David]

Le 16/07/2016 à 11:11, Lefteris Tsintjelis a écrit :
> On 16/07/2016 11:35, Jim Reid wrote:
>> That wouldn’t have worked anyway.
>>
>> Assuming a reverse lookup of an IP address returns a name -- a big if
>> -- there’s no guarantee that name has any relation to whatever domain
>> name is in the MAIL FROM.
>>
>> For instance, lots of organisations outsource their email to
>> specialised hosting/service providers. A MAIL FROM might well say
>> example.com while the reverse lookup returns some name in
>> $emailprovider.com for a server that’s delivering email sent by
>> bazillions of customers.
>
> No guarantees of course but the majority of valid MX servers return a
> name.
>
> It would have worked for smaller orgs where they usually have one MX
> server and a relativly simple mail and domain setup. Besides, this would
> have been a white listing only method so no harm done other than the
> additional DNS lookups but I guess it is not postscreen's job for this.
>
MX are for *incoming* messages. No garantee at all that outgoing servers
are the same. For big companies, it is quite standard to use different
servers for incoming and outgoing messages.

Re: Postscreen white listing based on MX, SPF

[Lefteris Tsintjelis]

On 16/07/2016 11:35, Jim Reid wrote:

That wouldn’t have worked anyway.

Assuming a reverse lookup of an IP address returns a name -- a big if
-- there’s no guarantee that name has any relation to whatever domain
name is in the MAIL FROM.

For instance, lots of organisations outsource their email to
specialised hosting/service providers. A MAIL FROM might well say
example.com while the reverse lookup returns some name in
$emailprovider.com for a server that’s delivering email sent by
bazillions of customers.


No guarantees of course but the majority of valid MX servers return a
name.

It would have worked for smaller orgs where they usually have one MX
server and a relativly simple mail and domain setup. Besides, this would
have been a white listing only method so no harm done other than the
additional DNS lookups but I guess it is not postscreen's job for this.

Re: Postscreen white listing based on MX, SPF

[Jim Reid]

> On 16 Jul 2016, at 02:50, Lefteris Tsintjelis  wrote:
> 
> I was thinking it more in simple DNS terms only and a simply reverse
> look up of the IP and then extract the domain from there but it is not
> possible without the FROM.

That wouldn’t have worked anyway.

Assuming a reverse lookup of an IP address returns a name -- a big if -- 
there’s no guarantee that name has any relation to whatever domain name is in 
the MAIL FROM.

For instance, lots of organisations outsource their email to specialised 
hosting/service providers. A MAIL FROM might well say example.com while the 
reverse lookup returns some name in $emailprovider.com for a server that’s 
delivering email sent by bazillions of customers.