Hey Marc,
glad to hear to great news.
After your explanation I know can understand your hassle. As said, for
me james is just running fully external on my root-server at OVH - and
my backup is really just for receiving when I bring down my root for
maintanance (weekly security updates). That's why I never encountered
the very issues you speaking about. Guess that's what you have to deal
with when you use james in an office / corporate network where james
is running on the main server (or maybe even on another machine just
communicating over the main-router/-server).
I've started using Citadel/UX - but they dropped support for SuSE at
some point so I just needed an easy to use replacement as I still
can't get other main MTAs like postfix, sendmail, exim, others - to
work properly. Most promising was once a try with a tutorial specific
written for then-up-to-date versions for I think it was back for 13.1
or so with postfix and postfix-admin - but I also failed with this
step-by-step guide. Also, what no-other MTA than james could offer me
is one thing: not only using database for user management - but also
for mail storage. This way I can easy backup by a database dump.
I fully aggree with you on the mentioned matcher/mailet vs smtp-auth -
espacially as it is mentioned in the doc itsefl. Also the
"master-override" might be useful - for what ever odd reason.
So, let's get started with let's encrypt then:
This is what I use the get a new certificate:
import org.shredzone.acme4j.Session;
import java.security.KeyPair;
import org.shredzone.acme4j.util.KeyPairUtils;
import java.io.FileReader;
import org.shredzone.acme4j.Account;
import org.shredzone.acme4j.AccountBuilder;
import java.net.URL;
import org.shredzone.acme4j.Login;
import org.shredzone.acme4j.Account;
import org.shredzone.acme4j.Order;
import org.shredzone.acme4j.Authorization;
import org.shredzone.acme4j.challenge.Dns01Challenge;
import org.shredzone.acme4j.Status;
import org.shredzone.acme4j.util.CSRBuilder;
import org.shredzone.acme4j.Certificate;
import java.io.FileWriter;
public final class Acme
{
public final static void main(final String... args) throws
Exception
{
Session session=new Session("acme://letsencrypt.org");
KeyPair accountKeyPair=KeyPairUtils.readKeyPair(new
FileReader("account-openssl.key"));
Account account=(new
AccountBuilder()).onlyExisting().useKeyPair(accountKeyPair).create(session);
URL accountLocationUrl=account.getLocation();
Login login=session.login(accountLocationUrl,
accountKeyPair);
Order
order=account.newOrder().domains("cryptearth.de",
"*.cryptearth.de").create();
for(Authorization auth : order.getAuthorizations())
processAuth(auth);
KeyPair domainKeyPair=KeyPairUtils.readKeyPair(new
FileReader("server-openssl.key"));
CSRBuilder csrBuilder=new CSRBuilder();
csrBuilder.addDomain("cryptearth.de");
csrBuilder.addDomain("*.cryptearth.de");
csrBuilder.sign(domainKeyPair);
order.execute(csrBuilder.getEncoded());
while(order.getStatus()!=Status.VALID)
{
System.out.println(order.getStatus());
if(order.getStatus()==Status.INVALID)
throw new RuntimeException("invalid");
Thread.sleep(3000L);
order.update();
}
System.out.println(order.getStatus());
Certificate certificate=order.getCertificate();
System.out.println(certificate.getLocation());
FileWriter fileWriter=new FileWriter("chain.crt");
certificate.writeCertificate(fileWriter);
fileWriter.flush();
fileWriter.close();
}
private final static void processAuth(Authorization auth)
throws Exception
{
Dns01Challenge
challenge=auth.findChallenge(Dns01Challenge.TYPE);
System.out.println(auth.getDomain());
System.out.println(challenge.getDigest());
System.in.read();
challenge.trigger();
while(challenge.getStatus()!=Status.VALID)
{
System.out.println(challenge.getStatus());
if(challenge.getStatus()==Status.INVALID)
throw new RuntimeException("invalid");
Thread.sleep(3000L);
challenge.update();
}
System.out.println(challenge.getStatus());
}
}
This requires the acme4j, bouncycastle, jose4j and slf4j libs -
instructions can be found following from let's encrypt site to acme4j
github-repo. The two key-files are just two RSA4096 keypairs in
openssl-style pem. This is very important, as acme4j uses
bouncy-castles openssl-pem-reader.
There is a wired quirk in how different keys can be represented:
openssl only adds the numbers really used and a OID in front
java internal always uses a full RSA-private-CRT-key - even for public
keys - but all private values just set to 0 - and when exporting there
is no leading OID
So you have to supply the right type of keys - if you don't you get an
exception about wrong key type.
These lines uses the DNS-type of acme challenge - you can read up in
let's encrypt wiki where and how to set the data returned by this
code. For example, when runnig this code, I get one line I have to set
as _acme-challenge.cryptearth.de TXT - the wait a bit for my dns
provider to refresh the zone file - then just hit enter to pass the
first challenge for main domain cryptearth.de. The second is the same
but for wildcard *.cryptearth.de. It's easier to just setup you
certificate that way since it's supported by let's encrypt.
Then, chain.crt contains both, my server certificate and the let's
encrypt intermediate certificate. For step two, you first have to
split them into thier own files. Then you can run this code to create
the keystore:
import java.math.BigInteger;
import java.security.*;
import java.security.spec.*;
import java.security.cert.*;
import java.io.*;
public final class James
{
public final static void main(final String... args) throws Exception
{
KeyStore keyStore=KeyStore.getInstance("JKS");
keyStore.load(null, null);
DataInputStream din=new DataInputStream(new
FileInputStream(new File("tls.key")));
BigInteger p=new BigInteger(din.readUTF(), 16), q=new
BigInteger(din.readUTF(), 16);
din.close();
BigInteger N=p.multiply(q),
phi=p.subtract(BigInteger.ONE).multiply(q.subtract(BigInteger.ONE)),
e=BigInteger.valueOf(65537L), d=e.modInverse(phi),
dmp1=d.mod(p.subtract(BigInteger.ONE)),
dmq1=d.mod(q.subtract(BigInteger.ONE)), iqmp=q.modInverse(p);
PrivateKey
privateKey=KeyFactory.getInstance("RSA").generatePrivate(new
RSAPrivateCrtKeySpec(N, e, d, p, q, dmp1, dmq1, iqmp));
java.security.cert.Certificate
main=CertificateFactory.getInstance("X509").generateCertificate(new
FileInputStream(new File("server.crt")));
java.security.cert.Certificate
inter=CertificateFactory.getInstance("X509").generateCertificate(new
FileInputStream(new File("le-inter.crt")));
keyStore.setKeyEntry("james", privateKey,
"secret".toCharArray(), new java.security.cert.Certificate[] { main,
inter });
keyStore.store(new FileOutputStream(new File("james.jks")),
"secret".toCharArray());
}
}
You should replace the key-load bit - as this only fits my special
type of private key file - wich just contains of the two primes P and
Q - and is calculated on the fly - another way would be to use
java.security.spec.PKCS8EncodedKeySpec - I have to re-work this code
as it's over 4 years old now.
Important: Although java keystore supports different keys for keystore
itself and for each key - you have to use same for both - as the
config only allows to set one passphrase. This all comes down to
Microsoft - wich, as once leader of marketshare, enforced all others
to use same phrase for keystore and key itself - damn you Microsoft ...
After you created the keystore - head to the config files
smtpserver.xml, imapserver.xml and maybe pop3server.xml if you use
pop3 (wich I have disabled) and set "startTLS" to true and the secret
for your keystore. This way, you have enabled your smtp and imap to
accept STARTTLS from client and upgrade insecure connection up to
secured one.
One last step: to enable outgoing StartTLS when you sent a mail to
others, go into mailetcontainer.xml - to remotedelivery section - and
then right after the "outgoing" line - put
<startTLS>true</startTLS>
just right in there. This way, your james will try to StartTLS when
connected outgoin to other MX-servers. For example: gmail will tell
you if mail was received over encrypted connection or not.
Yes, this is also antoher long mail - but sadly these topics not or
only barely covert in current docs - so it took me long time to figure
all this out. I'm glad I'm now be able to share this knowledge to
others so they can get it done right fast without much hassel.
So long,
Matt
Am 20.02.2019 um 21:20 schrieb Marc Chamberlin:
Hi Matt - We need you to put all your wonderful replies and commentary
up on the James website as part of the documentation. Some of the stuff
there is pretty sparse and you are doing a great job of explaining
things! ;-) I will intersperse a few comments below -
On 02/20/2019 09:50 AM, cryptearth wrote:
Evening all, Matt here.
Marc, let's look at the doc:
"This is an anti-relay matcher/mailet combination
Emails sent from servers not in the network list are rejected as spam.
This is one method of preventing your server from being used as an
open relay. Make sure you understand how to prevent your server from
becoming an open relay before changing this configuration. See
also<authorizedAddresses>in SMTP Server
This matcher/mailet combination must come after local delivery has
been performed. Otherwise local users will not be able to receive
email from senders not in this remote address list.
If you are using this matcher/mailet you will probably want to update
the configuration to include your own network/addresses. The matcher
can be configured with a comma separated list of IP addresses
wildcarded IP subnets, and wildcarded hostname subnets.
e.g. "RemoteAddrNotInNetwork=127.0.0.1, abc.de.*, 192.168.0.*"
Understood. This is a perfectly valid approach to cutting down on spam
being sent through a James server.. It would be interesting to know if
this is the most commonly used approach, or whether most servers are
using SMTP authentication instead, or whether most servers are using
both methods. My argument is not against using this particular
matcher/mailet, but that the default configuration files should come
supplied and set up in a way that reflects the most common usage. To
restrict emails to only come from users on the local host, by default in
the supplied config file, seems to be awfully restrictive and uncommon
usage, but I am only guessing. My suspicion is that most folks using
James are going to use SMTP authentication, at least that is my own
personal experience, and for users to be on a LAN/WLAN.
So I am wondering if this matcher/mailet should not be enabled by
default and SMTP authentication should be enabled instead, by default. I
understand the need for James to start up safely, from the default
configurations, so as not to be an open relay by default.
If you are using SMTP authentication then you can (and generally
should) disable this matcher/mailet pair."
I think this relationship between using SMTP authentication and this
matcher/mailet should be automated. In other words, if SMTP
authentication is turned on then this matcher/mailet should be disabled
by default automatically. And vice/versa. I also think that the
administrator should be able to override this automated relationship,
with an explicitly set option, if for some reason both or neither
approaches are wanted.
Again, the real question is, what is the most common way James is being
configured, and how can mistakes, such as I made, be minimized. The goal
being to keep James robust and easy to manage.
So, as far as I understand it: "Don't touch it if you don't understand
it - but you should remove it anyway when smtp auth is used.". Guess
that's it for you.
I took the "Don't touch it" approach as much as I could. Trouble is I
didn't catch this somewhat hidden matcher/mailet nor did I expect that
the James server would come up with a very restrictive policy that was
preventing me from testing/using it from somewhere else on my LAN.
Especially after I had enabled SMTP authentication, which kinda implied,
at least to me, that I would be able to use James from across my LAN.
This is re-enforce by the observation the IMAP and POP3 were working
from across my LAN and made it difficult to understand why SMTP
wouldn't.
I've never encountered that as I only have my domain cryptearth.de in
domainlist - neither localhost nor other local entries. I've never
tried to send a mail to localhost - allthough, that's one part of my
own current thread about overwrite local service mails from
sendmail-nullclient used by apache and cron - but that's its own
topic. So still have this matcher/mailet in my config, allthough I
have smtp auth enabled.
So, as far as I understood your reply, you now finally got james up
and running so you can also send mails to others?
Yep! :-) And don't get me wrong, I am NOT complaining about Apache James
really, just throwing out some thoughts to think about, which might make
it easier for others following in my footsteps, in installing and
bringing up James. I am very impressed with the amount of work that has
obviously gone into developing James, and totally appreciate the amount
of support you and Benoit have given me!
I am going to work on getting SSL/TLS working with LetsEncrypt
certificates next... Marc..
Matt
Am 20.02.2019 um 16:59 schrieb Marc Chamberlin:
Morning Benoit ;-) This could get into being a philosophical
discussion
for certain! I have mixed feelings about customization of error
messages, and you are correct in saying I could change this particular
one. I have always approached software design with the attitude that
error handling and error messages should be carefully crafted so as to
guide users to a solution, not just tell them that something went
wrong.
Which is what this particular error message is doing when left in it's
current default state. We could change/customize it for our own users,
(actually I will just remove this mailet) but doing so leads to a
different issue. If everyone who installs James servers (or any other
application for that matter) is allowed to customize error messages
then
it leads to a non-standard environment. Often, when users encounter an
error message, that doesn't provide an understandable solution, they
will then Google it looking for a solution, hoping to find a guru or a
collective mind to provide one. Even in cases such as this, where the
solution will require the assistance of the James administrators to
solve this problem, the user needs to be told that he/she must contact
them AND what exactly they need to tell the administrators. I would
craft this message to say, "Your email server is rejecting your
request
to send your email messages. Please contact your Internet Service
Provider and/or IT administrator and tell them that your email
server is
rejecting your request to relay email because it is not configured to
accept email from your IP address. They need to check the
configuration
of the anti-relay matcher/mailet or remove this matcher/mailet from
the
server." In this way, both the user and the administrators have been
guided to a solution making it easier to resolve this problem. I am
not
sure that I would design this matcher/mailet to allow easy
customization
of the error message however, I think that should be only done
internally within the code itself. But you could convince me otherwise
if you can provide me with some compelling reasons to allow
customization.
Marc....
On 02/20/2019 12:15 AM, Benoit Tellier wrote:
Hi.
This is very true. But the technical knowledge limitation is not the
only one... There is also internationalization + text/plain
messages...
Note that "Bounce" mailet family allows a '<message>' field allowing
you
to maybe further explain this to non techie users you might have to
handle - and in the language of your choice, which is a big +.
Cheers,
Benoit
On 2/20/19 12:02 PM, Marc Chamberlin wrote:
Funny that I wasn't getting the notice "550 - Requested action not
taken: relaying denied" in a bounce email... (but even that is a
really
bad error message that most users will not understand nor know
what to
do about it.)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
