Hey there, Matt here again.
Long time since my last update on this topic, but now I got a pretty
easy solution.
On May 22nd OpenSUSE 15.1 was released, and with this new update I again
looked at my "problem" about how to deal with apache and other local
mail. Instead of my initial thoughts I hadn't to implement any myself,
but james also comes with anything needed. The two keywords here are the
CompositeMatcher and the Redirect mailet.
To recap: basically I want to filter any local mail to a special service
user except those from authed smtp and apache (wich is set to
webmas...@cryptearth.de in php.ini).
CompositeMatcher offers a way to easy implement this filter rule:
<matcher name="local-service"
match="org.apache.james.mailetcontainer.impl.matchers.And">
<matcher match="RemoteAddrInNetwork=127.0.0.0/8" />
<matcher match="org.apache.james.mailetcontainer.impl.matchers.Not">
<matcher match="SMTPAuthSuccessful" />
</matcher>
<matcher match="org.apache.james.mailetcontainer.impl.matchers.Not">
<matcher match="SenderIs=webmas...@cryptearth.de" />
</matcher>
</matcher>
I guess this speaks for itself, at least I guess anyone subscribed to
this mailing-list should understand what this basic logic does.
One side-note: on start up the code that creates the config only looks
for Matchers and Mailets in org.apache.james.transport.matchers.* - so
any other classes has to be referenced by its FQN.
As I now have all those mails I want to filter into the special mailbox
an easy Redirect does the trick about rewriting the recipient:
<mailet match="local-service" class="Redirect">
<to>local-serv...@cryptearth.de</to>
</mailet>
The very convenient thing for apache is that every outgoing mail is also
saved automatic in SENT as for any existing local user.
I only came up with this as I started to dig my way around the github
and wasn't satisfied with this compilcated way of looking up the API. So
I quickly asked google about creating a complete javadoc - mvn
javadoc:aggregate-jar did it. So I got an easy browsable api-doc.
About placement I inserted it right at top of root processor, even above
the postmaster mailet. As it works it can't be that wrong, but if there
is a better spot than right at top please tell me.
So, this concludes my question about how to combine james with sendmail
in a way sendmail is only the dumb nullclient forwarding all local
generated mail to james. As the Redirect mailet also offers other
rewrites of the headers I guess it could be used to do a bit more with a
bit more work on the configs.
Thanks for anyone helped about this topic - a huge help was the last
reply from Benoit about using the RemoteAddrInNetwork class.
So long,
Matt
Am 13.02.2019 um 11:38 schrieb cryptearth:
Hey there,
I don't know if my last reply was sent, but I couldn't find it by checking
web-archive. So apologies if this is a double-post.
About the matchers mentioned by Benoit:
I already begun to implement a RemoteAddrMatcher myself as I couldn't find any
while digging through the source, but I guess RemoteAddrInNetwork will do the
trick, allthough it's wired style of checking with some DNS-stuff involved
confuses me. Seems it comes down how the network is defined - I'll check the
documentation.
About the SenderIs and SenderHostIs matchers: I already tried them, but
strangely, in the 3.2.0 branch the only consider something as local when it's
in the domain-list - wich I also tried but results in fastfail if I don't add
the user - wich I want to avoid by overwriting the recipient so the
RecipientIsLocal matches for LocalDelivery.
As I dug through the source I also found those logical-combine matchers, wich
can group multiple matchers with the basic logic AND, OR and NOT. So I came up
with the idea of checking RemoteAddr and combine with not sender is what I set
for apache. Reason: I want to use webmaster as the user part. So I made sure
there is no local user matching this, this way I know if anything come from
local but is not webmaster has to be some local service wich I want to rewrite
the recipient for local catch.
Sadly, due to long shifts at work I hadn't time to test this yet.
I'll report back after I got some time to implement a SetRecipient mailet and
check if existing RemoteAddrInNetwork helps or if I also have to implement my
own.
So long,
Matt
---- Benoit Tellier schrieb ----
Hi,
Have a look to:
- SenderIsLocal (based on the mail address of the sender)
- SenderHostIs + SenderHostIsLocal (based on the domain of the sender)
- You can already do some matching on the IP of the sender using
RemoteAddrInNetwork. Assuming you have well defined IPs, that you can
configure in XML, this would do 100% the job you tried to hack.
Best regards,
Benoit TELLIER
On 2/4/19 1:55 AM, cryptearth wrote:
So I dug through some code and found some points where I might could get
started:
The interface org.apache.mailet.Mail offeres the methods one might could
write a matcher and a mailet on:
String org.apache.mailet.Mail.getRemoteHost()
String org.apache.mailet.Mail.getRemoteAddr()
void org.apache.mailet.Mail.setRecipients(Collection<MailAdress>)
As one can see, the two getRemoteXXX() methods return a String. This can
lead to java.net.InetAddress.getByName(String) and provided methods
InetAddress.isAnyLocalAddress(), InetAddress.isLinkLocalAddress() and
InetAddress.isLoopbackAddress(). Also one can compare
InetAddress.getLocalHost().getHostName() or .getHostAddress().
As for the rewrite, the already mentioned SetRecipient mailet suggeted
by Benoit could already do the trick, at least for my personal question.
What I still not undestand: How to configure a Matcher or a Mailet by
XML syntax?
As Benoit mention:
<mailet ...>
<recipient>...</recipient>
</mailet>
So, how I would get the value of the recipient-tag into my Matcher or
Mailet? Reason: For modeling a Matcher and Mailet, I would like to not
only write some lines for my specific case, but would like to make it
configureable by what one could write in the XML. For example: specify
for what the matcher should match, like sepcific sender, remote-host,
maybe specific recipients - or what the Mailet should does: like remove
any recipients and replace it only by one entry, or maybe based on the
source set some specific recipients. I guess this should be possible as
one could configure more or less complex rules via XML - wich then just
had to be parsed into corresponding ruleset handling the Mail and decide
if and how to modify it.
Thanks so far,
Matt
Am 31.01.2019 um 10:32 schrieb cryptearth:
Hello Benoit,
thanks for your enormous effort you take into trying to help us with
such questions.
About the JIRA you filed, I guess that's what I looking for, as I
thought such SetRecipient(s) would already exist.
I came also up with a bit simpler description about my problem: I know
the origin: localhost, I don't know the sender nor target, but I do
know, that if it comes from a specific sender (the one I set in apache
php.ini), it's recipient is extern and should processed normal
RemoteDelivery, as there is no localuser matching this specific sender.
Example: in apache php.ini the sender is set to
webmas...@cryptearth.de - there is no localuser matching this, and the
recipient is external. This get's processed as usual and a copy of the
send mail is out into SENT folder - as I want it to. When sendmail
gets mails from other services, namely cron, it drops in, for example,
as gla...@glados.cryptearth.de. Even glados.cryptearth.de is a FQHN,
it could also seen as sub-domain, but james thinks this is an external
target as galdos.cryptearth.de isn't added in the domainlist. Other
services instead drop like wwwrun@localhost (that's the default wehn
no -f paramter is set in apache php.ini). So, as there is no matching
domain either, it gets rejected right away.
Also, it seems there is missing an "inverted" matcher wich could be
used for something like: "if <from> is not <xyz>" - I could only find
matchers for "if <from> is <xyz>". I couldn't find an example for
combine matchers neither: "if network is localhost AND if from is not
webmas...@cryptearth.de" > rewrite recipient to
local-servi...@cryptearth.de
Thanks so far,
Matt
Am 31.01.2019 um 08:29 schrieb Benoit Tellier:
Don't worry, I already feel overwhelmed with a single application and
several collegues contributing on it with me!
Concerning the JPAMailRepository, thank you very much for your proposal!
I do believe it is really nice as it allows JPA users to no more rely on
file-system based storage API at all.
Here is the JIRA, https://issues.apache.org/jira/browse/JAMES-2656 I
would be more than happy to help you implementing this!
Cheers,
Benoit
On 1/31/19 1:24 PM, Garry Hurley wrote:
As soon as I have gotten our Direct Project implementation to work
with James 3.x.0 (I am compilinga against the 3.2.0 release jars), I
can look at working the JPA MailRepository angle. Unfortunately,
other production issues and upgrade tasks have taken priority. You
know how it is when you help support two or three applications.
Sent from my iPhone
On Jan 30, 2019, at 11:43 PM, Benoit Tellier
<btell...@linagora.com> wrote:
Hi Garry,
To answer your previous email:
At Linagora we rely on "LDAP user repository".
Yes, we are currently limited to "a mailAddres" -> "a mailbox
account".
Shared mailboxes is not supported so far. That being said
"RecipientRewrite tables" successfully enables concepts like "Alias",
"Mail forwarding", "Groups", "Domain redirection". The
mailAddress<->mailAccount mapping is done at that level. Note that no
implementation of LDAP based RRT is implemented so far, but such a
component would make perfect sense and could be
implemented/contributed
in the future.
I don't understand the "either we are moving away from file-based
repos
or we aren't". All components have data-base alternatives. For the
example of the mailQueue that you took, there is an old-fashion file
implementation that I'd personally like to deprecate along with all
the
other file-based-storages implemented in James. But an ActiveMQ
implementation is available as well as a RabbitMQ one.
"Dead letter" concept for "Mail processing" is implemented via
Mail-Repositories (/var/mail/error) and full support for
reprocessing is
granted (at least in guice products).
(And you are right, we miss a JPA MailRepository implementation...
This
sounds like a nice proposal of contribution!)
So I really do think that we are, as a project, moving forward to
newer
tech. If you think we are missing something, then this should of
course
be discussed.
And of course, on all these topics, all contributions are welcomed ;-)
Cheers,
Benoit TELLIER
On 1/31/19 11:18 AM, Garry Hurley wrote:
A couple of questions. First, what is your usersrepository? If you
store user info in the database, yes, you would have to store each
user in there. If you store it in LDAP, you could store each entry
as an inetOrgPerson object, or a subclass of that, with a mailbox
entry for that person. Then when you authenticate against LDAP,
you get the mailbox for delivery. The LDAP is easier to maintain,
but harder to work with and adds complexity if you don’t already
use it for something else. What seems like a trivial thing - a
user to mailbox mapping - to be implemented in the database is
missing. Even an ‘entity’ to mailbox mapping so that you could
link a mailbox to a group or an individual is missing. Also
missing from the database is a dead letter queue, which is still
file-based. I can understand the tradeoff of security for
compliance with old-school tech, but either we are moving away
from file-based repos or we aren’t.
Sent from my iPhone
On Jan 30, 2019, at 11:00 PM, Benoit Tellier
<btell...@linagora.com> wrote:
Hello Matt,
So as far as I understand your main need would be:
```
WHEN I send an email
THEN it is stored in *myMagicRecipient* INBOX
Regardless of original recipients
```
Quite surprisingly such a "Overwrite envelop" feature is missing...
Here is the JIRA:
https://issues.apache.org/jira/browse/MAILET-163 Don't
hesitate to PR this ;-)
(Combined with a "not RemoteHostIs" you would achieve the
rewritting you
need)
Cheers,
Benoit TELLIER
On 1/30/19 4:13 PM, cryptearth wrote:
Hello Benoit,
let me try to re-phrase my goal:
I've set up a clean install of opensuse 15.0 on my test-rig,
named it
glados, assigned it to my domain cryptearth.de, set sendmail as
nullclient along with james 3.2.0 and want to collect any mail
dropped
by sendmail into a specific mailbox. The catch: As I also want
to use
sendmail to support mail() command for php, e-mails dropped in by
apache should be routed extern.
So: sendmail dropps in mails depending on wich service calls it.
Cron
for example uses gla...@glados.cryptearth.de - apache instead
wwwrun@localhost - and my guess is that other services might use
other
combinations as well.
Goal: When a mail is dropped by apache, wich is set to
webmas...@cryptearth.de by using -f parameter in php.ini, these
mails
should go out as normal to they recipient. Any other mail should
get
collected into a special user, for example
local-servi...@cryptearth.de or so.
What I disconvered yet: When the domain wich is used by sendmail is
not in the domain-list, james thinks it's an external domain and
tries
remote delivery. If the domain is in the domain-list, but not
the user
name - it fast fails to "no user". And if the user for the
domain is
also added it gets delivered into inbox without double in sent.
It's
that last one I want, but without adding any specific user for each
services wich may use sendmail to try to local-deliver
status-mails,
wich sendmail usual would drop into /var/mail or ~/mail (or what
ever
it is default to).
I dug through mailets and recipient rewrite and got some hints, but
couldn't figure out some working yet. One think I come up with
is add
some to mailetcontainer wich re-writes recipient based on
remote-host,
wich for sendmail is localhost. Other idea I had: use some
re-write-rule based on local hostname, but you recommended against
using this way.
Another possible what someone come up with: add my own few lines
into
source and re-build - but I don't know where to add as it's all
very
abstract and I'm not that pro of Java.
Thanks to anyone in advance,
Matt
Am 30.01.2019 um 05:15 schrieb Benoit Tellier:
Hi Matt,
I read the all thread but feel quite confuse about what you try to
achieve.
Could you describe it again? (`When ... Then ...` syntax can
really
help!)
Don't be using regex rewrites - I guess nobody understand what
it does.
Maybe you are interested by domain rewrites? Or do you want
that all
mail received, whatever the recipient, goes to the same mailbox?
Cheers,
Benoit TELLIER
On 1/28/19 9:57 AM, cryptearth wrote:
Hey there, Matt here again.
So, I think a found a possible solution without any
source-file mods
but
only clever mailet-config in mailetcontainer.xml.
As I dug through the source I found the standard mailets for
RecipientIsLocal. Then I found RecipientIsRegex (I might need
some help
here). Also there is something called RewriteTable.
I set up a fresh opensuse leap 15 - cron sends mails as
<user>@<hostname>.<domain> - so I set up a small test-script
and looked
at log while running james in console mode: when the
domainlist doesn't
contain <hostname>.<domain> but only <domain> james thinks that
<hostname>.<domain> is external and tries RemoteDelivery. So I
added
<hostname>.<domain> to domainlist. As I just had the domain,
but not
the
user, the mail is denied as user doesn't exists for the
domain. When I
add the user as <localuser>@<hostname>.<domain> it gets local
delivered
without additional copy in sent folder. So as I don't want to
set up a
user for each localuser, but also want to avoid fastfail I
come up
with:
What if I could early check for <whatever>@<hostname>.<domain>
and
could
just re-write it to an existing account in form of
<user>@<domain>?
When
recipient is re-written before RecipientIsLocal is checked,
then this
should trigger and just deliver any local mail from what ever
service to
the mailbox I want to.
So I search through the source tree - but couldn't find
anything to
alter the recipient by mailet. All I found was the
recipientrewritetable.xml file and the AddRegexMapping command
available
in james-cli.sh. The xml-config doesn't seem to help, but the
AddRegexMapping command looks good, but I don't understand the
command
parameters:
AddRegexMapping <user> <domain> <regex>
Can someone get me some hint how to use this command? Or
where/how I
can
setup the recipient-rewrite before RecipientIsLocal is checked?
Thanks in advance,
Matt
Am 27.01.2019 um 15:20 schrieb cryptearth:
So, as far as I dug through the code, it seems to come down to
MailetContext.isLocalEmail(MailAddress) wich is checked by
RecipientIsLocal (wich I guess also involve somehow a check of
isLocalServer(Domain). In transport-processor there is the line
<mailet match="RecipientIsLocal" class="LocalDelivery" />
I looked into LocalDelivery.java, but I guess when this
matches it's
to late to perform the test I want to about where the mail
comes from
and where it should send. JamesMailetContext looks good, as it
contains isLocalServer(Domain), wich should also used to check
SenderIsLocal (wich shouldn't matter, as this test is
performed by
source 127.0.0.1). As far as I yet could identify, it all
comes down
to somehow tell the config, that whatever domain sendmail
uses (it's
either hostname or hostname.domain) is considered as
localServer. The
wild-card alias shouldn't be any harder to find or figure out
as I
simply have to rewrite anything that comes from local and is for
local, but isn't in localUserList, is re-written to whatever
address I
want.
I'll report back when I found the lines to alter.
Matt
Am 26.01.2019 um 22:57 schrieb cryptearth:
Currently I'm just using out-of-the-box after clean build.
I'll have
a look at the mentioned class and the mailetcontainer. Maybe
I'll
find my way around. Good James is written in Java - the one
language
I really know.
Thanks for the point in a possible direction. I'll report
back when I
got something usefull.
Matt
---- Garry Hurley schrieb ----
Matt, are you using a custom mailet, or working out of the
box? Take
a look at the source for LocalDelivery.java
<http://LocalDelivery.java>, and the configuration for local
mail in
your mailetcontainer.xml <http://mailetcontainer.xml> file.
You may
have to override some functionality to get the processing
you want.
Sent from my iPhone
On Jan 26, 2019, at 9:36 AM, cryptearth
<cryptea...@cryptearth.de
<mailto:cryptea...@cryptearth.de>> wrote:
Hey there, Matt here.
So I got sendmail working with james now (tried postfix,
but it
always complains forwarding to localhost isn't supported as
there is
no option to ignore false loopback detection), and when used
from
apache with additional -f parameter all works good. But as
there're
other services might drop mail into local queue (crontab
specificly)
I noticed, that I'm still missing something. My goal is
somesort of
wildcard alias so that no matter from wich user sendmail gets a
message, except the one I use for apache, it should redirect
them
internally to one of the user-accounts inside james itself.
I guess
there would be a simple solution if would like all mails to get
forwareded, but I don't think there is a blacklist syntax so
that
mails from specific services with non-local recipients got
send out.
Currently I'm using 3.2.0 <tel:3.2.0> final release commit.
I also
looked into the mailetcontainer.xml
<http://mailetcontainer.xml> file
and I guess it's possible in there to specify this rule:
redirect anything local, except address set in apache
config, to
user X, otherwise when from apache with non-local target ->
transport
Could anyone maybe give me a hint?
Other solution: redirect anything local from sendmail to
processing
and with another task process the mails like some short
lines of
additional Java.
Matt
---------------------------------------------------------------------
To unsubscribe, e-mail:
server-user-unsubscr...@james.apache.org
<mailto:server-user-unsubscr...@james.apache.org>
For additional commands, e-mail:
server-user-h...@james.apache.org
<mailto:server-user-h...@james.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail:
server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
