Ok, I got it now, surprise.
What went wrong: It worked the first time.
What's wrong with this: It shouldn't had happend and failed the first time.
Why: Cause the mailet Redirect causes a new mail to be spooled.
How I got this: The doc of Redirect mentions another mailet: Resend. The
doc of Resend contains a very important additional information that's
missing in the doc of Redirect:
WARNING: as the message (or a copy of it) is reinjected in the spool
without any modification, the preceding example is very likely to cause
a "configuration loop" in your system, unless some other mailet has
previously modified something (a header for instance) that could force
the resent message follow a different path so that it does not return
here unchanged.
So, somehow the test in the VM on friday somehow altered the mail or I
missed something in the mailet so it didn't got looped. As said: I don't
have the original image anymore so I can't check. More important: This
explains why the Exception is thrown and that it is indeed fully correct
as the same Mail gets looped and looped and looped and would loop
forever if there wouldn't be a check. The check itself is very simple:
When a new Mail object is created its "name" is set to "Mail +
System.currentTimeMillis() + UUID.randomUUID". Redirect alters this
"name" by adding a suffix and checks if this exceeds some length (this
is implemented in MailImpl itself) - and in the end throws an Exception
and fails.
As the same Mail object is duplicated ever again this ID gets longer and
longer and finally exceeds the limit.
I tried to play around with adding a header and catch it, but somehow
the HasHeader matcher doesn't work as planed. So I did the easy route:
What's the main thing I want to do? Rewrite the recipient. How can I
check if I already did this? By checking the recipient of course. So I
added this to my mailet-block:
<matcher name="local-service"
match="org.apache.james.mailetcontainer.impl.matchers.And">
//...
<matcher match="org.apache.james.mailetcontainer.impl.matchers.Not">
<matcher match="RecipientIs=local-serv...@cryptearth.de" />
</matcher>
</matcher>
The whole block now works like this:
- for all mail from 127.0.0.0/8
-- that are NOT smtp-auth
-- that are NOT from webmas...@cryptearth.de
-- that are NOT to local-serv...@cryptearth.de
-> Redirect to local-serv...@cryptearth.de
This way I avoid re-loop as when the duplicated mail is queued it
matches the specified recipient and isn't redirected again > loop broken.
I guess it would be wise to add IPv6, but as I set sendmail explicit to
IPv4 127.0.0.1 and any other code I may write will use smtp-auth I guess
it doesn't matter.
I hope I can mark this topic finally as done.
Matt
Am 28.05.2019 um 23:48 schrieb cryptearth:
I don't get it. I've tested as much as I could - and now always get
this error. Seems I got somewhat different last time it worked as I
can't get it to work at all now. So I guess it's really into how the
name gets generated - time to dig deep into code.
Matt
Am 28.05.2019 um 17:43 schrieb cryptearth:
Hey Garry,
I appreciate your effort. I don't think any of your questions are
stupid, in fact, as you said: let's go on systematically.
About the NICs: the VM emulates a Intel PRO/1000 MT Desktop
(82540EM). The physical host the VM runs on has a Intel(R) 82583V
Gigabit Network Connection, the laptop has two NICs: ethernet
Qualcomm Atheros QCA8172 Fast Ether and wireless Realtek RTL8188EE
Wireless Network Adapter. For the tests it doesn't matter if the NICs
connected or not. Also, as far as I tested it doesn't matter if the
IP is set via DHCP or static, IPv4 or IPv6 or both, assignment of
hostname by my router or manual set. I even went so far to clone the
VM image on the laptop and vise versa - after the system fiddled a
bit to bring the NICs up the Exception persist.
The tests work so far that if I sent a mail outbound it gets
delivered (for example to google, I run a back-up on a small
one-board-system - wich is still on openSUSE 42.3 and James 3.1-snap
from June 3rd 2018.
What I did not had tested yet (currently prepare the VM and laptop)
is a james 3.3 on opensuse 15.0. Maybe it could be something change
from opensuse 15.0 to 15.1 that causes this issue. The reason why I
think this could be is all tests I did so far was on opensuse 15.0
with james 3.3 and 3.4-snap from master I never had this issue. I
will try this next and report back after all tests complete. If it is
an issue with opensuse 15.1 I still can run 15.0 on all systems until
either next version of opensuse is released or somehow I could figure
it out.
So far,
Matt
Am 28.05.2019 um 16:54 schrieb Garry Hurley:
Hey Matt
Sorry I missed some of the earlier emails in the chain. However, we
can look at your VM versus real machine systematically. The obvious
first difference is the way the network cards are described and/or
used. For example, on my antsle VMs, they share through the host on
the 10.1.1 network. If I want them on one of my home networks, say
the 192.168.1.x network, I have to assign them a second ‘nic’ on
that network and set up a static IP on that nic (my configuration,
but it does work). Now, I was reading in the postfix documentation
that mail servers are supposed to have static IP addresses for
relaying purposes. So, the questions I have to ask are: Are your IP
addresses static or dynamic on the physical servers? Do you have a
DNS entry for each server? On your physical box, have you set up an
/etc/hosts file (Linux) or C:\Windows\System32\drivers\etc\hostsfile
entry (Windows) for your mail server? Have you allowed local traffic
(or actually all traffic) through on port 25 and the secure email
ports(465 I think)? Is your router firewall blocking these ports?
Yeah, I know these are stupid sounding questions but rule out
problems with the server before blaming the app. I have ended up
with egg on my face more than once because of a configuration faux
paus on my computer.
Sent from my iPhone
On May 27, 2019, at 3:35 PM, cryptearth <cryptea...@cryptearth.de>
wrote:
Hey there,
I ran another test on another real machine instead of a VM - and
surprisingly I encountered the same issue. So it seems somethings
different on a real machine than on a vm causing this issue. Anyone
knows how the "name" of a mail is composed in the first place and
how it could be valid if cloning it exceeds some limit?
Matt
Am 27.05.2019 um 18:19 schrieb cryptearth:
Hey Garry,
please use webarchive or server-user-thread.<x> to download the
whole thread - I already explained the reason why I "somewhat have
to" set it up this way.
Long story short: unix uses local mail as to report output of
background / service processes. It's up the process how this is
done wich results in an not-uniform combination of
<username>@<hostname>.<domain>. James tries to remote-deliver any
mail belong to a domain not listed in domainlist and fails to
treat root1.cryptearth.de - wich is the FQDN - as
<hostname>.<domain> but rather treats it as its own - and every
mail to a user not contained in userlist just gets rejected with
"no user/mailbox".
Solution: unify each mail dropped local, not smtpauth and not
specified manual in some sort of exclusion list.
As it would require huge re-write to make a database solution work
- and I only need one mail excluded - I just "hardwired" this one
and rely on smtpauth otherwise for decide between local and
remote-delivery.
It's NOT webmas...@cryptearth.de to local-serv...@cryptearth.de -
in fact these two are excluded as used to remote-delivery - but
<anything>@<hostname> (vm@root1) - <anything>@<hostname>.<domain>
(r...@root1.cryptearth.de) or any maybe not-existing
<anything>@<domain> (u...@cryptearth.de) wich hasn't authed (wich
isn't possible for non exisiting users) to in fact redirect them
to local-serv...@cryptearth.de. It's this non-uniformity paired
with how james behave about recipients not listed in domainlist or
userlist require me to rewrite those "local system mail".
I tested this on a VM - works like a dream - did the same to my
root - fails. I tried to work out where the instance of Mail gets
created and how the initial name is set - but wasn't able to yet.
It's like turning all SMTP stuff inside out to figure out how it
works. As james uses netty I maybe also need to dug into this
framework to figure out where the specific handler is called
forming the Mail instance from the read input after seeing
<CRLF>.<CRLF>. So maybe someone can point out where to start to
figure out how/where the initial name is set to determine why the
copy fails on the root although it's pretty much the same system
as in my VM.
Matt
Am 27.05.2019 um 15:36 schrieb Garry Hurley:
Why are you trying to redirect the mail from
‘webmas...@mydomain.com’ to ‘a.randome.u...@mydomain.com’ anyhow?
Isn’tthere an alias set up? I was just configuring postfix on
another machine, not my james server, and I noted a virtualiases
configuration for that purpose. I am recalling an aliases file in
the conf directory. Maybe I am wrong or getting my versions mixed
up (if not, maybe it should be that way). I think the rewrite you
are trying to do is for sending from ‘webmas...@mydomain.com’ to
‘my.other.em...@otherdomain.com’ and is actually overkill for
what you are doing.
Sent from my iPhone
On May 26, 2019, at 3:18 PM, cryptearth
<cryptea...@cryptearth.de> wrote:
Well, seems it doesn't work in the real world.
The way I mentioned worked in a VM so far - but after I upgraded
my root-server and did the same I know get this issue:
INFO | jvm 1 | 2019/05/26 20:58:02 | INFO 20:58:02,242 |
org.apache.james.protocols.netty.BasicChannelUpstreamHandler |
Connection established from 127.0.0.1
INFO | jvm 1 | 2019/05/26 20:58:02 | INFO 20:58:02,300 |
org.apache.james.domainlist.lib.AbstractDomainList | Local host
is: 127.0.0.1
INFO | jvm 1 | 2019/05/26 20:58:02 | INFO 20:58:02,423 |
org.apache.james.smtpserver.SendMailHandler | Successfully
spooled mail
Mail1558897082344-b3644bc7-10c4-46a7-b670-75cf4dc9fd94 from
MaybeSender{mailAddress=Optional[cryptea...@root1.cryptearth.de]}
on localhost/127.0.0.1 for [cryptea...@root1.cryptearth.de]
INFO | jvm 1 | 2019/05/26 20:58:02 | INFO 20:58:02,507 |
org.apache.james.protocols.netty.BasicChannelUpstreamHandler |
Connection closed for 127.0.0.1
INFO | jvm 1 | 2019/05/26 20:58:03 | ERROR 20:58:03,024 |
org.apache.james.mailetcontainer.impl.camel.CamelProcessor |
Exception calling Redirect: Unable to create a new message name:
too long. Possible loop in config.xml.
INFO | jvm 1 | 2019/05/26 20:58:03 |
javax.mail.MessagingException: Unable to create a new message
name: too long. Possible loop in config.xml.
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.server.core.MailImpl.detectPossibleLoop(MailImpl.java:344)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.server.core.MailImpl.deriveNewName(MailImpl.java:321)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.server.core.MailImpl.duplicate(MailImpl.java:103)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.transport.mailets.redirect.ProcessRedirectNotify.process(ProcessRedirectNotify.java:49)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.transport.mailets.Redirect.service(Redirect.java:475)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.mailetcontainer.impl.camel.CamelProcessor.process(CamelProcessor.java:81)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.mailetcontainer.impl.camel.CamelMailetProcessor$MailetContainerRouteBuilder.handleMailet(CamelMailetProcessor.java:178)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.mailetcontainer.impl.camel.CamelMailetProcessor$MailetContainerRouteBuilder.lambda$configure$0(CamelMailetProcessor.java:155)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:715)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:638)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:248)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.Splitter.process(Splitter.java:130)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.component.direct.DirectBlockingProducer.process(DirectBlockingProducer.java:53)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:186)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:86)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:541)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:506)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:369)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:506)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.impl.ProducerCache.send(ProducerCache.java:229)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:144)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:161)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:168)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.mailetcontainer.impl.camel.CamelMailetProcessor.service(CamelMailetProcessor.java:68)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.mailetcontainer.lib.AbstractStateCompositeProcessor.service(AbstractStateCompositeProcessor.java:84)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
org.apache.james.mailetcontainer.impl.JamesMailSpooler.lambda$run$0(JamesMailSpooler.java:163)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
INFO | jvm 1 | 2019/05/26 20:58:03 | at
java.lang.Thread.run(Thread.java:748)
INFO | jvm 1 | 2019/05/26 20:58:03 |
INFO | jvm 1 | 2019/05/26 20:58:03 | INFO 20:58:03,025 |
org.apache.james.transport.mailets.ToRepository | Storing mail
fd94-!5647419-!8598886-!8671760-!8516799-!7948334-!6362370-!1064056-!8846222
in MailRepositoryUrl{value=file://var/mail/error/}
This is the content of the filtered message:
cryptearth@localhost:~/james-server-app-3.3.0/var/mail/error>
cat
666439342D21353634373431392D21383539383838362D21383637313736302D21383531363739392D21373934383333342D21363336323337302D21313036343035362D2138383436323232.Repository.FileStreamStore
To: local-serv...@cryptearth.de
Message-ID: <201905261858.x4qiw2wp003...@root1.cryptearth.de>
MIME-Version: 1.0
Received: from localhost (EHLO root1.cryptearth.de) ([127.0.0.1])
by localhost (JAMES SMTP Server ) with ESMTP ID
-1238704049
for <cryptea...@root1.cryptearth.de>;
Sun, 26 May 2019 20:58:02 +0200 (CEST)
Received: (from cryptearth@localhost)
by root1.cryptearth.de (8.15.2/8.15.2/Submit) id
x4QIw2wp003888;
Sun, 26 May 2019 20:58:02 +0200
Date: Sun, 26 May 2019 20:58:02 +0200
From: "(Cron Daemon)" <cryptea...@root1.cryptearth.de>
Subject: Cron <cryptearth@root1> /home/cryptearth/bin/test.sh
Content-Type: text/plain; charset=UTF-8
Auto-Submitted: auto-generated
Precedence: bulk
X-Cron-Env: <XDG_SESSION_ID=4>
X-Cron-Env: <XDG_RUNTIME_DIR=/run/user/1000>
X-Cron-Env: <DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus>
X-Cron-Env: <LANG=de_DE.UTF-8>
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/home/cryptearth>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=cryptearth>
X-Cron-Env: <USER=cryptearth>
test
cryptearth@localhost:~/james-server-app-3.3.0/var/mail/error>
So, the redirect seems to work as far to change the recipient,
but then it can't be sent to the local user account. I'm unsure
if it's maybe some DNS/IP error as the way I re-installed the
system isn't supported by OVH. So maybe I made an error
somewhere. What's bothering me that the hostname sometimes keep
change back to localhost although it's correctly set anywhere.
Maybe I get in touch with OVH support
Matt
Am 24.05.2019 um 01:58 schrieb cryptearth:
Hey there, Matt here again.
Long time since my last update on this topic, but now I got a
pretty easy solution.
On May 22nd OpenSUSE 15.1 was released, and with this new
update I again looked at my "problem" about how to deal with
apache and other local mail. Instead of my initial thoughts I
hadn't to implement any myself, but james also comes with
anything needed. The two keywords here are the CompositeMatcher
and the Redirect mailet.
To recap: basically I want to filter any local mail to a
special service user except those from authed smtp and apache
(wich is set to webmas...@cryptearth.de in php.ini).
CompositeMatcher offers a way to easy implement this filter rule:
<matcher name="local-service"
match="org.apache.james.mailetcontainer.impl.matchers.And">
<matcher match="RemoteAddrInNetwork=127.0.0.0/8" />
<matcher
match="org.apache.james.mailetcontainer.impl.matchers.Not">
<matcher match="SMTPAuthSuccessful" />
</matcher>
<matcher
match="org.apache.james.mailetcontainer.impl.matchers.Not">
<matcher match="SenderIs=webmas...@cryptearth.de" />
</matcher>
</matcher>
I guess this speaks for itself, at least I guess anyone
subscribed to this mailing-list should understand what this
basic logic does.
One side-note: on start up the code that creates the config
only looks for Matchers and Mailets in
org.apache.james.transport.matchers.* - so any other classes
has to be referenced by its FQN.
As I now have all those mails I want to filter into the special
mailbox an easy Redirect does the trick about rewriting the
recipient:
<mailet match="local-service" class="Redirect">
<to>local-serv...@cryptearth.de</to>
</mailet>
The very convenient thing for apache is that every outgoing
mail is also saved automatic in SENT as for any existing local
user.
I only came up with this as I started to dig my way around the
github and wasn't satisfied with this compilcated way of
looking up the API. So I quickly asked google about creating a
complete javadoc - mvn javadoc:aggregate-jar did it. So I got
an easy browsable api-doc.
About placement I inserted it right at top of root processor,
even above the postmaster mailet. As it works it can't be that
wrong, but if there is a better spot than right at top please
tell me.
So, this concludes my question about how to combine james with
sendmail in a way sendmail is only the dumb nullclient
forwarding all local generated mail to james. As the Redirect
mailet also offers other rewrites of the headers I guess it
could be used to do a bit more with a bit more work on the
configs.
Thanks for anyone helped about this topic - a huge help was the
last reply from Benoit about using the RemoteAddrInNetwork class.
So long,
Matt
<truncated>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org
