Tracking each piece of mail through the system

[Jerry Malcolm]

Ive got a situation where some
legit mail is being categorized as spam. I run a few of the default spam
blockers that come with James. But I cant figure out which one is
causing a spam hit. Im familiar with all the various logs.
But there doesnt appear to be a single routing log that tracks a note
through all the processors/matchers/mailets.

Am I missing something? Is there an easy
way to enable some sort of tracking log?

Thanks.

Jerry

AntiVirus for Windows Server 2003?

[Jerry Malcolm]

 Slightly off topic... but I need to get an antivirus package to plug
into James via the mailet.  Norton's cheapest product that'll run on
Windows Server 2003 is $269 for 5 licenses (which I don't need).  I
have a single server.  Just currently stuck with Server 2003.  I found
eTrust antivirus from Computer Associates for $49 that says it'll run on
Server 2003.  But I'm not familiar with the product (I know CA, but not
the product).  Is there anything else better/cheaper? $49 is not a big
deal if the product will do what I need it to.  All I need it for is to
pass mail through it and detect virus attachments.

Recommendations/Experiences/Education?

Jerry


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Can't find DNS for domain when domain does exist

[Jerry Malcolm]

I'm getting a bounce when trying to send mail to a valid domain.  I get an
error in the log that says it can't find the dns entry.  The dns entry is
clearly there when I do an nslookup.

However, I went to dnsreport.com and got the following error when it
analyzed the DNS for the domain:

===

FAIL MX A lookups have no CNAMEs WARNING: One or more of your MX records
points to a CNAME. CNAMEs are prohibited in MX records, according to RFC974,
RFC1034 3.6.2, RFC1912 2.4, and RFC2181 10.3. The problem MX record(s) are:
mxmail.register.com.-rcom-outblaze-com.mr.outblaze.com.-205.158.62.206

===

OK, seems pretty obvious that there's something wrong with the DNS.  But is
this what's causing James to be unable to send the mail?  Even if the error
is with the destination, my customer tells me other people have no problem
emailing this person.  So the question is, how can make James not be so
overly sensitive to this problem?

Thanks.



-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Maintaining server names list automatically?

[Jerry Malcolm]

I host quite a few server names hosted with one James instance.  And server
names periodically come and go.  When I add a new client, I'd really like to
have an automated way to add the server name to the list.

I realize the brute force way is to write some sort of Perl or Rexx script
that parses the servername section out the config file and re-gen's the
config file with an updated list, then bounce the James server so it'll pick
up the change.  Seems awfully clumsy to have to do that.  But is that the
only way?  Is there a way to store the server list in a database table where
James can read it out and check periodically to see if the entries in the
table have changed?  Or am I doomed to have to write a script to gen the
config file?


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Preserving MAIL_IS_SEEN state?

[Jerry Malcolm]

I am desperately trying to figure out a way to access a message without 
having the MAIL_IS_SEEN flag set.   I have some background server apps 
that need to scan folders and do various automated maintenance tasks.  I 
do not want unread mail to be marked as seen just because the utility 
happened to pass over it.  Yet I can't seem to stop it from happening.  
I saw some posts on IMAP forums that said this was the way IMAP worked.  
Really?  There is absolute NO WAY to scan a folder and look at messages 
in a background utility and maintain the MAIL_IS_SEEN state if it hasn't 
really been seen by the user?


I have a test case that verifies what is happening:

-- direct SQL query to JAMES_Mail record in db to check MAIL_IS_SEEN 
(initially = 0)

-- folder.open(Folder.READ_ONLY);
-- message = ((UIDFolder)folder).getMessageByUID( uid );
-- direct SQL query to JAMES_Mail record in db to check MAIL_IS_SEEN ( 
changed to 1)


I figured that if I opened the folder in READ_ONLY mode, then, by 
definition of the term READ_ONLY I would not have 'change' authority, 
and therefore the MAIL_IS_SEEN flag would not be affected.  No 
difference.  Even opening a folder in READ_ONLY mode causes all of the 
mail to be set to 'SEEN'.


Am I missing something really obvious here?  I can't believe there is no 
benign way to scan a folder without JAMES automatically deciding that 
all mail has now been 'seen'.


Am I totally misunderstanding READ_ONLY mode of folders?  Is this a bug 
in JAMES?  Is there a totally different way to access a message via a 
Java program WITHOUT having JAMES auto-set it to 'seen'?  Or am I 
basically out of luck?


Thanks.

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Thunderbird Compatibility with JAMES 3.0?

[Jerry Malcolm]

Has anyone encountered problems using JAMES 3 IMAP with Mozilla 
Thunderbird client?  It appears to work fine with small-medium mailbox 
repositories.  But I have a client that has a large mail account with 
over 500,000 emails spread across a series of folders, and we are seeing 
some serious problems.  On Thunderbird, it sometimes takes 10 minutes to 
open the INBOX folder, and then when it does open and you select an 
email, it may take another 5-10 minutes to show the email.  This doesn't 
happen all the time.  But it happens on several Thunderbird clients.  
The inbox usually only has 100-200 emails in it, but there are quite a 
few mail rules that run on the INBOX in Thunderbird.  I am configured to 
access the same user account via my iPhone.  When Tbird is sitting there 
with the circle spinning forever, I can still open the folder and view 
the email on my iPhone with no problem.  So I don't think this is a 
JAMES problem.  But I'm curious if there are others using JAMES that are 
also using Thunderbird and, if so, are you seeing any similar behaviors?


Secondly, and more critical, moving a few emails between folders in 
Tbird has never been any problem at all.  However, I have tried several 
times moving a few hundred emails as a block from one folder to another 
folder and also trying to delete a large block of emails (send to 
trash).  With larger sets of email, it goes into an infinite loop and by 
the time I notice what is happening, I have 30-40+ copies of every email 
in the target and it just keeps growing.  I understand that 'move' 
really means 'copy' then 'delete' when everything has been copied.  Copy 
is looping/restarting and Delete never happens.  One time I  started it 
up and unfortunately walked away.  Came back and had like 20,000 emails 
in the target folder.  Fun to clean that out, and the only way I could 
do that was under-the-hood database work.  Ugly.


Even though currently it appears Tbird is messed up here, I'm wondering 
if JAMES is returning an error on some invalidly formatted email or 
something that is causing Tbird to go crazy.  If that's the case, 
perhaps I could add defensive code to JAMES to make Tbird not get so 
irritated, etc.


I'm going to be pursuing this on the Thunderbird forum as well since it 
is likely more Tbird issue than JAMES issue.  But at this point, it's 
neither a Tbird nor a JAMES issue... It's MY issue with my client.  So 
I'm just attacking all angles.  I'm just looking for information and 
more data if others have seen this.


Anybody want to talk about Thunderbird and JAMES 3?

Thx

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

James 3 Multiple Datasources?

[Jerry Malcolm]

I need to be able to access a different database from my default James 
mail db in a couple of matchers and mailets.  In james 2, I was able to 
easily add a 2nd datasource into the datasources tag block in the 
config file, and simple access the other datasource by name in my 
matcher/mailets.  In James 3, I know everything has changed to Spring 
with beans, etc.  I have been digging through the config files, the 
docs, and the source, and I can't figure out how to define an additional 
named datasource and access it.   I see throughout the config files 
where repositoryPath is using the original 'maildb' datasource name.  I 
assume 'maildb' refers to the db defined in 
james-database.properties.  But couldn't find any place where that 
correlation was specified.  Is the maildb datasource name not used 
anymore?


I need to define a 'db://myDataSource' like I did in J2.  I'm probably 
missing something obvious.  But I'm stuck.  How do I do that?


Thx.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

ToSenderFolder Mailet Failure (James 3)

[Jerry Malcolm]

I tried activating the ToSenderFolder mailet and could not get it to 
work.  I added some debug code and looked through the source, and I see 
where the problem is.


The mailet assumes that the username (user login name / repository name) 
is the sender's email address. (see code snippit at the bottom of this 
post). Where this 'could' be the case, and could be argued that it's 
that way in many situations, it's not required. My login id for my mail 
account is m...@mycompany.net. I have several employees that share 
this one account.  However, they send mail as j...@mycompany.net, 
b...@mycompany.net, and su...@mycompany.net.  I use the virtual recipient 
rewrite function to map all of these ids to the m...@mycompany.net 
repository.


The problem in the mailet is that it is trying to put sent mail in 
j...@mycompany.net/Sent, etc. which is not a valid repository.


In order to fix this, I need to figure out the real username that was 
used to log in to SMTP.   I suspect you're going to tell me that there's 
not a way in the world to get addressability to the real username inside 
a mailet, right...?  I guess the next option would be to use the 
recipient rewrite table in the db to map the sender to the repository.  
I'll look around for a recipient rewrite mapping class that I can use.  
Or I guess I could just brute-force it directly with a SQL query to the 
table in the db. Either way, I should be able to determine the real 
repository name so the mailet can know where to put the Sent message.


I believe there is a philosophical design problem currently in the 
mailet to assume the sender email address is the repository name. But 
since I need to go ahead and fix this, I'd like to know what the James 
team would suggest to be the 'right way' to implement the fix.


Suggestions?

Thx.

Jerry
-- code snippit--
String username;
try {
if (usersRepository.supportVirtualHosting()) {
username = sender.toString();
}
else {
username = sender.getLocalPart();
}
} catch (UsersRepositoryException e) {
throw new MessagingException(e.getMessage());
}

Re: ToSenderFolder Mailet Failure (James 3)

[Jerry Malcolm]

Update

I updated the ToSenderFolder mailet and got it to work properly with my 
account that uses RecipientRewrite.


I cloned some lines from the LocalDelivery mailet that sets up the rrt 
variable via inject.  In ToSenderFolder.doService(), I first call 
rrt.getMappings() using the sender email address.  If result != 
null, I pull the username string from the result.  If result=null, I 
fall back to the existing code that uses the sender email as the username.


Minimal testing so far.  But it appears to be working correctly now.

As I stated in my previous email, I think this is a bug in the mailet 
and not a feature request.  The mailet should handle by default the 
situation where the sender email address is NOT the username, and not 
require my custom hack to make it work.


Thanks.

Jerry


On 7/4/2014 8:30 PM, Jerry Malcolm wrote:
I tried activating the ToSenderFolder mailet and could not get it to 
work.  I added some debug code and looked through the source, and I 
see where the problem is.


The mailet assumes that the username (user login name / repository 
name) is the sender's email address. (see code snippit at the bottom 
of this post). Where this 'could' be the case, and could be argued 
that it's that way in many situations, it's not required. My login id 
for my mail account is m...@mycompany.net. I have several employees 
that share this one account.  However, they send mail as 
j...@mycompany.net, b...@mycompany.net, and su...@mycompany.net.  I use 
the virtual recipient rewrite function to map all of these ids to the 
m...@mycompany.net repository.


The problem in the mailet is that it is trying to put sent mail in 
j...@mycompany.net/Sent, etc. which is not a valid repository.


In order to fix this, I need to figure out the real username that was 
used to log in to SMTP.   I suspect you're going to tell me that 
there's not a way in the world to get addressability to the real 
username inside a mailet, right...?  I guess the next option would be 
to use the recipient rewrite table in the db to map the sender to the 
repository.  I'll look around for a recipient rewrite mapping class 
that I can use.  Or I guess I could just brute-force it directly with 
a SQL query to the table in the db. Either way, I should be able to 
determine the real repository name so the mailet can know where to put 
the Sent message.


I believe there is a philosophical design problem currently in the 
mailet to assume the sender email address is the repository name. But 
since I need to go ahead and fix this, I'd like to know what the James 
team would suggest to be the 'right way' to implement the fix.


Suggestions?

Thx.

Jerry
-- code snippit--
String username;
try {
if (usersRepository.supportVirtualHosting()) {
username = sender.toString();
}
else {
username = sender.getLocalPart();
}
} catch (UsersRepositoryException e) {
throw new MessagingException(e.getMessage());
}




-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2014.0.4592 / Virus Database: 3986/7800 - Release Date: 07/04/14



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Preserving MAIL_IS_SEEN state?

[Jerry Malcolm]

Hi Eric,

The RFC pretty doesn't specifically refer to the SEEN flag in reference 
to READ_ONLY state.  But it does say that:


  If the client is not permitted to modify the mailbox but is
  permitted read access, the mailbox is selected as read-only,

It doesn't specifically define READ-ONLY and what is and is not 
included.  But unless there are
some extenuating circumstances that I didn't come across, I think it's 
safe to assume that when
the mailbox is open in READ-ONLY state, nothing should be changed, 
including the SEEN flag.


But I'll admit I'm not intimately familiar with the RFC.  I was 
initially just hoping one of the
developers of JAMES 3 would be able to confirm that READ-ONLY should 
include the SEEN flag state.


If the developer says that there is actually a reason why SEEN should 
not fall under READ-ONLY state,
I'll just figure a way to work around it. But assuming READ-ONLY can be 
interpreted using the classic
definition, and the SEEN state should NOT be modified when READ-ONLY, 
then I believe there is a bug.

I can provide a test case.

Is it better just to provide some java source for the test case that can 
be compiled into an
executable jar?  Or do you prefer the compiled/built jar?  If that's the 
case, how do I deliver

it to you?

Alternatively, I have the entire J3 source. But there are a ton of 
packages and java files and I'm not
that familiar with the code.  If you can tell me the java class where 
the SEEN flag is set/cleared,
it'd probably be faster for me to just look at the code and see if I can 
see what is happening.


The initial question is 'should' SEEN come under READ-ONLY and not 
be changed in READ-ONLY state?


If so, we can proceed with figuring out why it does not appear to be 
behaving correctly.


Thanks.

Jerry

On 6/20/2014 12:27 AM, Eric Charles wrote:

Hi Jerry,

Thank for joining and sorry for the late answer.

The best way to tackle this is to point the rfc section where you think
james does not behave as expected. You could also upload a simple test
case that shows this.

Eric


On 05/18/2014 10:06 PM, Jerry Malcolm wrote:

I am desperately trying to figure out a way to access a message without
having the MAIL_IS_SEEN flag set.   I have some background server apps
that need to scan folders and do various automated maintenance tasks.  I
do not want unread mail to be marked as seen just because the utility
happened to pass over it.  Yet I can't seem to stop it from happening.
I saw some posts on IMAP forums that said this was the way IMAP worked.
Really?  There is absolute NO WAY to scan a folder and look at messages
in a background utility and maintain the MAIL_IS_SEEN state if it hasn't
really been seen by the user?

I have a test case that verifies what is happening:

-- direct SQL query to JAMES_Mail record in db to check MAIL_IS_SEEN
(initially = 0)
-- folder.open(Folder.READ_ONLY);
-- message = ((UIDFolder)folder).getMessageByUID( uid );
-- direct SQL query to JAMES_Mail record in db to check MAIL_IS_SEEN (
changed to 1)

I figured that if I opened the folder in READ_ONLY mode, then, by
definition of the term READ_ONLY I would not have 'change' authority,
and therefore the MAIL_IS_SEEN flag would not be affected.  No
difference.  Even opening a folder in READ_ONLY mode causes all of the
mail to be set to 'SEEN'.

Am I missing something really obvious here?  I can't believe there is no
benign way to scan a folder without JAMES automatically deciding that
all mail has now been 'seen'.

Am I totally misunderstanding READ_ONLY mode of folders?  Is this a bug
in JAMES?  Is there a totally different way to access a message via a
Java program WITHOUT having JAMES auto-set it to 'seen'?  Or am I
basically out of luck?

Thanks.

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2014.0.4592 / Virus Database: 3986/7797 - Release Date: 07/04/14



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

James 3 b4 HELO Override Not Working?

[Jerry Malcolm]

I have been running James 3 b4 for several months.  I was having some 
outbound mail rejected by gmail and started doing some research.  I did 
an email test using mail-tester.com and got the error message that my 
HELO did not match the rDNS.  It reports:


IP: 64.34.163.186
HELO: p2825577
rDNS: mail.jwmhosting.com

The p2825577 number is my Windows Server 2008 machine name.  I don't 
want to use that.


In the RemoteDelivery mailet config, I have the following:

mail.smtp.localhostmail.jwmhosting.com/mail.smtp.localhost

If I'm reading the instructions correctly, this should cause the 
specified value (mail.jwmhosting.com) to be sent on the HELO instead of 
the p282... number, correct?  But it isn't.


What am I doing wrong?

BTW...  I tried changing my rDNS to the p282... number.  rDNS's have to 
be a fully qualified domain name.  So it won't take it.  Fine... I then 
tried changing my Win Server 08 machine name to be mail.jwmhosting.com.  
No go on that either.  Can't have periods in Win machines names So 
I'm stuck.  Please advise


This is kinda urgent since I now realize that there's a distinct 
possibility that some of my clients' emails are not getting through due 
to this.  So thanks in advance for a quick reply to this.


Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: James 3 b4 HELO Override Not Working?

[Jerry Malcolm]

Robert,

Thank you so much for the quick reply.  I will probably start the 
process of moving up to the latest beta soon.  But i know it's not a 
trivial process, and I don't want to risk destabilizing the environment 
at this point.  Just need to get this HELO thing fixed quickly and keep 
the clients happy at least for another day


I see where I left of 'mail.' in the helloName in the smtpserver.xml.  
But that appears to only be used for inbound SMTP.   In the 
mailetcontainer template file, as part of the remoteDelivery mailet, 
there is this:


  !-- Set the HELO/EHLO name to use when connectiong to remote 
SMTP-Server --

  !--
mail.smtp.localhostmyMailServer/mail.smtp.localhost
--
   /mailet

Why are there two different places to specify the HELO name?  If the 
smtpserver.xml one is supposed to be used for inbound and outbound, this 
is redundant.  I know smtpserver.xml hello is used for inbound smtp. 
That is working. But it appears that James isn't using either one of 
these (the smtpserver hello def or the RemoteDelivery HELO def) for 
OUTBOUND smtp (which is the one I care about).  After changing the 
smtpserver.xml to mail.jwmhosting.com and verifying the remoteDelivery 
mailet HELO def is also mail.jwmhosting.com, mail-tester reports that 
it's still getting the Win Server machine name


At this point, I'm willing to hardcode the HELO in the java file and 
rebuild the jar.  But I'm not that familiar with the James source. By 
chance could you point me to a particular class or package inside James 
where the HELO command is sent on outbound?


Thanks again.

Jerry


On 10/27/2014 3:51 AM, Robert Munn wrote:

Your rDNS: mail.jwmhosting.com looks correct to me.

I just did a telnet check of your smtp server:

$ telnet
telnet  open mail.jwmhosting.com 25
telnet  helo foo.com

reply shows:
250 jwmhosting.com Hello foo.com [xx.xx.xx.xx]) - my ip address


What is important here is the helo name : jwmhosting.com. It should say 
mail.jwmhosting.com. This is a spammer flag and can get your mail rejected 
depending on the host. That is what the mail-tester service is complaining 
about.

You set this in smptserver.xml (smtpserver.conf in beta4?)

Look for the block below, change it :

helloName autodetect=“false”mail.jwmhosting.com/helloName


!-- This is the name used by the server to identify itself in the SMTP --
 !-- protocol.  If autodetect is TRUE, the server will discover its --
 !-- own host name and use that in the protocol.  If discovery fails, 
--
 !-- the value of 'localhost' is used.  If autodetect is FALSE, James 
--
 !-- will use the specified value. --
 !--
 helloName autodetect=truemyMailServer/helloName
  --

FYI, beta4 has a deadlocking problem, you might want to upgrade to beta 5. I 
have instructions on how to build it from source on YouTube:

https://www.youtube.com/channel/UChNyi0dlyanLLhp7cA9q5dA

or if you prefer a text tutorial, on my blog:

http://blog.bonnydoonmedia.com/post.cfm/building-apache-james-from-source



On Oct 26, 2014, at 10:44 PM, Jerry Malcolm techst...@malcolms.com wrote:


I have been running James 3 b4 for several months.  I was having some outbound 
mail rejected by gmail and started doing some research.  I did an email test 
using mail-tester.com and got the error message that my HELO did not match the 
rDNS.  It reports:

IP: 64.34.163.186
HELO: p2825577
rDNS: mail.jwmhosting.com

The p2825577 number is my Windows Server 2008 machine name.  I don't want to 
use that.

In the RemoteDelivery mailet config, I have the following:

mail.smtp.localhostmail.jwmhosting.com/mail.smtp.localhost

If I'm reading the instructions correctly, this should cause the specified 
value (mail.jwmhosting.com) to be sent on the HELO instead of the p282... 
number, correct?  But it isn't.

What am I doing wrong?

BTW...  I tried changing my rDNS to the p282... number.  rDNS's have to be a 
fully qualified domain name.  So it won't take it.  Fine... I then tried 
changing my Win Server 08 machine name to be mail.jwmhosting.com.  No go on 
that either.  Can't have periods in Win machines names So I'm stuck.  
Please advise

This is kinda urgent since I now realize that there's a distinct possibility 
that some of my clients' emails are not getting through due to this.  So thanks 
in advance for a quick reply to this.

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org





-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2015.0.5315 / Virus Database: 4189/8459 - Release Date: 10/26/14




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: James 3 b4 HELO Override Not Working?

[Jerry Malcolm]

More progress... But now I'm really stumped.  I dug into the 
remoteDelivery mailet source.  I did confirm that James is NOT using the 
smtpserver.xml 'hello' value at all for outbound HELO.  It is definitely 
using the config parms for the remoteDelivery mailet.


In the mailet, the outbound HELO value is set by javax.mail.Transport 
based on the 'mail.smtp.localhost' property passed in via the Properties 
object.  According to the Transport javadoc, it says it'll use the 
property value for HELO if it's set, and if it's not set, it'll use 
InetAddress.getLocalHost().getHostName().  Fine.  So I cloned the mailet 
so I could add log statements and do some debug.   I add two log 
statements right above the 'transport.sendMessage()' call in the 
RemoteDelivery mailet:


 log( JWMRemoteDelivery.deliver() mail.smtp.localhost -  + 
props.getProperty( mail.smtp.localhost ));


 log( JWMRemoteDelivery.deliver() 
InetAddress.getLocalHost().getHostName() -  + 
InetAddress.getLocalHost().getHostName() );


  transport.sendMessage(message, addr);

In the log, I get

INFO  09:52:19,480 | james.mailetcontext | JWMRemoteDelivery.deliver() 
mail.smtp.localhost - mail.jwmhosting.com


INFO  09:52:19,480 | james.mailetcontext | JWMRemoteDelivery.deliver() 
InetAddress.getLocalHost().getHostName() - p2825577


This is precisely what I expected to get.  BUT when the mail is 
sent, the p282 is sent in the HELO.


It appears that javax.mail.Transport is ignoring the property (or not 
recognizing that it is set).  But I'm pretty certain that a bug that is 
that blatant would not be hanging around unreported in a base java class 
like Transport.  But, then again, that's what I appear to be seeing.


Where am I going wrong?

Secondarily, anybody know how I can change what java reports back on the 
InetAddress call other than changing the machine name?  Is there a JVM 
parameter I can pass in?  If I can force that, problem solved for me 
(although it's still not working correctly).


Thanks again.

Jerry



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: James 3 b4 HELO Override Not Working? [unsigned]

[Jerry Malcolm]

The remoteDelivery mailet gets a session instance, and the session 
instance returns a geronimo 'transport' instance.  Not sure how it knows 
to find the class in the geronimo package.  But I did a 
transport.getClass().getPackage() in the mailet, and it returned the 
geronimo path.  So somehow it knows that geronimo is the provider.


Please email me the jar or post it where i can get it once you get it built.

Thank you so much!

Jerry


On 10/27/2014 12:56 PM, Robert Munn wrote:

Test build looks good. I have imported the geronimo code into Eclipse for 
editing.

I don’t see directly in the code where the geronimo javamail package is being 
called, but there are references to it in the file

james-karaf-features-3.0.0-beta5-SNAPSHOT-features.xml

 feature name=james-server-mailetcontainer-api 
version=3.0.0-beta5-SNAPSHOT
 
bundlemvn:org.apache.james/james-server-mailetcontainer-api/3.0.0-beta5-SNAPSHOT/bundle
 bundlemvn:org.apache.james/apache-mailet-api/2.5.1-SNAPSHOT/bundle
 
bundlemvn:org.apache.geronimo.javamail/geronimo-javamail_1.4_mail/1.8.3/bundle
 /feature


On Oct 27, 2014, at 9:09 AM, Bernd Waibel bwai...@intarsys.de wrote:


Hi Jerry

I am not using v3 but:

Could you try setting the parameter -Djava.mail.localhost=mail.jwmhosting.com 
in the java startup command line?

Most java.mail parameters are only parsed on startup by the vm.


Ciao.
Bernd



 Ursprüngliche Nachricht 
Von: Jerry Malcolm techst...@malcolms.com
Datum:
An: James Users List server-user@james.apache.org
Betreff: Re: James 3 b4 HELO Override Not Working?


More progress... But now I'm really stumped.  I dug into the
remoteDelivery mailet source.  I did confirm that James is NOT using the
smtpserver.xml 'hello' value at all for outbound HELO.  It is definitely
using the config parms for the remoteDelivery mailet.

In the mailet, the outbound HELO value is set by javax.mail.Transport
based on the 'mail.smtp.localhost' property passed in via the Properties
object.  According to the Transport javadoc, it says it'll use the
property value for HELO if it's set, and if it's not set, it'll use
InetAddress.getLocalHost().getHostName().  Fine.  So I cloned the mailet
so I could add log statements and do some debug.   I add two log
statements right above the 'transport.sendMessage()' call in the
RemoteDelivery mailet:

  log( JWMRemoteDelivery.deliver() mail.smtp.localhost -  +
props.getProperty( mail.smtp.localhost ));

  log( JWMRemoteDelivery.deliver()
InetAddress.getLocalHost().getHostName() -  +
InetAddress.getLocalHost().getHostName() );

   transport.sendMessage(message, addr);

In the log, I get

INFO  09:52:19,480 | james.mailetcontext | JWMRemoteDelivery.deliver()
mail.smtp.localhost - mail.jwmhosting.com

INFO  09:52:19,480 | james.mailetcontext | JWMRemoteDelivery.deliver()
InetAddress.getLocalHost().getHostName() - p2825577

This is precisely what I expected to get.  BUT when the mail is
sent, the p282 is sent in the HELO.

It appears that javax.mail.Transport is ignoring the property (or not
recognizing that it is set).  But I'm pretty certain that a bug that is
that blatant would not be hanging around unreported in a base java class
like Transport.  But, then again, that's what I appear to be seeing.

Where am I going wrong?

Secondarily, anybody know how I can change what java reports back on the
InetAddress call other than changing the machine name?  Is there a JVM
parameter I can pass in?  If I can force that, problem solved for me
(although it's still not working correctly).

Thanks again.

Jerry



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org





-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2015.0.5315 / Virus Database: 4189/8462 - Release Date: 10/27/14




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: James 3 b4 HELO Override Not Working?

[Jerry Malcolm]

mailet calls 'transport' which is an SMTPTransport instance. 
SMTPTransport calls SMTPConnection.getLocalHost() which is inherited 
from MailConnection.getLocalHost().


That method returns the InetAddress().getLocalHost().getHostName() 
first.  According to the spec, that should only occur IF the property 
mail.smtp.localhost is not set.  So i think the fix should be in that 
method.


The very first thing in the MailConnection.getLocalHost()  method should 
be something like:

---
if ( props.getProperty( mail.smtp.localhost ) != null ) return( 
props.getProperty( mail.smtp.localhost ));

---
Then let it try all of the other options if and only if this property is 
NOT set.


Jerry

On 10/27/2014 1:07 PM, Robert Munn wrote:

More info. SMPTConnection.java extends MailConnection.java which has a method 
with this signature:

   public boolean protocolConnect(String host, int port, String username, 
String password) throws MessagingException

If you fail to pass ‘host’ as an argument to the method, the host defaults to 
‘localhost’. A patch could be made there. A better patch would be to find the 
upstream code that is calling this method and patch it to send the correct 
hostname. I am hunting for that location, please let me know if you have any 
suggestions...




On Oct 27, 2014, at 10:25 AM, Jerry Malcolm techst...@malcolms.com wrote:


Thanks for jumping in and investigating.  This has gone from ugly to 
horrible  I think I have found the culprit.

I first had to figure out who is providing the class for SMTPTransport.   figured 
out that the package for SMTPTransport is: 
org.apache.geronimo.javamail.transport.smtp.  So it is NOT a base JVM 
implementation It's apache, and I am about to withdraw my statement that 'surely 
the transport class couldn't be ignoring a property.

Geronimo's SMTPTransport.java uses a class SMTPConnection which actually 
returns the localhost name.  And... no surprise at this point, it totally 
ignores the mail.smtp.localhost property. The property doesn't even appear 
anywher in the class.  First line in getLocalHost() pulls the 
InetAddress.getLocalHost().getHostName() value.

So half the problem is done... we know the culprit.  Now I guess (oh 
joy) I get to download the whole geronimo package, set up a build process, 
fix the code, and replace the jar on my server. (yuck).

That is... unless someone already has a VERY quick way to patch and rebuild 
geronimo.  Looking for volunteers :-)

In the meantime, I will also try adding that JVM parameter to try to override 
the Win Server machine name.

Still open to other suggestions for quicker workaround.

Thanks.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org





-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2015.0.5315 / Virus Database: 4189/8462 - Release Date: 10/27/14




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: James 3 b4 HELO Override Not Working?

[Jerry Malcolm]

The mailet only sees 'transport' which is cast to generic 
javax.mail.Transport.   The localhost methods and connection methods 
are all part of the geronimo impl.  You'll need to cast the transport to 
the geronimo package for the class, then see if you can get the 
connection object from the transport.  Might work.  But non-trivial.  
I'll play around with that was well.


Jerry

On 10/27/2014 1:41 PM, Robert Munn wrote:

Here is the base code. Note localHost is a property of the MailConnection 
class. Note also that there is a setLocalHost() method. I am going to try 
setting the localhost property from RemoteDelivery using setLocalHost( 
localHost ).

 public String getLocalHost() throws MessagingException {
 if (localHost == null) {

 try {
 localHost = InetAddress.getLocalHost().getHostName();
 } catch (UnknownHostException e) {
 // fine, we're misconfigured - ignore
 }

 if (localHost == null) {
 localHost = props.getProperty(MAIL_LOCALHOST);
 }

 if (localHost == null) {
 localHost = props.getSessionProperty(MAIL_LOCALHOST);
 }

 if (localHost == null) {
 throw new MessagingException(Can't get local hostname. 
 +  Please correctly configure JDK/DNS or set 
mail.smtp.localhost);
 }
 }

 return localHost;
 }

 
 /**

  * Explicitly set the local host information.
  *
  * @param localHost
  *The new localHost name.
  */
 public void setLocalHost(String localHost) {
 this.localHost = localHost;
 }




On Oct 27, 2014, at 11:28 AM, Jerry Malcolm techst...@malcolms.com wrote:


mailet calls 'transport' which is an SMTPTransport instance. SMTPTransport 
calls SMTPConnection.getLocalHost() which is inherited from MailConnection.getLocalHost().

That method returns the InetAddress().getLocalHost().getHostName() first.  
According to the spec, that should only occur IF the property 
mail.smtp.localhost is not set.  So i think the fix should be in that method.

The very first thing in the MailConnection.getLocalHost()  method should be 
something like:
---
if ( props.getProperty( mail.smtp.localhost ) != null ) return( props.getProperty( 
mail.smtp.localhost ));
---
Then let it try all of the other options if and only if this property is NOT 
set.

Jerry

On 10/27/2014 1:07 PM, Robert Munn wrote:

More info. SMPTConnection.java extends MailConnection.java which has a method 
with this signature:

   public boolean protocolConnect(String host, int port, String username, 
String password) throws MessagingException

If you fail to pass ‘host’ as an argument to the method, the host defaults to 
‘localhost’. A patch could be made there. A better patch would be to find the 
upstream code that is calling this method and patch it to send the correct 
hostname. I am hunting for that location, please let me know if you have any 
suggestions...




On Oct 27, 2014, at 10:25 AM, Jerry Malcolm techst...@malcolms.com wrote:


Thanks for jumping in and investigating.  This has gone from ugly to 
horrible  I think I have found the culprit.

I first had to figure out who is providing the class for SMTPTransport.   figured 
out that the package for SMTPTransport is: 
org.apache.geronimo.javamail.transport.smtp.  So it is NOT a base JVM 
implementation It's apache, and I am about to withdraw my statement that 'surely 
the transport class couldn't be ignoring a property.

Geronimo's SMTPTransport.java uses a class SMTPConnection which actually 
returns the localhost name.  And... no surprise at this point, it totally 
ignores the mail.smtp.localhost property. The property doesn't even appear 
anywher in the class.  First line in getLocalHost() pulls the 
InetAddress.getLocalHost().getHostName() value.

So half the problem is done... we know the culprit.  Now I guess (oh 
joy) I get to download the whole geronimo package, set up a build process, 
fix the code, and replace the jar on my server. (yuck).

That is... unless someone already has a VERY quick way to patch and rebuild 
geronimo.  Looking for volunteers :-)

In the meantime, I will also try adding that JVM parameter to try to override 
the Win Server machine name.

Still open to other suggestions for quicker workaround.

Thanks.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org




-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2015.0.5315 / Virus Database: 4189/8462 - Release Date: 10/27/14



-
To unsubscribe, e-mail

Re: James 3 b4 HELO Override Not Working?

[Jerry Malcolm]

Well, Robert, you get the prize It works (finally)!

Two changes required to RemoteDelivery mailet:

Add:
import org.apache.geronimo.javamail.transport.smtp.*;

Right after:
transport = session.getTransport(outgoingMailServer);

add:
((org.apache.geronimo.javamail.transport.smtp.SMTPTransport)transport).setLocalHost( 
mail.jwmhosting.com);


or be more generic and get the value from props.

The setLocalHost needs to be right after the object is instantiated 
since the HELO is sent on one of the first calls to transport.


This fixes it (thank you so much for your help).  But Geronimo is still 
not following the spec and honoring the property mail.smtp.localhost.  
It needs to be fixed in a subsequent release.


Basically everybody using James 3 needs to add this fix to the mailet, 
at least if running on a Windows server.  Or verify through 
mail-tester.com that the HELO is correct.  Going back to the orig 
problem, gmail was not delivering tons of my mail due to this.


Thanks again.

Now anybody offer any help on getting jDKIM working?  I've got it 
implemented and the DNS is set up.  But mail-tester is telling me the 
signature is invalid.  On to the next problem.


Jerry




On 10/27/2014 2:00 PM, Jerry Malcolm wrote:
The mailet only sees 'transport' which is cast to generic 
javax.mail.Transport.   The localhost methods and connection methods 
are all part of the geronimo impl.  You'll need to cast the transport 
to the geronimo package for the class, then see if you can get the 
connection object from the transport.  Might work.  But non-trivial.  
I'll play around with that was well.


Jerry

On 10/27/2014 1:41 PM, Robert Munn wrote:
Here is the base code. Note localHost is a property of the 
MailConnection class. Note also that there is a setLocalHost() 
method. I am going to try setting the localhost property from 
RemoteDelivery using setLocalHost( localHost ).


 public String getLocalHost() throws MessagingException {
 if (localHost == null) {

 try {
 localHost = InetAddress.getLocalHost().getHostName();
 } catch (UnknownHostException e) {
 // fine, we're misconfigured - ignore
 }

 if (localHost == null) {
 localHost = props.getProperty(MAIL_LOCALHOST);
 }

 if (localHost == null) {
 localHost = props.getSessionProperty(MAIL_LOCALHOST);
 }

 if (localHost == null) {
 throw new MessagingException(Can't get local 
hostname. 
 +  Please correctly configure JDK/DNS or 
set mail.smtp.localhost);

 }
 }

 return localHost;
 }

  /**
  * Explicitly set the local host information.
  *
  * @param localHost
  *The new localHost name.
  */
 public void setLocalHost(String localHost) {
 this.localHost = localHost;
 }




On Oct 27, 2014, at 11:28 AM, Jerry Malcolm techst...@malcolms.com 
wrote:


mailet calls 'transport' which is an SMTPTransport instance. 
SMTPTransport calls SMTPConnection.getLocalHost() which is inherited 
from MailConnection.getLocalHost().


That method returns the InetAddress().getLocalHost().getHostName() 
first.  According to the spec, that should only occur IF the 
property mail.smtp.localhost is not set.  So i think the fix should 
be in that method.


The very first thing in the MailConnection.getLocalHost() method 
should be something like:

---
if ( props.getProperty( mail.smtp.localhost ) != null ) return( 
props.getProperty( mail.smtp.localhost ));

---
Then let it try all of the other options if and only if this 
property is NOT set.


Jerry

On 10/27/2014 1:07 PM, Robert Munn wrote:
More info. SMPTConnection.java extends MailConnection.java which 
has a method with this signature:


   public boolean protocolConnect(String host, int port, String 
username, String password) throws MessagingException


If you fail to pass ‘host’ as an argument to the method, the host 
defaults to ‘localhost’. A patch could be made there. A better 
patch would be to find the upstream code that is calling this 
method and patch it to send the correct hostname. I am hunting for 
that location, please let me know if you have any suggestions...





On Oct 27, 2014, at 10:25 AM, Jerry Malcolm 
techst...@malcolms.com wrote:


Thanks for jumping in and investigating.  This has gone from ugly 
to horrible  I think I have found the culprit.


I first had to figure out who is providing the class for 
SMTPTransport.   figured out that the package for SMTPTransport 
is: org.apache.geronimo.javamail.transport.smtp.  So it is NOT a 
base JVM implementation It's apache, and I am about to 
withdraw my statement that 'surely the transport class couldn't be 
ignoring a property.


Geronimo's SMTPTransport.java

jDKIM Signature is invalid

[Jerry Malcolm]

With the HELO problem finally resolved and behind me, I'm on to the next 
issue that mail-tester.com is yelling at me about.  It say my DKIM 
signature is invalid.  I have tried replacing the public/private keys in 
my DNS and in the mailet config.  I've tried different syntax options 
for the DNS record, etc.  Still nothing. Normally, I would assume that 
it's something I'm doing wrong.  But after the HELO problem, I first 
want to verify that others are indeed using jDKIM with James 3 b5 
successfully.  (BTW... I said last night in a post that I'm on b4 
incorrect... i'm on b5).


My dns server is ISC BIND.  I'm assuming that could make a difference 
with defining the dkim public key in the dns record.


Is anyone using this successfully?

Here's the latest report I get from mail-tester.com:



The DKIM signature of your message is:

v=1;
d=jwmhosting.com;

b=ST3jn85HmyWjjPoIxojUKGMMCvtFzlrgqJiVwZ/bd5bzZuVWK/gviYIBXHFuc1iTZ/0NHdGW0TgRk9E7tRsXXPYM4yOP9zna7WTDC20VrsBu/LQbyktpsINAbRDzJbaNmSGjcLlAUODnaFLJPpwktpNkJu4EbFtKyYT3Exb4mfs=;
s=primary;
a=rsa-sha256;
bh=g3zLYH4xKxcPrHOD18z9YfpQcnk/GaJedfustWU5uGs=;
h=from:to:received:received;


Your public key is:

k=rsa;
p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDfRF6NEFFahhyOSiy+ufRU6PMvUVzP31wJ16zNaLKG8ipL1S8rIDr7ZjkGiUdwqZ6BjNTddbmcH4AxTupNhANyXvJs+MxyJbMN3mkYHYn1evJGVqT85DrM7XrYMSVPK3Y4dfY4vhokQwATv+BwrW7K8QuwrDgvu8JdxM0muS/7lQIDAQAB

Key length: 1024bits

Your DKIM signature is not valid

==

from my mailet config file:

 mailet match=All class=org.apache.james.jdkim.mailets.ConvertTo7Bit/

 mailet match=All class=org.apach1e.james.jdkim.mailets.DKIMSign
signatureTemplatev=1; s=primary; d=jwmhosting.com; 
h=from:to:received:received; a=rsa-sha256; bh=; b=;/signatureTemplate


privateKey
-BEGIN RSA PRIVATE KEY-
MIICXgIBAAKBgQDfRF6NEFFahhyOSiy+ufRU6PMvUVzP31wJ16zNaLKG8ipL1S8r
IDr7ZjkGiUdwqZ6BjNTddbmcH4AxTupNhANyXvJs+MxyJbMN3mkYHYn1evJGVqT8
...etc
...etc
ASnFebXvbrI3MQzrvgz3AkEAs1Tn0TfzsKmri6zrqJak1EDojHPdbPAjEFCpunt4
+dRCWhtnwEGZ1REeEBiQsk9CM24VNknO0uJOKF3ZYb3lFA==
-END RSA PRIVATE KEY-
/privateKey
  /mailet


Help??

Thx again.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: jDKIM Signature is invalid

[Jerry Malcolm]

Robert,

Thank you so much for all of your help on both of these issues.  You 
have been a lifesaver for me.  I just got a 10/10 score on 
mail-tester.com (hooray, finally!).  I couldn't have done this without you.


BTW re the public key in my DNS I regenerated the keys a few 
times desperately trying to figure out how to get it working.  One 
generator assigned its own selector.  So the DNS key had a different 
selector than the one I had in my earlier post.  It's all working now.


Thanks again.

Jerry


On 10/28/2014 3:16 AM, Robert Munn wrote:

I checked your DNS, looks like you removed your DKIM record.

Also, mxtoolbox.com says your SPF record type is deprecated.

I just got jDKIM working on my domain. Looks like another tutorial in the 
works. It took a little investigation, but mail-tester.com is happy now. Here 
is how you fix your broken sig problem.

  mailet match=All class=org.apache.james.jdkim.mailets.ConvertTo7Bit/

mailet match=All class=org.apache.james.jdkim.mailets.DKIMSign
  signatureTemplatev=1; s=201410281226.pm; d=legacyavatar.com; 
c=relaxed/relaxed; h=Message-ID:Date:Subject:From:To:MIME-Version:Content-Type; 
a=rsa-sha256; bh=; b=;/signatureTemplate

Note my changes to h= and addition of c= in my sig template. I hit the same 
error you hit before I made these changes, based on this article:

http://www.gettingemaildelivered.com/dkim-explained-how-to-set-up-and-use-domainkeys-identified-mail-effectively

Make sure these mailers are just before the RemoteDelivery mailet. Looks like 
you probably had that right already.


Thanks for making me aware of the sender issue and motivating me to add DKIM.




On Oct 27, 2014, at 3:30 PM, Jerry Malcolm techst...@malcolms.com wrote:



With the HELO problem finally resolved and behind me, I'm on to the next issue 
that mail-tester.com is yelling at me about.  It say my DKIM signature is 
invalid.  I have tried replacing the public/private keys in my DNS and in the 
mailet config.  I've tried different syntax options for the DNS record, etc.  
Still nothing. Normally, I would assume that it's something I'm doing wrong.  
But after the HELO problem, I first want to verify that others are indeed using 
jDKIM with James 3 b5 successfully.  (BTW... I said last night in a post that 
I'm on b4 incorrect... i'm on b5).

My dns server is ISC BIND.  I'm assuming that could make a difference with 
defining the dkim public key in the dns record.

Is anyone using this successfully?

Here's the latest report I get from mail-tester.com:



The DKIM signature of your message is:

v=1;
d=jwmhosting.com;

b=ST3jn85HmyWjjPoIxojUKGMMCvtFzlrgqJiVwZ/bd5bzZuVWK/gviYIBXHFuc1iTZ/0NHdGW0TgRk9E7tRsXXPYM4yOP9zna7WTDC20VrsBu/LQbyktpsINAbRDzJbaNmSGjcLlAUODnaFLJPpwktpNkJu4EbFtKyYT3Exb4mfs=;
s=primary;
a=rsa-sha256;
bh=g3zLYH4xKxcPrHOD18z9YfpQcnk/GaJedfustWU5uGs=;
h=from:to:received:received;


Your public key is:

k=rsa;
p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDfRF6NEFFahhyOSiy+ufRU6PMvUVzP31wJ16zNaLKG8ipL1S8rIDr7ZjkGiUdwqZ6BjNTddbmcH4AxTupNhANyXvJs+MxyJbMN3mkYHYn1evJGVqT85DrM7XrYMSVPK3Y4dfY4vhokQwATv+BwrW7K8QuwrDgvu8JdxM0muS/7lQIDAQAB

Key length: 1024bits

Your DKIM signature is not valid

==

from my mailet config file:

mailet match=All class=org.apache.james.jdkim.mailets.ConvertTo7Bit/

mailet match=All class=org.apach1e.james.jdkim.mailets.DKIMSign
signatureTemplatev=1; s=primary; d=jwmhosting.com; h=from:to:received:received; 
a=rsa-sha256; bh=; b=;/signatureTemplate

privateKey
-BEGIN RSA PRIVATE KEY-
MIICXgIBAAKBgQDfRF6NEFFahhyOSiy+ufRU6PMvUVzP31wJ16zNaLKG8ipL1S8r
IDr7ZjkGiUdwqZ6BjNTddbmcH4AxTupNhANyXvJs+MxyJbMN3mkYHYn1evJGVqT8
...etc
...etc
ASnFebXvbrI3MQzrvgz3AkEAs1Tn0TfzsKmri6zrqJak1EDojHPdbPAjEFCpunt4
+dRCWhtnwEGZ1REeEBiQsk9CM24VNknO0uJOKF3ZYb3lFA==
-END RSA PRIVATE KEY-
/privateKey
  /mailet


Help??

Thx again.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org





-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2015.0.5315 / Virus Database: 4189/8469 - Release Date: 10/28/14




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Converting BATV address to regular email address?

[Jerry Malcolm]

I received an email through my JAMES server from an address 
prvs=aakjadsfjasd=u...@domain.com.   This my first experience with 
Bounce Address Tag Validation format for email addresses.  As you might 
suspect, my mailets that do things based on the sender's email address 
all crashed and burned on this email.


I figure I can do some string manipulation each time I need the real 
sender address to get the true email address.  But I figured I'd at 
least ask if there are some service routines already existing in 
JavaMail / JAMES to handle BATV email addresses.


Does JAMES or JavaMail deal with BATV-form email addresses?

Thanks.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Tracking Mail After Folder Moves

[Jerry Malcolm]

Robert,

I guess that's always a possibility.  But it seems like the overhead 
would be huge:  Purchasing, installing, and learning a completely new 
database system, doubling the several terabytes of storage, both primary 
and backup to duplicate the entire existing mail db, writing and 
maintaining code to guarantee the databases are 100% in sync (which 
experience tells me they'll never stay 100% in sync, no matter how much 
code I write), and then writing the code to search and extract the data 
from the duplicate parallel database system All of this vs. adding 
another column to JAMES_MAIL table.   Not sure I want to go that route.  
I've already got the code that maintains the search keys and does the 
searches.  I simply need a way to retrieve the one record out of nearly 
a million records in the JAMES_MAIL table that contains the email that 
my search engine has found.


On 3/14/2015 6:18 PM, Robert Munn wrote:

Jerry,

Have you considered using an external system for this purpose?

Offhand, I would think you could use either a NoSQL store like Cassandra or a 
search system like Solr to maintain your searchable history. You could create a 
mailet to insert new emails into the external system, then you don’t need to 
modify James. Moving emails should not be a problem, even if James makes copies 
of them. Solr is easy to set up and use, and a few hundred thousand documents 
is a small document store for Solr to index, especially since your documents 
will not change once they are inserted.

Using Solr would also give you the benefits of sophisticated searching while 
still allowing you to store the full text of each email.

  


On Mar 13, 2015, at 2:08 PM, Jerry Malcolm techst...@malcolms.com wrote:



Benoit,

Thanks for the info.  Kinda what I was suspecting.  Here's what I've done so 
far...

My ultimate objective is to maintain a searchable index for all of the hundreds 
of thousands of emails stored in my JAMES mail db.  As previously discussed, 
this is only possible assuming I have a way to later locate a particular email 
that I have built an index for (assuming the user will move it around between 
folders...)

1) Step one was to add one more column to the JAMES_MAIL table for my own 
globally-unique UUID
2) When JAMES stores an email, this column defaults to -1, so I'll know it 
hasn't yet been indexed
3) A chron job runs hourly and creates an index for the new mail. It also adds 
the matching index records with all of the keyword info I want to track into my 
own separate index table.
4) I have code to process index queries and identify the UUID for the desired 
mail
5) I query the JAMES_MAIL table for the mail record using the UUID value.  I 
then extract the folder and ID info in that record.
6) Finally, I go back around to the 'front door' and use the standard IMAP 
interface with the folder and ID info to access the desired email for the user.

Granted, emails can be deleted.  I periodically clean out index entries for 
UUIDs that no longer exist.

This is all pretty much working.  But as you said, this is going to require 
remerging everything each time I upgrade JAMES. I'm not really thrilled with 
modifying the schema for JAMES db tables.  I wouldn't expect all of my indexing 
functionality to be in JAMES. But I would love to have JAMES maintain a single 
global UUID column in JAMES_MAIL.  That would make merging my functionality 
with JAMES much cleaner.

As I said, this is pretty much working now the way I described.  I just decided 
to bring it up here on the forum to make sure I'm not re-inventing the wheel or 
something by overlooking existing functionality in JAMES.  It appears now that 
I'm blazing new trails and not duplicating anything that's existing.  But if 
there's any talk in the future, I definitely want to keep up with discussions.

Thanks again.

Jerry

On 3/13/2015 11:42 AM, Benoit Tellier wrote:

Hi Jerry,

You are right ... This is what happens when you drag and drop an e-mail
in thunderbid from folder A to B :

  1 : Client receive a mail in folder A . The mail is identified by the
pair ( mailbox path + uid ). Mailbox path ( or mailbox Id ) is folder
specific. Uid is a long, per mailbox generated. It makes no sens alone.
Let say we have ( A : 36 ).

  2 : You perform the drag and drop

  3 : Thunderbird issue a UID COPY command.

  4 : So you have the exact same mail in B, let say ( B : 42 ).

  5 : James dispatch a Added event for ( B : 42 ) ( Here we don't know
where this mail came from )

  6 : Your client perform a UID EXPUNGE command on ( A : 36 ).

  7 : ( A : 36 ) is deleted

  8 : You have de delete event for ( A : 36 ) ( Here we don't know where
this mail came from )

Note that the events I quoted you triggers IDLE operation, and
thunderbird gets aware of what is happening. Then it reads ( B : 42 )
and displays it.

Well, to sum up :

  - You do not have global e-mail identifier that survives copy.
  - You can not base such a feature on event

Re: Tracking Mail After Folder Moves

[Jerry Malcolm]

Benoit,

Thanks for the info.  Kinda what I was suspecting.  Here's what I've 
done so far...


My ultimate objective is to maintain a searchable index for all of the 
hundreds of thousands of emails stored in my JAMES mail db.  As 
previously discussed, this is only possible assuming I have a way to 
later locate a particular email that I have built an index for (assuming 
the user will move it around between folders...)


1) Step one was to add one more column to the JAMES_MAIL table for my 
own globally-unique UUID
2) When JAMES stores an email, this column defaults to -1, so I'll know 
it hasn't yet been indexed
3) A chron job runs hourly and creates an index for the new mail. It 
also adds the matching index records with all of the keyword info I want 
to track into my own separate index table.
4) I have code to process index queries and identify the UUID for the 
desired mail
5) I query the JAMES_MAIL table for the mail record using the UUID 
value.  I then extract the folder and ID info in that record.
6) Finally, I go back around to the 'front door' and use the standard 
IMAP interface with the folder and ID info to access the desired email 
for the user.


Granted, emails can be deleted.  I periodically clean out index entries 
for UUIDs that no longer exist.


This is all pretty much working.  But as you said, this is going to 
require remerging everything each time I upgrade JAMES. I'm not really 
thrilled with modifying the schema for JAMES db tables.  I wouldn't 
expect all of my indexing functionality to be in JAMES. But I would love 
to have JAMES maintain a single global UUID column in JAMES_MAIL.  That 
would make merging my functionality with JAMES much cleaner.


As I said, this is pretty much working now the way I described.  I just 
decided to bring it up here on the forum to make sure I'm not 
re-inventing the wheel or something by overlooking existing 
functionality in JAMES.  It appears now that I'm blazing new trails and 
not duplicating anything that's existing.  But if there's any talk in 
the future, I definitely want to keep up with discussions.


Thanks again.

Jerry

On 3/13/2015 11:42 AM, Benoit Tellier wrote:

Hi Jerry,

You are right ... This is what happens when you drag and drop an e-mail
in thunderbid from folder A to B :

  1 : Client receive a mail in folder A . The mail is identified by the
pair ( mailbox path + uid ). Mailbox path ( or mailbox Id ) is folder
specific. Uid is a long, per mailbox generated. It makes no sens alone.
Let say we have ( A : 36 ).

  2 : You perform the drag and drop

  3 : Thunderbird issue a UID COPY command.

  4 : So you have the exact same mail in B, let say ( B : 42 ).

  5 : James dispatch a Added event for ( B : 42 ) ( Here we don't know
where this mail came from )

  6 : Your client perform a UID EXPUNGE command on ( A : 36 ).

  7 : ( A : 36 ) is deleted

  8 : You have de delete event for ( A : 36 ) ( Here we don't know where
this mail came from )

Note that the events I quoted you triggers IDLE operation, and
thunderbird gets aware of what is happening. Then it reads ( B : 42 )
and displays it.

Well, to sum up :

  - You do not have global e-mail identifier that survives copy.
  - You can not base such a feature on event

So what can you do ?

If I were you, I would do this :

  1 : to choose a MAILBOX implementation ( the one your client want to
use ? ),
   2 : generate an value on mapper's add operation  ( either a long (if
you want it sorted) or a UUID. )

  3 : Provides a custom message implementation with an accessor on this
value.

  3.5 : Every where in your mapper you need to use this new message type.

  4 : Upon message mapper copy calls, you cast the copied message into
your message type, and copy the field without modiffying it.

  5 : Here we are ( not that this value may not be unic as message can
get copied but not deleted ). You can just build it, and replaces the
old jar for your MAILBOX implementation with the new one, and restart
your James server ( yes it works ). Note : update the db schemas before
restarting James ;-)

Note that you do not need more : such a feature can not be accessed over
IMAP, but you can read it using an other application. So your are
commpelled to access it threw your mail's storage ( you said it was no
problem ... )

Don't worry, such a feature is not that hard to implement.

Drawbacks : you may have to merge it with other James releases. ( Or get
it accepted in the project ? ).


Hope it helps,

Benoit


Le 13/03/2015 16:50, Jerry Malcolm a écrit :

This is somewhat an IMAP question.  But also a JAMES implementation
question.  My client has a massive amount of mail that must be kept and
accessed.  They use Thunderbird and Outlook to do the normal mail
handling stuff.  No problems at all on the client side.  But on the back
end, I need to sort and organize and keep track of emails and be able to
pull them up using a web interface on demand, completely independent of
folders

Tracking Mail After Folder Moves

[Jerry Malcolm]

This is somewhat an IMAP question.  But also a JAMES implementation 
question.  My client has a massive amount of mail that must be kept and 
accessed.  They use Thunderbird and Outlook to do the normal mail 
handling stuff.  No problems at all on the client side.  But on the back 
end, I need to sort and organize and keep track of emails and be able to 
pull them up using a web interface on demand, completely independent of 
folders that they may currently be in.  In other words, I need to keep 
track of 'email x' and be able to find it at a later time no matter how 
many times the user moves it from folder to folder.


I believe I understand the philosophy of IMAP for the client is to find 
a folder, display the contents, refresh periodically and add/remove mail 
from its records for that folder as contents change.  Basically if the 
user moves a mail item from one folder to another, the first folder 
recognizes it's no longer there, and is done with it.  The other folder 
subsequently realizes it has a new email item and displays it.  But 
there is no knowledge that this is the same email.  Have I got it pretty 
much correct?


So... I realize I may be stretching/bending the intent of IMAP.  But 
that doesn't diminish the fact that I have the requirement.  I've dug 
through all of the database table schemas for JAMES and have a pretty 
good handle on how mail is stored and tracked internally. But I may have 
missed something.  So my main question is is there a way for me to 
permanently track an email item and be able to locate it at some point 
down the road even if it's been moved around folders several times?  
Basically, is there a global unique ID for every email stored?  BTW 
I'm not bound by having to use only IMAP.  I have no problem at all 
back-dooring to the JAMES database and writing code to use SQL to track 
through the database tables to find the email.  I just don't think there 
is anything unique/unchangeable that will allow me to permanently track 
a particular email.


Am I totally off the wall in considering something like this?  Seems a 
complete waste to have to duplicate a hundred gigs of mail data for my 
own archive when JAMES has a perfectly good copy of everything.


Suggestions?

Thanks.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: AW: Tracking Mail After Folder Moves [unsigned]

[Jerry Malcolm]

Hi Bernd,

Thanks for the response.  Here's the problem scenario A company has 
lots of dealing with their client, Joe.  Several different employees 
have to email Joe regularly.  So correspondence to Joe is spread across 
several mail accounts and several folders in each account (sent, inbox, 
archive 2013, archive 2014, etc, and some employees might have a Joe's 
stuff folder).  There are no restrictions as to what folder a 
particular email might be in (and possibly be moved to a different 
folder tomorrow).  The 'boss' wants to be able to see a list of 'all 
email to Joe'.


As mail arrives I extract relevant search criteria and store it in my 
search engine database.  So it is easy for me to assemble the list of 
emails to Joe.  The one thing I need. I need to be able then to 
query JAMES_MAIL and extract the actual mail records.  A simple, 
guaranteed, unchangeable UUID in JAMES_MAIL is ALL I need. I simply 
store the UUID as the key in my search engine.  (Once I find the record, 
I can determine the account/folder from the record and then use standard 
IMAP functions to access the mail item).


My main design points...

1) The UUID must be immutable,  folder independent, and folder-move 
independent.
2) I do not want to duplicate a near million mail entry db somewhere 
else.  I want to pull the mail from the existing JAMES_MAIL db.
3) New mail and deleted mail will be handled by search engine sync 
utilities and is not an issue


In summary my search engine finds the index/key record it wants 
It must then locate that  particular mail item in the JAMES_MAIL table


I'm currently generating a hash UUID including various fields such as 
from, to, subject, etc. that is working fairly well at generating a 
unique id/key.  It still feels like a hack.  And I'm sure there will be 
situations where the calculated hash is a dup from a different email.  
So it's not 100%.


As you originally theorized, the simplest solution would have been to 
have the db autogen an incrementing id.  But as you pointed out, the 
copy/delete on folder move kills that id.  Perhaps add a UUID header to 
the mail when it first comes in if such header does not exist.  Then 
always reflect that UUID header value to the JAMES_MAIL table's UUID 
field for db query use (??).  Headers remain intact in case of 
Thunderbird's  copy/delete, correct?


Thoughts?

Jerry

On 3/13/2015 6:15 PM, Bernd Waibel wrote:

Sorry,

Thought about again:
I think using a sequence is wrong. Cause Thunderbird makes a COPY, you will get a new 
UUID for the B:42 mail, and as I understand that is not what you need.

Greetings
Bernd

-Ursprüngliche Nachricht-
Von: Bernd Waibel [mailto:bwai...@intarsys.de]
Gesendet: Samstag, 14. März 2015 00:07
An: James Users List
Betreff: AW: Tracking Mail After Folder Moves [unsigned]

Hello Jerry,

just a few thoughts about alternatives (not sure I got your problem).

Why don't use a database sequence field or AUTO_INCREMENT field, instead of a 
UUID? And let the database handle the UUID creation?
But if you would like to use UUIDs: Make sure it is not part of a race 
condition.
As shortly described here for postgres sequences: 
http://www.neilconway.org/docs/sequences/
James is multithreaded.

Maybe the UUID field should be indexed, if you search for it often (a sequence 
field does not need to be indexed).

Maybe a database trigger on insert could create your index table. And another trigger 
could delete on delete.

You said, you will have a hourly delay of indexing when using cron. What 
happens, if a new mail arrives, and the user moves this mail immediately to 
another folder, before indexed, is this ok for your process?
It is just the way I handle my mails: on arrival I move the mails to a new 
folder (after reading).


But a good indexing solution implemented in James would be nice, too. ;-)


Greetings
Bernd

-Ursprüngliche Nachricht-
Von: Jerry Malcolm [mailto:techst...@malcolms.com]
Gesendet: Freitag, 13. März 2015 22:08
An: server-user@james.apache.org
Betreff: Re: Tracking Mail After Folder Moves

Benoit,

Thanks for the info.  Kinda what I was suspecting.  Here's what I've done so 
far...

My ultimate objective is to maintain a searchable index for all of the hundreds 
of thousands of emails stored in my JAMES mail db.  As previously discussed, 
this is only possible assuming I have a way to later locate a particular email 
that I have built an index for (assuming the user will move it around between 
folders...)

1) Step one was to add one more column to the JAMES_MAIL table for my own 
globally-unique UUID
2) When JAMES stores an email, this column defaults to -1, so I'll know it 
hasn't yet been indexed
3) A chron job runs hourly and creates an index for the new mail. It also adds 
the matching index records with all of the keyword info I want to track into my 
own separate index table.
4) I have code to process index queries and identify the UUID for the desired 
mail
5) I

Thunderbird Losing UIDs in Folders

[Jerry Malcolm]

I use JAMES 3 b5 with a relatively large mail IMAP database (500K mail 
items) across several accounts.  Most of my clients are using 
Thunderbird.  The problem I'm seeing may very likely be a Thunderbird 
problem.  But I'm just curious what might be going on. Once every few 
days, I'll go to one of the folders in Thunderbird that I have been 
accessing regularly and try to open a mail item.  I get a msg from 
Thunderbird that says something like Mail Server responded that UID is 
invalid.  From this point on, all mail in that folder is 
inaccessible.   I found that the way to fix it is to go the folder 
properties in Thunderbird and click Repair Folder. This basically 
wipes the folder contents and re-downloads the folder contents from 
James.  Everything is good again until it happens a day or so later on 
the same folder or another folder.  I have Thunderbird on several 
computers.  This consistently occurs on all instances of Thunderbird.


This is by far not a catastrophic problem. But it is a pain.  I'm just 
curious if anyone else is seeing this.  I'm pretty familiar with the 
inner workings of James 3 and the James 3 database structure.  Can 
someone give me a quick explanation of how the folder UIDs are 
generated.  Any idea why the UID would become invalid (unless 
Thunderbird just decided to start sending totally corrupted UIDs)?  
Finally, is there some quick debug logging parm I can enable the next 
time this happens just so I can get a better idea of what invalid UID is 
being sent and what James thinks about that uid?  Even it is totally a 
Mozilla problem, I'd like to arm myself with some good trace data to 
take to Mozilla.


Thanks for any assistance you can provide.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Thunderbird Losing UIDs in Folders

[Jerry Malcolm]

Thanks for the quick response, Kurt.  I guess I'll start prowling the 
T-bird forums and see if there's any talk about it there.


Jerry

On 3/3/2015 10:38 AM, Kurt Harders wrote:

Hi Jerry,
i had the same problem with Thunderbird (and some others like sometimes
not opening mail with attachment). This class of problems happens with
e.g. dovecot imap server too, and its a pure Thunderbird problem. Since
i switch to Linux using evolution i never had such problems. The
difference seems to be the amount of buffered/locally stored
information.

Regards, Kurt

Am Dienstag, den 03.03.2015, 10:21 -0600 schrieb Jerry Malcolm:

I use JAMES 3 b5 with a relatively large mail IMAP database (500K mail
items) across several accounts.  Most of my clients are using
Thunderbird.  The problem I'm seeing may very likely be a Thunderbird
problem.  But I'm just curious what might be going on. Once every few
days, I'll go to one of the folders in Thunderbird that I have been
accessing regularly and try to open a mail item.  I get a msg from
Thunderbird that says something like Mail Server responded that UID is
invalid.  From this point on, all mail in that folder is
inaccessible.   I found that the way to fix it is to go the folder
properties in Thunderbird and click Repair Folder. This basically
wipes the folder contents and re-downloads the folder contents from
James.  Everything is good again until it happens a day or so later on
the same folder or another folder.  I have Thunderbird on several
computers.  This consistently occurs on all instances of Thunderbird.

This is by far not a catastrophic problem. But it is a pain.  I'm just
curious if anyone else is seeing this.  I'm pretty familiar with the
inner workings of James 3 and the James 3 database structure.  Can
someone give me a quick explanation of how the folder UIDs are
generated.  Any idea why the UID would become invalid (unless
Thunderbird just decided to start sending totally corrupted UIDs)?
Finally, is there some quick debug logging parm I can enable the next
time this happens just so I can get a better idea of what invalid UID is
being sent and what James thinks about that uid?  Even it is totally a
Mozilla problem, I'd like to arm myself with some good trace data to
take to Mozilla.

Thanks for any assistance you can provide.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: James:: Move Succesfully Processed Email to a separte directory

[Jerry Malcolm]

There is a ToFolder mailet provided by James (I'm on James 3b5...). 
Unfortunately, it is not easily subclassable to set the target folder.  
But I just cloned the entire mailet from the James source, renamed it, 
and customized it.  You can then add whatever logic you need and define 
the folder that each mail item is to be placed into.


Jerry

On 3/27/2015 10:25 AM, Muhammad Ismail wrote:

Anyone ?

On Tue, Mar 24, 2015 at 8:30 PM, Muhammad Ismail it.is.ism...@gmail.com
wrote:


Hi

After processing an email successfully if change its state to some custom
processor  inside that processor can we copy/ move file to specified
directory  ?

Actually I am trying to figure out how we can copy or move the email which
get processed successfully to a separate directory.

--
Thanks







-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Gmail-style All Mail folder?

[Jerry Malcolm]

Is there any capability built into James to support a virtual All Mail 
folder like Gmail offers?  Gmail reports the All Mail folder to IMAP 
client as just another folder.  But Gmail rolls up the contents of all 
folders into this one virtual folder.


I have some massive business accounts with a very large rule base for 
sorting and storing inbound and outbound mail into various folders.   
When I get a buzz on my phone that new mail has arrived, I've got to 
search through a ton of folders trying to find the new mail that just 
came in.  I figure I could write a mailet to fork the routing and store 
a copy of each email in a common folder.  But I'm not really thrilled 
about doubling the size of the database with a copy of every email in a 
common folder.


Any thoughts on this?  Any ideas how difficult it would be to implement 
something like this, assuming it's not already there and I just missed 
it?  I might be willing to play around with adding this assuming it's 
not massive, and I can get a bit of guidance.


Thanks.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Problem with sending email to Yahoo, MSN, Hotmail and AOL accounts

[Jerry Malcolm]

Hi, Marc,

I have had many problems with gmail, yahoo, hotmail, etc bouncing my 
emails.  To say they are picky is an understatement.  If there is ANY 
problem with the structure of your email and/or your email setup, 
domain, dns, server IP address, and a bunch of other things , they are 
going to bounce it (with zero useful info to explain why). They also 
bounce according to how many emails you've sent depending on how 
'spammy' they think the email is.  They may let a few in. But if you 
send 20 'suspect' emails in a day, they'll start bouncing for a day or 
so, then let it start passing thru again.


I have found that the best solution is to make sure your emails and 
everything related to your outbound server are all completely clean 
according to spam analysis criteria.  A good place to start is 
mail-tester.com    Go to the web site and 
they'll give you a temp email address that you can send an email to.  
After you send it, click the link to analyze the mail.  They will grade 
your email including the sending server parameters and tell you how 
'good' your email is.  With detailed explanations of what is wrong.  My 
initial score was pretty bad.  It took some time and was a royal pain to 
fix all of the issues.  But I finally got the coveted 100% grade.  It 
appears gmail and others are finally happy with me and are letting my 
emails thru.


Jerry

On 9/23/2015 11:32 PM, Marc Chamberlin wrote:
Hi, I am running James 2.3.2 on a Linux (openSuSE 12.3) and recently 
encountered a problem that I am unable to resolve. I use the maillist 
mailet to manage a mail list for a number of users and now all emails 
being sent by the users to accounts at AOL, Yahoo, MSN, GMail, and 
Hotmail are bouncing. This is something new and I have not made any 
changes to James's configuration for a very long time. I have checked 
forward checking on reverse DNS look ups both via dig and on several 
different DNS checking websites and that is working correctly. The log 
files don't give me much help as far as I can decipher them, the 
mailet logs just indicate that there was a problem such as -


23/09/15 16:03:20 INFO  James.Mailet: RemoteDelivery: Exception 
delivering message (Mail1442594207467-7122-!518000-to-hotmail.com) - 
[EOF]
23/09/15 16:03:20 INFO  James.Mailet: RemoteDelivery: Temporary 
exception delivering mail (Mail1442594207467-7122-!518000-to-hotmail.com:
23/09/15 16:03:20 INFO  James.Mailet: RemoteDelivery: Bouncing message 
Mail1442594207467-7122-!518000-to-hotmail.com after 25 retries
23/09/15 16:03:20 INFO  James.Mailet: RemoteDelivery: Attempting 
delivery of Mail1442594207467-7122-!518000-to-msn.com to host 
mx4.hotmail.com. at 65.55.37.120 for addresses [some...@msn.com, 
someonee...@msn.com]


This log response varies a bit depending on the exact mail server that 
was the target of the email but looks similar enough to make me think 
all these failures are related.


I cannot make much sense out of what is stored in the other log files 
that relate to this particular problem though I do see a LOT of 
exceptions being reported with stack walkback traces. I admit I do not 
know enough about these log files to be able to make any sort of 
correlation between the errors I see in the mailet log files and these 
other errors being reported in other log files. So I am out of my 
depths, any kind guru want to help me resolve this?


Thanks in advance..Marc...




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Blowing up the tmp folder

[Jerry Malcolm]

I have been researching why it is taking  over a half hour for mysql to 
start up each time I have to reboot my server.  One post on a forum said 
to check c:\Windows\temp.  Apparently, if it gets a bunch of files in 
it, then mysql startup slows to a crawl.  Ok... I checked that temp 
folder.  Found over 1,000,000 (not exaggerating over 1 million!) 
spamassassin temp files. The files were named 
".spamassassintmp" plus nearly a million more 
"imap.msg" files.  (Just my opinion... but nearly 2 
million files probably qualifies as 'a bunch' as stated in the mysql 
forum)  It's taking over an hour just to erase the 2M files


I installed this entire server image clean a little over a year ago.  I 
am running James 3 beta 5.  I do have spamassassin installed, I'm 
running every email through it.  I have no doubt that I have had several 
million emails run through spamassassin over the past year.  I'm just 
curious what it causing it not to clean up after itself.  Is there some 
configuration item I've inadvertently set that's telling SpamAssassin to 
leave droppings behind?  I don't know if the imap files are related to 
spamassassin or not.  But the imap prefix pretty much means it has 
something to do with my James 3 configuration.


At this point, I know the simplest fix is to run a daily chron job to 
clean the temp folder, now that I know what is happening.  But I can't 
believe this is working as designed.  Do I have a more serious problem 
with spamassassin and/or James that this is a symptom of?


Thanks.

Jerry

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Training SpamAssassin with JAMES

[Jerry Malcolm]

I am using JAMES with a mySQL mail repository.  I would like to be able 
to train the Bayesian component of SpamAssassin using sa-learn or some 
equivalent.  All the documentation I can find on sa-learn assumes MBox 
format files for the folders containing spam/ham.   To this point I 
can't find any documentation related to integrating sa-learn with JAMES 
using a mySQL repository.


Long ago before I was using SpamAssassin, I was using the 
BayesianAnalysisFeeder mailet.  But that is not for SpamAssassin, is 
it?  Is there an equivalent somewhere?  Or just in general, what is the 
typical way everyone who integrates SpamAssassin with JAMES going about 
the process of training SpamAssassin?


BTW... I'm using JAMES 3.0 B5.  But I'm assuming that the answer applies 
to all versions.


Thanks.

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: SpamAssassin Flow

[Jerry Malcolm]

Benoit,

Thanks for the response.  But the default conf files do not have a 
'hasHeader' matcher following it.  That's what confused me.  Is that 
just an omission in the default conf file? (James 3 Beta 5).


Also, is there some documentation on how to integrate sa-learn with the 
James database when using mySQL repository and not using file-based mbox 
repository (i.e. point sa-learn to a James mySQL spam/ham folder)?



On 8/13/2016 2:30 AM, Benoit Tellier wrote:

Hi Jerry,

 From the javadoc of SpamAssassin mailet, it is written :

   " If the message exceeds the threshold, the header X-Spam-Flag will be
added with the value of YES"

I think you can then use a simple HasHeader.

I hope it helps.

Regards,

Benoit Tellier

Le 13/08/2016 à 00:43, Jerry Malcolm a écrit :

I just discovered my SpamAssassin flow is not working correctly. The
more I researched it, the more I can't figure out what is correct (James
3).  In the sample conf, there is the  for SpamAssassin.  But
there are no matchers after it to check the X-Spam-Status header and
route it based on threshold.  The next comment says something like 'send
remaining mails to'.  This implies that SpamAssassin somehow eats
the mail it determines is spam.  But I don't see that in the
SpamAssassin.java mailet source code.

Basically what I expected to see was some sort of matcher that looked at
the SpamAssassin header: X-Spam-Status=Yes, hits=24.0 required=5.0 and
routed based on Yes/No in the header.  But there is no matcher following
the SpamAssassin mailet.

I know I'm missing something obvious.  What is happening here that I'm
not seeing?  And what is the correct way to route to a Spam processor if
the header is a 'YES'?

Thanks.


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2016.0.7752 / Virus Database: 4633/12799 - Release Date: 08/12/16



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

SpamAssassin Flow

[Jerry Malcolm]

I just discovered my SpamAssassin flow is not working correctly. The 
more I researched it, the more I can't figure out what is correct (James 
3).  In the sample conf, there is the  for SpamAssassin.  But 
there are no matchers after it to check the X-Spam-Status header and 
route it based on threshold.  The next comment says something like 'send 
remaining mails to'.  This implies that SpamAssassin somehow eats 
the mail it determines is spam.  But I don't see that in the 
SpamAssassin.java mailet source code.


Basically what I expected to see was some sort of matcher that looked at 
the SpamAssassin header: X-Spam-Status=Yes, hits=24.0 required=5.0 and 
routed based on Yes/No in the header.  But there is no matcher following 
the SpamAssassin mailet.


I know I'm missing something obvious.  What is happening here that I'm 
not seeing?  And what is the correct way to route to a Spam processor if 
the header is a 'YES'?


Thanks.


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: JAMES website?

[Jerry Malcolm]

I understand how spf works.  I need the javadoc to the jspf classes so I 
can program to them.



On 9/25/2016 1:20 PM, Luuk wrote:

On 25-09-16 19:52, Jerry Malcolm wrote:
Thanks for the info.  I'm curious, though, why the need to take the 
old site down before having the new site up.


I'm trying to find the documentation for using jspf.  Is there an 
alternate site that might have that documentation until the new james 
site is up?





not james specific, but specific to spf:
http://www.openspf.org/



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2016.0.7797 / Virus Database: 4656/13081 - Release Date: 
09/25/16



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Training SpamAssassin with JAMES

[Jerry Malcolm]

I tried to get answers to this and other questions on the SpamAssassin 
forum.  They basically told me to pound sand and to get a 'real' MTA 
instead of JAMES.  I'm not throwing out everything just to be able to do 
better spam filtering.  But I'm back here looking for answers.  Is 
ANYBODY using SpamAssassin with JAMES?



On 8/14/2016 5:40 PM, Jerry Malcolm wrote:
I am using JAMES with a mySQL mail repository.  I would like to be 
able to train the Bayesian component of SpamAssassin using sa-learn or 
some equivalent.  All the documentation I can find on sa-learn assumes 
MBox format files for the folders containing spam/ham.   To this point 
I can't find any documentation related to integrating sa-learn with 
JAMES using a mySQL repository.


Long ago before I was using SpamAssassin, I was using the 
BayesianAnalysisFeeder mailet.  But that is not for SpamAssassin, is 
it?  Is there an equivalent somewhere?  Or just in general, what is 
the typical way everyone who integrates SpamAssassin with JAMES going 
about the process of training SpamAssassin?


BTW... I'm using JAMES 3.0 B5.  But I'm assuming that the answer 
applies to all versions.


Thanks.

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2016.0.7752 / Virus Database: 4647/12832 - Release Date: 
08/18/16



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

JAMES website?

[Jerry Malcolm]

Is the JAMES website getting some work done?  I get a 404 on 
http://james.apache.org/



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: JAMES website?

[Jerry Malcolm]

Thanks for the info.  I'm curious, though, why the need to take the old 
site down before having the new site up.


I'm trying to find the documentation for using jspf.  Is there an 
alternate site that might have that documentation until the new james 
site is up?



On 9/25/2016 12:38 PM, Ioan Eugen Stan wrote:

I just read an email from Eric Charles. He moved the website to
gitpubsub and was working on publishing it.

Maybe this is the reason?


On 25.09.2016 19:39, Jerry Malcolm wrote:

Is the JAMES website getting some work done?  I get a 404 on
http://james.apache.org/


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2016.0.7797 / Virus Database: 4656/13081 - Release Date: 09/25/16



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Ramifications of Deleting 'var' in James 3?

[Jerry Malcolm]

I've been on James 3 for nearly 3 years.  It's a rare occurrence, but 
there have been a few times that the var 'store' has become corrupted.  
Basically, JAMES siezes up, and nothing will get it going again.  I have 
found in these situations (basically through trial and error) that 
deleting the var directory fixes it.  Since deleting var causes JAMES to 
start functioning again, I'm going to assume that there was something 
corrupted in that folder.


After deleting var, I'm not observing any apparent repercussions.  But I 
gotta believe that there cannot be "zero" damage to killing a folder.


Today, it happened again.   From the log, it appears that mail was still 
coming in.  But IMAP was dead.  I tried bouncing JAMES and MySQL.  No 
change.  Tried rebooting the entire box.  Also, no change.  The errors 
below were showing up in the log.   I removed the var folder (actually 
just renamed it), and everything is back to running fine.


My one question what damage did I do?  Did I lose mail that might 
have come in during the several hours it froze before I noticed the 
problem?  BTW... it's James 3.0b5 running on WinServer2008 r2.


Log entries:

WARN  16:44:07,985 | org.apache.activemq.store.amq.AMQPersistenceAdapter 
| The ReferenceStore is not valid - recovering ...


... with several of these entries:

Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.io.ByteArrayOutputStream.grow(Unknown Source)
at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)
at java.io.ByteArrayOutputStream.write(Unknown Source)
at 
javax.mail.internet.MimeMultipart.readTillFirstBoundary(MimeMultipart.java:316)

..

ERROR 16:46:16,968 | james.mailspooler | Exception processing mail while 
spooling Unable to process mail 
Mail1482864159130-1b87a776-1f41-41d5-951b-ab7b2ed94d04 
(org.apache.camel.CamelExecutionException: Exception occurred during 
execution on the exchange: Exchange[Message: 
org.apache.james.core.MailImpl@c96e9e])
javax.mail.MessagingException: Unable to process mail 
Mail1482864159130-1b87a776-1f41-41d5-951b-ab7b2ed94d04 
(org.apache.camel.CamelExecutionException: Exception occurred during 
execution on the exchange: Exchange[Message: 
org.apache.james.core.MailImpl@c96e9e])
at 
org.apache.james.mailetcontainer.impl.camel.CamelMailetProcessor.service(CamelMailetProcessor.java:65)
at 
org.apache.james.mailetcontainer.lib.AbstractStateCompositeProcessor.service(AbstractStateCompositeProcessor.java:98)
at 
org.apache.james.mailetcontainer.impl.JamesMailSpooler$1.run(JamesMailSpooler.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown 
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown 
Source)

at java.lang.Thread.run(Unknown Source)


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Moving Archived Messages Out of Database

[Jerry Malcolm]

I'm using MySQL.

I'd love to get better performance on large folders.  However, I'd 
prefer to make clients move old archive folders completely off the 
server and store locally (the way POP3 does).  Since IMAP doesn't 
support doing that directly, I'm looking for suggestions on how to move 
hundreds of thousands of emails off the server completely.


Thanks.

Jerry

On 3/27/2017 2:37 AM, Matthieu Baechler wrote:

Hi,

I can confirm what Benoit said : making large mailboxes fast is not a 
easy task.


However, it's not impossible to do, it's just a matter of preparing 
the data for SELECT to being fast.


What mailbox backend are you using ?

Regards,




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Training SpamAssassin with JAMES

[Jerry Malcolm]

I moved to other problems for a while.  But my uncaught spam is getting 
worse by the day.  SpamAssassin states in its own docs that it going to 
be useless without training (i.e. SA-LEARN).  So I'm back again with 
same problem and question.  SA-LEARN can't interact with the JAMES db 
mail store.  It only deals with file system mail folders/files.  It 
baffles me that I cannot find a SINGLE JAMES user that is using 
SpamAssassin with JAMES.  Is that really the case, or are users just not 
getting around to responding?


Does ANYBODY know how to train the bayesian filter in SpamAssassin 
integrated with JAMES?  If anybody understands the SpamAssassin API 
(assuming there is one), I'll looking into writing a Mailet to do the 
work.  I and all my mail clients are drowning in uncaught spam right now.


By chance is there a utility that will dump a JAMES IMAP user folder 
into a file system MBOX folder?  I 'guess' I could copy a db folder into 
a file system folder, then run sa-learn on it, then erase it.  Sounds 
like duct tape and baling wire... But I HAVE to do SOMETHING.


Please give me some advice.

Thanks.

Jerry


On 8/18/2016 2:20 PM, Jerry Malcolm wrote:
I tried to get answers to this and other questions on the SpamAssassin 
forum.  They basically told me to pound sand and to get a 'real' MTA 
instead of JAMES.  I'm not throwing out everything just to be able to 
do better spam filtering.  But I'm back here looking for answers.  Is 
ANYBODY using SpamAssassin with JAMES?



On 8/14/2016 5:40 PM, Jerry Malcolm wrote:
I am using JAMES with a mySQL mail repository.  I would like to be 
able to train the Bayesian component of SpamAssassin using sa-learn 
or some equivalent. All the documentation I can find on sa-learn 
assumes MBox format files for the folders containing spam/ham.   To 
this point I can't find any documentation related to integrating 
sa-learn with JAMES using a mySQL repository.


Long ago before I was using SpamAssassin, I was using the 
BayesianAnalysisFeeder mailet.  But that is not for SpamAssassin, is 
it?  Is there an equivalent somewhere?  Or just in general, what is 
the typical way everyone who integrates SpamAssassin with JAMES going 
about the process of training SpamAssassin?


BTW... I'm using JAMES 3.0 B5.  But I'm assuming that the answer 
applies to all versions.


Thanks.

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2016.0.7752 / Virus Database: 4647/12832 - Release Date: 
08/18/16



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


---
This email has been checked for viruses by AVG.
http://www.avg.com




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: OOM on Specific Emails

[Jerry Malcolm]

I've decided to take this in steps.  My first objective is to just kill 
the email the first time it causes an OOM and at least stop it from 
going into an infinite loop trying to re-process it. My plan is to 
modify SetMimeHeader.java and add to to catch block.  But I need help on 
what I can do to kill the email.  Is there anything I can set in the 
Mail object or the MimeMessage object that will cause JAMES to just kill 
it or at least get it into a suspended state so JAMES won't restart it?


Thx.

BTW... my next step is to analyze the message in that same catch block 
before killing it and hopefully figure out the characteristic that is 
causing the OOM and simply kill it before the OOM can even occur.  But 
that's phase 2  Just want to stop the infinite loop first.



On 4/25/2017 11:39 AM, Jerry Malcolm wrote:

Benoit,

Thanks so much for the information.  This has definitely been a 
frustration.  I'm glad that there is some hope at getting around it.


You mentioned a SizeGreaterThan matcher.  Two questions

1) How can I determine the threshold/limit size?  I don't want to kill 
off good emails just because they are a bit large.  But I want to 
definitely get rid of the ones that are causing the problem.


2) If I do get a hit, what processor do I call that will discard the 
email and not risk further processing that might still cause the OOM?


Thanks again for the help.

Jerry


On 4/21/2017 11:57 PM, Benoit Tellier wrote:

Hi,

You are processing a too large email. SetMimeHeader mailet is modifying
your email, but fails allocating more resources.

This may be caused to a missing pre-allocation parameter in
ByteArrayOutputStream. This sounds from the stacktrace like a limitation
of the javax implementation. (You might get very long headers on this
mail, I guess)

We had some recent work with javax, and it might be interesting to know
if this limitation is still here.

You get the error on a recurrent bases as James is marking processing as
failed and will re-attend it. You might want to position a
"SizeGreaterThan" matcher to defend against this.

Hope this helps,

Thanks for the report,

Benoit

Le 22/04/2017 à 05:27, Jerry Malcolm a écrit :
I have James R3B5.  It's been working fine for over 2 years.  But in 
the

last few months I randomly start getting repeated OutOfMemory
exceptions.   The only fix I've found is to delete the var folder and
reboot.  The OOM entry in the log is always preceded by an error 
related

to one particular email.  The error and an OOM repeat continually for
that email until I delete the var folder.

Since it will sometimes go a week or two without an OOM and then for a
few days it will happen every few hours, I'm pretty certain is has 
to do
with some specific spam message coming in that JAMES is not 
handling.  I

don't know enough about the inner workings of JAMES to interpret this
stack trace.  I can make a patch and rebuild if I can just figure out
where to patch a workaround for this.  Can someone point me in the 
right

direction?

Log file: (this block is replicated 20-25+ times in the log file for 
the

same msg id;  Looks to me like the camel processor is in an infinite
loop until OOM occurs)

BTW... my heap is now set to 1024m.  I kept increasing it hoping for a
change.  But it looks like its going to fill up no matter what size I
make it.

ERROR 15:04:19,491 | org.apache.camel.processor.DefaultErrorHandler |
Failed delivery for (MessageId: ID-p3965917-56475-1492805025650-0-1 on
ExchangeId: ID-p3965917-56475-1492805025650-0-62). Exhausted after
delivery attempt: 1 caught: org.apache.camel.CamelExecutionException:
Exception occurred during execution on the exchange: Exchange[Message:
org.apache.james.core.MailImpl@bf4cc8]
org.apache.camel.CamelExecutionException: Exception occurred during
execution on the exchange: Exchange[Message:
org.apache.james.core.MailImpl@bf4cc8]
 at
org.apache.camel.util.ObjectHelper.wrapCamelExecutionException(ObjectHelper.java:1287) 



 at
org.apache.camel.impl.DefaultExchange.setException(DefaultExchange.java:282) 



 at
org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:64) 



 at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) 



 at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99) 



 at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) 



 at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73) 



 at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99) 



 at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90) 



 at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcess

OOM on Specific Emails

[Jerry Malcolm]

I have James R3B5.  It's been working fine for over 2 years.  But in the 
last few months I randomly start getting repeated OutOfMemory 
exceptions.   The only fix I've found is to delete the var folder and 
reboot.  The OOM entry in the log is always preceded by an error related 
to one particular email.  The error and an OOM repeat continually for 
that email until I delete the var folder.


Since it will sometimes go a week or two without an OOM and then for a 
few days it will happen every few hours, I'm pretty certain is has to do 
with some specific spam message coming in that JAMES is not handling.  I 
don't know enough about the inner workings of JAMES to interpret this 
stack trace.  I can make a patch and rebuild if I can just figure out 
where to patch a workaround for this.  Can someone point me in the right 
direction?


Log file: (this block is replicated 20-25+ times in the log file for the 
same msg id;  Looks to me like the camel processor is in an infinite 
loop until OOM occurs)


BTW... my heap is now set to 1024m.  I kept increasing it hoping for a 
change.  But it looks like its going to fill up no matter what size I 
make it.


ERROR 15:04:19,491 | org.apache.camel.processor.DefaultErrorHandler | 
Failed delivery for (MessageId: ID-p3965917-56475-1492805025650-0-1 on 
ExchangeId: ID-p3965917-56475-1492805025650-0-62). Exhausted after 
delivery attempt: 1 caught: org.apache.camel.CamelExecutionException: 
Exception occurred during execution on the exchange: Exchange[Message: 
org.apache.james.core.MailImpl@bf4cc8]
org.apache.camel.CamelExecutionException: Exception occurred during 
execution on the exchange: Exchange[Message: 
org.apache.james.core.MailImpl@bf4cc8]
at 
org.apache.camel.util.ObjectHelper.wrapCamelExecutionException(ObjectHelper.java:1287)
at 
org.apache.camel.impl.DefaultExchange.setException(DefaultExchange.java:282)
at 
org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:64)
at 
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at 
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at 
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at 
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at 
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at 
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at 
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:73)
at 
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at 
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at 
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at 
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)
at 
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at 
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:334)
at 
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:220)
at 
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
at 
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at 
org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:303)
at 
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at 
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at 
org.apache.camel.processor.ChoiceProcessor.process(ChoiceProcessor.java:81)
at 
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at 
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at 
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at 
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:73)
at 
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at 
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
at 
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
at 
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)
at 
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
at 

Re: OOM on Specific Emails

[Jerry Malcolm]

Benoit,

Thanks so much for the information.  This has definitely been a 
frustration.  I'm glad that there is some hope at getting around it.


You mentioned a SizeGreaterThan matcher.  Two questions

1) How can I determine the threshold/limit size?  I don't want to kill 
off good emails just because they are a bit large.  But I want to 
definitely get rid of the ones that are causing the problem.


2) If I do get a hit, what processor do I call that will discard the 
email and not risk further processing that might still cause the OOM?


Thanks again for the help.

Jerry


On 4/21/2017 11:57 PM, Benoit Tellier wrote:

Hi,

You are processing a too large email. SetMimeHeader mailet is modifying
your email, but fails allocating more resources.

This may be caused to a missing pre-allocation parameter in
ByteArrayOutputStream. This sounds from the stacktrace like a limitation
of the javax implementation. (You might get very long headers on this
mail, I guess)

We had some recent work with javax, and it might be interesting to know
if this limitation is still here.

You get the error on a recurrent bases as James is marking processing as
failed and will re-attend it. You might want to position a
"SizeGreaterThan" matcher to defend against this.

Hope this helps,

Thanks for the report,

Benoit

Le 22/04/2017 à 05:27, Jerry Malcolm a écrit :

I have James R3B5.  It's been working fine for over 2 years.  But in the
last few months I randomly start getting repeated OutOfMemory
exceptions.   The only fix I've found is to delete the var folder and
reboot.  The OOM entry in the log is always preceded by an error related
to one particular email.  The error and an OOM repeat continually for
that email until I delete the var folder.

Since it will sometimes go a week or two without an OOM and then for a
few days it will happen every few hours, I'm pretty certain is has to do
with some specific spam message coming in that JAMES is not handling.  I
don't know enough about the inner workings of JAMES to interpret this
stack trace.  I can make a patch and rebuild if I can just figure out
where to patch a workaround for this.  Can someone point me in the right
direction?

Log file: (this block is replicated 20-25+ times in the log file for the
same msg id;  Looks to me like the camel processor is in an infinite
loop until OOM occurs)

BTW... my heap is now set to 1024m.  I kept increasing it hoping for a
change.  But it looks like its going to fill up no matter what size I
make it.

ERROR 15:04:19,491 | org.apache.camel.processor.DefaultErrorHandler |
Failed delivery for (MessageId: ID-p3965917-56475-1492805025650-0-1 on
ExchangeId: ID-p3965917-56475-1492805025650-0-62). Exhausted after
delivery attempt: 1 caught: org.apache.camel.CamelExecutionException:
Exception occurred during execution on the exchange: Exchange[Message:
org.apache.james.core.MailImpl@bf4cc8]
org.apache.camel.CamelExecutionException: Exception occurred during
execution on the exchange: Exchange[Message:
org.apache.james.core.MailImpl@bf4cc8]
 at
org.apache.camel.util.ObjectHelper.wrapCamelExecutionException(ObjectHelper.java:1287)

 at
org.apache.camel.impl.DefaultExchange.setException(DefaultExchange.java:282)

 at
org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:64)

 at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)

 at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)

 at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)

 at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)

 at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)

 at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)

 at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:73)

 at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)

 at
org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)

 at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)

 at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)

 at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)

 at
org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:334)

 at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:220)

 at
org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcess

Meaning of "Got : ##" in IMapServer log?

[Jerry Malcolm]

Some of my users are getting timeouts periodically in Thunderbird and 
other mail clients when trying to access their mail folders. I enabled 
DEBUG on IMAP (James R3B5).  I'm seeing sessions lasting several minutes 
with an ongoing "tag" count.   I have a pretty good grasp of the IMAP 
API and have written some standalone apps that interface with IMAP.  But 
I don't really understand what Tbird is doing.Is this 'tag' just 
some ndx that Tbird keeps incrementing, and does Tbird just keep a 
session open forever? I've got to wait until the hang occurs again on 
Tbird for some client and then determine if the logs differ.  But until 
that time, I'd like a better grasp of what I'm seeing in the log below.  
Can someone explain what is flowing back and forth below? Also, is DEBUG 
the most granular I can go in JAMES log4j?  I'd love to see when each 
command completes and not just when it starts.


DEBUG 18:22:44,249 | james.imapserver | ID=27822739 Got : 21
DEBUG 18:22:44,249 | james.imapserver | ID=27822739 Got : IDLE
DEBUG 18:22:44,249 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f

DEBUG 18:24:44,359 | james.imapserver | ID=27822739 Got : 22
DEBUG 18:24:44,359 | james.imapserver | ID=27822739 Got : noop
DEBUG 18:24:44,359 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f

DEBUG 18:24:44,406 | james.imapserver | ID=27822739 Got : 23
DEBUG 18:24:44,406 | james.imapserver | ID=27822739 Got : UID
DEBUG 18:24:44,406 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f
DEBUG 18:24:44,406 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f

DEBUG 18:24:44,515 | james.imapserver | ID=27822739 Got : 24
DEBUG 18:24:44,515 | james.imapserver | ID=27822739 Got : IDLE
DEBUG 18:24:44,515 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f

DEBUG 18:25:42,569 | james.imapserver | ID=27822739 Got : 25
DEBUG 18:25:42,569 | james.imapserver | ID=27822739 Got : check
DEBUG 18:25:42,571 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f

DEBUG 18:25:42,619 | james.imapserver | ID=27822739 Got : 26
DEBUG 18:25:42,619 | james.imapserver | ID=27822739 Got : UID
DEBUG 18:25:42,621 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f
DEBUG 18:25:42,641 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f

DEBUG 18:25:42,735 | james.imapserver | ID=27822739 Got : 27
DEBUG 18:25:42,735 | james.imapserver | ID=27822739 Got : IDLE
DEBUG 18:25:42,735 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f

DEBUG 18:27:42,839 | james.imapserver | ID=27822739 Got : 28
DEBUG 18:27:42,839 | james.imapserver | ID=27822739 Got : noop
DEBUG 18:27:42,841 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f

DEBUG 18:27:42,888 | james.imapserver | ID=27822739 Got : 29
DEBUG 18:27:42,888 | james.imapserver | ID=27822739 Got : UID
DEBUG 18:27:42,890 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f
DEBUG 18:27:42,896 | james.imapserver | ID=27822739 Loaded mailbox 
#private:fred@f

DEBUG 18:27:43,003 | james.imapserver | ID=27822739 Got : 30
DEBUG 18:27:43,003 | james.imapserver | ID=27822739 Got : IDLE


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

james_mail_userflag not being managed

[Jerry Malcolm]

I'm running r3b5.  I have a reasonably large repository with ~1M mail 
records.  I just noticed that the james_mail_userflag table has 1.2 
BILLION records... approximately 1000 records for each email record.  I 
do use user flags a lot.  But I would expect a max of one user flag 
record for each email.  It appears that when email records are deleted, 
(e.g. spam cleanup), then associated userflag records are not being deleted.


Fortunately, I have an imap utility that can rebuild the flags. So I 
emptied out the user flag table. (it cut the database size in half) 
and let the utility rebuild the flags.  But obviously, this is not the 
correct process.


Is this a known issue?

Thanks.

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Forwarding Mailet

[Jerry Malcolm]

If you host both domains on the same server, you can add records to the 
JAMES_Recipient_Rewrite table.  Also, there is a Resend mailet that you 
can use to send to outside domains.  Depending on you specific 
requirements you might need to write a subclass of it to customize it to 
your needs.



On 11/2/2017 4:09 PM, Mark Gordon wrote:

Is there a mailet already written that takes any inbound mail to:
x...@blabla.com and forwards it to x...@zzz.com?

That is it forwards to the same user at a different domain.  The use case
is that we use a domain that we send email out in the From address but this
domain has no real email addresses.  So the from address would be
x...@mail.example.com  so if the customer replied we would want the email to
be forwarded to x...@example.com

Thanks,
-Mark




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Multiple Databases?

[Jerry Malcolm]

I realize this is a completely off-the-wall question... but has there 
been any discussion about breaking the JAMES db into multiple 
databases?  I have a bunch of clients, each with a bunch of accounts.  
And they all want to archive all of their mail and be able to access it 
through the same mail client.  As the years go by, the db just keeps 
growing and growing.  This became acutely obvious when I was recently 
forced by my server provider to migrate to newer hardware, and I 
realized I had to have mail down for a long time while I transferred a 
60gB+ file across a slow connection to the new server.  The db size is 
also making daily backups a problem.


The reality is that 90% of the mail is archived into 'year' folders for 
each of the accounts which are basically "read-only" now.  Only a 
relatively small amount of mail is truly in dynamically updated 
folders.  If there was a way to store "/archives/2002" folders through 
"/archives/2017" folders in one db and all of the other folders in 
another db, it would make backup and migration a much simpler task.


Ok, I'm pretty sure that isn't in the immediate plan.  But just let me 
put my vote in.  Alternatively, is there any alternative to having one 
ever-growing mail db?  Is there some trick with the db server that can 
present one logical db from multiple db files?  (I know a lot about 
databases... but there's still a bunch I don't know).  Has anyone else 
faced/addressed this problem?  Or is the answer to just live with it?


Thanks.

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Java Update Disaster

[Jerry Malcolm]

James has been working for months untouched.=C2=A0 This morning I needed to=
=20
reboot the server for other reasons.=C2=A0 When it came back, James would n=
ot=20
start.=C2=A0 It was giving me errors saying camel xml file had syntax error=
s=20
and couldn't parse, etc.=C2=A0 This really blindsided me.=C2=A0 I hadn't do=
ne a=20
thing to change anything.=C2=A0=C2=A0 The root cause of the error was somet=
hing=20
like some jaxb exception class couldn't be found.=C2=A0=C2=A0 I did some go=
ogling=20
and found that jaxb is removed from java SE 9+ (and moved to EE).=C2=A0 I=
=20
checked my java paths, and they still show java 8.=C2=A0 So that "couldn't =
be=20
the problem" Then I remembered a week or so ago, java started=20
yelling at me that it was out of date.=C2=A0 I never auto upgrade if things=
=20
are working.=C2=A0 But when I thought I was dismissing the popup, it said=
=20
something about old files removed.=C2=A0 Bottom line is java apparently=20
upgraded me to java 9 inside the java 8 folder. Maybe not fully... but=20
in any case, jaxb was no longer there.=C2=A0 And it didn't show up until I=
=20
rebooted.

In the forum research someone said that the jaxb files are still there=20
in 9, but you have to call them out.=C2=A0 Add an additional argument:=20
"--add-modules java.xml.bind" on the java command makes them available.

So... I don't really know what the java update did under the covers.=C2=A0=
=20
But it broke james.=C2=A0 Finally, after adding the above arg to the java=
=20
command line, things started working again.=C2=A0 So if you are using Java9=
=20
or higher, or if java appears to have updated you against your wishes,=20
this java command line arg should fix james.


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: New user, so far so good .. just one problem.

[Jerry Malcolm]

Gmail can be harsh... Any little thing wrong, and they chunk your 
mail.   Just a suggestion if you haven't already done so, go to 
http://www.mail-tester.com/ and then send an email to the address they 
give you.  They'll grade your email.  I've found if you don't get a 10 
out of 10, you'll have problems delivering to gmail.  If you get a 10 
score, then at least you can be confident that your problems aren't in 
that area.



On 12/27/2018 1:27 PM, John Dale wrote:

Greetings!

I'm just getting acquainted with James.

I have it up and running and listening.

I was able to send mail to james from gmail.

I fixed the relaying issue and added 1.1.1.1 as the james dns server.

My message back to gmail, though, gets spooled, but never is delivered.

I made sure my reverse DNS was correct for the mail server.

There are no ERROR level messages in the log files.

27/12/18 18:46:12 INFO  smtpserver: Successfully spooled mail
Mail1545936372121-1 from j...@mydomain.com on 75.76.161.194 for
[jcdw...@gmail.com]

Looking forward to hardening my installation if I can get this to work.

Thanks!

John

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Blacklist site started flagging everything as spam this morning

[Jerry Malcolm]

I am running 3.0b5.  Everything has been running pretty much without a 
hitch for a couple of years now.  Around 5:30 this morning (11:30GMT) 
all inbound mail started being flagged as spam.  After digging through 
the logs and my conf files, I finally isolated it to the njabl.org 
site.  It was flagging all sender domains including gmail.com.  I 
commented the matcher block (below) out of my conf file, and finally 
everything is flowing again.



    spam
    550 Requested action not taken: rejected - see 
http://njabl.org/



Is anyone else seeing this?  This site is rating the sending domain/ip, 
and therefore shouldn't be rating anything to do with my domain, 
correct?  I'll just keep this commented out until I figure out what's 
happening.


Ideas?

Thanks,

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Windows Service Problem

[Jerry Malcolm]

I have had James v3B5 running for several years on a WinServer16 box.  
In the last couple of weeks, the service has failed to start.  I can 
open a command prompt and run James with the 'run' bat file from there 
without any problem.  I've tried rebooting.  I uninstalled the service 
and reinstalled it.  No luck.  I haven't made any changes to the system 
other than the mandatory Windows updates.  But at this point, I can't 
get the service to start. The last two times I tried it, I got error 
messages that meant nothing.  The most recent one said it didn't have 
enough memory. I've got 16GB, and it's running about 45%.  Perhaps 
somebody can decipher the log below.  Not much recent info on google 
about the 'unable to allocate a console for the service' message.  The 
wrapper header output is: Java Service Wrapper Community Edition 64-bit 
3.5.24.


Anybody got any suggestions?  Thx.  Log:

C:\james-3.0.0-b5\log>type wrapper.log
STATUS | wrapper  | 2019/08/01 13:06:26 | --> Wrapper Started as Service
ERROR  | wrapper  | 2019/08/01 13:06:26 | ERROR: Unable to allocate a 
console for the service: The specified resource name cannot be found in 
the image file. (0x716)

STATUS | wrapper  | 2019/08/01 13:06:26 | <-- Wrapper Stopped
STATUS | wrapperm | 2019/08/02 19:06:58 | Apache James :: Server :: App 
service removed.
STATUS | wrapperm | 2019/08/02 19:07:12 | Apache James :: Server :: App 
service installed.

STATUS | wrapper  | 2019/08/02 19:07:26 | --> Wrapper Started as Service
ERROR  | wrapper  | 2019/08/02 19:07:26 | ERROR: Unable to allocate a 
console for the service: Not enough storage is available to process this 
command. (0x8)

STATUS | wrapper  | 2019/08/02 19:07:26 | <-- Wrapper Stopped

Re: James with AWS SES mail receiving

[Jerry Malcolm]

I'm using an EC2 with tomcat and jdbc with RDS with no problems. So that 
shouldn't be a problem.  My main concern was whether or not AWS blocks 
port 25.  At one time, goDaddy dedicated servers were forced to go 
through a goDaddy proxy for port 25.  The troll under the bridge would 
only send a certain number of emails each day, so emails would take 
several days to arrive if the limit was reached,  and they would charge 
outrageous fees if you sent too many emails out.  I can't afford any 
situation like that.  As long as AWS doesn't proxy port 25, I'm not 
worried. But not surprising that goDaddy didn't widely publicize their 
little proxy scam.  So just a bit paranoid Just wondering if anybody 
had first hand experience with James in AWS.


On 8/12/2019 11:39 PM, Tellier Benoit wrote:

If there is a JDBC driver for RDS then that is the way to go.

Am I missing something?

Cheers,

Benoit

On 05/08/2019 23:24, Jerry Malcolm wrote:

I'm looking at moving off of my dedicated server to AWS, which means
moving JAMES.  Interesting concepts in this thread below from a year or
so ago about using SES.  But all I want to do is get JAMES up and
running in an AWS EC2 with an RDS with as little rip up and effort as
possible.   Assuming I copy my current JAMES build to EC2, set up the
RDS, install SpamAssassin, and open the appropriate ports, are there any
gotchas lurking that I need to be aware of?  (Still just getting my feet
wet with AWS).

Somewhat off topic... what size EC2 is recommended?  Do I simply start
small and creep up until the EC2 no longer pegs the meter? Anybody have
an experience with what size EC2 to select?

Thanks.

Jerry

On 6/11/2018 8:16 AM, Jeremy T. Bouse wrote:

Benoit,

  Yes it can send to a SMTP endpoint but the FROM address has to be an
approved email address/domain for SES to be able to send it which means
having to modify the envelope headers. Further to that I'm looking at
the possibility of running James in containers on AWS ECS which would be
on a private subnet so it wouldn't be reachable directly without going
through a load-balancer.


On 6/11/2018 12:44 AM, Benoit Tellier wrote:

Hi Jeremy,

Can't AWS SES send these messages to a SMTP endpoint? This way it will
work without any further development.

That being said, I consider the feature you propose extremely
interesting, as it will provide alternatives to SMTP for applications
sending emails.

In my opinion, you would need to implement a new component in James
listening on AWS SNS, upon messages retrieving the mails, and then
enqueue them in James internal MailQueue.

Cheers,

Benoit Tellier

Le 11/06/2018 à 01:53, Jeremy T. Bouse a écrit :

  Has anyone thought about how to possibly make use of AWS SES email
receiving to accept inbound email and get it passed along into James?
With AWS SES email receiving you can have it save the actual message to
an S3 bucket and then fire off an SNS topic or Lambda function so the
question would really be how to trigger James to be able to process the
alert and ingest the message from S3. I've been thinking about it as a
possible email solution for myself and it seems like it should be
possible but I've not yet been able to determine if there's already an
easy method to do so or if it'll take some development to make it work.
Wanted to see if anyone else had thought about it.


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Building James 3.3.0 in Eclipse

[Jerry Malcolm]

Raphael,

Thanks for the guidance.  I'm closer now.  But still not there. After a 
full day of trial and error on this, I settled on the following steps:


1) On a clean eclipse workspace, File-->Import-->Git --- I selected 
"clone URI" and entered the James GIT clone URI: 
https://github.com/apache/james-project.git


2) I selected the 3.3.x branch (I guess that's what I want... (??) ) and 
said to also clone submodules.  It downloaded to a folder.  It created 
an eclipse project/package named james-server. But the project only had 
3 files: model.di, model.notation, and model.uml.


3) On a whim, I decided to import again using "Maven existing project" 
and pointed to the extracted folder.  That seemed to work for the most part.


It initially gave me 10 errors saying "Plugin execution not covered by 
lifecycle configuration" and 16 more errors with "Project configuration 
not up to date with pom.xml".  All with varying pom.xml files.  I did a 
"quick fix" and it got rid of the 'not up to date' errors.  I googled 
the lifecycle error and it said to edit all of the pom.xml files and add 
 tags around the plugin tags.  I tried in on a couple 
of the pom.xml files, and it did fix the errors (???).


I then did a build, and it added one more error, something about an 
invalid mojo tag in a pom descriptor.


Just for fun, I did a clean build, and that was a huge mistake. I now 
have 340 errors where it can't resolve DTOs in FlagsTest, 
EventSerializer in EventDispatcher, etc.  Apparently 'clean' wipes out 
source files as well.  Or it wipes out class files that are not part of 
the full build.


So that's where I am.  I can edit a bunch of pom.xml files and add 
tags.  I can hopefully figure out how to fix mojo.  And I can hope to 
remember to NEVER do a clean build again.  But this just doesn't seem 
right that I have to go in and modify a bunch of source files just to 
get a clean build from a ship-level source package.  I have zero 
confidence if I have to make all of these change that I really have an 
'official' 3.3.x James.


Suggestions?  Where did I stray from the correct path?  I'm beginning to 
feel like I belong in the class with the slow kids


On 9/2/2019 6:04 PM, Jerry Malcolm wrote:


On 9/2/2019 2:55 AM, Raphael OUAZANA wrote:

Hi Jerry,

Don't worry, Eclipse is still supported as a development environment 
and currently used by some developers (even if we have some issues in 
incoming 3.4 regarding compatibility between Eclipse and some Scala 
code).


The usual workflow to work in a dev environment is the following:
- clone the git repository
- checkout the branch / tag you want to work on
- import the project in Eclipse

This workflow should work perfectly well.

The issue you are encountering is because as you are taking the 
sources directly from the released version, you don't have any .git 
repository as you would have if you had cloned the project. This is 
not a big harm, but it seems to make failing the git-commit-id-plugin.
So if you remove this plugin from the main pom.xml, I guess your 
problem would be solved.


Regards,
Raphaël.

Le 2019-09-01 06:16, Jerry Malcolm a écrit :

Thanks for the info, Garry.  But I'm still confused.  Are you saying
that it is no longer possible to build from the source on Eclipse?  I
realize the team may be pushing docker.  But there are still
non-docker binaries that got built someway.  I just want to recreate
those binaries with my code additions.  But first I want to just build
with the official source untouched and verify that works. Then I'll
add my changes and rebuild.

You said the .git files are from the .git repository.  I don't doubt
that.  But I'm not a git guru.  I'm still not sure how to fix the
errors I'm getting (~400+ of these):

org.apache.maven.plugin.MojoExecutionException: .git directory is not
found! Please specify a valid [dotGitDirectory] in your pom.xml

I do not know anything about pom syntax.  I have no clue how/where to
"specify a valid [dotGitDirectory] in your pom.xml" (times 400)
Does this error mean that there isn't a gitDirectory defined in the
pom.xml and there should be? Or does it mean that one is defined
somewhere and the specified directory doesn't exist on my file
system?  I guess it'd be too much to ask for this error message to
include the directory it found and which of the 400 pom.xml files the
error refers to. (Sorry... I'm just frustrated at less-than-useful
error messages).

Did I download an invalid source zip?  Is there a different zip (or
additional zips that contain the correct pom files? ) I simply went to
github and downloaded the latest source zip file.

Where do I start?  I really don't want to have to learn how to build
on Linux unless there's an official word that says Eclipse is no
longer supported as a build platform.  I need to use Eclipse to debug
my code.  I just want to download the correct source, import it in

Re: "Getting Started" with JAMES using Docker

[Jerry Malcolm]

Thanks for the detailed info.  I have no real desire to get into Docker 
at this point (likely will at a later date).  I was just enticed by the 
statement on the James home page: (http://james.apache.org/index.html) 
that says I can download and install James in one statement:


*=
*

*STEPS YOU HAVE TO FOLLOW:*
Pull and run the James image with the following single command:

|docker run -p "25:25" -p "143:143" linagora/james-jpa-sample:3.3.0|

Then, connect this image with for instance, Thunderbird.



I just did what it said.  I'll try dropping off the 3.3.0. But I feel 
like we're just shooting in the dark trying to guess at the package name 
docker is looking for.  Is there a way to list the package names that 
docker is aware of?


Thx

Jerry

On 8/29/2019 2:53 AM, Aleksandar Stoisavljevic wrote:

Hi Jerry,

I'll try to be brief and not to come into many details of how Docker is
working.
For that, I suggest that you go through some video materials and/or to find
some
really extensive documentation on Internet (blogs, posts, Docker
documentation, books, etc.)

In the meantime, I'll try to explain what happened in your case.

Obviously you've installed Docker correctly as we can see from command
response.

There are actually two concepts in Docker that you have to distinguish:
Docker Image and
Docker Container.
Docker Container is "running version" of Docker Image.

When you stated:

[ec2-user@ip-172-31-47-236 ~]$ docker run -p "25:25" -p "143:143"
linagora/james-jpa-sample:3.3.0

It actually means that it will try to run Docker Container, listening on
host ports 25 and 143, and Container should
be of a Docker Image "linagora/james-jpa-sample:3.3.0".

That image, "linagora/james-jpa-sample:3.3.0", should be availabe either
locally or placed somewhere on Internet, so
Docker can access it and grab it. For this kind of Docker Image store,
usually people use Docker Hub (https://hub.docker.com/),
but there are also other Docker Repositories (private or public).

As you don't have it locally, that Docker Image
("linagora/james-jpa-sample:3.3.0"), therefore you've got that information
in reponse:
"Unable to find image 'linagora/james-jpa-sample:3.3.0' locally".

At that moment, Docker Engine will try to grab that image from Docker Hub.

But unfortunately, I don't see that 3.3.0 version of that image exists.
You can go to URL (https://hub.docker.com/r/linagora/james-jpa-sample/tags)
for yourself and make sure that it doesn't exists.

I don't know how did you get that 3.3.0, maybe it is in some outdated
documentation, but I suggest that you try either with latest tag
(so the command would be  docker run -p "25:25" -p "143:143"
linagora/james-jpa-sample) or if you really have some reasons for 3.3.X
then I would try with docker run -p "25:25" -p "143:143"
linagora/james-jpa-sample:3.3.X

Let's try that and then just reply to this thread.


Than You



On Thu, Aug 29, 2019 at 5:18 AM Jerry Malcolm 
wrote:


I was preparing to do a trial run at getting James up and running on an
AWS EC2 instance.  I went to the James home page and saw there is a
1-line command to download and install it.  However, to use that one
line there is apparently an assumption of a full understanding of Docker
and a pre-existing installation of Docker.  Docker isn't automatically
installed with Amazon Linux in an EC2.  I was able to install it with
yum.  But when I entered the command from the JAMES getting started
page, this is what I got:

[ec2-user@ip-172-31-47-236 ~]$ docker run -p "25:25" -p "143:143"
linagora/james-jpa-sample:3.3.0
Unable to find image 'linagora/james-jpa-sample:3.3.0' locally
docker: Error response from daemon: manifest for
linagora/james-jpa-sample:3.3.0 not found.
See 'docker run --help'.

Since there's no url to download james, I assume that the manifest that
it is saying that it can't find contains all of that useful info.  I
know nothing about how Docker works.  Where would I find the missing
manifest, and if it was expected to automatically come with Docker as
implied with the "1-line-download-install" statement, what did I do wrong?

Thx

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: "Getting Started" with JAMES using Docker

[Jerry Malcolm]

Hi, Aleksanda,

I really appreciate your information.  I believe that someone realized 
the image wasn't there and quietly got it posted overnight.  When I 
reran the command it worked, at least somewhat.


Just an unsolicited opinion for the James folks... I'm not sure that one 
line command as the first thing a new user will see is quite ready for 
prime-time.   As I mentioned in my first post yesterday, there is an 
implied assumption that the user understands docker and has it 
installed.  Nothing against docker. But it has not yet attained 
ubiquity.  My guess is a lot of new James users will have no idea what 
it is or where to get it, either for Linux or Windows. And if that's the 
case, there's no alternative pointer on that home page for non-docker 
install. Next, the docker install failed for me because port 25 was 
already in use since aws ec2 Linux automatically installs sendmail and 
starts it up.  Maybe all new James users will implicitly understand what 
that error means and how to fix it.  Finally, docker got to the point of 
creating the dummy user ids and locked up.  Had to kill my puTTY 
session.  No idea how to try to start/stop the james service and see how 
much, if any of it, is installed and runable.


I'm not trying to be negative.  That one line install is great if it 
really works for the majority.  But if a user (even a veteran of JAMES 
for ~15 years like me) has the problems I've encountered, I would 
strongly suggest a detailed followup page to go to when the one line 
install crashes and burns.  If I wasn't a seasoned user of James, and 
rather a first-timer with this supposedly 'super easy' experience to 
get it up and running, I might decide to go another direction for my MTA 
needs.  I think we can head that off with a pointer to more info and 
what to do if things don't go right on that one-line install.


Thx

Jerry


On 8/29/2019 2:53 AM, Aleksandar Stoisavljevic wrote:

Hi Jerry,

I'll try to be brief and not to come into many details of how Docker is
working.
For that, I suggest that you go through some video materials and/or to find
some
really extensive documentation on Internet (blogs, posts, Docker
documentation, books, etc.)

In the meantime, I'll try to explain what happened in your case.

Obviously you've installed Docker correctly as we can see from command
response.

There are actually two concepts in Docker that you have to distinguish:
Docker Image and
Docker Container.
Docker Container is "running version" of Docker Image.

When you stated:

[ec2-user@ip-172-31-47-236 ~]$ docker run -p "25:25" -p "143:143"
linagora/james-jpa-sample:3.3.0

It actually means that it will try to run Docker Container, listening on
host ports 25 and 143, and Container should
be of a Docker Image "linagora/james-jpa-sample:3.3.0".

That image, "linagora/james-jpa-sample:3.3.0", should be availabe either
locally or placed somewhere on Internet, so
Docker can access it and grab it. For this kind of Docker Image store,
usually people use Docker Hub (https://hub.docker.com/),
but there are also other Docker Repositories (private or public).

As you don't have it locally, that Docker Image
("linagora/james-jpa-sample:3.3.0"), therefore you've got that information
in reponse:
"Unable to find image 'linagora/james-jpa-sample:3.3.0' locally".

At that moment, Docker Engine will try to grab that image from Docker Hub.

But unfortunately, I don't see that 3.3.0 version of that image exists.
You can go to URL (https://hub.docker.com/r/linagora/james-jpa-sample/tags)
for yourself and make sure that it doesn't exists.

I don't know how did you get that 3.3.0, maybe it is in some outdated
documentation, but I suggest that you try either with latest tag
(so the command would be  docker run -p "25:25" -p "143:143"
linagora/james-jpa-sample) or if you really have some reasons for 3.3.X
then I would try with docker run -p "25:25" -p "143:143"
linagora/james-jpa-sample:3.3.X

Let's try that and then just reply to this thread.


Than You



On Thu, Aug 29, 2019 at 5:18 AM Jerry Malcolm 
wrote:


I was preparing to do a trial run at getting James up and running on an
AWS EC2 instance.  I went to the James home page and saw there is a
1-line command to download and install it.  However, to use that one
line there is apparently an assumption of a full understanding of Docker
and a pre-existing installation of Docker.  Docker isn't automatically
installed with Amazon Linux in an EC2.  I was able to install it with
yum.  But when I entered the command from the JAMES getting started
page, this is what I got:

[ec2-user@ip-172-31-47-236 ~]$ docker run -p "25:25" -p "143:143"
linagora/james-jpa-sample:3.3.0
Unable to find image 'linagora/james-jpa-sample:3.3.0' locally
docker: Error response from daemon: manifest for
linagora/james-jpa-sample:3.3.0 not found.
See 'docker 

Re: Building James 3.3.0 in Eclipse

[Jerry Malcolm]

Thanks for the info, Garry.  But I'm still confused.  Are you saying 
that it is no longer possible to build from the source on Eclipse?  I 
realize the team may be pushing docker.  But there are still non-docker 
binaries that got built someway.  I just want to recreate those binaries 
with my code additions.  But first I want to just build with the 
official source untouched and verify that works.  Then I'll add my 
changes and rebuild.


You said the .git files are from the .git repository.  I don't doubt 
that.  But I'm not a git guru.  I'm still not sure how to fix the errors 
I'm getting (~400+ of these):


org.apache.maven.plugin.MojoExecutionException: .git directory is not 
found! Please specify a valid [dotGitDirectory] in your pom.xml


I do not know anything about pom syntax.  I have no clue how/where to 
"specify a valid [dotGitDirectory] in your pom.xml" (times 400)  
Does this error mean that there isn't a gitDirectory defined in the 
pom.xml and there should be? Or does it mean that one is defined 
somewhere and the specified directory doesn't exist on my file system?  
I guess it'd be too much to ask for this error message to include the 
directory it found and which of the 400 pom.xml files the error refers 
to. (Sorry... I'm just frustrated at less-than-useful error messages).


Did I download an invalid source zip?  Is there a different zip (or 
additional zips that contain the correct pom files? ) I simply went to 
github and downloaded the latest source zip file.


Where do I start?  I really don't want to have to learn how to build on 
Linux unless there's an official word that says Eclipse is no longer 
supported as a build platform.  I need to use Eclipse to debug my code.  
I just want to download the correct source, import it into Eclipse, and 
build it.  I just can't understand why I can't get anywhere near a 
successful build.  What am I doing wrong?


Jerry

On 8/31/2019 5:25 PM, Garry Hurley wrote:

Jerry, I had to download the source, run configure and make on a Linux system. 
The problem is that the source is made to build onto a docker system on 
Windows. Those of us who choose not to run docker and run a real operating 
system often have issues. Those .git files are from the .git repository. If you 
did a git clone, create your workspace from the local git repo or import 
existing maven projects into eclipse from the repo. This serves two purposes. 
1, it allows you to keep your project up to date and 2, it allows you to commit 
if you have any changes to the existing code you want to push up for others.

I haven’t built anything after 3.3.0 was finalizedbecause the government 
finally approved 3.3.0 for use (ironically, they had been insisting that 
3.0.4beta was safe to use in production and I outright refused to put that in 
place).

Sent from my iPhone


On Aug 31, 2019, at 6:02 PM, Jerry Malcolm  wrote:

Something else I just found... the link to a tutorial at 
http://blog.bonnydoonmedia.com/post.cfm/walkthrough-tutorials-compiling-apache-james-v3-with-eclipse
 in the Eclipse build section is dead.  No DNS entry.



On 8/31/2019 4:54 PM, Jerry Malcolm wrote:
Correction... the build instructions page link should be: 
https://james.apache.org/server/3/dev-build.html



On 8/31/2019 4:52 PM, Jerry Malcolm wrote:
I'm trying to build the latest stable James 3.3.0 in Eclipse.

-- The link to the 3.3.0 source zip on the downloads page is dead 
(https://www.apache.org/dyn/closer.lua/james/server/3.3.0/james-server-sources-3.3.0.zip)

-- I figured out the correct link: 
http://mirrors.gigenet.com/apache/james/server/3.3.0/james-project-3.3.0-src.zip
 and downloaded the zip.  But the zip contains ONLY the primary source files 
and no pom.xml, etc files.

-- I finally went to git and downloaded a zip that contained everything.

-- But the fun just kept happening... I tried to import the project into Eclipse, but got 
over 500 errors.   A large chunk of these errors was that every pom file has an invalid 
or missing a "dotGit" folder name.

-- Also, I tried following the instructions here: 
http://mirrors.gigenet.com/apache/james/server/3.3.0/james-project-3.3.0-src.zip -- In 
the section titled "prepare projects" it says to copy files from: 
server/container/spring/src/main/config/james and from 
server/container/src/main/config/james Apparently, the folder structures have completely 
changed since that was written. Neither of those folder exist, and it wasn't obvious 
where they moved to.

-- I know enough about eclipse to get by.  I know nothing about maven (and for 
now would like to keep it as close to that way as possible).  I just want to 
add a few tweaks to James 3.3 that I added back in v3b5 and get the latest 
james into production. I'm sure there's something missing or some setting wrong 
in Eclipse or in the Maven plug-in.  I did a complete clean install of Eclipse 
(Java EE version) and Maven on a different machine just to m

Re: "Getting Started" with JAMES using Docker

[Jerry Malcolm]

Hi Tellier,

Thanks for the response.  You're suggestion is a perfect solution.  Just 
add the option to the quick start: if you know docker, do this way.  If 
you want do it the traditional way, here's the steps.


Thanks.

Jerry

On 8/30/2019 12:04 AM, Tellier Benoit wrote:

Hi Jerry,

Yes thanks to your report, the image had been updated. Thanks!


Just an unsolicited opinion for the James folks...

there is an implied assumption that the user understands docker and

has it installed.  Nothing against docker. But it has not yet attained
ubiquity.

Well, thanks for the feedback.

IMO you see it the wrong way.

IF you have/are-familiar-with docker you can get a one
line try - which is awesome.

If you do not have it, you can follow one of the many non-docker tutorials.

CF: http://james.apache.org/server/install.html

Regarding "windows support" we are actively looking for some
contributors... Most of us don't have a license...

I'm not going to answer docker sysadmin related issues here.

Maybe the key output would be to enrich the **Get started with James**
section in order to also link "non-docker install options".

Would you agree with such a proposal?

Thanks again for your feedback, it's really helpful.

Best regards,

Benoit

On 30/08/2019 11:40, Jerry Malcolm wrote:

Hi, Aleksanda,

I really appreciate your information.  I believe that someone realized
the image wasn't there and quietly got it posted overnight.  When I
reran the command it worked, at least somewhat.

Just an unsolicited opinion for the James folks... I'm not sure that one
line command as the first thing a new user will see is quite ready for
prime-time.   As I mentioned in my first post yesterday, there is an
implied assumption that the user understands docker and has it
installed.  Nothing against docker. But it has not yet attained
ubiquity.  My guess is a lot of new James users will have no idea what
it is or where to get it, either for Linux or Windows. And if that's the
case, there's no alternative pointer on that home page for non-docker
install. Next, the docker install failed for me because port 25 was
already in use since aws ec2 Linux automatically installs sendmail and
starts it up.  Maybe all new James users will implicitly understand what
that error means and how to fix it.  Finally, docker got to the point of
creating the dummy user ids and locked up.  Had to kill my puTTY
session.  No idea how to try to start/stop the james service and see how
much, if any of it, is installed and runable.

I'm not trying to be negative.  That one line install is great if it
really works for the majority.  But if a user (even a veteran of JAMES
for ~15 years like me) has the problems I've encountered, I would
strongly suggest a detailed followup page to go to when the one line
install crashes and burns.  If I wasn't a seasoned user of James, and
rather a first-timer with this supposedly 'super easy' experience to
get it up and running, I might decide to go another direction for my MTA
needs.  I think we can head that off with a pointer to more info and
what to do if things don't go right on that one-line install.

Thx

Jerry


On 8/29/2019 2:53 AM, Aleksandar Stoisavljevic wrote:

Hi Jerry,

I'll try to be brief and not to come into many details of how Docker is
working.
For that, I suggest that you go through some video materials and/or to
find
some
really extensive documentation on Internet (blogs, posts, Docker
documentation, books, etc.)

In the meantime, I'll try to explain what happened in your case.

Obviously you've installed Docker correctly as we can see from command
response.

There are actually two concepts in Docker that you have to distinguish:
Docker Image and
Docker Container.
Docker Container is "running version" of Docker Image.

When you stated:

[ec2-user@ip-172-31-47-236 ~]$ docker run -p "25:25" -p "143:143"
linagora/james-jpa-sample:3.3.0

It actually means that it will try to run Docker Container, listening on
host ports 25 and 143, and Container should
be of a Docker Image "linagora/james-jpa-sample:3.3.0".

That image, "linagora/james-jpa-sample:3.3.0", should be availabe either
locally or placed somewhere on Internet, so
Docker can access it and grab it. For this kind of Docker Image store,
usually people use Docker Hub (https://hub.docker.com/),
but there are also other Docker Repositories (private or public).

As you don't have it locally, that Docker Image
("linagora/james-jpa-sample:3.3.0"), therefore you've got that
information
in reponse:
"Unable to find image 'linagora/james-jpa-sample:3.3.0' locally".

At that moment, Docker Engine will try to grab that image from Docker
Hub.

But unfortunately, I don't see that 3.3.0 version of that image exists.
You can go to URL
(https://hub.docker.com/r/linagora/james-jpa-sample/tags)
for yourself and make sure that it doesn't exists.

I don't know how did you get

Building James 3.3.0 in Eclipse

[Jerry Malcolm]

I'm trying to build the latest stable James 3.3.0 in Eclipse.

-- The link to the 3.3.0 source zip on the downloads page is dead 
(https://www.apache.org/dyn/closer.lua/james/server/3.3.0/james-server-sources-3.3.0.zip)


-- I figured out the correct link: 
http://mirrors.gigenet.com/apache/james/server/3.3.0/james-project-3.3.0-src.zip 
and downloaded the zip.  But the zip contains ONLY the primary source 
files and no pom.xml, etc files.


-- I finally went to git and downloaded a zip that contained everything.

-- But the fun just kept happening... I tried to import the project into 
Eclipse, but got over 500 errors.   A large chunk of these errors was 
that every pom file has an invalid or missing a "dotGit" folder name.


-- Also, I tried following the instructions here: 
http://mirrors.gigenet.com/apache/james/server/3.3.0/james-project-3.3.0-src.zip 
-- In the section titled "prepare projects" it says to copy files from: 
server/container/spring/src/main/config/james and from 
server/container/src/main/config/james Apparently, the folder structures 
have completely changed since that was written.  Neither of those folder 
exist, and it wasn't obvious where they moved to.


-- I know enough about eclipse to get by.  I know nothing about maven 
(and for now would like to keep it as close to that way as possible).  I 
just want to add a few tweaks to James 3.3 that I added back in v3b5 and 
get the latest james into production.  I'm sure there's something 
missing or some setting wrong in Eclipse or in the Maven plug-in.  I did 
a complete clean install of Eclipse (Java EE version) and Maven on a 
different machine just to make sure there were no issues with my 
original Eclipse.  No change.


Any help on resolving the hundreds of import errors and where to find 
all of these files that have moved since the instructions were written 
will be appreciated.   I'm at a total loss.  I really just need to get 
this thing to build.


Thanks.

Jerry

Re: Building James 3.3.0 in Eclipse

[Jerry Malcolm]

Correction... the build instructions page link should be: 
https://james.apache.org/server/3/dev-build.html



On 8/31/2019 4:52 PM, Jerry Malcolm wrote:

I'm trying to build the latest stable James 3.3.0 in Eclipse.

-- The link to the 3.3.0 source zip on the downloads page is dead 
(https://www.apache.org/dyn/closer.lua/james/server/3.3.0/james-server-sources-3.3.0.zip) 



-- I figured out the correct link: 
http://mirrors.gigenet.com/apache/james/server/3.3.0/james-project-3.3.0-src.zip 
and downloaded the zip.  But the zip contains ONLY the primary source 
files and no pom.xml, etc files.


-- I finally went to git and downloaded a zip that contained everything.

-- But the fun just kept happening... I tried to import the project 
into Eclipse, but got over 500 errors.   A large chunk of these errors 
was that every pom file has an invalid or missing a "dotGit" folder name.


-- Also, I tried following the instructions here: 
http://mirrors.gigenet.com/apache/james/server/3.3.0/james-project-3.3.0-src.zip 
-- In the section titled "prepare projects" it says to copy files 
from: server/container/spring/src/main/config/james and from 
server/container/src/main/config/james Apparently, the folder 
structures have completely changed since that was written. Neither of 
those folder exist, and it wasn't obvious where they moved to.


-- I know enough about eclipse to get by.  I know nothing about maven 
(and for now would like to keep it as close to that way as possible).  
I just want to add a few tweaks to James 3.3 that I added back in v3b5 
and get the latest james into production.  I'm sure there's something 
missing or some setting wrong in Eclipse or in the Maven plug-in.  I 
did a complete clean install of Eclipse (Java EE version) and Maven on 
a different machine just to make sure there were no issues with my 
original Eclipse.  No change.


Any help on resolving the hundreds of import errors and where to find 
all of these files that have moved since the instructions were written 
will be appreciated.   I'm at a total loss.  I really just need to get 
this thing to build.


Thanks.

Jerry




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Building James 3.3.0 in Eclipse

[Jerry Malcolm]

Something else I just found... the link to a tutorial at 
http://blog.bonnydoonmedia.com/post.cfm/walkthrough-tutorials-compiling-apache-james-v3-with-eclipse 
in the Eclipse build section is dead.  No DNS entry.



On 8/31/2019 4:54 PM, Jerry Malcolm wrote:
Correction... the build instructions page link should be: 
https://james.apache.org/server/3/dev-build.html



On 8/31/2019 4:52 PM, Jerry Malcolm wrote:

I'm trying to build the latest stable James 3.3.0 in Eclipse.

-- The link to the 3.3.0 source zip on the downloads page is dead 
(https://www.apache.org/dyn/closer.lua/james/server/3.3.0/james-server-sources-3.3.0.zip) 



-- I figured out the correct link: 
http://mirrors.gigenet.com/apache/james/server/3.3.0/james-project-3.3.0-src.zip 
and downloaded the zip.  But the zip contains ONLY the primary source 
files and no pom.xml, etc files.


-- I finally went to git and downloaded a zip that contained everything.

-- But the fun just kept happening... I tried to import the project 
into Eclipse, but got over 500 errors.   A large chunk of these 
errors was that every pom file has an invalid or missing a "dotGit" 
folder name.


-- Also, I tried following the instructions here: 
http://mirrors.gigenet.com/apache/james/server/3.3.0/james-project-3.3.0-src.zip 
-- In the section titled "prepare projects" it says to copy files 
from: server/container/spring/src/main/config/james and from 
server/container/src/main/config/james Apparently, the folder 
structures have completely changed since that was written. Neither of 
those folder exist, and it wasn't obvious where they moved to.


-- I know enough about eclipse to get by.  I know nothing about maven 
(and for now would like to keep it as close to that way as 
possible).  I just want to add a few tweaks to James 3.3 that I added 
back in v3b5 and get the latest james into production. I'm sure 
there's something missing or some setting wrong in Eclipse or in the 
Maven plug-in.  I did a complete clean install of Eclipse (Java EE 
version) and Maven on a different machine just to make sure there 
were no issues with my original Eclipse.  No change.


Any help on resolving the hundreds of import errors and where to find 
all of these files that have moved since the instructions were 
written will be appreciated.   I'm at a total loss.  I really just 
need to get this thing to build.


Thanks.

Jerry




-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Building James 3.3.0 in Eclipse

[Jerry Malcolm]

On 9/2/2019 2:55 AM, Raphael OUAZANA wrote:

Hi Jerry,

Don't worry, Eclipse is still supported as a development environment 
and currently used by some developers (even if we have some issues in 
incoming 3.4 regarding compatibility between Eclipse and some Scala 
code).


The usual workflow to work in a dev environment is the following:
- clone the git repository
- checkout the branch / tag you want to work on
- import the project in Eclipse

This workflow should work perfectly well.

The issue you are encountering is because as you are taking the 
sources directly from the released version, you don't have any .git 
repository as you would have if you had cloned the project. This is 
not a big harm, but it seems to make failing the git-commit-id-plugin.
So if you remove this plugin from the main pom.xml, I guess your 
problem would be solved.


Regards,
Raphaël.

Le 2019-09-01 06:16, Jerry Malcolm a écrit :

Thanks for the info, Garry.  But I'm still confused.  Are you saying
that it is no longer possible to build from the source on Eclipse?  I
realize the team may be pushing docker.  But there are still
non-docker binaries that got built someway.  I just want to recreate
those binaries with my code additions.  But first I want to just build
with the official source untouched and verify that works.  Then I'll
add my changes and rebuild.

You said the .git files are from the .git repository.  I don't doubt
that.  But I'm not a git guru.  I'm still not sure how to fix the
errors I'm getting (~400+ of these):

org.apache.maven.plugin.MojoExecutionException: .git directory is not
found! Please specify a valid [dotGitDirectory] in your pom.xml

I do not know anything about pom syntax.  I have no clue how/where to
"specify a valid [dotGitDirectory] in your pom.xml" (times 400)
Does this error mean that there isn't a gitDirectory defined in the
pom.xml and there should be? Or does it mean that one is defined
somewhere and the specified directory doesn't exist on my file
system?  I guess it'd be too much to ask for this error message to
include the directory it found and which of the 400 pom.xml files the
error refers to. (Sorry... I'm just frustrated at less-than-useful
error messages).

Did I download an invalid source zip?  Is there a different zip (or
additional zips that contain the correct pom files? ) I simply went to
github and downloaded the latest source zip file.

Where do I start?  I really don't want to have to learn how to build
on Linux unless there's an official word that says Eclipse is no
longer supported as a build platform.  I need to use Eclipse to debug
my code.  I just want to download the correct source, import it into
Eclipse, and build it.  I just can't understand why I can't get
anywhere near a successful build.  What am I doing wrong?

Jerry

On 8/31/2019 5:25 PM, Garry Hurley wrote:
Jerry, I had to download the source, run configure and make on a 
Linux system. The problem is that the source is made to build onto a 
docker system on Windows. Those of us who choose not to run docker 
and run a real operating system often have issues. Those .git files 
are from the .git repository. If you did a git clone, create your 
workspace from the local git repo or import existing maven projects 
into eclipse from the repo. This serves two purposes. 1, it allows 
you to keep your project up to date and 2, it allows you to commit 
if you have any changes to the existing code you want to push up for 
others.


I haven’t built anything after 3.3.0 was finalizedbecause the 
government finally approved 3.3.0 for use (ironically, they had been 
insisting that 3.0.4beta was safe to use in production and I 
outright refused to put that in place).


Sent from my iPhone

On Aug 31, 2019, at 6:02 PM, Jerry Malcolm  
wrote:


Something else I just found... the link to a tutorial at 
http://blog.bonnydoonmedia.com/post.cfm/walkthrough-tutorials-compiling-apache-james-v3-with-eclipse 
in the Eclipse build section is dead.  No DNS entry.




On 8/31/2019 4:54 PM, Jerry Malcolm wrote:
Correction... the build instructions page link should be: 
https://james.apache.org/server/3/dev-build.html




On 8/31/2019 4:52 PM, Jerry Malcolm wrote:
I'm trying to build the latest stable James 3.3.0 in Eclipse.

-- The link to the 3.3.0 source zip on the downloads page is dead 
(https://www.apache.org/dyn/closer.lua/james/server/3.3.0/james-server-sources-3.3.0.zip)


-- I figured out the correct link: 
http://mirrors.gigenet.com/apache/james/server/3.3.0/james-project-3.3.0-src.zip 
and downloaded the zip.  But the zip contains ONLY the primary 
source files and no pom.xml, etc files.


-- I finally went to git and downloaded a zip that contained 
everything.


-- But the fun just kept happening... I tried to import the 
project into Eclipse, but got over 500 errors.   A large chunk of 
these errors was that every pom file has an invalid or missing a 
"dotGit" folder name.


-- Als

"Getting Started" with JAMES using Docker

[Jerry Malcolm]

I was preparing to do a trial run at getting James up and running on an 
AWS EC2 instance.  I went to the James home page and saw there is a 
1-line command to download and install it.  However, to use that one 
line there is apparently an assumption of a full understanding of Docker 
and a pre-existing installation of Docker.  Docker isn't automatically 
installed with Amazon Linux in an EC2.  I was able to install it with 
yum.  But when I entered the command from the JAMES getting started 
page, this is what I got:


[ec2-user@ip-172-31-47-236 ~]$ docker run -p "25:25" -p "143:143" 
linagora/james-jpa-sample:3.3.0

Unable to find image 'linagora/james-jpa-sample:3.3.0' locally
docker: Error response from daemon: manifest for 
linagora/james-jpa-sample:3.3.0 not found.

See 'docker run --help'.

Since there's no url to download james, I assume that the manifest that 
it is saying that it can't find contains all of that useful info.  I 
know nothing about how Docker works.  Where would I find the missing 
manifest, and if it was expected to automatically come with Docker as 
implied with the "1-line-download-install" statement, what did I do wrong?


Thx

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: Building James 3.3.0 in Eclipse

[Jerry Malcolm]

Any help on this would be appreciated.  I'm dead in the water now on this.

Thx.

Jerry

On 9/2/2019 10:40 PM, Jerry Malcolm wrote:

Raphael,

Thanks for the guidance.  I'm closer now.  But still not there. After 
a full day of trial and error on this, I settled on the following steps:


1) On a clean eclipse workspace, File-->Import-->Git --- I selected 
"clone URI" and entered the James GIT clone URI: 
https://github.com/apache/james-project.git


2) I selected the 3.3.x branch (I guess that's what I want... (??) ) 
and said to also clone submodules.  It downloaded to a folder. It 
created an eclipse project/package named james-server. But the project 
only had 3 files: model.di, model.notation, and model.uml.


3) On a whim, I decided to import again using "Maven existing project" 
and pointed to the extracted folder.  That seemed to work for the most 
part.


It initially gave me 10 errors saying "Plugin execution not covered by 
lifecycle configuration" and 16 more errors with "Project 
configuration not up to date with pom.xml".  All with varying pom.xml 
files.  I did a "quick fix" and it got rid of the 'not up to date' 
errors.  I googled the lifecycle error and it said to edit all of the 
pom.xml files and add  tags around the plugin tags.  
I tried in on a couple of the pom.xml files, and it did fix the errors 
(???).


I then did a build, and it added one more error, something about an 
invalid mojo tag in a pom descriptor.


Just for fun, I did a clean build, and that was a huge mistake. I now 
have 340 errors where it can't resolve DTOs in FlagsTest, 
EventSerializer in EventDispatcher, etc.  Apparently 'clean' wipes out 
source files as well.  Or it wipes out class files that are not part 
of the full build.


So that's where I am.  I can edit a bunch of pom.xml files and add 
tags.  I can hopefully figure out how to fix mojo.  And I can hope to 
remember to NEVER do a clean build again.  But this just doesn't seem 
right that I have to go in and modify a bunch of source files just to 
get a clean build from a ship-level source package.  I have zero 
confidence if I have to make all of these change that I really have an 
'official' 3.3.x James.


Suggestions?  Where did I stray from the correct path?  I'm beginning 
to feel like I belong in the class with the slow kids


On 9/2/2019 6:04 PM, Jerry Malcolm wrote:


On 9/2/2019 2:55 AM, Raphael OUAZANA wrote:

Hi Jerry,

Don't worry, Eclipse is still supported as a development environment 
and currently used by some developers (even if we have some issues 
in incoming 3.4 regarding compatibility between Eclipse and some 
Scala code).


The usual workflow to work in a dev environment is the following:
- clone the git repository
- checkout the branch / tag you want to work on
- import the project in Eclipse

This workflow should work perfectly well.

The issue you are encountering is because as you are taking the 
sources directly from the released version, you don't have any .git 
repository as you would have if you had cloned the project. This is 
not a big harm, but it seems to make failing the git-commit-id-plugin.
So if you remove this plugin from the main pom.xml, I guess your 
problem would be solved.


Regards,
Raphaël.

Le 2019-09-01 06:16, Jerry Malcolm a écrit :

Thanks for the info, Garry.  But I'm still confused.  Are you saying
that it is no longer possible to build from the source on Eclipse?  I
realize the team may be pushing docker.  But there are still
non-docker binaries that got built someway.  I just want to recreate
those binaries with my code additions.  But first I want to just build
with the official source untouched and verify that works. Then I'll
add my changes and rebuild.

You said the .git files are from the .git repository.  I don't doubt
that.  But I'm not a git guru.  I'm still not sure how to fix the
errors I'm getting (~400+ of these):

org.apache.maven.plugin.MojoExecutionException: .git directory is not
found! Please specify a valid [dotGitDirectory] in your pom.xml

I do not know anything about pom syntax.  I have no clue how/where to
"specify a valid [dotGitDirectory] in your pom.xml" (times 400)
Does this error mean that there isn't a gitDirectory defined in the
pom.xml and there should be? Or does it mean that one is defined
somewhere and the specified directory doesn't exist on my file
system?  I guess it'd be too much to ask for this error message to
include the directory it found and which of the 400 pom.xml files the
error refers to. (Sorry... I'm just frustrated at less-than-useful
error messages).

Did I download an invalid source zip?  Is there a different zip (or
additional zips that contain the correct pom files? ) I simply went to
github and downloaded the latest source zip file.

Where do I start?  I really don't want to have to learn how to build
on Linux unless there's an official word that says Eclipse is no
longer

Re: AdministratorId Documentation

[Jerry Malcolm]

Tellier,

Sorry, I'm still missing something.  I downloaded the commons-net 
package and looked at the IMAPMail sample code.  It says to put the 
id/pw on the url: "Usage: IMAPMail 
imap[s]://username:password@server/");  I assume I would put my admin 
id:pw on the url.  But then where do I specify the userid for the actual 
inbox I want to access?


On 9/12/2019 11:01 PM, Tellier Benoit wrote:

If you are using commons-net, I succeeded to write:

https://gist.github.com/chibenwa/abd12fd6c0b06cadd1de591e3ac792b9

That should be helping you!

On 13/09/2019 10:40, Tellier Benoit wrote:

Wich library are you using?

On 13/09/2019 10:33, Jerry Malcolm wrote:

Thanks for the info, Tellier.  I kinda lost you on the mpt tests... I
was looking for how to change the following code to include an
administrator id.  I only have one field in the store.connect() method
for a user id, but I have an administrator id and the userid for the
target mailbox.  How do I pass in both?

    try
    {
   session = Session.getInstance(props, null);
   session.setDebug(debug);
   store = session.getStore(protocol);
   store.connect(host, port, user, password);
    }
    catch( Exception e)
    {
   log( "IMAPClient.connect() exception: " + e );
   throw(e);
    }

On 9/12/2019 10:10 PM, Tellier Benoit wrote:

Hello Jerry,

With the `administratorId`, you are able to use IMAP impersonation.

IE to log in as another user.

You should define it within usersrepository.xml. To see related "reading
config" code: AbstractUsersRepository is the way to go.

Agree that this needs example and documentation. I will do it straight
away.

For your IMAP call, MPT tests are going to be a gold mine, presenting
you the exact syntax.
mpt/impl/imap-mailbox/core/src/main/resources/org/apache/james/imap/scripts/AuthenticatePlain.test


Contains:

C: 0007 AUTHENTICATE "PLAIN" {36+}
# delegate\0imapuser\0password
C: ZGVsZWdhdGUAaW1hcHVzZXIAcGFzc3dvcmQ=
S: 0007 OK AUTHENTICATE completed.

# Ensure we are delegate
C: 0008 SELECT delegate
SUB {
S: \* FLAGS .*
S: \* .* EXISTS
S: \* .* RECENT
S: \* OK \[UIDVALIDITY .*\] UIDs valid
S: \* OK \[PERMANENTFLAGS .*\] Limited
S: \* OK \[HIGHESTMODSEQ .*\] Highest
S: \* OK \[UIDNEXT .*\] Predicted next UID
}
S: 0008 OK \[READ-WRITE\] SELECT completed\.

Regarding your development issues, I'm really sorry. My environment is
based on IntelliJ, and I encounter no issue.

To be able to compile, I use maven 3.6.2 with JDK 11.

Best regards,

Benoit Tellier


On 13/09/2019 09:53, Jerry Malcolm wrote:

Since it appears I'm not going to be able to build James 3.3.0 in the
foreseeable future, I'm now moving to plan d, e, f, or whatever... .I've
lost count.

My goal now is to assess my possibilities of using James 3.3.0 binaries
as-is and discarding the functionality I had hoped to re-add to 3.3.0
from 30b5 or moving the functionality to mailets that I can compile
outside of doing a full James build.

One thing I had added to beta5 was the ability to have a super-user,
master-key id that could be used in my custom external imap mailbox
maintenance utilities.  I've spent several hours digging through the
3.3.0 source code, and I see code referencing "administratorId".  From
what I can deduce from the code, the Authenticator and Authorizator
classes seem to accept this administrator id as the equivalent of the
master key I added to b5.  So can I access UserABC's mailbox through
imap but use my administratorId and password?  If that's the case, then
that's exactly what I need.  The problem is I can't find any
documentation on how to use administratorId.  Where do I define it?  And
how do I pass it on imap calls using the javax.mail.Store and other
classes in javax.mail that I use to access imap?

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: serve

Re: James Database Import

[Jerry Malcolm]

Thanks so much for all of the advice.  I think I've got a resolution.

1) First misassumption... mySQL makes no attempt to order tables based 
on foreign key dependency.  I just assumed the tables would be ordered 
correctly (bad habit assuming).  But due to possible cyclic 
dependencies, mySQL just says "you're on your own".


2) This 'should' be no problem if I could force foreign-key checking off 
during import.  But apparently it's turned back on somewhere in the dump 
file, and I don't want to try to find/buy an editor that I can trust to 
successfully edit an 80GB+ dump file. So this one will just have to 
remain a mystery.


3) The idea of just removing the foreign key constraint before export 
would be a solution.  However, I'm just trying to get a sandbox database 
imported now so I can test the new installation. The old db is still 
live.  And I'm a bit concerned about 
removing/re-adding/removing/re-adding that constraint from the live 
production database and/or taking that database completely offline to 
export the sandbox and then again for the final transfer export.


4) I realize now that the reason the import worked on my Windows machine 
is that I was reimporting to an existing database.  The import deletes a 
table and rebuilds it.  But it deletes tables one at a time.  So when it 
was rebuilding the mail table, the old version of the dependent mailbox 
table still existed.  So the keys were found.  In a totally clean import 
the mailbox table does not exist.  Hence the key errors.  So I exported 
a separate file containing ONLY the mailbox table and imported it into 
the new database.  That finally made the main import happy.  I didn't 
modify the main dump file even though I no longer needed it to import 
the mailbox table.  I just let it delete the mailbox table and rebuild 
it again.


So... on to testing the AWS installation.  I'll be back

Jerry

On 9/16/2019 7:34 AM, Garry Hurley wrote:

Jerry

It looks like the export did not export the data in the proper order. You are 
trying to put mail into the mail table that is assigned to mailboxes that do 
not exist yet. Three options:
1, create ‘dummy mailboxes’ for those messages to fall into. This may be the 
faster workaround for the import to work.
2, take your export script apart and import the mailboxes before the messages. 
This has a problem if, for example, you had deleted mailboxes in your old 
database and the mailbox id does not match the previous one. That means the 
mail messages will be inappropriately assigned.  You may have to go theough and 
reassign the mailbox ids in the mail table.
3, Remove the delete cascade constraint until you have the data imported, then 
put it back. I assume your AWS instance is not live in production yet, so this 
may or may not be possible.

Sent from my iPhone


On Sep 15, 2019, at 10:07 PM, Tellier Benoit  wrote:

Hi Jerry,

James is relying on an ORM (openjpa) for database persistance.

We do test things on top of a H2 database, and (a ltle bit) on top
of PostgresSQL & derby database.

SQL database compatibility is a tough topic, we had some issues in the
past with some reserved name, too long table name etc... on top of
Microsoft SQL server.

Not too mention that ORM is also a tricky piece of software:

- OpenJPA do not mention Apache RDS in their examples

Do you succeed to run a fresh James install on top of RDS?

Best regards,

Benoit Tellier


On 15/09/2019 10:46, Jerry Malcolm wrote:
This is a bit off-topic... more of a database question... but it IS
about the James database.  Hopefully somebody understands the James
schema details enough to recognize the errors below and explain why the
import is failing.  I posted this in the AWS RDS forum.  But no help
there so far I may have to write a best-selling novel about my saga
of getting James migrated to Amazon... if I ever actually succeed... :-(

I have an 80GB mySQLDump file containing an Apache JAMES mail database.
I am currently on a Windows server (mySQL 5.7.21). The dump file was
generated from that server. Just to test the file, I round-tripped it
back into the same mysql server with no errors. But I've been trying for
a week to get it loaded into an Amazon RDS. I've tried both mySQL and
aurora instances. I've tried small instances and large instances. I get
the same errors. Since the errors relate to foreign keys, I tried
turning off foreign key checking, and STILL got the errors. I'm
completely stuck. The fact that it reads successfully into a windows
mysql 5.7.21 and won't read into an RDS mysql 5.7.22 completely baffles
me. Here are a few lines from the output. It repeats this way until I
stop it:

Thx
Jerry

Log:
ERROR 1452 (23000): Cannot add or update a child row: a foreign key
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1`
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON
DELETE CASCADE)
ERROR 2006 (HY000): MySQL server has gone away
No conn

James Database Import

[Jerry Malcolm]

This is a bit off-topic... more of a database question... but it IS 
about the James database.  Hopefully somebody understands the James 
schema details enough to recognize the errors below and explain why the 
import is failing.  I posted this in the AWS RDS forum.  But no help 
there so far I may have to write a best-selling novel about my saga 
of getting James migrated to Amazon... if I ever actually succeed... :-(


I have an 80GB mySQLDump file containing an Apache JAMES mail database. 
I am currently on a Windows server (mySQL 5.7.21). The dump file was 
generated from that server. Just to test the file, I round-tripped it 
back into the same mysql server with no errors. But I've been trying for 
a week to get it loaded into an Amazon RDS. I've tried both mySQL and 
aurora instances. I've tried small instances and large instances. I get 
the same errors. Since the errors relate to foreign keys, I tried 
turning off foreign key checking, and STILL got the errors. I'm 
completely stuck. The fact that it reads successfully into a windows 
mysql 5.7.21 and won't read into an RDS mysql 5.7.22 completely baffles 
me. Here are a few lines from the output. It repeats this way until I 
stop it:


Thx
Jerry

Log:
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)

ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 628
Current database: mail3x

ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 629
Current database: mail3x

ERROR 2006 (HY000): MySQL server has gone away
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 630
Current database: mail3x

ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)
ERROR 1452 (23000): Cannot add or update a child row: a foreign key 
constraint fails (`mail3x`.`james_mail`, CONSTRAINT `james_mail_ibfk_1` 
FOREIGN KEY (`MAILBOX_ID`) REFERENCES `james_mailbox` (`MAILBOX_ID`) ON 
DELETE CASCADE)

two datasources?

[Jerry Malcolm]

A couple of my mailets need to access a different database other than 
the base james db.  I've got the main db set up in the 
james-database.properties file.  Currently I'm just hardcoding the 
database connection to the other db in the mailet.  Just curious if 
there's a better architected process where I can define two datasources 
in james-database.properties and reference the alternate datasource when 
needed.  Couldn't see anything as to how to do that.   Not a big deal.  
Just trying to clean up a few things as I migrate to the new version.



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: IMAP Error

[Jerry Malcolm]

Something that might be relevant to this problem, but the messages make 
no more sense than the other one... A little earlier in the log, I have 
a bunch of these messages:


174  James  INFO   [main] openjpa.Runtime - Starting OpenJPA 3.0.0
207  James  INFO   [main] openjpa.jdbc.JDBC - Using dictionary class 
"org.apache.openjpa.jdbc.sql.MySQLDictionary".
552  James  INFO   [main] openjpa.jdbc.JDBC - Connected to MySQL version 
5.5 using JDBC driver MySQL Connector Java version 
mysql-connector-java-5.1.34 ( Revision: 
jess.bal...@oracle.com-20141014163213-wqbwpf1ok2kvo1om ).
586  James  WARN   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded 
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
587  James  WARN   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage" 
listed in the openjpa.MetaDataFactory configuration property could not 
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
688  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded 
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
689  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage" 
listed in the openjpa.MetaDataFactory configuration property could not 
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
702  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded 
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.


On 9/16/2019 10:28 PM, Jerry Malcolm wrote:
The fun just keeps happening I was finally able to bring up the 
new installation of James 3.3.0 with a sandbox copy of my real 
database.  I created a new test email account in James and configured 
it in my local Thunderbird instance.  So far, so good. But now every 
time Thunderbird contacts James for an IMAP update, I get the error 
below in the James log.  I'm not really familiar with jpa.  I searched 
through the source and found quite a few places that the 
"findMessagesInMailbox" name is referenced.  So not really sure why it 
says it isn't defined.  Any ideas?  This is with the downloaded 
binaries, not a re-built version.



org.apache.openjpa.persistence.ArgumentException:  There is no query
with the name "findMessagesInMailbox" defined for any of the known
persistent classes:
[org.apache.james.mailbox.jpa.mail.model.JPAMailbox,
org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage,
org.apache.james.mailbox.jpa.mail.model.JPAUserFlag,
org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage,
org.apache.james.domainlist.jpa.model.JPADomain,
org.apache.james.user.jpa.model.JPAUser,
org.apache.james.mailbox.jpa.mail.model.JPAProperty,
org.apache.james.mailbox.jpa.user.model.JPASubscription,
org.apache.james.rrt.jpa.model.JPARecipientRewrite].


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: James Database Import

[Jerry Malcolm]

Absolutely... will do.  If I survive :-)

On 9/16/2019 9:24 PM, Tellier Benoit wrote:

Nice,

Thanks for the feedback!

I believe writing a short blog-post that we could then share with the
community, once you get this up and running will be really valuable.

Regards,

Benoit

On 17/09/2019 00:27, Jerry Malcolm wrote:

Thanks so much for all of the advice.  I think I've got a resolution.

1) First misassumption... mySQL makes no attempt to order tables based
on foreign key dependency.  I just assumed the tables would be ordered
correctly (bad habit assuming).  But due to possible cyclic
dependencies, mySQL just says "you're on your own".

2) This 'should' be no problem if I could force foreign-key checking off
during import.  But apparently it's turned back on somewhere in the dump
file, and I don't want to try to find/buy an editor that I can trust to
successfully edit an 80GB+ dump file. So this one will just have to
remain a mystery.

3) The idea of just removing the foreign key constraint before export
would be a solution.  However, I'm just trying to get a sandbox database
imported now so I can test the new installation. The old db is still
live.  And I'm a bit concerned about
removing/re-adding/removing/re-adding that constraint from the live
production database and/or taking that database completely offline to
export the sandbox and then again for the final transfer export.

4) I realize now that the reason the import worked on my Windows machine
is that I was reimporting to an existing database.  The import deletes a
table and rebuilds it.  But it deletes tables one at a time.  So when it
was rebuilding the mail table, the old version of the dependent mailbox
table still existed.  So the keys were found.  In a totally clean import
the mailbox table does not exist.  Hence the key errors.  So I exported
a separate file containing ONLY the mailbox table and imported it into
the new database.  That finally made the main import happy.  I didn't
modify the main dump file even though I no longer needed it to import
the mailbox table.  I just let it delete the mailbox table and rebuild
it again.

So... on to testing the AWS installation.  I'll be back

Jerry

On 9/16/2019 7:34 AM, Garry Hurley wrote:

Jerry

It looks like the export did not export the data in the proper order.
You are trying to put mail into the mail table that is assigned to
mailboxes that do not exist yet. Three options:
1, create ‘dummy mailboxes’ for those messages to fall into. This may
be the faster workaround for the import to work.
2, take your export script apart and import the mailboxes before the
messages. This has a problem if, for example, you had deleted
mailboxes in your old database and the mailbox id does not match the
previous one. That means the mail messages will be inappropriately
assigned.  You may have to go theough and reassign the mailbox ids in
the mail table.
3, Remove the delete cascade constraint until you have the data
imported, then put it back. I assume your AWS instance is not live in
production yet, so this may or may not be possible.

Sent from my iPhone


On Sep 15, 2019, at 10:07 PM, Tellier Benoit 
wrote:

Hi Jerry,

James is relying on an ORM (openjpa) for database persistance.

We do test things on top of a H2 database, and (a ltle bit) on top
of PostgresSQL & derby database.

SQL database compatibility is a tough topic, we had some issues in the
past with some reserved name, too long table name etc... on top of
Microsoft SQL server.

Not too mention that ORM is also a tricky piece of software:

- OpenJPA do not mention Apache RDS in their examples

Do you succeed to run a fresh James install on top of RDS?

Best regards,

Benoit Tellier


On 15/09/2019 10:46, Jerry Malcolm wrote:
This is a bit off-topic... more of a database question... but it IS
about the James database.  Hopefully somebody understands the James
schema details enough to recognize the errors below and explain why the
import is failing.  I posted this in the AWS RDS forum.  But no help
there so far I may have to write a best-selling novel about my saga
of getting James migrated to Amazon... if I ever actually succeed...
:-(

I have an 80GB mySQLDump file containing an Apache JAMES mail database.
I am currently on a Windows server (mySQL 5.7.21). The dump file was
generated from that server. Just to test the file, I round-tripped it
back into the same mysql server with no errors. But I've been trying
for
a week to get it loaded into an Amazon RDS. I've tried both mySQL and
aurora instances. I've tried small instances and large instances. I get
the same errors. Since the errors relate to foreign keys, I tried
turning off foreign key checking, and STILL got the errors. I'm
completely stuck. The fact that it reads successfully into a windows
mysql 5.7.21 and won't read into an RDS mysql 5.7.22 completely baffles
me. Here are a few lines from the output. It repeats this way until I
stop it:

Thx
Jer

Re: two datasources?

[Jerry Malcolm]

No the main mailet in question is 100% custom.  It is a mailet that 
decides what folder to put the mail item in.  I access a completely 
separate client database and determine if the sender is a client vs. a 
vendor vs. an employee and store the mail in a different folder based on 
the result.  The client database is completely unrelated to james.  It's 
just a lookup.  Since this mailet is pure custom, I really don't care to 
do any more integration into James other than mailet interface.  My main 
goal was simply to be able to store the database name, id/pw, url, etc 
in a database properties file and get it out of the mailet code and/or 
the mailet flow xml files.   Again, not a big deal.


On 9/16/2019 9:49 PM, Tellier Benoit wrote:

Hi Jerry,

Are you speaking of JDBC mailets (whiteList for instance)

These mailets are currently deprecated (or will be) as they hard code
their database, are not standard, not tested and not documented. Note
that they are considered as 'experimental' ( cf
http://james.apache.org/server/dev-provided-mailets.html ).

I would discourage their use (unless you really need these features) and
call for contribution:

  - Write a generic storage API for them
  - Rewrite these mailets on top of this storage API
  - Provide JPA storage for these storage APIs
  - Bind stuff in Guice/Spring

A bit of work, but if you really need it, I would be pleased to provide
such guidance.

On 17/09/2019 09:39, Jerry Malcolm wrote:

A couple of my mailets need to access a different database other than
the base james db.  I've got the main db set up in the
james-database.properties file.  Currently I'm just hardcoding the
database connection to the other db in the mailet.  Just curious if
there's a better architected process where I can define two datasources
in james-database.properties and reference the alternate datasource when
needed.  Couldn't see anything as to how to do that.   Not a big deal.
Just trying to clean up a few things as I migrate to the new version.


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

IMAP Error

[Jerry Malcolm]

The fun just keeps happening I was finally able to bring up the new 
installation of James 3.3.0 with a sandbox copy of my real database.  I 
created a new test email account in James and configured it in my local 
Thunderbird instance.  So far, so good. But now every time Thunderbird 
contacts James for an IMAP update, I get the error below in the James 
log.  I'm not really familiar with jpa.  I searched through the source 
and found quite a few places that the "findMessagesInMailbox" name is 
referenced.  So not really sure why it says it isn't defined.  Any 
ideas?  This is with the downloaded binaries, not a re-built version.



org.apache.openjpa.persistence.ArgumentException:  There is no query
with the name "findMessagesInMailbox" defined for any of the known
persistent classes:
[org.apache.james.mailbox.jpa.mail.model.JPAMailbox,
org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage,
org.apache.james.mailbox.jpa.mail.model.JPAUserFlag,
org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage,
org.apache.james.domainlist.jpa.model.JPADomain,
org.apache.james.user.jpa.model.JPAUser,
org.apache.james.mailbox.jpa.mail.model.JPAProperty,
org.apache.james.mailbox.jpa.user.model.JPASubscription,
org.apache.james.rrt.jpa.model.JPARecipientRewrite].


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: 3.0.0 on Linux

[Jerry Malcolm]

Garry,

I'm all for taking the shortest route to the latest code.  Here is my 
full saga from a previous post:


1) On a clean eclipse workspace, File-->Import-->Git --- I selected 
"clone URI" and entered the James GIT clone URI: 
https://github.com/apache/james-project.git


2) I selected the 3.3.x branch (I guess that's what I want... (??) ) and 
said to also clone submodules.  It downloaded to a folder. It created an 
eclipse project/package named james-server. But the project only had 3 
files: model.di, model.notation, and model.uml.


3) On a whim, I decided to import again using "Maven existing project" 
and pointed to the extracted folder.  That seemed to work for the most 
part.


It initially gave me 10 errors saying "Plugin execution not covered by 
lifecycle configuration" and 16 more errors with "Project configuration 
not up to date with pom.xml".  All with varying pom.xml files.  I did a 
"quick fix" and it got rid of the 'not up to date' errors.  I googled 
the lifecycle error and it said to edit all of the pom.xml files and add 
 tags around the plugin tags.  I tried in on a couple 
of the pom.xml files, and it did fix the errors (???).


I then did a build, and it added one more error, something about an 
invalid mojo tag in a pom descriptor.


Just for fun, I did a clean build, and that was a huge mistake. I now 
have 340 errors where it can't resolve DTOs in FlagsTest, 
EventSerializer in EventDispatcher, etc.  Apparently 'clean' wipes out 
source files as well.  Or it wipes out class files that are not part of 
the full build.


So that's where I am.  I can edit a bunch of pom.xml files and add 
tags.  I can hopefully figure out how to fix mojo.  And I can hope to 
remember to NEVER do a clean build again.  But this just doesn't seem 
right that I have to go in and modify a bunch of source files just to 
get a clean build from a ship-level source package.  I have zero 
confidence if I have to make all of these change that I really have an 
'official' 3.3.x James.


Suggestions?  Where did I stray from the correct path?  I'm beginning to 
feel like I belong in the class with the slow kids


On 9/9/2019 11:13 AM, Garry Hurley wrote:

Let's go to your James 3.3.0 problem.  I ran into some issues with James
early on, and that is why I signed up on this list.  What is the issue with
3.3.0?  What is the error you are getting?

On Sun, Sep 8, 2019 at 2:20 PM Jerry Malcolm  wrote:


Hi Garry,

You missed my point.  I've been trying to get help on building 3.3.0 for
over a week with no response (see my long list of posts).  Since I can't
get anyone to respond to my 3.3.0 build problem, I had to look for an
alternative.  I currently have a 3.0.0 beta 5 install that has worked
for years on a Windows server.  My only alternative until I get my 3.3.0
build problems resolved is to move that v3b5 to Linux since my windows
server is going away.  I have added custom code.  So I can't just
download the binaries.  I would love to move to 3.3.0.  But until I can
figure out my build problems and/or get someone to respond to my
questions, this is my only choice.  Now my current ancient v3b5 that
worked on Windows fails on Linux with OpenJPA issues (see prev post).

I realize this is a completely volunteer forum.  But I'm dying here.
I've got to get some version of james running on my AWS Linux box.  Can
someone please find time to help?  I'll take help on my 3.0.0 b5 OpenJPA
issue and/or my 3.3.0 Eclipse build problems... just SOMETHING.

Thx

Jerry

On 9/8/2019 2:31 AM, Garry Hurley wrote:

3.0.0 is kinda old. In fact, IIRC, it never made it out of Beta stage.

3.3.0 works and is stable, so unless you have some reason to use an older
version, you should consider updating.

Sent from my iPhone


On Sep 8, 2019, at 12:07 AM, Jerry Malcolm 

wrote:

I'm still waiting and hoping somebody can please explain why I can't

build 3.3.0 on Eclipse.  (See my other thread).  But I'm on a deadline.  y
windows server is going away soon and I HAVE to get a mail solution onto
AWS.  (ANYBODY?? PLEASE HELP!)

So in meantime, plan-B is to copy my current ancient v3b5 installation

off of my Windows server to my AWS Linux instance. Since James is 100% java
(good ol' "write once -- run anywhere"..., right?) I figured I could just
copy the entire James installation folder and fix a couple of backslashes
in file paths in config files, and it should run perfectly.  Wrong-o.  I
literally copied the folder precisely, and changed to run.sh instead of
run.bat.  After a few expected startup messages, I get a bunch of OpenJPA
warnings, followed by a bunch of errors telling me foreign keys are missing
from my database, etc. (See log excerpt below).  I went back to the running
Windows instance. There is no reference anywhere in the Windows James log
to jpa. Since the conf and lib folders are identical (as far as I can tell)
between these two instances,

Re: 3.0.0 on Linux

[Jerry Malcolm]

Garry,

A shorter explanation of what's happening trying to build 3.3... I 
decided to start totally from scratch on a new machine.  I installed JDK 
1.8. Next I installed the latest enterprise developers version of 
Eclipse.  Next I used the regular import-->git project, clone uri from 
https://github.com/apache/james-project.git.  I deselected all branches 
except 3.3.x and downloaded.  Finally I then did 
'import-->maven-->existing maven project and pointed it to the git 
download folder.  During the import it displayed 201 errors: "Plugin 
execution not covered by lifecycle configuration", 300+ warnings saying 
something about 'progress and finally did a popup about not finding mojo 
in the tools.jar file.


This takes about 5 minutes to reproduce.  If these steps are wrong to 
download and build 3.3, that should be obvious to someone.  If these 
steps are right, can someone try the same process and see if they get 
the same errors?  If they do, then someone who knows something is going 
to have to explain it.  I think it's just you and me on this forum now.  
But hopefully, somebody from JAMES development will come back and 
finally explain what's happening.



On 9/9/2019 11:38 AM, Jerry Malcolm wrote:

Garry,

I'm all for taking the shortest route to the latest code.  Here is my 
full saga from a previous post:


1) On a clean eclipse workspace, File-->Import-->Git --- I selected 
"clone URI" and entered the James GIT clone URI: 
https://github.com/apache/james-project.git


2) I selected the 3.3.x branch (I guess that's what I want... (??) ) 
and said to also clone submodules.  It downloaded to a folder. It 
created an eclipse project/package named james-server. But the project 
only had 3 files: model.di, model.notation, and model.uml.


3) On a whim, I decided to import again using "Maven existing project" 
and pointed to the extracted folder.  That seemed to work for the most 
part.


It initially gave me 10 errors saying "Plugin execution not covered by 
lifecycle configuration" and 16 more errors with "Project 
configuration not up to date with pom.xml".  All with varying pom.xml 
files.  I did a "quick fix" and it got rid of the 'not up to date' 
errors.  I googled the lifecycle error and it said to edit all of the 
pom.xml files and add  tags around the plugin tags.  
I tried in on a couple of the pom.xml files, and it did fix the errors 
(???).


I then did a build, and it added one more error, something about an 
invalid mojo tag in a pom descriptor.


Just for fun, I did a clean build, and that was a huge mistake. I now 
have 340 errors where it can't resolve DTOs in FlagsTest, 
EventSerializer in EventDispatcher, etc.  Apparently 'clean' wipes out 
source files as well.  Or it wipes out class files that are not part 
of the full build.


So that's where I am.  I can edit a bunch of pom.xml files and add 
tags.  I can hopefully figure out how to fix mojo.  And I can hope to 
remember to NEVER do a clean build again.  But this just doesn't seem 
right that I have to go in and modify a bunch of source files just to 
get a clean build from a ship-level source package.  I have zero 
confidence if I have to make all of these change that I really have an 
'official' 3.3.x James.


Suggestions?  Where did I stray from the correct path?  I'm beginning 
to feel like I belong in the class with the slow kids


On 9/9/2019 11:13 AM, Garry Hurley wrote:

Let's go to your James 3.3.0 problem.  I ran into some issues with James
early on, and that is why I signed up on this list.  What is the 
issue with

3.3.0?  What is the error you are getting?

On Sun, Sep 8, 2019 at 2:20 PM Jerry Malcolm  
wrote:



Hi Garry,

You missed my point.  I've been trying to get help on building 3.3.0 
for

over a week with no response (see my long list of posts). Since I can't
get anyone to respond to my 3.3.0 build problem, I had to look for an
alternative.  I currently have a 3.0.0 beta 5 install that has worked
for years on a Windows server.  My only alternative until I get my 
3.3.0

build problems resolved is to move that v3b5 to Linux since my windows
server is going away.  I have added custom code.  So I can't just
download the binaries.  I would love to move to 3.3.0.  But until I can
figure out my build problems and/or get someone to respond to my
questions, this is my only choice.  Now my current ancient v3b5 that
worked on Windows fails on Linux with OpenJPA issues (see prev post).

I realize this is a completely volunteer forum.  But I'm dying here.
I've got to get some version of james running on my AWS Linux box.  Can
someone please find time to help?  I'll take help on my 3.0.0 b5 
OpenJPA

issue and/or my 3.3.0 Eclipse build problems... just SOMETHING.

Thx

Jerry

On 9/8/2019 2:31 AM, Garry Hurley wrote:

3.0.0 is kinda old. In fact, IIRC, it never made it out of Beta stage.
3.3.0 works and is stable, so unless you have some

3.0.0 on Linux

[Jerry Malcolm]

I'm still waiting and hoping somebody can please explain why I can't 
build 3.3.0 on Eclipse.  (See my other thread).  But I'm on a deadline.  
My windows server is going away soon and I HAVE to get a mail solution 
onto AWS.  (ANYBODY?? PLEASE HELP!)


So in meantime, plan-B is to copy my current ancient v3b5 installation 
off of my Windows server to my AWS Linux instance. Since James is 100% 
java (good ol' "write once -- run anywhere"..., right?) I figured I 
could just copy the entire James installation folder and fix a couple of 
backslashes in file paths in config files, and it should run perfectly.  
Wrong-o.  I literally copied the folder precisely, and changed to run.sh 
instead of run.bat.  After a few expected startup messages, I get a 
bunch of OpenJPA warnings, followed by a bunch of errors telling me 
foreign keys are missing from my database, etc. (See log excerpt 
below).  I went back to the running Windows instance. There is no 
reference anywhere in the Windows James log to jpa. Since the conf and 
lib folders are identical (as far as I can tell) between these two 
instances, I'm baffled.  Why is OpenJPA getting upset with the Linux 
install and not the Windows install. My first thought was something with 
case-sensitive file names. But I can't find any of those situations.  
I'm not familiar with OpenJPA configuration.  So these errors mean 
nothing to me.  What did I do wrong? (BTW... both instances point to the 
same db... so I know the database is not corrupt).


Can anybody help with this problem (or better... help me with the 3.3.0 
Eclipse disaster... I don't want to remain on v3b5 forever).


Thx.

Log entries:

INFO  02:35:01,219 | james.dnsservice | Adding autodiscovered search 
path ec2.internal.

INFO  02:35:01,220 | james.dnsservice | DNS Server is: 172.31.0.2
INFO  02:35:01,226 | james.dnsservice | Registered cache, resolver and 
search paths as DNSJava defaults

185  James  INFO   [main] openjpa.Runtime - Starting OpenJPA 2.2.1
215  James  INFO   [main] openjpa.jdbc.JDBC - Using dictionary class 
"org.apache.openjpa.jdbc.sql.MySQLDictionary".
630  James  WARN   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded 
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
631  James  WARN   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage" 
listed in the openjpa.MetaDataFactory configuration property could not 
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
723  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded 
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
723  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage" 
listed in the openjpa.MetaDataFactory configuration property could not 
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
735  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded 
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
735  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage" 
listed in the openjpa.MetaDataFactory configuration property could not 
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
740  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded 
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
740  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage" 
listed in the openjpa.MetaDataFactory configuration property could not 
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
746  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded 
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.

.

Caused by: org.springframework.beans.factory.BeanCreationException: 
Error creating bean with name 'domainlist': Initialization of bean 
failed; nested exception is 
org.springframework.beans.FatalBeanException: Unable to execute 
lifecycle method on beandomainlist; nested exception is 
 
org.apache.openjpa.persistence.PersistenceException: Table 
"JAMES_MAIL_PROPERTY" has a foreign key to table "james_mail" that has 
not been generated.  You must run the schema generator on all 
inter-related tables at once

log4j Configuration

[Jerry Malcolm]

For the problem du jour when I start JAMES, all of the expected log 
files get created.  The main log output  goes to the console and to the 
james-server.log file as expected.  No problem there.  But the remainder 
of the log files never get touched.  No matter what happens, every file 
in the log folder other than james-server.log is empty.  I've tried 
changing log4j.properties and setting everything to DEBUG level.  The 
console log dumps a bunch more info while in DEBUG mode (which confirms 
I'm changing the correct log4j file).  But every other log file is still 
empty.  I used to see my mailet output in the main console and 
james-server.log in b5.  I'm not seeing any of that anywhere. Again, 
I've played around with log4j.properties (I made sure I was using the 
latest 3.3.0 log4j.properties file).  Nothing.  What am I doing wrong?  
How can I get log data to start going into those other files?



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: IMAP Error

[Jerry Malcolm]

Well this one wasn't as difficult as I expected.  Turns out during all 
of the transition and merging I ended up with the conf/META-INF folder 
from b5.  Since that folder contains all of the JPA stuff, 3.3.0 was not 
at all happy with v3b5 JPA files.


Just a note for continuing to document my adventures Once I swapped 
over the META-INF folder, things initially got much worse.  The JPA 
stuff is used to update all of the table schemas from 3.0 to 3.3 
version.  In the process of updating, it tries to add foreign keys to 
JAMES_MAIL_PROPERTY and JAMES_MAIL_USERFLAG tables.  Both of those 
failed miserably.  Digging into those database tables, I realized I had 
32 million (yup... 32 million) userFlag records with null MAIL_ID  and 
MAIL_UID fields that had no hope of matching up with fields in 
JAMES_MAIL.  And there were four records in JAMES_MAIL_PROPERTY that 
weren't null, but still didn't have a match in JAMES_MAIL.  So after 
finding and deleting those 4 as well as the 32 million null records, 
James was able to successfully complete the JPA startup stuff.  I guess 
this is a good thing that I'm forced to clean up problems with the 
current database.  This migration glacier is moving a millimeter at a 
time



On 9/17/2019 1:26 AM, Tellier Benoit wrote:

Hi Jerry,

To explain it shortly:

  1. In OpenJPA NamedQuery are directly attached to the POJO being
persited, here AbstractJPAMailboxMessage
  2. Using these annotations (named query) openjpa plugin enhance code
upon compilation
  3. Then the magic happens

Here because openjpa do not load correctly persisted objects, it then
fails at executing related queries.

Why OpenJPA fails loading related metadata? - the log is missing
explanation...

And of course...
  - What is the exact James version?
  - JDBC mysql driver version?
  - MySQL version?

This would enable us to be testing this locally, see if we can reproduce
it with basic setup...

Cheers,

Benoit

On 17/09/2019 10:36, Jerry Malcolm wrote:

Something that might be relevant to this problem, but the messages make
no more sense than the other one... A little earlier in the log, I have
a bunch of these messages:

174  James  INFO   [main] openjpa.Runtime - Starting OpenJPA 3.0.0
207  James  INFO   [main] openjpa.jdbc.JDBC - Using dictionary class
"org.apache.openjpa.jdbc.sql.MySQLDictionary".
552  James  INFO   [main] openjpa.jdbc.JDBC - Connected to MySQL version
5.5 using JDBC driver MySQL Connector Java version
mysql-connector-java-5.1.34 ( Revision:
jess.bal...@oracle.com-20141014163213-wqbwpf1ok2kvo1om ).
586  James  WARN   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in
the openjpa.MetaDataFactory configuration property could not be loaded
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
587  James  WARN   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage"
listed in the openjpa.MetaDataFactory configuration property could not
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
688  James  INFO   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in
the openjpa.MetaDataFactory configuration property could not be loaded
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
689  James  INFO   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage"
listed in the openjpa.MetaDataFactory configuration property could not
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
702  James  INFO   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in
the openjpa.MetaDataFactory configuration property could not be loaded
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.

On 9/16/2019 10:28 PM, Jerry Malcolm wrote:

The fun just keeps happening I was finally able to bring up the
new installation of James 3.3.0 with a sandbox copy of my real
database.  I created a new test email account in James and configured
it in my local Thunderbird instance.  So far, so good. But now every
time Thunderbird contacts James for an IMAP update, I get the error
below in the James log.  I'm not really familiar with jpa.  I searched
through the source and found quite a few places that the
"findMessagesInMailbox" name is referenced.  So not really sure why it
says it isn't defined.  Any ideas?  This is with the downloaded
binaries, not a re-built version.


org.apache.openjpa.persistence.ArgumentException:  There is no query
with the name "findMessagesInMailbox" defined for any of the known
persistent classes:
[org.apache.james.mailbox.jpa.mail.model.JPAMailbox,
org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage,
org.apache.james.mailbox.jpa.mail.model.JPAUserFlag,
org.apache.james.mailbox.jpa.mail.m

Re: log4j Configuration

[Jerry Malcolm]

I'll try that shortly.  But that doesn't explain why all of the logger 
output files are empty, no matter what the log4j settings are.  This was 
after 30 minutes of running with every log level throughout 
log4j.properties set to DEBUG.  Shouldn't at least some of these have 
some sort of log data, such as imapserver.log, smtpserver.log, etc?


 9/17/2019  21:33   0  dnsservice.log
 9/17/2019  21:33   0  domainlist.log
 9/17/2019  21:33   0  fetchmail.log
 9/17/2019  21:33   0  imapserver.log
 9/17/2019  21:33   0  lmtpserver.log
 9/17/2019  21:33   0  mailboxmanager.log
 9/17/2019  21:33   0  mailetcontainer.log
 9/17/2019  21:33   0  mailqueuefactory.log
 9/17/2019  21:33   0  mailrepositorystore.log
 9/17/2019  21:33   0  pop3server.log
 9/17/2019  21:33   0  usersrepository.log
 9/17/2019  21:33   0  virtualusertable.log

On 9/17/2019 10:49 PM, Tellier Benoit wrote:

I inlined my answers...

Regards,

On 18/09/2019 10:32, Jerry Malcolm wrote:

Thanks for the reply.  But I did not use my old log4j.properties.  I
copied the one from the downloaded 3.3.0 zip file.  Although, it looks
almost identical to the beta5 one.  Any chance the wrong one is included
in the package?

I think the default log4j file was never updated.

Contribution welcomed BTW...


I'm fine with logger-per-mailet.  However, I'm hoping there's a way to
see the flow for an email item.  It's just as important at times to see
that various matchers passed or didn't pass on an item as it goes
through the flow.  Is there still a logger that shows the flow?

The approach regarding this is to use the MDC for doing some
contextualized logging (SLF4J API)

Within James guice application, we rely on logback, and a
"ElasticSearch" appender. Logs can then be filtered by 'mail' property.

You will then see all of the logs related to the processing of that very
email.

I'm not a log4j expert but there should be a way of doing something
somehow similar...


Bottom line, though, I don't know what to put in the log4j file to make
the mailet logs show up.  Do you have a link handy to the correct log4j
in github?

I don't think we have...

I'm not an expert here but let's play a bit:

In log4j.properties I would replace line 124

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER

By

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER
log4j.logger.org.apache.james.transport=INFO, MAILETCONTAINER

(Rationals: all mailets are located within the
"org.apache.james.transport.mailets" &
"org.apache.james.transport.matchers" packages...


Thx


On 9/17/2019 10:23 PM, Tellier Benoit wrote:

Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered that behavior, and create one logger per
mailet class for bundled mailets.

In the end, this enabled simpler logging filtering - on a per mailet
basis, got rid of the imperfect logging facade exposed in the mailet API
(no placeholder for instance), and also killed some "debug"
configuration parameters.

I believe that your log4j configuration file inherited from beta5 needs
to be revisited to adapt these changes.

Hope it helps.

Benoit

On 18/09/2019 09:10, Jerry Malcolm wrote:

Not good news for me, however.  My mailet flow that worked in v3b5 is
crashing now, and none of the log output I'm writing from my mailets is
showing up anywhere.  I've GOT to see that log info some place in order
to debug both my mailets as well as my mailet flow that is somehow no
longer compatible.  All I get is a 'mail received' message, and an error
saying it can't launch one of my mailets because there is a recursive
loop in my flow (there is not, or at least there was not when it running
in v3b5).

There's got to be a way to see mailet log data.  Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:

It’s not you. I have the same issue. I think those other log files are
just there for backwards compatibility to make users feel good, since
they never seem to be written to. “No news is good news” as the saying
goes.

Sent from my iPhone


On Sep 17, 2019, at 4:34 PM, Jerry Malcolm 
wrote:

For the problem du jour when I start JAMES, all of the expected
log files get created.  The main log output  goes to the console and
to the james-server.log file as expected.  No problem there.  But the
remainder of the log files never get touched.  No matter what
happens, every file in the log folder other than james-server.log is
empty.  I've tried changing log4j.properties and setting everything
to DEBUG level.  The console log dumps a bunch more info while in
DEBUG mode (which confirms I'm changing the correct log4j file).  But
every other log file is 

Re: log4j Configuration

[Jerry Malcolm]

I added the transport line as you suggested and sent a test email 
inbound.  The console showed it coming in, as does james-server.log.  
But still, the only log file in the logs directory that is not empty is 
james-server.  Nothing is going to any of these files.


On 9/17/2019 10:49 PM, Tellier Benoit wrote:

I inlined my answers...

Regards,

On 18/09/2019 10:32, Jerry Malcolm wrote:

Thanks for the reply.  But I did not use my old log4j.properties.  I
copied the one from the downloaded 3.3.0 zip file.  Although, it looks
almost identical to the beta5 one.  Any chance the wrong one is included
in the package?

I think the default log4j file was never updated.

Contribution welcomed BTW...


I'm fine with logger-per-mailet.  However, I'm hoping there's a way to
see the flow for an email item.  It's just as important at times to see
that various matchers passed or didn't pass on an item as it goes
through the flow.  Is there still a logger that shows the flow?

The approach regarding this is to use the MDC for doing some
contextualized logging (SLF4J API)

Within James guice application, we rely on logback, and a
"ElasticSearch" appender. Logs can then be filtered by 'mail' property.

You will then see all of the logs related to the processing of that very
email.

I'm not a log4j expert but there should be a way of doing something
somehow similar...


Bottom line, though, I don't know what to put in the log4j file to make
the mailet logs show up.  Do you have a link handy to the correct log4j
in github?

I don't think we have...

I'm not an expert here but let's play a bit:

In log4j.properties I would replace line 124

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER

By

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER
log4j.logger.org.apache.james.transport=INFO, MAILETCONTAINER

(Rationals: all mailets are located within the
"org.apache.james.transport.mailets" &
"org.apache.james.transport.matchers" packages...


Thx


On 9/17/2019 10:23 PM, Tellier Benoit wrote:

Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered that behavior, and create one logger per
mailet class for bundled mailets.

In the end, this enabled simpler logging filtering - on a per mailet
basis, got rid of the imperfect logging facade exposed in the mailet API
(no placeholder for instance), and also killed some "debug"
configuration parameters.

I believe that your log4j configuration file inherited from beta5 needs
to be revisited to adapt these changes.

Hope it helps.

Benoit

On 18/09/2019 09:10, Jerry Malcolm wrote:

Not good news for me, however.  My mailet flow that worked in v3b5 is
crashing now, and none of the log output I'm writing from my mailets is
showing up anywhere.  I've GOT to see that log info some place in order
to debug both my mailets as well as my mailet flow that is somehow no
longer compatible.  All I get is a 'mail received' message, and an error
saying it can't launch one of my mailets because there is a recursive
loop in my flow (there is not, or at least there was not when it running
in v3b5).

There's got to be a way to see mailet log data.  Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:

It’s not you. I have the same issue. I think those other log files are
just there for backwards compatibility to make users feel good, since
they never seem to be written to. “No news is good news” as the saying
goes.

Sent from my iPhone


On Sep 17, 2019, at 4:34 PM, Jerry Malcolm 
wrote:

For the problem du jour when I start JAMES, all of the expected
log files get created.  The main log output  goes to the console and
to the james-server.log file as expected.  No problem there.  But the
remainder of the log files never get touched.  No matter what
happens, every file in the log folder other than james-server.log is
empty.  I've tried changing log4j.properties and setting everything
to DEBUG level.  The console log dumps a bunch more info while in
DEBUG mode (which confirms I'm changing the correct log4j file).  But
every other log file is still empty.  I used to see my mailet output
in the main console and james-server.log in b5.  I'm not seeing any
of that anywhere. Again, I've played around with log4j.properties (I
made sure I was using the latest 3.3.0 log4j.properties file).
Nothing.  What am I doing wrong?  How can I get log data to start
going into those other files?


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional command

Re: log4j Configuration

[Jerry Malcolm]

Thanks for the reply.  But I did not use my old log4j.properties.  I 
copied the one from the downloaded 3.3.0 zip file.  Although, it looks 
almost identical to the beta5 one.  Any chance the wrong one is included 
in the package?


I'm fine with logger-per-mailet.  However, I'm hoping there's a way to 
see the flow for an email item.  It's just as important at times to see 
that various matchers passed or didn't pass on an item as it goes 
through the flow.  Is there still a logger that shows the flow?


Bottom line, though, I don't know what to put in the log4j file to make 
the mailet logs show up.  Do you have a link handy to the correct log4j 
in github?


Thx


On 9/17/2019 10:23 PM, Tellier Benoit wrote:

Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered that behavior, and create one logger per
mailet class for bundled mailets.

In the end, this enabled simpler logging filtering - on a per mailet
basis, got rid of the imperfect logging facade exposed in the mailet API
(no placeholder for instance), and also killed some "debug"
configuration parameters.

I believe that your log4j configuration file inherited from beta5 needs
to be revisited to adapt these changes.

Hope it helps.

Benoit

On 18/09/2019 09:10, Jerry Malcolm wrote:

Not good news for me, however.  My mailet flow that worked in v3b5 is
crashing now, and none of the log output I'm writing from my mailets is
showing up anywhere.  I've GOT to see that log info some place in order
to debug both my mailets as well as my mailet flow that is somehow no
longer compatible.  All I get is a 'mail received' message, and an error
saying it can't launch one of my mailets because there is a recursive
loop in my flow (there is not, or at least there was not when it running
in v3b5).

There's got to be a way to see mailet log data.  Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:

It’s not you. I have the same issue. I think those other log files are
just there for backwards compatibility to make users feel good, since
they never seem to be written to. “No news is good news” as the saying
goes.

Sent from my iPhone


On Sep 17, 2019, at 4:34 PM, Jerry Malcolm 
wrote:

For the problem du jour when I start JAMES, all of the expected
log files get created.  The main log output  goes to the console and
to the james-server.log file as expected.  No problem there.  But the
remainder of the log files never get touched.  No matter what
happens, every file in the log folder other than james-server.log is
empty.  I've tried changing log4j.properties and setting everything
to DEBUG level.  The console log dumps a bunch more info while in
DEBUG mode (which confirms I'm changing the correct log4j file).  But
every other log file is still empty.  I used to see my mailet output
in the main console and james-server.log in b5.  I'm not seeing any
of that anywhere. Again, I've played around with log4j.properties (I
made sure I was using the latest 3.3.0 log4j.properties file).
Nothing.  What am I doing wrong?  How can I get log data to start
going into those other files?


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: log4j Configuration

[Jerry Malcolm]

Not good news for me, however.  My mailet flow that worked in v3b5 is 
crashing now, and none of the log output I'm writing from my mailets is 
showing up anywhere.  I've GOT to see that log info some place in order 
to debug both my mailets as well as my mailet flow that is somehow no 
longer compatible.  All I get is a 'mail received' message, and an error 
saying it can't launch one of my mailets because there is a recursive 
loop in my flow (there is not, or at least there was not when it running 
in v3b5).


There's got to be a way to see mailet log data.  Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:

It’s not you. I have the same issue. I think those other log files are just 
there for backwards compatibility to make users feel good, since they never 
seem to be written to. “No news is good news” as the saying goes.

Sent from my iPhone


On Sep 17, 2019, at 4:34 PM, Jerry Malcolm  wrote:

For the problem du jour when I start JAMES, all of the expected log files 
get created.  The main log output  goes to the console and to the 
james-server.log file as expected.  No problem there.  But the remainder of the 
log files never get touched.  No matter what happens, every file in the log 
folder other than james-server.log is empty.  I've tried changing 
log4j.properties and setting everything to DEBUG level.  The console log dumps 
a bunch more info while in DEBUG mode (which confirms I'm changing the correct 
log4j file).  But every other log file is still empty.  I used to see my mailet 
output in the main console and james-server.log in b5.  I'm not seeing any of 
that anywhere. Again, I've played around with log4j.properties (I made sure I 
was using the latest 3.3.0 log4j.properties file).  Nothing.  What am I doing 
wrong?  How can I get log data to start going into those other files?


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: log4j Configuration

[Jerry Malcolm]

Without a way to see my mailet log data, I have no way of knowing if my 
mailets were even called, if they succeeded, if they failed.  Right now, 
all I know is they went into a processor and disappeared.  This is a 
serious problem for me.  I really need to have a way to see this data.  
If it's a log4j.properties change, I just need to know what to change.  
I'm dead in the water.



On 9/17/2019 11:08 PM, Jerry Malcolm wrote:
I added the transport line as you suggested and sent a test email 
inbound.  The console showed it coming in, as does james-server.log.  
But still, the only log file in the logs directory that is not empty 
is james-server.  Nothing is going to any of these files.


On 9/17/2019 10:49 PM, Tellier Benoit wrote:

I inlined my answers...

Regards,

On 18/09/2019 10:32, Jerry Malcolm wrote:

Thanks for the reply.  But I did not use my old log4j.properties.  I
copied the one from the downloaded 3.3.0 zip file.  Although, it looks
almost identical to the beta5 one.  Any chance the wrong one is 
included

in the package?

I think the default log4j file was never updated.

Contribution welcomed BTW...


I'm fine with logger-per-mailet. However, I'm hoping there's a way to
see the flow for an email item.  It's just as important at times to see
that various matchers passed or didn't pass on an item as it goes
through the flow.  Is there still a logger that shows the flow?

The approach regarding this is to use the MDC for doing some
contextualized logging (SLF4J API)

Within James guice application, we rely on logback, and a
"ElasticSearch" appender. Logs can then be filtered by 'mail' property.

You will then see all of the logs related to the processing of that very
email.

I'm not a log4j expert but there should be a way of doing something
somehow similar...


Bottom line, though, I don't know what to put in the log4j file to make
the mailet logs show up.  Do you have a link handy to the correct log4j
in github?

I don't think we have...

I'm not an expert here but let's play a bit:

In log4j.properties I would replace line 124

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER

By

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER
log4j.logger.org.apache.james.transport=INFO, MAILETCONTAINER

(Rationals: all mailets are located within the
"org.apache.james.transport.mailets" &
"org.apache.james.transport.matchers" packages...


Thx


On 9/17/2019 10:23 PM, Tellier Benoit wrote:

Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered that behavior, and create one 
logger per

mailet class for bundled mailets.

In the end, this enabled simpler logging filtering - on a per mailet
basis, got rid of the imperfect logging facade exposed in the 
mailet API

(no placeholder for instance), and also killed some "debug"
configuration parameters.

I believe that your log4j configuration file inherited from beta5 
needs

to be revisited to adapt these changes.

Hope it helps.

Benoit

On 18/09/2019 09:10, Jerry Malcolm wrote:

Not good news for me, however.  My mailet flow that worked in v3b5 is
crashing now, and none of the log output I'm writing from my 
mailets is
showing up anywhere.  I've GOT to see that log info some place in 
order

to debug both my mailets as well as my mailet flow that is somehow no
longer compatible.  All I get is a 'mail received' message, and an 
error

saying it can't launch one of my mailets because there is a recursive
loop in my flow (there is not, or at least there was not when it 
running

in v3b5).

There's got to be a way to see mailet log data.  Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:
It’s not you. I have the same issue. I think those other log 
files are
just there for backwards compatibility to make users feel good, 
since
they never seem to be written to. “No news is good news” as the 
saying

goes.

Sent from my iPhone


On Sep 17, 2019, at 4:34 PM, Jerry Malcolm 
wrote:

For the problem du jour when I start JAMES, all of the expected
log files get created.  The main log output  goes to the console 
and
to the james-server.log file as expected.  No problem there.  
But the

remainder of the log files never get touched.  No matter what
happens, every file in the log folder other than 
james-server.log is

empty.  I've tried changing log4j.properties and setting everything
to DEBUG level.  The console log dumps a bunch more info while in
DEBUG mode (which confirms I'm changing the correct log4j 
file).  But
every other log file is still empty.  I used to see my mailet 
output

in the main console and james-server.log in b5.  I'm not seeing any
of that anywhere. Again, I've played around with 
log4j.properties (I

made sure I was using the latest 3.3.0 log4j.properties file).
Nothing.  Wh

Re: AdministratorId Documentation

[Jerry Malcolm]

Ignore my previous question.  I dug further and realized that your auth 
method is a lower-level replacement for the login method in the 
commons.net sample.  Still may have other questions... But I think I'm 
good on this one for now.


Thx


On 9/13/2019 12:43 PM, Jerry Malcolm wrote:

Tellier,

Sorry, I'm still missing something.  I downloaded the commons-net 
package and looked at the IMAPMail sample code.  It says to put the 
id/pw on the url: "Usage: IMAPMail 
imap[s]://username:password@server/");  I assume I would put my admin 
id:pw on the url.  But then where do I specify the userid for the 
actual inbox I want to access?


On 9/12/2019 11:01 PM, Tellier Benoit wrote:

If you are using commons-net, I succeeded to write:

https://gist.github.com/chibenwa/abd12fd6c0b06cadd1de591e3ac792b9

That should be helping you!

On 13/09/2019 10:40, Tellier Benoit wrote:

Wich library are you using?

On 13/09/2019 10:33, Jerry Malcolm wrote:

Thanks for the info, Tellier.  I kinda lost you on the mpt tests... I
was looking for how to change the following code to include an
administrator id.  I only have one field in the store.connect() method
for a user id, but I have an administrator id and the userid for the
target mailbox.  How do I pass in both?

    try
    {
   session = Session.getInstance(props, null);
   session.setDebug(debug);
   store = session.getStore(protocol);
   store.connect(host, port, user, password);
    }
    catch( Exception e)
    {
   log( "IMAPClient.connect() exception: " + e );
   throw(e);
    }

On 9/12/2019 10:10 PM, Tellier Benoit wrote:

Hello Jerry,

With the `administratorId`, you are able to use IMAP impersonation.

IE to log in as another user.

You should define it within usersrepository.xml. To see related 
"reading

config" code: AbstractUsersRepository is the way to go.

Agree that this needs example and documentation. I will do it 
straight

away.

For your IMAP call, MPT tests are going to be a gold mine, presenting
you the exact syntax.
mpt/impl/imap-mailbox/core/src/main/resources/org/apache/james/imap/scripts/AuthenticatePlain.test 




Contains:

C: 0007 AUTHENTICATE "PLAIN" {36+}
# delegate\0imapuser\0password
C: ZGVsZWdhdGUAaW1hcHVzZXIAcGFzc3dvcmQ=
S: 0007 OK AUTHENTICATE completed.

# Ensure we are delegate
C: 0008 SELECT delegate
SUB {
S: \* FLAGS .*
S: \* .* EXISTS
S: \* .* RECENT
S: \* OK \[UIDVALIDITY .*\] UIDs valid
S: \* OK \[PERMANENTFLAGS .*\] Limited
S: \* OK \[HIGHESTMODSEQ .*\] Highest
S: \* OK \[UIDNEXT .*\] Predicted next UID
}
S: 0008 OK \[READ-WRITE\] SELECT completed\.

Regarding your development issues, I'm really sorry. My 
environment is

based on IntelliJ, and I encounter no issue.

To be able to compile, I use maven 3.6.2 with JDK 11.

Best regards,

Benoit Tellier


On 13/09/2019 09:53, Jerry Malcolm wrote:
Since it appears I'm not going to be able to build James 3.3.0 in 
the
foreseeable future, I'm now moving to plan d, e, f, or 
whatever... .I've

lost count.

My goal now is to assess my possibilities of using James 3.3.0 
binaries
as-is and discarding the functionality I had hoped to re-add to 
3.3.0

from 30b5 or moving the functionality to mailets that I can compile
outside of doing a full James build.

One thing I had added to beta5 was the ability to have a super-user,
master-key id that could be used in my custom external imap mailbox
maintenance utilities.  I've spent several hours digging through the
3.3.0 source code, and I see code referencing "administratorId".  
From

what I can deduce from the code, the Authenticator and Authorizator
classes seem to accept this administrator id as the equivalent of 
the

master key I added to b5.  So can I access UserABC's mailbox through
imap but use my administratorId and password?  If that's the 
case, then

that's exactly what I need.  The problem is I can't find any
documentation on how to use administratorId.  Where do I define 
it?  And

how do I pass it on imap calls using the javax.mail.Store and other
classes in javax.mail that I use to access imap?

Jerry


- 


To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional c

Re: 3.0.0 on Linux

[Jerry Malcolm]

Hi Garry,

You missed my point.  I've been trying to get help on building 3.3.0 for 
over a week with no response (see my long list of posts).  Since I can't 
get anyone to respond to my 3.3.0 build problem, I had to look for an 
alternative.  I currently have a 3.0.0 beta 5 install that has worked 
for years on a Windows server.  My only alternative until I get my 3.3.0 
build problems resolved is to move that v3b5 to Linux since my windows 
server is going away.  I have added custom code.  So I can't just 
download the binaries.  I would love to move to 3.3.0.  But until I can 
figure out my build problems and/or get someone to respond to my 
questions, this is my only choice.  Now my current ancient v3b5 that 
worked on Windows fails on Linux with OpenJPA issues (see prev post).


I realize this is a completely volunteer forum.  But I'm dying here.  
I've got to get some version of james running on my AWS Linux box.  Can 
someone please find time to help?  I'll take help on my 3.0.0 b5 OpenJPA 
issue and/or my 3.3.0 Eclipse build problems... just SOMETHING.


Thx

Jerry

On 9/8/2019 2:31 AM, Garry Hurley wrote:

3.0.0 is kinda old. In fact, IIRC, it never made it out of Beta stage. 3.3.0 
works and is stable, so unless you have some reason to use an older version, 
you should consider updating.

Sent from my iPhone


On Sep 8, 2019, at 12:07 AM, Jerry Malcolm  wrote:

I'm still waiting and hoping somebody can please explain why I can't build 
3.3.0 on Eclipse.  (See my other thread).  But I'm on a deadline.  My windows 
server is going away soon and I HAVE to get a mail solution onto AWS.  
(ANYBODY?? PLEASE HELP!)

So in meantime, plan-B is to copy my current ancient v3b5 installation off of my Windows 
server to my AWS Linux instance. Since James is 100% java (good ol' "write once -- 
run anywhere"..., right?) I figured I could just copy the entire James installation 
folder and fix a couple of backslashes in file paths in config files, and it should run 
perfectly.  Wrong-o.  I literally copied the folder precisely, and changed to run.sh 
instead of run.bat.  After a few expected startup messages, I get a bunch of OpenJPA 
warnings, followed by a bunch of errors telling me foreign keys are missing from my 
database, etc. (See log excerpt below).  I went back to the running Windows instance. 
There is no reference anywhere in the Windows James log to jpa. Since the conf and lib 
folders are identical (as far as I can tell) between these two instances, I'm baffled.  
Why is OpenJPA getting upset with the Linux install and not the Windows install. My first 
thought was something with case-sensitive file names. But I can't find any of those 
situations.  I'm not familiar with OpenJPA configuration.  So these errors mean nothing 
to me.  What did I do wrong? (BTW... both instances point to the same db... so I know the 
database is not corrupt).

Can anybody help with this problem (or better... help me with the 3.3.0 Eclipse 
disaster... I don't want to remain on v3b5 forever).

Thx.

Log entries:

INFO  02:35:01,219 | james.dnsservice | Adding autodiscovered search path 
ec2.internal.
INFO  02:35:01,220 | james.dnsservice | DNS Server is: 172.31.0.2
INFO  02:35:01,226 | james.dnsservice | Registered cache, resolver and search 
paths as DNSJava defaults
185  James  INFO   [main] openjpa.Runtime - Starting OpenJPA 2.2.1
215  James  INFO   [main] openjpa.jdbc.JDBC - Using dictionary class 
"org.apache.openjpa.jdbc.sql.MySQLDictionary".
630  James  WARN   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in the 
openjpa.MetaDataFactory configuration property could not be loaded by 
sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
631  James  WARN   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded by 
sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
723  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in the 
openjpa.MetaDataFactory configuration property could not be loaded by 
sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
723  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage" listed in 
the openjpa.MetaDataFactory configuration property could not be loaded by 
sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
735  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in the 
openjpa.MetaDataFactory configuration property could not be loaded by 
sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
735  James  INFO   [main] openjpa.MetaData - The class 
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAM

AdministratorId Documentation

[Jerry Malcolm]

Since it appears I'm not going to be able to build James 3.3.0 in the 
foreseeable future, I'm now moving to plan d, e, f, or whatever... .I've 
lost count.


My goal now is to assess my possibilities of using James 3.3.0 binaries 
as-is and discarding the functionality I had hoped to re-add to 3.3.0 
from 30b5 or moving the functionality to mailets that I can compile 
outside of doing a full James build.


One thing I had added to beta5 was the ability to have a super-user, 
master-key id that could be used in my custom external imap mailbox 
maintenance utilities.  I've spent several hours digging through the 
3.3.0 source code, and I see code referencing "administratorId".  From 
what I can deduce from the code, the Authenticator and Authorizator 
classes seem to accept this administrator id as the equivalent of the 
master key I added to b5.  So can I access UserABC's mailbox through 
imap but use my administratorId and password?  If that's the case, then 
that's exactly what I need.  The problem is I can't find any 
documentation on how to use administratorId.  Where do I define it?  And 
how do I pass it on imap calls using the javax.mail.Store and other 
classes in javax.mail that I use to access imap?


Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

30b5 to 3.3.0 Database Changes?

[Jerry Malcolm]

Quick question... does 3.3.0 do any upward migration of an existing 
database that burns bridges for accessing it by 3.0.beta5?  In other 
words, if I start up 3.3.0 using my existing database and I find 
problems, can I successfully revert back to using beta5?  Or does 3.3.0 
do irreversible modifications to the database?



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Intercepting delivery success/fail

[Jerry Malcolm]

Another mod I had made to v3b5 was to intercept the final result of 
success/fail of delivery.  I have a requirement to externally log the 
"mail was successfully transferred to " 
message and also after all of the retries are exhausted, the final 
'fail' message.


Back in v3b5 I cloned the RemoteDelivery mailet and located the 
appropriate locations where the successful handoff was logged and where 
the final fail was logged.  It looks like that RemoteDelivery mailet was 
totally rewritten somewhere between b5 and 3.3.  Mail is put on a queue 
in remoteDelivery, and all the work is now handled elsewhere.


My question is simply this: What is the easiest/best/correct way to hook 
in so that I can add my logging code on the successful handoff and also 
on the final retry fail?  If I can do this without requiring a full 
rebuild, that's desired considering my disaster at build attempts.  But 
if the only way to add this required functionality is to modify core 
code, sooner or later I'll have to figure out my build problems.  Either 
way, I need to know what you recommend for hooking these two points.


Thx



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: AdministratorId Documentation

[Jerry Malcolm]

Thanks for the info, Tellier.  I kinda lost you on the mpt tests... I 
was looking for how to change the following code to include an 
administrator id.  I only have one field in the store.connect() method 
for a user id, but I have an administrator id and the userid for the 
target mailbox.  How do I pass in both?


   try
   {
  session = Session.getInstance(props, null);
  session.setDebug(debug);
  store = session.getStore(protocol);
  store.connect(host, port, user, password);
   }
   catch( Exception e)
   {
  log( "IMAPClient.connect() exception: " + e );
  throw(e);
   }

On 9/12/2019 10:10 PM, Tellier Benoit wrote:

Hello Jerry,

With the `administratorId`, you are able to use IMAP impersonation.

IE to log in as another user.

You should define it within usersrepository.xml. To see related "reading
config" code: AbstractUsersRepository is the way to go.

Agree that this needs example and documentation. I will do it straight away.

For your IMAP call, MPT tests are going to be a gold mine, presenting
you the exact syntax.
mpt/impl/imap-mailbox/core/src/main/resources/org/apache/james/imap/scripts/AuthenticatePlain.test

Contains:

C: 0007 AUTHENTICATE "PLAIN" {36+}
# delegate\0imapuser\0password
C: ZGVsZWdhdGUAaW1hcHVzZXIAcGFzc3dvcmQ=
S: 0007 OK AUTHENTICATE completed.

# Ensure we are delegate
C: 0008 SELECT delegate
SUB {
S: \* FLAGS .*
S: \* .* EXISTS
S: \* .* RECENT
S: \* OK \[UIDVALIDITY .*\] UIDs valid
S: \* OK \[PERMANENTFLAGS .*\] Limited
S: \* OK \[HIGHESTMODSEQ .*\] Highest
S: \* OK \[UIDNEXT .*\] Predicted next UID
}
S: 0008 OK \[READ-WRITE\] SELECT completed\.

Regarding your development issues, I'm really sorry. My environment is
based on IntelliJ, and I encounter no issue.

To be able to compile, I use maven 3.6.2 with JDK 11.

Best regards,

Benoit Tellier


On 13/09/2019 09:53, Jerry Malcolm wrote:

Since it appears I'm not going to be able to build James 3.3.0 in the
foreseeable future, I'm now moving to plan d, e, f, or whatever... .I've
lost count.

My goal now is to assess my possibilities of using James 3.3.0 binaries
as-is and discarding the functionality I had hoped to re-add to 3.3.0
from 30b5 or moving the functionality to mailets that I can compile
outside of doing a full James build.

One thing I had added to beta5 was the ability to have a super-user,
master-key id that could be used in my custom external imap mailbox
maintenance utilities.  I've spent several hours digging through the
3.3.0 source code, and I see code referencing "administratorId".  From
what I can deduce from the code, the Authenticator and Authorizator
classes seem to accept this administrator id as the equivalent of the
master key I added to b5.  So can I access UserABC's mailbox through
imap but use my administratorId and password?  If that's the case, then
that's exactly what I need.  The problem is I can't find any
documentation on how to use administratorId.  Where do I define it?  And
how do I pass it on imap calls using the javax.mail.Store and other
classes in javax.mail that I use to access imap?

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: AdministratorId Documentation

[Jerry Malcolm]

I believe I'm just using the basic javax.mail.* that comes with JDK 
1.8.   Do I need to move to a later imap implementation package?



On 9/12/2019 10:40 PM, Tellier Benoit wrote:

Wich library are you using?

On 13/09/2019 10:33, Jerry Malcolm wrote:

Thanks for the info, Tellier.  I kinda lost you on the mpt tests... I
was looking for how to change the following code to include an
administrator id.  I only have one field in the store.connect() method
for a user id, but I have an administrator id and the userid for the
target mailbox.  How do I pass in both?

    try
    {
   session = Session.getInstance(props, null);
   session.setDebug(debug);
   store = session.getStore(protocol);
   store.connect(host, port, user, password);
    }
    catch( Exception e)
    {
   log( "IMAPClient.connect() exception: " + e );
   throw(e);
    }

On 9/12/2019 10:10 PM, Tellier Benoit wrote:

Hello Jerry,

With the `administratorId`, you are able to use IMAP impersonation.

IE to log in as another user.

You should define it within usersrepository.xml. To see related "reading
config" code: AbstractUsersRepository is the way to go.

Agree that this needs example and documentation. I will do it straight
away.

For your IMAP call, MPT tests are going to be a gold mine, presenting
you the exact syntax.
mpt/impl/imap-mailbox/core/src/main/resources/org/apache/james/imap/scripts/AuthenticatePlain.test


Contains:

C: 0007 AUTHENTICATE "PLAIN" {36+}
# delegate\0imapuser\0password
C: ZGVsZWdhdGUAaW1hcHVzZXIAcGFzc3dvcmQ=
S: 0007 OK AUTHENTICATE completed.

# Ensure we are delegate
C: 0008 SELECT delegate
SUB {
S: \* FLAGS .*
S: \* .* EXISTS
S: \* .* RECENT
S: \* OK \[UIDVALIDITY .*\] UIDs valid
S: \* OK \[PERMANENTFLAGS .*\] Limited
S: \* OK \[HIGHESTMODSEQ .*\] Highest
S: \* OK \[UIDNEXT .*\] Predicted next UID
}
S: 0008 OK \[READ-WRITE\] SELECT completed\.

Regarding your development issues, I'm really sorry. My environment is
based on IntelliJ, and I encounter no issue.

To be able to compile, I use maven 3.6.2 with JDK 11.

Best regards,

Benoit Tellier


On 13/09/2019 09:53, Jerry Malcolm wrote:

Since it appears I'm not going to be able to build James 3.3.0 in the
foreseeable future, I'm now moving to plan d, e, f, or whatever... .I've
lost count.

My goal now is to assess my possibilities of using James 3.3.0 binaries
as-is and discarding the functionality I had hoped to re-add to 3.3.0
from 30b5 or moving the functionality to mailets that I can compile
outside of doing a full James build.

One thing I had added to beta5 was the ability to have a super-user,
master-key id that could be used in my custom external imap mailbox
maintenance utilities.  I've spent several hours digging through the
3.3.0 source code, and I see code referencing "administratorId".  From
what I can deduce from the code, the Authenticator and Authorizator
classes seem to accept this administrator id as the equivalent of the
master key I added to b5.  So can I access UserABC's mailbox through
imap but use my administratorId and password?  If that's the case, then
that's exactly what I need.  The problem is I can't find any
documentation on how to use administratorId.  Where do I define it?  And
how do I pass it on imap calls using the javax.mail.Store and other
classes in javax.mail that I use to access imap?

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: James with AWS SES mail receiving

[Jerry Malcolm]

I'm looking at moving off of my dedicated server to AWS, which means 
moving JAMES.  Interesting concepts in this thread below from a year or 
so ago about using SES.  But all I want to do is get JAMES up and 
running in an AWS EC2 with an RDS with as little rip up and effort as 
possible.   Assuming I copy my current JAMES build to EC2, set up the 
RDS, install SpamAssassin, and open the appropriate ports, are there any 
gotchas lurking that I need to be aware of?  (Still just getting my feet 
wet with AWS).


Somewhat off topic... what size EC2 is recommended?  Do I simply start 
small and creep up until the EC2 no longer pegs the meter? Anybody have 
an experience with what size EC2 to select?


Thanks.

Jerry

On 6/11/2018 8:16 AM, Jeremy T. Bouse wrote:

Benoit,

     Yes it can send to a SMTP endpoint but the FROM address has to be an
approved email address/domain for SES to be able to send it which means
having to modify the envelope headers. Further to that I'm looking at
the possibility of running James in containers on AWS ECS which would be
on a private subnet so it wouldn't be reachable directly without going
through a load-balancer.


On 6/11/2018 12:44 AM, Benoit Tellier wrote:

Hi Jeremy,

Can't AWS SES send these messages to a SMTP endpoint? This way it will
work without any further development.

That being said, I consider the feature you propose extremely
interesting, as it will provide alternatives to SMTP for applications
sending emails.

In my opinion, you would need to implement a new component in James
listening on AWS SNS, upon messages retrieving the mails, and then
enqueue them in James internal MailQueue.

Cheers,

Benoit Tellier

Le 11/06/2018 à 01:53, Jeremy T. Bouse a écrit :

     Has anyone thought about how to possibly make use of AWS SES email
receiving to accept inbound email and get it passed along into James?
With AWS SES email receiving you can have it save the actual message to
an S3 bucket and then fire off an SNS topic or Lambda function so the
question would really be how to trigger James to be able to process the
alert and ingest the message from S3. I've been thinking about it as a
possible email solution for myself and it seems like it should be
possible but I've not yet been able to determine if there's already an
easy method to do so or if it'll take some development to make it work.
Wanted to see if anyone else had thought about it.


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: log4j Configuration

[Jerry Malcolm]

Any chance I could get someone to take a look at this problem? There's 
no way for me to move forward until I can see my mailet logs.  I tried 
to dig into slf4j and log4j.  But I just don't have enough background in 
that to figure out what is happening.  And since I don't have a full 
james set up on an IDE where I can set breakpoints, I have no way to 
figure out why all log data is just being discarded regardless of the 
log4j.properties settings.  With the appropriate tools it seems like 
it'd be pretty easy to see why log4j is not writing any data.  But I 
just don't have the tools. Can somebody in James development please take 
a few minutes and see if they figure it out?  I'm sure it's probably 
something trivial as properties file settings.  But without some 
guidance, I'm shooting in the dark.


Help, please?

Jerry

On 9/18/2019 10:18 AM, Jerry Malcolm wrote:
Without a way to see my mailet log data, I have no way of knowing if 
my mailets were even called, if they succeeded, if they failed. Right 
now, all I know is they went into a processor and disappeared.  This 
is a serious problem for me.  I really need to have a way to see this 
data.  If it's a log4j.properties change, I just need to know what to 
change.  I'm dead in the water.



On 9/17/2019 11:08 PM, Jerry Malcolm wrote:
I added the transport line as you suggested and sent a test email 
inbound.  The console showed it coming in, as does james-server.log.  
But still, the only log file in the logs directory that is not empty 
is james-server. Nothing is going to any of these files.


On 9/17/2019 10:49 PM, Tellier Benoit wrote:

I inlined my answers...

Regards,

On 18/09/2019 10:32, Jerry Malcolm wrote:

Thanks for the reply.  But I did not use my old log4j.properties.  I
copied the one from the downloaded 3.3.0 zip file. Although, it looks
almost identical to the beta5 one.  Any chance the wrong one is 
included

in the package?

I think the default log4j file was never updated.

Contribution welcomed BTW...


I'm fine with logger-per-mailet. However, I'm hoping there's a way to
see the flow for an email item.  It's just as important at times to 
see

that various matchers passed or didn't pass on an item as it goes
through the flow.  Is there still a logger that shows the flow?

The approach regarding this is to use the MDC for doing some
contextualized logging (SLF4J API)

Within James guice application, we rely on logback, and a
"ElasticSearch" appender. Logs can then be filtered by 'mail' property.

You will then see all of the logs related to the processing of that 
very

email.

I'm not a log4j expert but there should be a way of doing something
somehow similar...

Bottom line, though, I don't know what to put in the log4j file to 
make
the mailet logs show up.  Do you have a link handy to the correct 
log4j

in github?

I don't think we have...

I'm not an expert here but let's play a bit:

In log4j.properties I would replace line 124

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER

By

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER
log4j.logger.org.apache.james.transport=INFO, MAILETCONTAINER

(Rationals: all mailets are located within the
"org.apache.james.transport.mailets" &
"org.apache.james.transport.matchers" packages...


Thx


On 9/17/2019 10:23 PM, Tellier Benoit wrote:

Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered that behavior, and create one 
logger per

mailet class for bundled mailets.

In the end, this enabled simpler logging filtering - on a per mailet
basis, got rid of the imperfect logging facade exposed in the 
mailet API

(no placeholder for instance), and also killed some "debug"
configuration parameters.

I believe that your log4j configuration file inherited from beta5 
needs

to be revisited to adapt these changes.

Hope it helps.

Benoit

On 18/09/2019 09:10, Jerry Malcolm wrote:
Not good news for me, however.  My mailet flow that worked in 
v3b5 is
crashing now, and none of the log output I'm writing from my 
mailets is
showing up anywhere.  I've GOT to see that log info some place in 
order
to debug both my mailets as well as my mailet flow that is 
somehow no
longer compatible.  All I get is a 'mail received' message, and 
an error
saying it can't launch one of my mailets because there is a 
recursive
loop in my flow (there is not, or at least there was not when it 
running

in v3b5).

There's got to be a way to see mailet log data. Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:
It’s not you. I have the same issue. I think those other log 
files are
just there for backwards compatibility to make users feel good, 
since
they never seem to be written to. “No news is good news” as the 
saying

goes.

Sent from my iPhone

Re: JMX Access?

[Jerry Malcolm]

The way I understand tunneling, with SSH tunneling, I am technically 
connecting locally.   I use PuTTY on my local machine.  But I set up a 
secure tunnel to port  on the remote host.  When I connect to 
localhost: on my machine it's the same as connecting to 
localhost: on the target machine.  As long as the tunnel is secure 
(SSL), there's no outside access to JMX.   At least that's the way the 
videos explain it for accessing remote Tomcat via a PuTTY tunnel.  And 
Tomcat access works.   Is the exception in James startup log stating 
that the connection is refused ok to ignore.  Seems that's not a good 
thing.


On 9/26/2019 12:40 AM, Tellier Benoit wrote:

Hi Jerry,

You need to be aware James restrict JMX access to localhost.

As you know, the James command line is currently using JMX.

JMX is known-to-be unsecure: read
https://www.cvedetails.com/cve/CVE-2017-12628/ for instance. Exposing
JMX only to localHost is a mitigation.

Could this be the root of your JMX problem?

On 26/09/2019 12:03, Jerry Malcolm wrote:

[...]> So I decided to use JMX to see if there were any
MBeans exposed that might help me figure out why there is no logging.
And, of course with the way everything else has gone, I can't connect to
James JMX.  JConsole will not connect.  It works fine accessing Tomcat
using the same approach. But nothing with James.  I'm using a PuTTY
tunnel directly to the Amazon EC2 instance.

I went back to an absolute pristine install of JAMES 3.3.0 right out of
the downloaded zip file.  I made absolutely zero changes to the
install/config.  Just unpack and run.  No JMX access with out-of-the-box
James either.  JConsole hangs for about a minute, and gives me an error
that it can't connect.

Then I made one change to log4j.properties and bumped everything to
DEBUG.  Thankfully, the one log file that actually works in this release
is James-server.log.  I see the following error when James is apparently
starting up JMX.  The 172.31 IP address it's failing on is my EC2
public IP.  But the jmx.properties file says to connect to
127.0.0.1:.  I never explicitly configured the 172.31... address
anywhere in James.

What's causing this:

INFO  04:29:47,638 |
org.springframework.remoting.rmi.RmiRegistryFactoryBean | Looking for
RMI registry at port '', using custom socket factory
DEBUG 04:29:47,642 |
org.springframework.remoting.rmi.RmiRegistryFactoryBean | RMI registry
access threw exception
java.rmi.ConnectException: Connection refused to host: 172.31.32.236;
nested exception is:
     java.net.ConnectException: Connection refused (Connection refused)
     at
sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
     at
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
     at
sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
     at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:338)
     at
sun.rmi.registry.RegistryImpl_Stub.list(RegistryImpl_Stub.java:85)
     at
org.springframework.remoting.rmi.RmiRegistryFactoryBean.testRegistry(RmiRegistryFactoryBean.java:281)

     at
org.springframework.remoting.rmi.RmiRegistryFactoryBean.getRegistry(RmiRegistryFactoryBean.java:222)

     at
org.springframework.remoting.rmi.RmiRegistryFactoryBean.getRegistry(RmiRegistryFactoryBean.java:193)

     at
org.springframework.remoting.rmi.RmiRegistryFactoryBean.afterPropertiesSet(RmiRegistryFactoryBean.java:164)

     at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1573)

     at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1511)

     at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)

     at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)

     at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)

     at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)

     at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)

     at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)

     at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:283)

     at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)

     at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:

Re: JMX Access?

[Jerry Malcolm]

I apologize that my posts have appeared course. I'm very frustrated and 
nervous about my client base.  I committed my business to James around 
2004, I believe.  I upgraded from v2 to v3b5 in 2014 with relatively 
minor issues.  When I started what I thought would be an even less 
stressful migration in early August to 3.3.0, I was not prepared for 
nearly 2 months of effort and still no way of knowing how to get 3.3.0 
to work.  The date is fast approaching that my current server is going 
away.  And if I don't have 3.3.0 up and running, I get to tell a whole 
lot of people that they're not going to be getting mail and my business 
folds. This is pretty scary.  I completely understand this is 
voluntary.  I simply would like some people more knowledgeable about 
James than me to dialog about whether problems are actually release 
content problems or user error.  I can handle user errors.  I am far 
from knowledgeable about everything in James, and I make mistakes.  But 
when I open the zip and set all log levels to DEBUG and all the log 
files are empty, I would hope someone on the developer/contributor team 
would be willing to take a look and see if I'm right.  If I'm right, 
then a suggested workaround would be huge.  If I'm wrong, someone set me 
straight. Again, I'm sorry for the frustrated posts.  It's the total 
silence that is difficult when my source of income is on the line.


I'm willing to pay for support.  Please contact me privately and let me 
know the cost would be to find out why I can't see any log output for 
any loggers other than james-server.


On 9/26/2019 12:35 AM, Tellier Benoit wrote:

Hi Jerry
Well,

  1. You are asking sometime hard questions that people can't answers
without significant work overhead.

  2. This is OpenSource. This is Free. This is 'best effort' given a
limited time range.

Please read again carefully the Apache foundation mailing list guide
lines: https://www.apache.org/foundation/policies/conduct.html

I personally do not find the style of your previous email being
'friendly' with the work contributors do. And you are also not patient
with the answers the community provides you for free.

I'd like to mention that some community member will by the way be very
happy to offer a paid support (for example me as part of Linagora
company) might you be unhappy with the support provided as part of the
community.

And for your information, yes, we are using James 3.3.0.

I will answer your JMX question in an other email.

Best regards

On 26/09/2019 12:03, Jerry Malcolm wrote:

I figure if I keep asking questions, sooner or later I'm going to hit
one that SOMEBODY can help me withThe issues keep backing up.  And
my deadline is way too close Is anyone using James 3.3.0?  Yet I'm
the only one having these out-of-the-box problems?  This migration has
been an absolute nightmare.

I still have absolutely NO way to see any logging data for the mailets,
smtp, imap, etc logs.   So I decided to use JMX to see if there were any
MBeans exposed that might help me figure out why there is no logging.
And, of course with the way everything else has gone, I can't connect to
James JMX.  JConsole will not connect.  It works fine accessing Tomcat
using the same approach. But nothing with James.  I'm using a PuTTY
tunnel directly to the Amazon EC2 instance.

I went back to an absolute pristine install of JAMES 3.3.0 right out of
the downloaded zip file.  I made absolutely zero changes to the
install/config.  Just unpack and run.  No JMX access with out-of-the-box
James either.  JConsole hangs for about a minute, and gives me an error
that it can't connect.

Then I made one change to log4j.properties and bumped everything to
DEBUG.  Thankfully, the one log file that actually works in this release
is James-server.log.  I see the following error when James is apparently
starting up JMX.  The 172.31 IP address it's failing on is my EC2
public IP.  But the jmx.properties file says to connect to
127.0.0.1:.  I never explicitly configured the 172.31... address
anywhere in James.

What's causing this:

INFO  04:29:47,638 |
org.springframework.remoting.rmi.RmiRegistryFactoryBean | Looking for
RMI registry at port '', using custom socket factory
DEBUG 04:29:47,642 |
org.springframework.remoting.rmi.RmiRegistryFactoryBean | RMI registry
access threw exception
java.rmi.ConnectException: Connection refused to host: 172.31.32.236;
nested exception is:
     java.net.ConnectException: Connection refused (Connection refused)
     at
sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
     at
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
     at
sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
     at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:338)
     at
sun.rmi.registry.RegistryImpl_Stub.list(RegistryImpl_Stub.java:85

Re: JMX Access?

[Jerry Malcolm]

Sai,

Thanks for the quick response.  But in the latest 3.3.0 download, there 
is not a james.sh file.  I've been using run.sh.  There is a 
james-cli.sh.  But I have not been using it.  But both the run.sh and 
james-cli.sh files have those two parameters already specified in the 
JAVA command.  So I don't think that could be what's wrong.


Any idea what is causing the exception?

And just out of curiosity, if you are on James 3.3.0, do any of the log 
files in the logs directory have any data in them?  Even out of the box, 
with every log level set to DEBUG, for me the only file that doesn't 
have zero bytes is james-server-log.  My bigger problem than JMX is that 
all of the logging data is being discarded.


Jerry

On 9/26/2019 12:21 AM, Saibabu Vallurupalli wrote:

Hi Jerry,

you need to add JVM options before starting the James server. See the
two parameters below:
Edit james.sh in vi editor or whichever you are comfortable with and
then add  "-Dcom.sun.management.jmxremote=true" and
"-Dcom.sun.management.jmxremote.authenticate=false", see below the way
how I did.

case "$1" in
 start)
 echo Starting James
 nohup java -classpath
"james-server-jpa-guice.jar:james-server-jpa-guice.lib/*:james-server-jpa-guice.lib"
-javaagent:james-server-jpa-guice.lib/openjpa-3.0.0.jar
-Dcom.sun.management.jmxremote=true
-Dcom.sun.management.jmxremote.authenticate=false
-Dlogback.configurationFile=conf/logback.xml -Dworking.directory=.
org.apache.james.JPAJamesServerMain > /dev/null 2>&1  &

Do the same for restart) option too just in case if you restart you
may loose JMX option.

Then the JMX gets enabled. The script has issues, these parameters
should be available by default.

Hope this helps and you can meet your deadline now :-)

Let me know how it goes.

Thanks,
Sai

On Thu, Sep 26, 2019 at 1:11 AM Jerry Malcolm  wrote:

I figure if I keep asking questions, sooner or later I'm going to hit
one that SOMEBODY can help me with  The issues keep backing up.  And
my deadline is way too close Is anyone using James 3.3.0?  Yet I'm
the only one having these out-of-the-box problems?  This migration has
been an absolute nightmare.

I still have absolutely NO way to see any logging data for the mailets,
smtp, imap, etc logs.   So I decided to use JMX to see if there were any
MBeans exposed that might help me figure out why there is no logging.
And, of course with the way everything else has gone, I can't connect to
James JMX.  JConsole will not connect.  It works fine accessing Tomcat
using the same approach. But nothing with James.  I'm using a PuTTY
tunnel directly to the Amazon EC2 instance.

I went back to an absolute pristine install of JAMES 3.3.0 right out of
the downloaded zip file.  I made absolutely zero changes to the
install/config.  Just unpack and run.  No JMX access with out-of-the-box
James either.  JConsole hangs for about a minute, and gives me an error
that it can't connect.

Then I made one change to log4j.properties and bumped everything to
DEBUG.  Thankfully, the one log file that actually works in this release
is James-server.log.  I see the following error when James is apparently
starting up JMX.  The 172.31 IP address it's failing on is my EC2
public IP.  But the jmx.properties file says to connect to
127.0.0.1:.  I never explicitly configured the 172.31... address
anywhere in James.

What's causing this:

INFO  04:29:47,638 |
org.springframework.remoting.rmi.RmiRegistryFactoryBean | Looking for
RMI registry at port '', using custom socket factory
DEBUG 04:29:47,642 |
org.springframework.remoting.rmi.RmiRegistryFactoryBean | RMI registry
access threw exception
java.rmi.ConnectException: Connection refused to host: 172.31.32.236;
nested exception is:
  java.net.ConnectException: Connection refused (Connection refused)
  at
sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
  at
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
  at
sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
  at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:338)
  at
sun.rmi.registry.RegistryImpl_Stub.list(RegistryImpl_Stub.java:85)
  at
org.springframework.remoting.rmi.RmiRegistryFactoryBean.testRegistry(RmiRegistryFactoryBean.java:281)
  at
org.springframework.remoting.rmi.RmiRegistryFactoryBean.getRegistry(RmiRegistryFactoryBean.java:222)
  at
org.springframework.remoting.rmi.RmiRegistryFactoryBean.getRegistry(RmiRegistryFactoryBean.java:193)
  at
org.springframework.remoting.rmi.RmiRegistryFactoryBean.afterPropertiesSet(RmiRegistryFactoryBean.java:164)
  at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1573)
  at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.

JMX Access?

[Jerry Malcolm]

I figure if I keep asking questions, sooner or later I'm going to hit 
one that SOMEBODY can help me with  The issues keep backing up.  And 
my deadline is way too close Is anyone using James 3.3.0?  Yet I'm 
the only one having these out-of-the-box problems?  This migration has 
been an absolute nightmare.


I still have absolutely NO way to see any logging data for the mailets, 
smtp, imap, etc logs.   So I decided to use JMX to see if there were any 
MBeans exposed that might help me figure out why there is no logging.   
And, of course with the way everything else has gone, I can't connect to 
James JMX.  JConsole will not connect.  It works fine accessing Tomcat 
using the same approach. But nothing with James.  I'm using a PuTTY 
tunnel directly to the Amazon EC2 instance.


I went back to an absolute pristine install of JAMES 3.3.0 right out of 
the downloaded zip file.  I made absolutely zero changes to the 
install/config.  Just unpack and run.  No JMX access with out-of-the-box 
James either.  JConsole hangs for about a minute, and gives me an error 
that it can't connect.


Then I made one change to log4j.properties and bumped everything to 
DEBUG.  Thankfully, the one log file that actually works in this release 
is James-server.log.  I see the following error when James is apparently 
starting up JMX.  The 172.31 IP address it's failing on is my EC2 
public IP.  But the jmx.properties file says to connect to 
127.0.0.1:.  I never explicitly configured the 172.31... address 
anywhere in James.


What's causing this:

INFO  04:29:47,638 | 
org.springframework.remoting.rmi.RmiRegistryFactoryBean | Looking for 
RMI registry at port '', using custom socket factory
DEBUG 04:29:47,642 | 
org.springframework.remoting.rmi.RmiRegistryFactoryBean | RMI registry 
access threw exception
java.rmi.ConnectException: Connection refused to host: 172.31.32.236; 
nested exception is:

    java.net.ConnectException: Connection refused (Connection refused)
    at 
sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
    at 
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
    at 
sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)

    at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:338)
    at 
sun.rmi.registry.RegistryImpl_Stub.list(RegistryImpl_Stub.java:85)
    at 
org.springframework.remoting.rmi.RmiRegistryFactoryBean.testRegistry(RmiRegistryFactoryBean.java:281)
    at 
org.springframework.remoting.rmi.RmiRegistryFactoryBean.getRegistry(RmiRegistryFactoryBean.java:222)
    at 
org.springframework.remoting.rmi.RmiRegistryFactoryBean.getRegistry(RmiRegistryFactoryBean.java:193)
    at 
org.springframework.remoting.rmi.RmiRegistryFactoryBean.afterPropertiesSet(RmiRegistryFactoryBean.java:164)
    at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1573)
    at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1511)
    at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
    at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
    at 
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
    at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
    at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
    at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
    at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:283)
    at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
    at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:620)
    at 
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:942)
    at 
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
    at 
org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:139)
    at 
org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:93)
    at 
org.apache.james.container.spring.context.JamesServerApplicationContext.(JamesServerApplicationContext.java:40)
    at 

Re: log4j Configuration

[Jerry Malcolm]

Thank you so much.  This new log4j.properties file did solve the problem 
that the logs were not getting written.   When I finally saw the mailet 
log, I realized that this wasn't the real problem. Even though there was 
some mailet logging data now written to the log file, my own mailet 
log(..) output was still missing.  But the good news is that I figured 
out what was happening.  I decided to start researching up the mailet 
class hierarchy.   I found the log(..) methods in GenericMailet.   Both 
variations of log(...) were flagged as "@deprecated Prefer using SLF4J 
LoggingFactory to get a Logger in each class"  (actually it should say 
LoggerFactory, not LoggingFactory... but that's not a big deal). So I 
cloned the SLF4J code from GenericMailet to one my mailets, and viola... 
I got my mailet log output.  Not sure why the GenericMailet logger was 
eating all of the output.  I tried adding a line to log4j: 
log4j.logger.org.apache.james.mailet.base=DEBUG, MAILETCONTAINER.  But 
that didn't seem to make a difference.


But the GenericMailet's log(..) problem is not critical to me anymore 
assuming that the SLF4J updates to all of my mailets work as the test did.


Thanks.

Jerry

On 9/26/2019 2:53 AM, Tellier Benoit wrote:

Check https://github.com/linagora/james-project/pull/2718

This should solve your issue.

On 20/09/2019 09:18, Jerry Malcolm wrote:

Any chance I could get someone to take a look at this problem? There's
no way for me to move forward until I can see my mailet logs.  I tried
to dig into slf4j and log4j.  But I just don't have enough background in
that to figure out what is happening.  And since I don't have a full
james set up on an IDE where I can set breakpoints, I have no way to
figure out why all log data is just being discarded regardless of the
log4j.properties settings.  With the appropriate tools it seems like
it'd be pretty easy to see why log4j is not writing any data.  But I
just don't have the tools. Can somebody in James development please take
a few minutes and see if they figure it out?  I'm sure it's probably
something trivial as properties file settings.  But without some
guidance, I'm shooting in the dark.

Help, please?

Jerry

On 9/18/2019 10:18 AM, Jerry Malcolm wrote:

Without a way to see my mailet log data, I have no way of knowing if
my mailets were even called, if they succeeded, if they failed. Right
now, all I know is they went into a processor and disappeared.  This
is a serious problem for me.  I really need to have a way to see this
data.  If it's a log4j.properties change, I just need to know what to
change.  I'm dead in the water.


On 9/17/2019 11:08 PM, Jerry Malcolm wrote:

I added the transport line as you suggested and sent a test email
inbound.  The console showed it coming in, as does james-server.log.
But still, the only log file in the logs directory that is not empty
is james-server. Nothing is going to any of these files.

On 9/17/2019 10:49 PM, Tellier Benoit wrote:

I inlined my answers...

Regards,

On 18/09/2019 10:32, Jerry Malcolm wrote:

Thanks for the reply.  But I did not use my old log4j.properties.  I
copied the one from the downloaded 3.3.0 zip file. Although, it looks
almost identical to the beta5 one.  Any chance the wrong one is
included
in the package?

I think the default log4j file was never updated.

Contribution welcomed BTW...


I'm fine with logger-per-mailet. However, I'm hoping there's a way to
see the flow for an email item.  It's just as important at times to
see
that various matchers passed or didn't pass on an item as it goes
through the flow.  Is there still a logger that shows the flow?

The approach regarding this is to use the MDC for doing some
contextualized logging (SLF4J API)

Within James guice application, we rely on logback, and a
"ElasticSearch" appender. Logs can then be filtered by 'mail' property.

You will then see all of the logs related to the processing of that
very
email.

I'm not a log4j expert but there should be a way of doing something
somehow similar...


Bottom line, though, I don't know what to put in the log4j file to
make
the mailet logs show up.  Do you have a link handy to the correct
log4j
in github?

I don't think we have...

I'm not an expert here but let's play a bit:

In log4j.properties I would replace line 124

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER

By

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER
log4j.logger.org.apache.james.transport=INFO, MAILETCONTAINER

(Rationals: all mailets are located within the
"org.apache.james.transport.mailets" &
"org.apache.james.transport.matchers" packages...


Thx


On 9/17/2019 10:23 PM, Tellier Benoit wrote:

Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered tha

3.4.0 AWS S3 Storage

[Jerry Malcolm]

I moved to 3.4.0 today with no major hurdles.  I'm thrilled with the 
announcement about using AWS S3 for blob storage.  This is huge for me.  
I figure it'll reduce my 80+gB SQL database by about 95%.  That will 
help with storage cost as well as migration, backup, and just about 
everything dealing with mail.


I need to be fully up and running with my current configuration before I 
start attempting anything new. But I'd really like to start reading 
about it and figuring out a path to get there.  I know it was just 
announced today.  So if docs are still thin, I understand.  But  I 
couldn't find a link to documentation on how to setup/configure/use the 
S3 function.  Sorry if I overlooked it. Just wanted to know if there is 
a document, a sample, or anything that'll help my education process on this?


Thanks for adding this.  It's huge!

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

(Resolved): Moving Servers Wipes Out iPhone's Mail Accounts

[Jerry Malcolm]

Finally, after 2 weeks, iPhones have all of their mail.  The problem was 
that my Lucene index was corrupted. Or to be more precise, only emails 
from the last two weeks since I first started up James on the new 
environment were indexed. Even after renaming (effectively removing) the 
/var/store/lucene folder, the index would not fully rebuild.  Same 
problem... but worse now... now it showed only mail that came in over 
the last 10 minutes since I cleaned the lucene index folder and lost the 
last 2 weeks' mail that actually was indexed.  On a whim, I changed 
indexer.xml to use lazyIndex instead of luceneIndex... voila all of 
the folders were fully indexed, and iPhone has all of the mail for each 
folder.  Couldn't find much documentation on lazyIndex vs. luceneIndex.  
But at this point, I think I'll take having mail back on my client's 
iPhones over any downside to using lazyIndex. But educate me if 
there are serious downsides to using lazyIndex.


Thunderbird populates folders using a different imap command which, 
apparently, goes straight to the JPA database instead of using the 
Lucene index.  So that's why the problem didn't appear in TBird.


I'm going to start another thread in the dev group specifically to 
discuss the Lucene issue.


Jerry

On 10/31/2019 12:35 PM, Jerry Malcolm wrote:
(I'm moving this thread to the dev forum since it appears now to not 
be a user problem)


More data now... Since iPhone doesn't load the correct email list and 
TBird does, I traced the precise same scenario with the same email 
account with TBird as the previous iPhone trace in the post below.  As 
I suspected, in order to do an initial full sync on a folder, TBird is 
asking for the list of UIDs in a different way. Note from the two traces:


TBird: UID fetch 1:* (FLAGS)
    Response: 1 FETCH (FLAGS (\\Seen) UID 5801)  (...followed by many 
other UIDs including the ones in iPhone search below)

---
iPhone: UID SEARCH RETURN (COUNT) 1:* NOT DELETED
 Response: ESEARCH (TAG "39") UID ALL 7776,7955:7970,7975:8033

The two queries above are for the same email account/folder. Again, 
I'm not an IMAP expert.  But it appears that both should return the 
same list of UIDs even though they ask for it differently.


Summary... the TBird query gets all UIDs for the folder starting at 
5801 as expected.  The iPhone query gets a subset of UIDs starting at 
7776.  This continues to support my current theory that the code that 
processes UID SEARCH RETURN (COUNT) 1:* is not correct.


If someone knows what class processes these imap queries, please let 
me know.  I'm going to dig into the code now.  But a few pointers to 
the package/classes would be helpful.


Thunderbird trace:

127 Request: 19 select "INBOX"
129,Response: * FLAGS (\\Answered \\Deleted \\Draft \\Flagged \\Seen 
"MailFlagBit1 "MailFlagBit0 NonJunk "label3 "label1 "Forwarded)

130,Response: 19 OK [READ-WRITE] SELECT completed.
132 Request: 20 getquotaroot "INBOX"
134 Response: * QUOTAROOT "INBOX" #private@malcolms.com
135,Response: 20 OK GETQUOTAROOT completed.
137,Request: 21 UID fetch 1:* (FLAGS)
139,Response: * 1 FETCH (FLAGS (\\Seen) UID 5801)
140,Response: * 2 FETCH (FLAGS (\\Seen) UID 5802)
141,Response: * 4 FETCH (FLAGS (\\Seen) UID 5804)

On 10/29/2019 3:12 PM, Jerry Malcolm wrote:
Ok, I need an IMAP expert Below is a very brief trace of the 
communications between iPhone mail and JAMES (3.4).  I completely 
deleted an account on my iPhone, then recreated it while in airplane 
mode to make sure I didn't miss any communications in my trace.  I 
started the trace, exited airplane mode and let the iPhone do an 
initial sync with the account.  The inbox folder in this account has 
over 1000 emails going back to early 2019.


I'm not an expert in IMAP.  But it appears that the iPhone mail app 
requests all of the emails 1:* (see line 812), but JAMES returns a 
single id plus two ranges (line 813).  But the total count JAMES 
reports is nowhere near the full 1000.  Subsequently (line 822), 
iPhone requests the emails JAMES told it about in line 813.  From 
what I can tell, the problem is in line 813. JAMES did not report all 
of the actual mail that is in the INBOX, which is exactly what I'm 
seeing on the phone.  Shouldn't JAMES return all 1000+ email ids that 
exist in INBOX?  Or am I reading the IMAP trace incorrectly?


Note that the only emails JAMES reports came in AFTER I migrated to 
the new server and to the James 3.4 from v3b5.  It appears that there 
is something different about pre-existing emails in the mailbox 
folder that is causing JAMES not to recognize them. But this problem 
ONLY exists on the iPhone mail app. Thunderbird gets all of the mail 
for the inbox.  So is iPhone must be sending a slightly different 
request command syntax than other clients do (??)  Just speculating


 798 IMAP Request: 37 SELECT INBOX
 802 IMAP Response: 37 OK [READ

Re: Setting "administratorid"?

[Jerry Malcolm]

Yes, I will definitely contribute to this.  A whole section on how to 
use delegation will be very helpful to other users.


On 11/4/2019 3:38 AM, Tellier Benoit wrote:

I just created an issue regarding this.

https://issues.apache.org/jira/browse/JAMES-2963

Jerry, would you have time to contribute this missing documentation?

Regards,

Benoit

On 04/11/2019 16:27, Tellier Benoit wrote:

Answers inlined,

Regards,

Benoit

On 30/10/2019 11:37, Jerry Malcolm wrote:

"enableVirtualHosting" is another config parameter that's set in this
same method.  So I decided to search around and see where that is set.
I found it, and just taking a wild guess, is this correct?

Maybe we need to further explain this.

When virtual hosting is enabled, usernames are composed of a local part
and a domain part. al...@domain.tld and al...@company.org are two
distinct users, with different login and distinct mailboxes. Creating a
user without domain part is forbidden.


When virtual hosting is turned of, usernames are only composed of a
local part. All domains handled by James (check domain list) can be used
as domain part of their mail address. al...@domain.tld and
al...@company.org are two mail address belonging to a same "alice" user,
with same login and mailboxes. Creating users with domain parts is
forbidden.

I will add the following indication within the documentation as it might
get really useful to understand this. Thanks for highlighting this weak
point!



     MD5

Try more secure algorithms, you should give a go to SHA-512.

We should update default configuration accordingly as MD5 hashing is not
acceptable.


     true
ad...@myhost.com


This took a whole lot of digging to figure out.  I really think the
default usersrepository.xml should have at least commented-out lines
defining where/how to set the administrator id.

I would be glad to have you contribute this!


I'll add it to my list if this is indeed the correct implementation.

Jerry


On 10/29/2019 11:27 PM, Jerry Malcolm wrote:

I see in AbstractUsersRepository where "adminstratorid" is pulled from
somewhere in the configuration:

     administratorId =
Optional.ofNullable(configuration.getString("administratorId"));

But I can't find any place to set it in the shipped configuration
files.  Where is the administrator id supposed to be defined, and what
is the correct syntax to define it?

Thx

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org

Re: AdministratorId Documentation

[Jerry Malcolm]

This is a followup to an earlier discussion.  I have a very large number 
of imap utilities that I use to maintain my clients' accounts and 
mailboxes.  They were all written using javax.mail's version of the IMAP 
interface.   Your code reference below was very helpful.  But I realized 
I would have to change all of my javax.mail code to commons.net.  It was 
going to be worth it if I could get the delegate function.  But I still 
was dreading the amount of migration/testing work.


So I did a bit of playing around looking at what was really 
happening to send the admin id, admin pw, and target account in the 
login.  Turns out in the javax.mail package, if you use:


      store.connect(host, port, user + "\0" + adminId, adminPW);

it logs in correctly with delegation.  So as much as I appreciate your 
code :-),  a one-line change to my javax.mail login code won out over 
almost a full rewrite of all my utilities.


Jerry


On 9/12/2019 11:01 PM, Tellier Benoit wrote:

If you are using commons-net, I succeeded to write:

https://gist.github.com/chibenwa/abd12fd6c0b06cadd1de591e3ac792b9

That should be helping you!

On 13/09/2019 10:40, Tellier Benoit wrote:

Wich library are you using?

On 13/09/2019 10:33, Jerry Malcolm wrote:

Thanks for the info, Tellier.  I kinda lost you on the mpt tests... I
was looking for how to change the following code to include an
administrator id.  I only have one field in the store.connect() method
for a user id, but I have an administrator id and the userid for the
target mailbox.  How do I pass in both?

    try
    {
   session = Session.getInstance(props, null);
   session.setDebug(debug);
   store = session.getStore(protocol);
   store.connect(host, port, user, password);
    }
    catch( Exception e)
    {
   log( "IMAPClient.connect() exception: " + e );
   throw(e);
    }

On 9/12/2019 10:10 PM, Tellier Benoit wrote:

Hello Jerry,

With the `administratorId`, you are able to use IMAP impersonation.

IE to log in as another user.

You should define it within usersrepository.xml. To see related "reading
config" code: AbstractUsersRepository is the way to go.

Agree that this needs example and documentation. I will do it straight
away.

For your IMAP call, MPT tests are going to be a gold mine, presenting
you the exact syntax.
mpt/impl/imap-mailbox/core/src/main/resources/org/apache/james/imap/scripts/AuthenticatePlain.test


Contains:

C: 0007 AUTHENTICATE "PLAIN" {36+}
# delegate\0imapuser\0password
C: ZGVsZWdhdGUAaW1hcHVzZXIAcGFzc3dvcmQ=
S: 0007 OK AUTHENTICATE completed.

# Ensure we are delegate
C: 0008 SELECT delegate
SUB {
S: \* FLAGS .*
S: \* .* EXISTS
S: \* .* RECENT
S: \* OK \[UIDVALIDITY .*\] UIDs valid
S: \* OK \[PERMANENTFLAGS .*\] Limited
S: \* OK \[HIGHESTMODSEQ .*\] Highest
S: \* OK \[UIDNEXT .*\] Predicted next UID
}
S: 0008 OK \[READ-WRITE\] SELECT completed\.

Regarding your development issues, I'm really sorry. My environment is
based on IntelliJ, and I encounter no issue.

To be able to compile, I use maven 3.6.2 with JDK 11.

Best regards,

Benoit Tellier


On 13/09/2019 09:53, Jerry Malcolm wrote:

Since it appears I'm not going to be able to build James 3.3.0 in the
foreseeable future, I'm now moving to plan d, e, f, or whatever... .I've
lost count.

My goal now is to assess my possibilities of using James 3.3.0 binaries
as-is and discarding the functionality I had hoped to re-add to 3.3.0
from 30b5 or moving the functionality to mailets that I can compile
outside of doing a full James build.

One thing I had added to beta5 was the ability to have a super-user,
master-key id that could be used in my custom external imap mailbox
maintenance utilities.  I've spent several hours digging through the
3.3.0 source code, and I see code referencing "administratorId".  From
what I can deduce from the code, the Authenticator and Authorizator
classes seem to accept this administrator id as the equivalent of the
master key I added to b5.  So can I access UserABC's mailbox through
imap but use my administratorId and password?  If that's the case, then
that's exactly what I need.  The problem is I can't find any
documentation on how to use administratorId.  Where do I define it?  And
how do I pass it on imap calls using the javax.mail.Store and other
classes in javax.mail that I use to access imap?

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr..

Re: Setting "administratorid"?

[Jerry Malcolm]

Hi Benoit,

I wasn't actually questioning enableVirtualHosting.  I was simply trying 
to figure out where to put the admin id.  I noticed that the adminId was 
pulled from the configuration in the next line after the 
enableVirtualHosting flag was pulled.  So I was just deducing that 
perhaps they should be side by side in the same config file.  I tried 
that, and it worked.


My only recommendation was that we add a 'dummy' commented-only 
 line to the default config file, so others can easily 
find where to set it.


Jerry

On 11/4/2019 3:27 AM, Tellier Benoit wrote:

Answers inlined,

Regards,

Benoit

On 30/10/2019 11:37, Jerry Malcolm wrote:

"enableVirtualHosting" is another config parameter that's set in this
same method.  So I decided to search around and see where that is set.
I found it, and just taking a wild guess, is this correct?

Maybe we need to further explain this.

When virtual hosting is enabled, usernames are composed of a local part
and a domain part. al...@domain.tld and al...@company.org are two
distinct users, with different login and distinct mailboxes. Creating a
user without domain part is forbidden.


When virtual hosting is turned of, usernames are only composed of a
local part. All domains handled by James (check domain list) can be used
as domain part of their mail address. al...@domain.tld and
al...@company.org are two mail address belonging to a same "alice" user,
with same login and mailboxes. Creating users with domain parts is
forbidden.

I will add the following indication within the documentation as it might
get really useful to understand this. Thanks for highlighting this weak
point!



     MD5

Try more secure algorithms, you should give a go to SHA-512.

We should update default configuration accordingly as MD5 hashing is not
acceptable.


     true
ad...@myhost.com


This took a whole lot of digging to figure out.  I really think the
default usersrepository.xml should have at least commented-out lines
defining where/how to set the administrator id.

I would be glad to have you contribute this!


I'll add it to my list if this is indeed the correct implementation.

Jerry


On 10/29/2019 11:27 PM, Jerry Malcolm wrote:

I see in AbstractUsersRepository where "adminstratorid" is pulled from
somewhere in the configuration:

     administratorId =
Optional.ofNullable(configuration.getString("administratorId"));

But I can't find any place to set it in the shipped configuration
files.  Where is the administrator id supposed to be defined, and what
is the correct syntax to define it?

Thx

Jerry


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org