Re: log4j Configuration

2019-09-17 Thread Jerry Malcolm
I added the transport line as you suggested and sent a test email 
inbound.  The console showed it coming in, as does james-server.log.  
But still, the only log file in the logs directory that is not empty is 
james-server.  Nothing is going to any of these files.


On 9/17/2019 10:49 PM, Tellier Benoit wrote:

I inlined my answers...

Regards,

On 18/09/2019 10:32, Jerry Malcolm wrote:

Thanks for the reply.  But I did not use my old log4j.properties.  I
copied the one from the downloaded 3.3.0 zip file.  Although, it looks
almost identical to the beta5 one.  Any chance the wrong one is included
in the package?

I think the default log4j file was never updated.

Contribution welcomed BTW...


I'm fine with logger-per-mailet.  However, I'm hoping there's a way to
see the flow for an email item.  It's just as important at times to see
that various matchers passed or didn't pass on an item as it goes
through the flow.  Is there still a logger that shows the flow?

The approach regarding this is to use the MDC for doing some
contextualized logging (SLF4J API)

Within James guice application, we rely on logback, and a
"ElasticSearch" appender. Logs can then be filtered by 'mail' property.

You will then see all of the logs related to the processing of that very
email.

I'm not a log4j expert but there should be a way of doing something
somehow similar...


Bottom line, though, I don't know what to put in the log4j file to make
the mailet logs show up.  Do you have a link handy to the correct log4j
in github?

I don't think we have...

I'm not an expert here but let's play a bit:

In log4j.properties I would replace line 124

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER

By

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER
log4j.logger.org.apache.james.transport=INFO, MAILETCONTAINER

(Rationals: all mailets are located within the
"org.apache.james.transport.mailets" &
"org.apache.james.transport.matchers" packages...


Thx


On 9/17/2019 10:23 PM, Tellier Benoit wrote:

Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered that behavior, and create one logger per
mailet class for bundled mailets.

In the end, this enabled simpler logging filtering - on a per mailet
basis, got rid of the imperfect logging facade exposed in the mailet API
(no placeholder for instance), and also killed some "debug"
configuration parameters.

I believe that your log4j configuration file inherited from beta5 needs
to be revisited to adapt these changes.

Hope it helps.

Benoit

On 18/09/2019 09:10, Jerry Malcolm wrote:

Not good news for me, however.  My mailet flow that worked in v3b5 is
crashing now, and none of the log output I'm writing from my mailets is
showing up anywhere.  I've GOT to see that log info some place in order
to debug both my mailets as well as my mailet flow that is somehow no
longer compatible.  All I get is a 'mail received' message, and an error
saying it can't launch one of my mailets because there is a recursive
loop in my flow (there is not, or at least there was not when it running
in v3b5).

There's got to be a way to see mailet log data.  Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:

It’s not you. I have the same issue. I think those other log files are
just there for backwards compatibility to make users feel good, since
they never seem to be written to. “No news is good news” as the saying
goes.

Sent from my iPhone


On Sep 17, 2019, at 4:34 PM, Jerry Malcolm 
wrote:

For the problem du jour when I start JAMES, all of the expected
log files get created.  The main log output  goes to the console and
to the james-server.log file as expected.  No problem there.  But the
remainder of the log files never get touched.  No matter what
happens, every file in the log folder other than james-server.log is
empty.  I've tried changing log4j.properties and setting everything
to DEBUG level.  The console log dumps a bunch more info while in
DEBUG mode (which confirms I'm changing the correct log4j file).  But
every other log file is still empty.  I used to see my mailet output
in the main console and james-server.log in b5.  I'm not seeing any
of that anywhere. Again, I've played around with log4j.properties (I
made sure I was using the latest 3.3.0 log4j.properties file).
Nothing.  What am I doing wrong?  How can I get log data to start
going into those other files?


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: log4j Configuration

2019-09-17 Thread Jerry Malcolm
I'll try that shortly.  But that doesn't explain why all of the logger 
output files are empty, no matter what the log4j settings are.  This was 
after 30 minutes of running with every log level throughout 
log4j.properties set to DEBUG.  Shouldn't at least some of these have 
some sort of log data, such as imapserver.log, smtpserver.log, etc?


 9/17/2019  21:33   0  dnsservice.log
 9/17/2019  21:33   0  domainlist.log
 9/17/2019  21:33   0  fetchmail.log
 9/17/2019  21:33   0  imapserver.log
 9/17/2019  21:33   0  lmtpserver.log
 9/17/2019  21:33   0  mailboxmanager.log
 9/17/2019  21:33   0  mailetcontainer.log
 9/17/2019  21:33   0  mailqueuefactory.log
 9/17/2019  21:33   0  mailrepositorystore.log
 9/17/2019  21:33   0  pop3server.log
 9/17/2019  21:33   0  usersrepository.log
 9/17/2019  21:33   0  virtualusertable.log

On 9/17/2019 10:49 PM, Tellier Benoit wrote:

I inlined my answers...

Regards,

On 18/09/2019 10:32, Jerry Malcolm wrote:

Thanks for the reply.  But I did not use my old log4j.properties.  I
copied the one from the downloaded 3.3.0 zip file.  Although, it looks
almost identical to the beta5 one.  Any chance the wrong one is included
in the package?

I think the default log4j file was never updated.

Contribution welcomed BTW...


I'm fine with logger-per-mailet.  However, I'm hoping there's a way to
see the flow for an email item.  It's just as important at times to see
that various matchers passed or didn't pass on an item as it goes
through the flow.  Is there still a logger that shows the flow?

The approach regarding this is to use the MDC for doing some
contextualized logging (SLF4J API)

Within James guice application, we rely on logback, and a
"ElasticSearch" appender. Logs can then be filtered by 'mail' property.

You will then see all of the logs related to the processing of that very
email.

I'm not a log4j expert but there should be a way of doing something
somehow similar...


Bottom line, though, I don't know what to put in the log4j file to make
the mailet logs show up.  Do you have a link handy to the correct log4j
in github?

I don't think we have...

I'm not an expert here but let's play a bit:

In log4j.properties I would replace line 124

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER

By

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER
log4j.logger.org.apache.james.transport=INFO, MAILETCONTAINER

(Rationals: all mailets are located within the
"org.apache.james.transport.mailets" &
"org.apache.james.transport.matchers" packages...


Thx


On 9/17/2019 10:23 PM, Tellier Benoit wrote:

Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered that behavior, and create one logger per
mailet class for bundled mailets.

In the end, this enabled simpler logging filtering - on a per mailet
basis, got rid of the imperfect logging facade exposed in the mailet API
(no placeholder for instance), and also killed some "debug"
configuration parameters.

I believe that your log4j configuration file inherited from beta5 needs
to be revisited to adapt these changes.

Hope it helps.

Benoit

On 18/09/2019 09:10, Jerry Malcolm wrote:

Not good news for me, however.  My mailet flow that worked in v3b5 is
crashing now, and none of the log output I'm writing from my mailets is
showing up anywhere.  I've GOT to see that log info some place in order
to debug both my mailets as well as my mailet flow that is somehow no
longer compatible.  All I get is a 'mail received' message, and an error
saying it can't launch one of my mailets because there is a recursive
loop in my flow (there is not, or at least there was not when it running
in v3b5).

There's got to be a way to see mailet log data.  Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:

It’s not you. I have the same issue. I think those other log files are
just there for backwards compatibility to make users feel good, since
they never seem to be written to. “No news is good news” as the saying
goes.

Sent from my iPhone


On Sep 17, 2019, at 4:34 PM, Jerry Malcolm 
wrote:

For the problem du jour when I start JAMES, all of the expected
log files get created.  The main log output  goes to the console and
to the james-server.log file as expected.  No problem there.  But the
remainder of the log files never get touched.  No matter what
happens, every file in the log folder other than james-server.log is
empty.  I've tried changing log4j.properties and setting everything
to DEBUG level.  The console log dumps a bunch more info while in
DEBUG mode (which confirms I'm changing the correct log4j file).  But
every other log file is still empty.  I used to see my mailet output
in 

Re: log4j Configuration

2019-09-17 Thread Tellier Benoit
I inlined my answers...

Regards,

On 18/09/2019 10:32, Jerry Malcolm wrote:
> Thanks for the reply.  But I did not use my old log4j.properties.  I
> copied the one from the downloaded 3.3.0 zip file.  Although, it looks
> almost identical to the beta5 one.  Any chance the wrong one is included
> in the package?

I think the default log4j file was never updated.

Contribution welcomed BTW...

> I'm fine with logger-per-mailet.  However, I'm hoping there's a way to
> see the flow for an email item.  It's just as important at times to see
> that various matchers passed or didn't pass on an item as it goes
> through the flow.  Is there still a logger that shows the flow?

The approach regarding this is to use the MDC for doing some
contextualized logging (SLF4J API)

Within James guice application, we rely on logback, and a
"ElasticSearch" appender. Logs can then be filtered by 'mail' property.

You will then see all of the logs related to the processing of that very
email.

I'm not a log4j expert but there should be a way of doing something
somehow similar...

> Bottom line, though, I don't know what to put in the log4j file to make
> the mailet logs show up.  Do you have a link handy to the correct log4j
> in github?

I don't think we have...

I'm not an expert here but let's play a bit:

In log4j.properties I would replace line 124

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER

By

log4j.logger.org.apache.james.mailprocessor=INFO, MAILETCONTAINER
log4j.logger.org.apache.james.transport=INFO, MAILETCONTAINER

(Rationals: all mailets are located within the
"org.apache.james.transport.mailets" &
"org.apache.james.transport.matchers" packages...

> Thx
> 
> 
> On 9/17/2019 10:23 PM, Tellier Benoit wrote:
>> Maybe I could help on that one:
>>
>> The mailet logging had been migrated to a standard SLF4J approach.
>>
>> Before that, all mailet logs went though a single logger, exposed as
>> part of the MailetContext object.
>>
>> Before 3.0.0 release we altered that behavior, and create one logger per
>> mailet class for bundled mailets.
>>
>> In the end, this enabled simpler logging filtering - on a per mailet
>> basis, got rid of the imperfect logging facade exposed in the mailet API
>> (no placeholder for instance), and also killed some "debug"
>> configuration parameters.
>>
>> I believe that your log4j configuration file inherited from beta5 needs
>> to be revisited to adapt these changes.
>>
>> Hope it helps.
>>
>> Benoit
>>
>> On 18/09/2019 09:10, Jerry Malcolm wrote:
>>> Not good news for me, however.  My mailet flow that worked in v3b5 is
>>> crashing now, and none of the log output I'm writing from my mailets is
>>> showing up anywhere.  I've GOT to see that log info some place in order
>>> to debug both my mailets as well as my mailet flow that is somehow no
>>> longer compatible.  All I get is a 'mail received' message, and an error
>>> saying it can't launch one of my mailets because there is a recursive
>>> loop in my flow (there is not, or at least there was not when it running
>>> in v3b5).
>>>
>>> There's got to be a way to see mailet log data.  Tellier?
>>>
>>>
>>> On 9/17/2019 9:02 PM, Garry Hurley wrote:
 It’s not you. I have the same issue. I think those other log files are
 just there for backwards compatibility to make users feel good, since
 they never seem to be written to. “No news is good news” as the saying
 goes.

 Sent from my iPhone

> On Sep 17, 2019, at 4:34 PM, Jerry Malcolm 
> wrote:
>
> For the problem du jour when I start JAMES, all of the expected
> log files get created.  The main log output  goes to the console and
> to the james-server.log file as expected.  No problem there.  But the
> remainder of the log files never get touched.  No matter what
> happens, every file in the log folder other than james-server.log is
> empty.  I've tried changing log4j.properties and setting everything
> to DEBUG level.  The console log dumps a bunch more info while in
> DEBUG mode (which confirms I'm changing the correct log4j file).  But
> every other log file is still empty.  I used to see my mailet output
> in the main console and james-server.log in b5.  I'm not seeing any
> of that anywhere. Again, I've played around with log4j.properties (I
> made sure I was using the latest 3.3.0 log4j.properties file).
> Nothing.  What am I doing wrong?  How can I get log data to start
> going into those other files?
>
>
> -
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
>
 -
 To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
 For additional commands, e-mail: server-user-h...@james.apache.org

>>> 

Re: log4j Configuration

2019-09-17 Thread Jerry Malcolm
Thanks for the reply.  But I did not use my old log4j.properties.  I 
copied the one from the downloaded 3.3.0 zip file.  Although, it looks 
almost identical to the beta5 one.  Any chance the wrong one is included 
in the package?


I'm fine with logger-per-mailet.  However, I'm hoping there's a way to 
see the flow for an email item.  It's just as important at times to see 
that various matchers passed or didn't pass on an item as it goes 
through the flow.  Is there still a logger that shows the flow?


Bottom line, though, I don't know what to put in the log4j file to make 
the mailet logs show up.  Do you have a link handy to the correct log4j 
in github?


Thx


On 9/17/2019 10:23 PM, Tellier Benoit wrote:

Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered that behavior, and create one logger per
mailet class for bundled mailets.

In the end, this enabled simpler logging filtering - on a per mailet
basis, got rid of the imperfect logging facade exposed in the mailet API
(no placeholder for instance), and also killed some "debug"
configuration parameters.

I believe that your log4j configuration file inherited from beta5 needs
to be revisited to adapt these changes.

Hope it helps.

Benoit

On 18/09/2019 09:10, Jerry Malcolm wrote:

Not good news for me, however.  My mailet flow that worked in v3b5 is
crashing now, and none of the log output I'm writing from my mailets is
showing up anywhere.  I've GOT to see that log info some place in order
to debug both my mailets as well as my mailet flow that is somehow no
longer compatible.  All I get is a 'mail received' message, and an error
saying it can't launch one of my mailets because there is a recursive
loop in my flow (there is not, or at least there was not when it running
in v3b5).

There's got to be a way to see mailet log data.  Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:

It’s not you. I have the same issue. I think those other log files are
just there for backwards compatibility to make users feel good, since
they never seem to be written to. “No news is good news” as the saying
goes.

Sent from my iPhone


On Sep 17, 2019, at 4:34 PM, Jerry Malcolm 
wrote:

For the problem du jour when I start JAMES, all of the expected
log files get created.  The main log output  goes to the console and
to the james-server.log file as expected.  No problem there.  But the
remainder of the log files never get touched.  No matter what
happens, every file in the log folder other than james-server.log is
empty.  I've tried changing log4j.properties and setting everything
to DEBUG level.  The console log dumps a bunch more info while in
DEBUG mode (which confirms I'm changing the correct log4j file).  But
every other log file is still empty.  I used to see my mailet output
in the main console and james-server.log in b5.  I'm not seeing any
of that anywhere. Again, I've played around with log4j.properties (I
made sure I was using the latest 3.3.0 log4j.properties file).
Nothing.  What am I doing wrong?  How can I get log data to start
going into those other files?


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: log4j Configuration

2019-09-17 Thread Tellier Benoit
Maybe I could help on that one:

The mailet logging had been migrated to a standard SLF4J approach.

Before that, all mailet logs went though a single logger, exposed as
part of the MailetContext object.

Before 3.0.0 release we altered that behavior, and create one logger per
mailet class for bundled mailets.

In the end, this enabled simpler logging filtering - on a per mailet
basis, got rid of the imperfect logging facade exposed in the mailet API
(no placeholder for instance), and also killed some "debug"
configuration parameters.

I believe that your log4j configuration file inherited from beta5 needs
to be revisited to adapt these changes.

Hope it helps.

Benoit

On 18/09/2019 09:10, Jerry Malcolm wrote:
> Not good news for me, however.  My mailet flow that worked in v3b5 is
> crashing now, and none of the log output I'm writing from my mailets is
> showing up anywhere.  I've GOT to see that log info some place in order
> to debug both my mailets as well as my mailet flow that is somehow no
> longer compatible.  All I get is a 'mail received' message, and an error
> saying it can't launch one of my mailets because there is a recursive
> loop in my flow (there is not, or at least there was not when it running
> in v3b5).
> 
> There's got to be a way to see mailet log data.  Tellier?
> 
> 
> On 9/17/2019 9:02 PM, Garry Hurley wrote:
>> It’s not you. I have the same issue. I think those other log files are
>> just there for backwards compatibility to make users feel good, since
>> they never seem to be written to. “No news is good news” as the saying
>> goes.
>>
>> Sent from my iPhone
>>
>>> On Sep 17, 2019, at 4:34 PM, Jerry Malcolm 
>>> wrote:
>>>
>>> For the problem du jour when I start JAMES, all of the expected
>>> log files get created.  The main log output  goes to the console and
>>> to the james-server.log file as expected.  No problem there.  But the
>>> remainder of the log files never get touched.  No matter what
>>> happens, every file in the log folder other than james-server.log is
>>> empty.  I've tried changing log4j.properties and setting everything
>>> to DEBUG level.  The console log dumps a bunch more info while in
>>> DEBUG mode (which confirms I'm changing the correct log4j file).  But
>>> every other log file is still empty.  I used to see my mailet output
>>> in the main console and james-server.log in b5.  I'm not seeing any
>>> of that anywhere. Again, I've played around with log4j.properties (I
>>> made sure I was using the latest 3.3.0 log4j.properties file). 
>>> Nothing.  What am I doing wrong?  How can I get log data to start
>>> going into those other files?
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>>> For additional commands, e-mail: server-user-h...@james.apache.org
>>>
>> -
>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-user-h...@james.apache.org
>>
> 
> -
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
> 

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: log4j Configuration

2019-09-17 Thread Jerry Malcolm
Not good news for me, however.  My mailet flow that worked in v3b5 is 
crashing now, and none of the log output I'm writing from my mailets is 
showing up anywhere.  I've GOT to see that log info some place in order 
to debug both my mailets as well as my mailet flow that is somehow no 
longer compatible.  All I get is a 'mail received' message, and an error 
saying it can't launch one of my mailets because there is a recursive 
loop in my flow (there is not, or at least there was not when it running 
in v3b5).


There's got to be a way to see mailet log data.  Tellier?


On 9/17/2019 9:02 PM, Garry Hurley wrote:

It’s not you. I have the same issue. I think those other log files are just 
there for backwards compatibility to make users feel good, since they never 
seem to be written to. “No news is good news” as the saying goes.

Sent from my iPhone


On Sep 17, 2019, at 4:34 PM, Jerry Malcolm  wrote:

For the problem du jour when I start JAMES, all of the expected log files 
get created.  The main log output  goes to the console and to the 
james-server.log file as expected.  No problem there.  But the remainder of the 
log files never get touched.  No matter what happens, every file in the log 
folder other than james-server.log is empty.  I've tried changing 
log4j.properties and setting everything to DEBUG level.  The console log dumps 
a bunch more info while in DEBUG mode (which confirms I'm changing the correct 
log4j file).  But every other log file is still empty.  I used to see my mailet 
output in the main console and james-server.log in b5.  I'm not seeing any of 
that anywhere. Again, I've played around with log4j.properties (I made sure I 
was using the latest 3.3.0 log4j.properties file).  Nothing.  What am I doing 
wrong?  How can I get log data to start going into those other files?


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org


-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: log4j Configuration

2019-09-17 Thread Garry Hurley
It’s not you. I have the same issue. I think those other log files are just 
there for backwards compatibility to make users feel good, since they never 
seem to be written to. “No news is good news” as the saying goes. 

Sent from my iPhone

> On Sep 17, 2019, at 4:34 PM, Jerry Malcolm  wrote:
> 
> For the problem du jour when I start JAMES, all of the expected log files 
> get created.  The main log output  goes to the console and to the 
> james-server.log file as expected.  No problem there.  But the remainder of 
> the log files never get touched.  No matter what happens, every file in the 
> log folder other than james-server.log is empty.  I've tried changing 
> log4j.properties and setting everything to DEBUG level.  The console log 
> dumps a bunch more info while in DEBUG mode (which confirms I'm changing the 
> correct log4j file).  But every other log file is still empty.  I used to see 
> my mailet output in the main console and james-server.log in b5.  I'm not 
> seeing any of that anywhere. Again, I've played around with log4j.properties 
> (I made sure I was using the latest 3.3.0 log4j.properties file).  Nothing.  
> What am I doing wrong?  How can I get log data to start going into those 
> other files?
> 
> 
> -
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
> 

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



log4j Configuration

2019-09-17 Thread Jerry Malcolm
For the problem du jour when I start JAMES, all of the expected log 
files get created.  The main log output  goes to the console and to the 
james-server.log file as expected.  No problem there.  But the remainder 
of the log files never get touched.  No matter what happens, every file 
in the log folder other than james-server.log is empty.  I've tried 
changing log4j.properties and setting everything to DEBUG level.  The 
console log dumps a bunch more info while in DEBUG mode (which confirms 
I'm changing the correct log4j file).  But every other log file is still 
empty.  I used to see my mailet output in the main console and 
james-server.log in b5.  I'm not seeing any of that anywhere. Again, 
I've played around with log4j.properties (I made sure I was using the 
latest 3.3.0 log4j.properties file).  Nothing.  What am I doing wrong?  
How can I get log data to start going into those other files?



-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: IMAP Error

2019-09-17 Thread Jerry Malcolm
Well this one wasn't as difficult as I expected.  Turns out during all 
of the transition and merging I ended up with the conf/META-INF folder 
from b5.  Since that folder contains all of the JPA stuff, 3.3.0 was not 
at all happy with v3b5 JPA files.


Just a note for continuing to document my adventures Once I swapped 
over the META-INF folder, things initially got much worse.  The JPA 
stuff is used to update all of the table schemas from 3.0 to 3.3 
version.  In the process of updating, it tries to add foreign keys to 
JAMES_MAIL_PROPERTY and JAMES_MAIL_USERFLAG tables.  Both of those 
failed miserably.  Digging into those database tables, I realized I had 
32 million (yup... 32 million) userFlag records with null MAIL_ID  and 
MAIL_UID fields that had no hope of matching up with fields in 
JAMES_MAIL.  And there were four records in JAMES_MAIL_PROPERTY that 
weren't null, but still didn't have a match in JAMES_MAIL.  So after 
finding and deleting those 4 as well as the 32 million null records, 
James was able to successfully complete the JPA startup stuff.  I guess 
this is a good thing that I'm forced to clean up problems with the 
current database.  This migration glacier is moving a millimeter at a 
time



On 9/17/2019 1:26 AM, Tellier Benoit wrote:

Hi Jerry,

To explain it shortly:

  1. In OpenJPA NamedQuery are directly attached to the POJO being
persited, here AbstractJPAMailboxMessage
  2. Using these annotations (named query) openjpa plugin enhance code
upon compilation
  3. Then the magic happens

Here because openjpa do not load correctly persisted objects, it then
fails at executing related queries.

Why OpenJPA fails loading related metadata? - the log is missing
explanation...

And of course...
  - What is the exact James version?
  - JDBC mysql driver version?
  - MySQL version?

This would enable us to be testing this locally, see if we can reproduce
it with basic setup...

Cheers,

Benoit

On 17/09/2019 10:36, Jerry Malcolm wrote:

Something that might be relevant to this problem, but the messages make
no more sense than the other one... A little earlier in the log, I have
a bunch of these messages:

174  James  INFO   [main] openjpa.Runtime - Starting OpenJPA 3.0.0
207  James  INFO   [main] openjpa.jdbc.JDBC - Using dictionary class
"org.apache.openjpa.jdbc.sql.MySQLDictionary".
552  James  INFO   [main] openjpa.jdbc.JDBC - Connected to MySQL version
5.5 using JDBC driver MySQL Connector Java version
mysql-connector-java-5.1.34 ( Revision:
jess.bal...@oracle.com-20141014163213-wqbwpf1ok2kvo1om ).
586  James  WARN   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in
the openjpa.MetaDataFactory configuration property could not be loaded
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
587  James  WARN   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage"
listed in the openjpa.MetaDataFactory configuration property could not
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
688  James  INFO   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in
the openjpa.MetaDataFactory configuration property could not be loaded
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
689  James  INFO   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage"
listed in the openjpa.MetaDataFactory configuration property could not
be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
702  James  INFO   [main] openjpa.MetaData - The class
"org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in
the openjpa.MetaDataFactory configuration property could not be loaded
by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.

On 9/16/2019 10:28 PM, Jerry Malcolm wrote:

The fun just keeps happening I was finally able to bring up the
new installation of James 3.3.0 with a sandbox copy of my real
database.  I created a new test email account in James and configured
it in my local Thunderbird instance.  So far, so good. But now every
time Thunderbird contacts James for an IMAP update, I get the error
below in the James log.  I'm not really familiar with jpa.  I searched
through the source and found quite a few places that the
"findMessagesInMailbox" name is referenced.  So not really sure why it
says it isn't defined.  Any ideas?  This is with the downloaded
binaries, not a re-built version.


org.apache.openjpa.persistence.ArgumentException:  There is no query
with the name "findMessagesInMailbox" defined for any of the known
persistent classes:
[org.apache.james.mailbox.jpa.mail.model.JPAMailbox,
org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage,
org.apache.james.mailbox.jpa.mail.model.JPAUserFlag,
org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage,
org.apache.james.domainlist.jpa.model.JPADomain,

Re: two datasources?

2019-09-17 Thread Garry Hurley
Jerry, 

I have had moderate success with creating a separate properties file with your 
database info in it. You can use a simple properties file with key-value pairs 
and then use a class with a name like MailetProperties to read the file into 
your application. I recommend this approach so that you have the capability to 
add more properties and use them in more places. 

Sent from my iPhone

> On Sep 17, 2019, at 2:14 AM, Tellier Benoit  wrote:
> 
> If I were you, I'd use only mailet configuration properties for this.
> 
>> On 17/09/2019 10:18, Jerry Malcolm wrote:
>> No the main mailet in question is 100% custom.  It is a mailet that
>> decides what folder to put the mail item in.  I access a completely
>> separate client database and determine if the sender is a client vs. a
>> vendor vs. an employee and store the mail in a different folder based on
>> the result.  The client database is completely unrelated to james.  It's
>> just a lookup.  Since this mailet is pure custom, I really don't care to
>> do any more integration into James other than mailet interface.  My main
>> goal was simply to be able to store the database name, id/pw, url, etc
>> in a database properties file and get it out of the mailet code and/or
>> the mailet flow xml files.   Again, not a big deal.
>> 
>>> On 9/16/2019 9:49 PM, Tellier Benoit wrote:
>>> Hi Jerry,
>>> 
>>> Are you speaking of JDBC mailets (whiteList for instance)
>>> 
>>> These mailets are currently deprecated (or will be) as they hard code
>>> their database, are not standard, not tested and not documented. Note
>>> that they are considered as 'experimental' ( cf
>>> http://james.apache.org/server/dev-provided-mailets.html ).
>>> 
>>> I would discourage their use (unless you really need these features) and
>>> call for contribution:
>>> 
>>>   - Write a generic storage API for them
>>>   - Rewrite these mailets on top of this storage API
>>>   - Provide JPA storage for these storage APIs
>>>   - Bind stuff in Guice/Spring
>>> 
>>> A bit of work, but if you really need it, I would be pleased to provide
>>> such guidance.
>>> 
 On 17/09/2019 09:39, Jerry Malcolm wrote:
 A couple of my mailets need to access a different database other than
 the base james db.  I've got the main db set up in the
 james-database.properties file.  Currently I'm just hardcoding the
 database connection to the other db in the mailet.  Just curious if
 there's a better architected process where I can define two datasources
 in james-database.properties and reference the alternate datasource when
 needed.  Couldn't see anything as to how to do that.   Not a big deal.
 Just trying to clean up a few things as I migrate to the new version.
 
 
 -
 To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
 For additional commands, e-mail: server-user-h...@james.apache.org
 
>>> -
>>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>>> For additional commands, e-mail: server-user-h...@james.apache.org
>>> 
>> 
>> -
>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-user-h...@james.apache.org
>> 
> 
> -
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
> 

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: IMAP Error

2019-09-17 Thread Tellier Benoit
Hi Jerry,

To explain it shortly:

 1. In OpenJPA NamedQuery are directly attached to the POJO being
persited, here AbstractJPAMailboxMessage
 2. Using these annotations (named query) openjpa plugin enhance code
upon compilation
 3. Then the magic happens

Here because openjpa do not load correctly persisted objects, it then
fails at executing related queries.

Why OpenJPA fails loading related metadata? - the log is missing
explanation...

And of course...
 - What is the exact James version?
 - JDBC mysql driver version?
 - MySQL version?

This would enable us to be testing this locally, see if we can reproduce
it with basic setup...

Cheers,

Benoit

On 17/09/2019 10:36, Jerry Malcolm wrote:
> Something that might be relevant to this problem, but the messages make
> no more sense than the other one... A little earlier in the log, I have
> a bunch of these messages:
> 
> 174  James  INFO   [main] openjpa.Runtime - Starting OpenJPA 3.0.0
> 207  James  INFO   [main] openjpa.jdbc.JDBC - Using dictionary class
> "org.apache.openjpa.jdbc.sql.MySQLDictionary".
> 552  James  INFO   [main] openjpa.jdbc.JDBC - Connected to MySQL version
> 5.5 using JDBC driver MySQL Connector Java version
> mysql-connector-java-5.1.34 ( Revision:
> jess.bal...@oracle.com-20141014163213-wqbwpf1ok2kvo1om ).
> 586  James  WARN   [main] openjpa.MetaData - The class
> "org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in
> the openjpa.MetaDataFactory configuration property could not be loaded
> by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
> 587  James  WARN   [main] openjpa.MetaData - The class
> "org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage"
> listed in the openjpa.MetaDataFactory configuration property could not
> be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
> 688  James  INFO   [main] openjpa.MetaData - The class
> "org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in
> the openjpa.MetaDataFactory configuration property could not be loaded
> by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
> 689  James  INFO   [main] openjpa.MetaData - The class
> "org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage"
> listed in the openjpa.MetaDataFactory configuration property could not
> be loaded by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
> 702  James  INFO   [main] openjpa.MetaData - The class
> "org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage" listed in
> the openjpa.MetaDataFactory configuration property could not be loaded
> by sun.misc.Launcher$AppClassLoader@764c12b6; ignoring.
> 
> On 9/16/2019 10:28 PM, Jerry Malcolm wrote:
>> The fun just keeps happening I was finally able to bring up the
>> new installation of James 3.3.0 with a sandbox copy of my real
>> database.  I created a new test email account in James and configured
>> it in my local Thunderbird instance.  So far, so good. But now every
>> time Thunderbird contacts James for an IMAP update, I get the error
>> below in the James log.  I'm not really familiar with jpa.  I searched
>> through the source and found quite a few places that the
>> "findMessagesInMailbox" name is referenced.  So not really sure why it
>> says it isn't defined.  Any ideas?  This is with the downloaded
>> binaries, not a re-built version.
>>
>> 
>> org.apache.openjpa.persistence.ArgumentException:  There is no query
>> with the name "findMessagesInMailbox" defined for any of the known
>> persistent classes:
>> [org.apache.james.mailbox.jpa.mail.model.JPAMailbox,
>> org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMessage,
>> org.apache.james.mailbox.jpa.mail.model.JPAUserFlag,
>> org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMessage,
>> org.apache.james.domainlist.jpa.model.JPADomain,
>> org.apache.james.user.jpa.model.JPAUser,
>> org.apache.james.mailbox.jpa.mail.model.JPAProperty,
>> org.apache.james.mailbox.jpa.user.model.JPASubscription,
>> org.apache.james.rrt.jpa.model.JPARecipientRewrite].
>>
>>
>> -
>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-user-h...@james.apache.org
>>
> 
> -
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
> 

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org



Re: two datasources?

2019-09-17 Thread Tellier Benoit
If I were you, I'd use only mailet configuration properties for this.

On 17/09/2019 10:18, Jerry Malcolm wrote:
> No the main mailet in question is 100% custom.  It is a mailet that
> decides what folder to put the mail item in.  I access a completely
> separate client database and determine if the sender is a client vs. a
> vendor vs. an employee and store the mail in a different folder based on
> the result.  The client database is completely unrelated to james.  It's
> just a lookup.  Since this mailet is pure custom, I really don't care to
> do any more integration into James other than mailet interface.  My main
> goal was simply to be able to store the database name, id/pw, url, etc
> in a database properties file and get it out of the mailet code and/or
> the mailet flow xml files.   Again, not a big deal.
> 
> On 9/16/2019 9:49 PM, Tellier Benoit wrote:
>> Hi Jerry,
>>
>> Are you speaking of JDBC mailets (whiteList for instance)
>>
>> These mailets are currently deprecated (or will be) as they hard code
>> their database, are not standard, not tested and not documented. Note
>> that they are considered as 'experimental' ( cf
>> http://james.apache.org/server/dev-provided-mailets.html ).
>>
>> I would discourage their use (unless you really need these features) and
>> call for contribution:
>>
>>   - Write a generic storage API for them
>>   - Rewrite these mailets on top of this storage API
>>   - Provide JPA storage for these storage APIs
>>   - Bind stuff in Guice/Spring
>>
>> A bit of work, but if you really need it, I would be pleased to provide
>> such guidance.
>>
>> On 17/09/2019 09:39, Jerry Malcolm wrote:
>>> A couple of my mailets need to access a different database other than
>>> the base james db.  I've got the main db set up in the
>>> james-database.properties file.  Currently I'm just hardcoding the
>>> database connection to the other db in the mailet.  Just curious if
>>> there's a better architected process where I can define two datasources
>>> in james-database.properties and reference the alternate datasource when
>>> needed.  Couldn't see anything as to how to do that.   Not a big deal.
>>> Just trying to clean up a few things as I migrate to the new version.
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>>> For additional commands, e-mail: server-user-h...@james.apache.org
>>>
>> -
>> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
>> For additional commands, e-mail: server-user-h...@james.apache.org
>>
> 
> -
> To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
> For additional commands, e-mail: server-user-h...@james.apache.org
> 

-
To unsubscribe, e-mail: server-user-unsubscr...@james.apache.org
For additional commands, e-mail: server-user-h...@james.apache.org