from:"Benoit Tellier"



[ 
https://issues.apache.org/jira/browse/JAMES-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851783#comment-17851783
 ] 

Benoit Tellier commented on JAMES-3897:
---

Author contacted off-channel confirmed me this can be closed.

> IP filtering with CrowdSec
> --
>
> Key: JAMES-3897
> URL: https://issues.apache.org/jira/browse/JAMES-3897
> Project: James Server
>  Issue Type: Sub-task
>    Reporter: Benoit Tellier
>Priority: Major
>  Time Spent: 13h 50m
>  Remaining Estimate: 0h
>
> Thanks to recommandation from a collegue Xavier GUIMARD, I discovered 
> CrowdSec ( https://www.crowdsec.net/ ).
> CrowdSec is a free, modern & collaborative behavior detection engine, coupled 
> with a global IP reputation network, based on IA behaviour refinement.
> Develop a third-party plugin for questionning crowdSec:
>  - Create a SMTP EHLO hook questionning via a REST call the CrowdSec local 
> agent
>  - Create a mailet questionning via a REST call the CrowdSec local agent
>  - Create a mailet to provision local CrowdSec database (for highest level of 
> spam for instance)
>  - Think about the interfaces we would need to question CrowdSec upon 
> incoming IMAP connections
>  - Externalize behaviour linked to failed login attempts (sleep, 3 failure 
> connection closure) as configurable extensible plugins.
>  - Use it to manage IP reporting to crowdSec, especially upon failed 
> authentications (~fail2ban). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-4042) Task manager: issues with long tasks



[ 
https://issues.apache.org/jira/browse/JAMES-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851782#comment-17851782
 ] 

Benoit Tellier commented on JAMES-4042:
---

Thanks for the pointer, I forgot about that.

I did a quick code review...

The additional information logic is there (great), cancellation also seems like 
it would work (future is recorded into SerialTaskManagerWorker::runningTasks), 
termination signalling is done too (SerialTaskManagerWorker::runTask).

Do you confirm? I would likely contribute some more tests onto AsyncTasks 
handling before pushing this further...

Also, I do not like the idea to encode the knowlege of "how a task should be 
run" into it's type system. This makes it hard to do this conditionnally. Maybe 
making this a method of the Task interface (defaulting to synchronous) would 
allow to make this a task parameter and yield a greater freedom?

> Task manager: issues with long tasks
> 
>
> Key: JAMES-4042
> URL: https://issues.apache.org/jira/browse/JAMES-4042
> Project: James Server
>  Issue Type: Improvement
>  Components: rabbitmq, task
>Affects Versions: master, 3.8.2
>Reporter: Benoit Tellier
>Priority: Major
>
> See JAMES-3955
> Today a tasks per default obey a 1 day consumer timeout (configurable!)
> Yet, we might have some tasks like deduplication that might take longer than 
> a day to complete...
> As of today tihis means such a task would complete but very likely crash the 
> consumer.
> Likely, getting such long jobs in the first place might not be the best idea.
> Things like deduplication could be packaged in separate binaries that one 
> could run independently of James thus avoiding the issue altogether... We 
> could allow this as an alternative way to run such long tasks.
> Moreover, I think if needed we could write a Proof Of Concept of running the 
> GC on top of something like Apache Spark to leverage parallele computations.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3552) Enforce session advertised limits



[ 
https://issues.apache.org/jira/browse/JAMES-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851697#comment-17851697
 ] 

Benoit Tellier commented on JAMES-3552:
---

maxObjectsInGet maxObjectsInSet maxSizeAttachmentsPerEmail had been implemented.

MaxConcurrentRequests and MaxConcurrentRequests still are hard coded.

> Enforce session advertised limits
> -
>
> Key: JAMES-3552
> URL: https://issues.apache.org/jira/browse/JAMES-3552
> Project: James Server
>  Issue Type: Sub-task
>  Components: JMAP
>        Reporter: Benoit Tellier
>Assignee: Antoine Duprat
>Priority: Major
> Fix For: 3.6.0
>
>
> We advertize a bunch of limits in the session:
> {code:java}
> "urn:ietf:params:jmap:core" : {
>   "maxConcurrentUpload" : 4,
>   "maxSizeRequest" : 1000,
>   "maxConcurrentRequests" : 4,
>   "maxCallsInRequest" : 16,
>   "maxObjectsInGet" : 500,
>   "maxObjectsInSet" : 500
> },
> "urn:ietf:params:jmap:mail" : {
>   "maxMailboxesPerEmail" : 1000,
>   "maxMailboxDepth" : null,
>   "maxSizeAttachmentsPerEmail" : 2000
> }
> {code}
>  - maxMailboxesPerEmail can likely be null if not enforced
> Regarding maxConcurrentUpload and maxConcurrentRequests enforcing these 
> limitations in a distributed environment is a non trivial task... Maybe via 
> the use of external API management solutions? In which case we should allow 
> an administrator to configure those values according to system wide setting...
> maxSizeRequest, maxCallsInRequest, maxObjectsInGet, maxObjectsInSet, 
> maxSizeAttachmentsPerEmail would likely only need some simple validation 
> logic to be written. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3564) Bugfixes to 3.6.x branch



[ 
https://issues.apache.org/jira/browse/JAMES-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851695#comment-17851695
 ] 

Benoit Tellier commented on JAMES-3564:
---

3.6.x branch is no longer maintained as far as I can tell.

Can we close this ticket?

> Bugfixes to 3.6.x branch
> 
>
> Key: JAMES-3564
> URL: https://issues.apache.org/jira/browse/JAMES-3564
> Project: James Server
>  Issue Type: Improvement
>Affects Versions: 3.6.0
>Reporter: Juhan Aasaru
>Priority: Major
> Fix For: 3.6.1
>
>
> I created this task
>  * to keep track of all the pull requests that have been made to master 
> arfter 3.6.x
>  * and decide (and discuss) which ones should also get to 3.6.x branch (and 
> which should not)
> ||PR to master||PR to 3.6.x||Ticket||
> |[#362|https://github.com/apache/james-project/pull/362]|[#382 
> |https://github.com/apache/james-project/pull/382] 
> [(/)|https://github.com/apache/james-project/pull/384]|JAMES-3537|
> |[#374|https://github.com/apache/james-project/pull/374]|[#383 
> |https://github.com/apache/james-project/pull/383] 
> [(/)|https://github.com/apache/james-project/pull/384]|JAMES-3432|
> |[#377|https://github.com/apache/james-project/pull/377]|[#404|https://github.com/apache/james-project/pull/404]
>  (/)|JAMES-3557|
> |[#378|https://github.com/apache/james-project/pull/378]|[#392|https://github.com/apache/james-project/pull/392]
>  (/)|JAMES-3558|
> |[#379|https://github.com/apache/james-project/pull/379]|[#384 
> |https://github.com/apache/james-project/pull/384] (/)|JAMES-3556|
> |[#221|https://github.com/apache/james-project/pull/221] 
> [#396|https://github.com/apache/james-project/pull/396]|[#393|https://github.com/apache/james-project/pull/393]
>  (/) [#411|https://github.com/apache/james-project/pull/411] (/)  |JAMES-3261|
> |[#394|https://github.com/apache/james-project/pull/394]|[#404|https://github.com/apache/james-project/pull/404]
>  (/)|JAMES-3566|
> |[#395|https://github.com/apache/james-project/pull/395]|[#404|https://github.com/apache/james-project/pull/404]
>  (/)|JAMES-3525|
> |[#405|https://github.com/apache/james-project/pull/405] |TODO|JAMES-3569|
> |[#407|https://github.com/apache/james-project/pull/407] 
> |[#410|https://github.com/apache/james-project/pull/410] (/) |JAMES-3571|
> What not to merge and why
> ||PR to master||Why not||
> |[#386|https://github.com/apache/james-project/pull/386]|not targeted to 
> james repo|
> |[#397|https://github.com/apache/james-project/pull/397/]|documentation not 
> releated to a realease|
> |[#398|https://github.com/apache/james-project/pull/398/]|No need for website 
> typo|
> |[#401|https://github.com/apache/james-project/pull/401/]|Enhancement only|
> |[#385|https://github.com/apache/james-project/pull/385/]|MDN/send is a new 
> feature|
> I plan to work on this myself.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3567) Apache James 3.6 has Critical Vulnerability in dependent libs



 [ 
https://issues.apache.org/jira/browse/JAMES-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3567.
-
Resolution: Fixed

Mentioned dependencies are either updated or removed.

Closing.

> Apache James 3.6 has Critical Vulnerability in dependent libs
> -
>
> Key: JAMES-3567
> URL: https://issues.apache.org/jira/browse/JAMES-3567
> Project: James Server
>  Issue Type: Improvement
>  Components: James Core
>Affects Versions: 3.6.0
> Environment: Docker Image: - apache/james:distributed-3.6.0 
>Reporter: Rikin Patel
>Priority: Major
>  Labels: vulnerability
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> /root/james-server-cassandra-guice.lib/netty-3.10.6.Final.jar: -
>     ->  HttpObjectDecoder.java in Netty before 4.1.44 allows a Content-Length 
> header to be accompanied by a second Content-Length header, or by a 
> Transfer-Encoding header
>     -> HttpObjectDecoder.java in Netty before 4.1.44 allows an HTTP header 
> that lacks a colon, which might be interpreted as a separate header with an 
> incorrect syntax, or might be interpreted as an "invalid fold.". Impacted 
> Image File(s): /root/james-server-cassandra-guice.lib/netty-3.10.6.Final.jar
> /root/james-server-cassandra-guice.lib/jgroups-3.6.13.Final.jar
>     ->  JGroups before 4.0 does not require the proper headers for the 
> ENCRYPT and AUTH protocols from nodes joining the cluster, which allows 
> remote attackers to bypass security restrictions and send and receive 
> messages within the cluster via unspecified vectors..



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3572) SMTP server does not recognize LF only separator



[ 
https://issues.apache.org/jira/browse/JAMES-3572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851693#comment-17851693
 ] 

Benoit Tellier commented on JAMES-3572:
---

Well in the face of SMTP smuggling, Apache James took a stricter stance 
regarding this: only accept CRLF and reject non compliant LF that might be a 
parser differential and an attack in the waiting.

As such this ticket no longer is relevant. I would close it. Feel free to 
reopen if I am wrong.

> SMTP server does not recognize LF only separator
> 
>
> Key: JAMES-3572
> URL: https://issues.apache.org/jira/browse/JAMES-3572
> Project: James Server
>  Issue Type: Bug
>  Components: SMTPServer
>Affects Versions: 3.6.0
> Environment: James: built from james-project-3.6.0 tag
> OS: Linux
>Reporter: Karsten Otto
>Priority: Major
> Fix For: 3.7.0, 3.6.1
>
>
> When I send a mail via SMTP and the blank line between headers and body 
> consists only of an LF instead of CR LF, then James does not recognize it as 
> the separator. Instead, it stores any subsequent body as part of the header 
> blob, I assume as continuation of the last header line.
> I also observed this in DSN bounces, where the last part had the 
> rfc822-headers type but included the mail body, when it was sent with an 
> LF-only separator.
> I am not sure if this is indeed a bug or just a strict interpretation of the 
> standard. If this is not a bug in your opinion, please feel free to close 
> this ticket.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3572) SMTP server does not recognize LF only separator



 [ 
https://issues.apache.org/jira/browse/JAMES-3572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3572.
-
Resolution: Fixed

> SMTP server does not recognize LF only separator
> 
>
> Key: JAMES-3572
> URL: https://issues.apache.org/jira/browse/JAMES-3572
> Project: James Server
>  Issue Type: Bug
>  Components: SMTPServer
>Affects Versions: 3.6.0
> Environment: James: built from james-project-3.6.0 tag
> OS: Linux
>Reporter: Karsten Otto
>Priority: Major
> Fix For: 3.7.0, 3.6.1
>
>
> When I send a mail via SMTP and the blank line between headers and body 
> consists only of an LF instead of CR LF, then James does not recognize it as 
> the separator. Instead, it stores any subsequent body as part of the header 
> blob, I assume as continuation of the last header line.
> I also observed this in DSN bounces, where the last part had the 
> rfc822-headers type but included the mail body, when it was sent with an 
> LF-only separator.
> I am not sure if this is indeed a bug or just a strict interpretation of the 
> standard. If this is not a bug in your opinion, please feel free to close 
> this ticket.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3582) Message streaming doesn't work



[ 
https://issues.apache.org/jira/browse/JAMES-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851690#comment-17851690
 ] 

Benoit Tellier commented on JAMES-3582:
---

Proposition: drop not working JPA implementations

I'll do this in the coming weeks on my free time.

> Message streaming doesn't work
> --
>
> Key: JAMES-3582
> URL: https://issues.apache.org/jira/browse/JAMES-3582
> Project: James Server
>  Issue Type: Bug
>  Components: data, jpa, Spring Container
>Affects Versions: 3.6.0
>Reporter: Cosimo Damiano Prete
>Priority: Major
>
> Setting {{openjpa.streaming=true}} and using the 
> {{JPAStreamingMailboxMessage}} entity on the Spring version (against 
> PostgreSQL) leads to
> ```
> jvm 1| Caused by: org.apache.openjpa.persistence.ArgumentException: No 
> metadata was found for type "class 
> org.apache.james.mailbox.jpa.mail.model.openjpa.JPAMailboxMessage". The class 
> does not appear in the list of persistent types: 
> [org.apache.james.mailbox.jpa.mail.model.JPAUserFlag, 
> org.apache.james.mailbox.jpa.quota.model.MaxUserStorage, 
> org.apache.james.mailbox.jpa.mail.model.openjpa.AbstractJPAMailboxMessage, 
> org.apache.james.mailbox.jpa.quota.model.JpaCurrentQuota, 
> org.apache.james.mailbox.jpa.mail.model.JPAMailboxAnnotation, 
> org.apache.james.mailbox.jpa.user.model.JPASubscription, 
> org.apache.james.rrt.jpa.model.JPARecipientRewrite, 
> org.apache.james.mailbox.jpa.quota.model.MaxDefaultStorage, 
> org.apache.james.mailbox.jpa.mail.model.JPAMailbox, 
> org.apache.james.mailbox.jpa.quota.model.MaxUserMessageCount, 
> org.apache.james.domainlist.jpa.model.JPADomain, 
> org.apache.james.user.jpa.model.JPAUser, 
> org.apache.james.mailbox.jpa.mail.model.JPAProperty, 
> org.apache.james.mailbox.jpa.mail.model.openjpa.JPAStreamingMailboxMessage, 
> org.apache.james.mailbox.jpa.quota.model.MaxDefaultMessageCount].
> jvm 1|  at 
> org.apache.openjpa.meta.MetaDataRepository.getMetaDataInternal(MetaDataRepository.java:419)
>  ~[openjpa-3.1.2.jar:3.1.2]
> jvm 1|  at 
> org.apache.openjpa.meta.MetaDataRepository.getMetaData(MetaDataRepository.java:394)
>  ~[openjpa-3.1.2.jar:3.1.2]
> jvm 1|  at 
> org.apache.openjpa.kernel.BrokerImpl.persistInternal(BrokerImpl.java:2769) 
> ~[openjpa-3.1.2.jar:3.1.2]
> jvm 1|  at 
> org.apache.openjpa.kernel.BrokerImpl.persist(BrokerImpl.java:2707) 
> ~[openjpa-3.1.2.jar:3.1.2]
> jvm 1|  at 
> org.apache.openjpa.kernel.BrokerImpl.persist(BrokerImpl.java:2690) 
> ~[openjpa-3.1.2.jar:3.1.2]
> jvm 1|  at 
> org.apache.openjpa.kernel.BrokerImpl.persist(BrokerImpl.java:2592) 
> ~[openjpa-3.1.2.jar:3.1.2]
> jvm 1|  at 
> org.apache.openjpa.kernel.DelegatingBroker.persist(DelegatingBroker.java:1197)
>  ~[openjpa-3.1.2.jar:3.1.2]
> jvm 1|  at 
> org.apache.openjpa.persistence.EntityManagerImpl.persist(EntityManagerImpl.java:837)
>  ~[openjpa-3.1.2.jar:3.1.2]
> jvm 1|  at 
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
> jvm 1|  at 
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
> jvm 1|  at 
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 
> ~[?:?]
> jvm 1|  at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
> jvm 1|  at 
> org.springframework.orm.jpa.ExtendedEntityManagerCreator$ExtendedEntityManagerInvocationHandler.invoke(ExtendedEntityManagerCreator.java:347)
>  ~[spring-orm-4.3.25.RELEASE.jar:4.3.25.RELEASE]
> jvm 1|  at com.sun.proxy.$Proxy81.persist(Unknown Source) ~[?:?]
> jvm 1|  at 
> org.apache.james.mailbox.jpa.mail.JPAMessageMapper.save(JPAMessageMapper.java:361)
>  ~[apache-james-mailbox-jpa-3.6.0.jar:3.6.0]
> jvm 1|  at 
> org.apache.james.mailbox.jpa.mail.JPAMessageMapper.add(JPAMessageMapper.java:293)
>  ~[apache-james-mailbox-jpa-3.6.0.jar:3.6.0]
> jvm 1|  at 
> org.apache.james.mailbox.store.MessageStorer$WithoutAttachment.lambda$appendMessageToStore$0(MessageStorer.java:135)
>  ~[apache-james-mailbox-store-3.6.0.jar:3.6.0]
> jvm 1|  at 
> org.apache.james.mailbox.store.transaction.TransactionalMapper.execute(TransactionalMapper.java:38)
>  ~[apache-james-mailbox-store-3.6.0.jar:3.6.0]
> jvm 1|  at 
> org.apache.james.mailbox.store.MessageStorer$WithoutAttachment.appendMessageToStore(MessageStorer.java:133)
>  ~[apache-james-mailbox-store-3.6.0.jar:3.6.0]
> jvm 1|  at 
> org.apache.james.mailbox.store.StoreMessageManager.lambda$createA

[jira] [Closed] (JAMES-3730) Upgraded from Legacy JavaMail to Jakarta Mail 2.0.1/Activation 2.0.1



 [ 
https://issues.apache.org/jira/browse/JAMES-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3730.
-
Resolution: Fixed

Legal answer: all clear, EPL 2 is compatible

This work had recently been contributed to apache James.

> Upgraded from Legacy JavaMail to Jakarta Mail 2.0.1/Activation 2.0.1
> 
>
> Key: JAMES-3730
> URL: https://issues.apache.org/jira/browse/JAMES-3730
> Project: James Server
>  Issue Type: Improvement
>Reporter: Glennos
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> For more information, see: https://github.com/apache/james-project/pull/923



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3740) IMAP UID <-> MSN mapping occupies too much memory



[ 
https://issues.apache.org/jira/browse/JAMES-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851687#comment-17851687
 ] 

Benoit Tellier commented on JAMES-3740:
---

Feedback on this?

With...
... related memory leaks fixed
... compact uid <=> MSN m,appings implemented
This is less of a concern IMO. I run a 10.000 user platform with less than 350 
MB footprint all james servers combined...

Yet we could experiment alternative UID -> MSN mappings.

> IMAP UID <-> MSN mapping occupies too much memory
> -
>
> Key: JAMES-3740
> URL: https://issues.apache.org/jira/browse/JAMES-3740
> Project: James Server
>  Issue Type: Improvement
>  Components: IMAPServer
>Affects Versions: 3.7.0
>Reporter: Benoit Tellier
>Priority: Major
> Attachments: Screenshot from 2022-03-30 17-39-37.png, Screenshot from 
> 2022-04-08 17-33-35.png
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> h3. What is UID <-> MSN mapping ?
> In IMAP RFC-3501 there is two ways one addresses a message:
>  - By its UID (Unique ID) that is unique (until UID_VALIDITY changes...)
>  - By its MSN (Message Sequence  Number) which is the (mutable) position of a 
> message in the mailbox.
> We then need:
>  - Given a UID return its MSN which is for instance compulsory upon EXPUNGED 
> notifications when QRESYNCH is not enabled.
>  - Given a MSN based request we need to convert it back to a UID (rare).
> We do store the list of UIDs, sorted, in RAM and perform binarysearches to 
> resolve those.
> h3. What is the impact on heap?
> Each uid is wrapped in a MessageUID object. This object wrapping comes with 
> an overhead of at least 12 bytes in addition to the 8 bytes payload (long). 
> Quick benchmarks shows it's actually worse: 10 million uids did take up to 
> 275 MB.
> {code:java}
> @Test
> void measureHeapUsage() throws InterruptedException {
> int count =1000;
> testee.addAll(IntStream.range(0, count)
> .mapToObj(i -> MessageUid.of(i + 1))
> .collect(Collectors.toList()));
> Thread.sleep(1000);
> System.out.println("GCing");
> System.gc();
> Thread.sleep(1000);
> 
> System.out.println(ManagementFactory.getMemoryMXBean().getHeapMemoryUsage().getUsed());
> }
> {code}
> Now, from let's take a classical production deployment I get:
>  - Some users have up to 2.5 million messages in their INBOX
>  - I can get an average of 100.000 messages for each user
> So for a small scale deployment, we are already "consuming" ~300 MB of memory 
> just for the UID <-> mapping.
> Scaling to 1.000 users on a single James instance we clearly see that HEAP 
> consumption will start being a problem (~3GB) without even speaking of target 
> of 10.000 users per James I do have in mind.
> It's worth mentioning that IMAP being statefull, and UID <-> MSN mapping 
> attached to a selected mailbox, such a mapping is long lived:
>  - Multiple small objects would need to be copied individually by the GC, 
> putting pressure during long gen
>  - Those long lived object will eventually be promoted to old gen, thus the 
> more there is the longer the resulting stop-the-world GC pauses will be.
> h3. Temporary fix ?
> We can get rid of the object boxing in UidMsnConverter by using primitive 
> type collections for instance provided by fastutils project.
> The same bench was down to 84MB.
> Also, we could get things more compact by using an INT representation of 
> UIDs. (Those are most of the case below 2 billions, to be above this there 
> need to be more than 2 billion emails transiting through one's mailbox which 
> is highly unlikely). A fallback to "long" storage can be setted up if a UID 
> above 2 billion is observed.
> This such a compact int storage we are down to 46MB.
> So taking the prior mentioned numbers we could expect a 1.000 people 
> deployment to require ~400 MB and a larger scale 10.000 people deployment on 
> a single James to consume up to 4GB. Not that enjoyable but definitly more 
> manageable.
> Please note that primitive collections are more GC friendly as their elements 
> are manages together, as a single object (backing array).
> h3. What other mail servers do
> I found references to Dovecote, which does a similar algorithm compared to 
> us: binary search on a list of uids. The noticeable difference is that this 
> list of UIDs is held on disk and not in memory as we do.
> References: 
> https://doc.dovecot.org/developer_manua

[jira] [Closed] (JAMES-3741) SSL: sporadic failure of new connection failure under load in 3.7.x



 [ 
https://issues.apache.org/jira/browse/JAMES-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3741.
-
Resolution: Fixed

Solved for latest releases.

Documented for 3.7.x

Let's move on.

> SSL: sporadic failure of new connection failure under load in 3.7.x
> ---
>
> Key: JAMES-3741
> URL: https://issues.apache.org/jira/browse/JAMES-3741
> Project: James Server
>  Issue Type: Improvement
>  Components: IMAPServer, POP3Server, SMTPServer
>Affects Versions: 3.7.0
>    Reporter: Benoit Tellier
>Priority: Major
> Fix For: master
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Exception seen on 3.7.x... Context performance tests with several new SSL 
> connections opened per seconds (high concurrency).
> {code:java}
> 21:34:28.460 [WARN ] o.a.j.i.n.ImapChannelUpstreamHandler - Error while 
> processing imap request
> javax.crypto.BadPaddingException: Insufficient buffer remaining for AEAD 
> cipher fragment (2). Needs to be more than tag size (16)
>   at 
> java.base/sun.security.ssl.SSLCipher$T13GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1894)
>   at 
> java.base/sun.security.ssl.SSLEngineInputRecord.decodeInputRecord(SSLEngineInputRecord.java:240)
>   at 
> java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:197)
>   at 
> java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:160)
>   at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
>   ... 24 common frames omitted
> {code}
> Can be reliably reproduced by opening many new SSL connections concurrently:
> {code:java}
> @Nested
> class Toto {
> IMAPServer imapServer;
> int port;
> @BeforeEach
> void setup() throws Exception {
> HierarchicalConfiguration config = 
> ConfigLoader.getConfig(ClassLoaderUtils.getSystemResourceAsSharedStream("imapSSL.xml"));
> imapServer = createImapServer(config);
> port = imapServer.getListenAddresses().get(0).getPort();
> }
> @AfterEach
> void tearDown() {
> if (imapServer != null) {
> imapServer.destroy();
> }
> }
> @Test
> void test() throws Exception {
> ConcurrentTestRunner.builder()
> .operation((a, b) -> {
> IMAPSClient imapsClient = imapsImplicitClient(port);
> final boolean capability = imapsClient.capability();
> assertThat(capability).isTrue();
> final boolean close = imapsClient.close();
> })
> .threadCount(10)
> .operationCount(3000)
> .runSuccessfullyWithin(Duration.ofMinutes(10));
> }
> private IMAPSClient imapsImplicitClient(int port) throws Exception {
> IMAPSClient client = new IMAPSClient(true, 
> BogusSslContextFactory.getClientContext());
> 
> client.setTrustManager(BogusTrustManagerFactory.getTrustManagers()[0]);
> client.connect("127.0.0.1", port);
> return client;
> }
> }
> {code}
> and `imapSSL.xml` being: 
> {code:java}
> 
> imapserver
> 0.0.0.0:9993
> 
> private.key
> certs.self-signed.csr
> 123456
> 
> 
> true
> true
> 
> 
> {code}
> Interestingly enough the Netty4 migration post 3.7.x fixed the issue.
> Thus it will be fixed in later releases yet it seemed interesting to me to 
> document the issue.
> I propose to add a non-regression test on master.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3742) MIME-Version header is changed, breaking DKIM signatures



[ 
https://issues.apache.org/jira/browse/JAMES-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851682#comment-17851682
 ] 

Benoit Tellier commented on JAMES-3742:
---

2 years of inactivity on this important topic.

I propose to move forward, implement the proposed change AND add a system 
variable in order to allow bypassing this just in case someone find a 
legitimate reason to change mime versions...

We could further improve the proposed code in order to check the mime version 
is 1.0 with optional comment. and not set it again if this is 1.0.

> MIME-Version header is changed, breaking DKIM signatures
> 
>
> Key: JAMES-3742
> URL: https://issues.apache.org/jira/browse/JAMES-3742
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 2.3.2
>Reporter: Paul Chown
>Priority: Major
>
> We are using James to forward mails from external email servers to customer 
> accounts.
> Some email clients create emails with the following headers:
> {noformat}
> Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.60.0.1.1\))
> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
>         d=.gappssmtp.com; s=20210112;
>         
> h=from:content-transfer-encoding:mime-version:subject:message-id:date:to;
>         bh=...;
>         b=...{noformat}
> So the message is signed by the originator including the {{Mime-Version}} 
> header. The problem is that if we set an additional header with 
> {{SetMimeHeaderHandler}} or invoke the SpamAssassin service then the method 
> {{MimeMessage.updateHeaders()}} is invoked which sets a new mime version 
> header:
> {noformat}
> MIME-Version: 1.0{noformat}
> thereby breaking the DKIM signature from the original sender. We can add our 
> own signature using the new header, but this isn't aligned with the sender 
> domain so it doesn't help. It's probably debatable whether the original Mac 
> mail client mime version header is valid, but it is what it is.
> This seems to be completely broken, but the behaviour is baked into the 
> {{MimeMessage}} class and we can't see any way around it. How can we preserve 
> the original mime-version header and the DKIM signature?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3894) Antora build mis-fonctionning - Distributed server



[ 
https://issues.apache.org/jira/browse/JAMES-3894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851680#comment-17851680
 ] 

Benoit Tellier commented on JAMES-3894:
---

We will merge the distributed module back  in 
https://github.com/apache/james-project/tree/master/docs as this remains a 
massive pain.

> Antora build mis-fonctionning - Distributed server
> --
>
> Key: JAMES-3894
> URL: https://issues.apache.org/jira/browse/JAMES-3894
> Project: James Server
>  Issue Type: Bug
>  Components: site
>        Reporter: Benoit Tellier
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> The Antora build for the distributed server randomly fails, leading to 
> missing online docimentation.
> Investigate "why" it fails and propose a fix.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3897) IP filtering with CrowdSec



[ 
https://issues.apache.org/jira/browse/JAMES-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851678#comment-17851678
 ] 

Benoit Tellier commented on JAMES-3897:
---

Can this be closed?

> IP filtering with CrowdSec
> --
>
> Key: JAMES-3897
> URL: https://issues.apache.org/jira/browse/JAMES-3897
> Project: James Server
>  Issue Type: Sub-task
>    Reporter: Benoit Tellier
>Priority: Major
>  Time Spent: 13h 50m
>  Remaining Estimate: 0h
>
> Thanks to recommandation from a collegue Xavier GUIMARD, I discovered 
> CrowdSec ( https://www.crowdsec.net/ ).
> CrowdSec is a free, modern & collaborative behavior detection engine, coupled 
> with a global IP reputation network, based on IA behaviour refinement.
> Develop a third-party plugin for questionning crowdSec:
>  - Create a SMTP EHLO hook questionning via a REST call the CrowdSec local 
> agent
>  - Create a mailet questionning via a REST call the CrowdSec local agent
>  - Create a mailet to provision local CrowdSec database (for highest level of 
> spam for instance)
>  - Think about the interfaces we would need to question CrowdSec upon 
> incoming IMAP connections
>  - Externalize behaviour linked to failed login attempts (sleep, 3 failure 
> connection closure) as configurable extensible plugins.
>  - Use it to manage IP reporting to crowdSec, especially upon failed 
> authentications (~fail2ban). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3920) SPF record: not found for host



[ 
https://issues.apache.org/jira/browse/JAMES-3920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851677#comment-17851677
 ] 

Benoit Tellier commented on JAMES-3920:
---

This is fixed on JSPF side.

I propose myself to shoot a JSPF release and then update James accordingly atop 
the future JSPF release.

> SPF record: not found for host
> --
>
> Key: JAMES-3920
> URL: https://issues.apache.org/jira/browse/JAMES-3920
> Project: James Server
>  Issue Type: Bug
>Reporter: Günter Paul
>Priority: Major
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> I tested "apache-jspf-resolver-1.0.3.jar" with newest source code.
> Here an output:
> {{Received-SPF: none (spfCheck: 209.85.217.49 is neither permitted nor denied 
> by domain of googlemail.com) client-ip=209.85.217.49; 
> envelope-from=xx....@googlemail.com; helo=googlemail.com;}}
> I tested it with other mailserver to, the same problem.
> After I changed the class "org.apache.james.jspf.impl.SPF".  In the 
> constructor I changed the "executor" form "AsynchronousSPFExecutor" to 
> "SynchronousSPFExecutor" and the problem was solved.
> this.executor = new SynchronousSPFExecutor(dnsProbe);
> {{It'a a workaround, not a solution.}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3929) Upgrade opensearch-java client



 [ 
https://issues.apache.org/jira/browse/JAMES-3929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3929.
-
Resolution: Fixed

Done though maybe another batch of upgrades shall be planned.

> Upgrade opensearch-java client 
> ---
>
> Key: JAMES-3929
> URL: https://issues.apache.org/jira/browse/JAMES-3929
> Project: James Server
>  Issue Type: Improvement
>Reporter: René Cordier
>Priority: Major
> Fix For: 3.9.0
>
>  Time Spent: 5h 20m
>  Remaining Estimate: 0h
>
> We should upgrade and fix code accordingly the opensearch-java client from 
> 2.1.0 to its latest version, 2.6.0.
> Also, referring to the official doc: 
> [https://opensearch.org/docs/latest/clients/java/]
> It seems the opensearch-java client now has its own http client transport. We 
> need to try to refactor the code to use this instead of the transport client 
> privded by the opensearch-rest-client dependency, which would allow us to 
> drop it. 
> Then we need to do some performance tests to measure if such adoption has any 
> impact or not. If the tests are conclusive enough, we should adopt it and 
> drop the opensearch-rest-client dependency once and for all.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3939) Apache James server UTF8/IDN support



[ 
https://issues.apache.org/jira/browse/JAMES-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851670#comment-17851670
 ] 

Benoit Tellier commented on JAMES-3939:
---

Hello. No James do not yet support IDN.

Contributions are welcome on this, if you are interested.
If exposing this extension we might want to ensure:

 - Those emails can be read in an IMAP4 compliant way
 - Those emails can well be relayed to third party MXes. And what happens with 
/ without SMTPUTF8 support.

> Apache James server UTF8/IDN support
> 
>
> Key: JAMES-3939
> URL: https://issues.apache.org/jira/browse/JAMES-3939
> Project: James Server
>  Issue Type: Bug
>  Components: configuration
>Affects Versions: 3.7.3
>Reporter: Evgeny Agronsky
>Priority: Major
>
> I'm running Postfix MTA with content filter based on Apache James, whenever 
> Postfix relays a mail to the content-filter, for example with *{{From}}* 
> field containing UTF8 characters, it fails to deliver the mail to Apache 
> James with error:
>  
> {code:java}
> SMTPUTF8 is required, but was not offered by host{code}
>  
> Postfix for example declares UTF8 support in it's documentation and is able 
> to handle the message.
> Unfortunately I could not find any documentation whatsoever on how to enable 
> SMTPUTF8 support on James, so I'm not sure if it's supported/unsupported/bug. 
> Does James support IDN?
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3941) NPE with health checks



 [ 
https://issues.apache.org/jira/browse/JAMES-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3941.
-
Resolution: Fixed

Latest James release did decrease the logging level of such events so that port 
scan based healthcheck no longer trigger a warning/error.

> NPE with health checks
> --
>
> Key: JAMES-3941
> URL: https://issues.apache.org/jira/browse/JAMES-3941
> Project: James Server
>  Issue Type: Bug
>  Components: app
>Affects Versions: 3.8.0
> Environment: AWS
>Reporter: vincent PIRAT
>Priority: Minor
>
> When James is installed on an EC2 instance behind a Network loadbalancer, it 
> looks like health checks are generating NPE when James tries to retrieve IP 
> info from the connection.
> Does not seem to have functional impact but generates lot of noise.
> James is configured with 2 open ports (143 for Imap and 465 for Smtp).
>  
> {code:java}
> Sep 18 12:58:53 aws-instance ec2: 18-Sep-2023 12:58:53.053 ERROR 
> [smtpserver-io-2] 
> org.apache.james.protocols.netty.BasicChannelInboundHandler.exceptionCaught:271
>  - Unable to process request
> Sep 18 12:58:53 aws-instance ec2: java.lang.NullPointerException: null
> Sep 18 12:58:53 aws-instance ec2: at 
> org.apache.james.protocols.smtp.SMTPSessionImpl.(SMTPSessionImpl.java:44)
>  ~[protocols-smtp-3.8.0.jar:3.8.0]
> Sep 18 12:58:53 aws-instance ec2: at 
> org.apache.james.smtpserver.ExtendedSMTPSession.(ExtendedSMTPSession.java:37)
>  ~[james-server-protocols-smtp-3.8.0.jar:3.8.0]
> Sep 18 12:58:53 aws-instance ec2: at 
> org.apache.james.smtpserver.netty.SMTPServer$1.newSession(SMTPServer.java:216)
>  ~[james-server-protocols-smtp-3.8.0.jar:3.8.0]
> Sep 18 12:58:53 aws-instance ec2: at 
> org.apache.james.protocols.netty.BasicChannelInboundHandler.createSession(BasicChannelInboundHandler.java:244)
>  ~[protocols-netty-3.8.0.jar:3.8.0]
> Sep 18 12:58:53 aws-instance ec2: at 
> org.apache.james.protocols.netty.BasicChannelInboundHandler.channelActive(BasicChannelInboundHandler.java:96)
>  ~[protocols-netty-3.8.0.jar:3.8.0]
> Sep 18 12:58:53 aws-instance ec2: at 
> org.apache.james.smtpserver.netty.SMTPChannelInboundHandler.channelActive(SMTPChannelInboundHandler.java:63)
>  ~[james-server-protocols-smtp-3.8.0.jar:3.8.0]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:262)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:238)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:231)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:69)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.handler.timeout.IdleStateHandler.channelActive(IdleStateHandler.java:271)
>  ~[netty-handler-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:260)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:238)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:231)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:69)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:260)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.AbstractChannelHandlerContext.access$200(AbstractChannelHandlerContext.java:61)
>  ~[netty-transport-4.1.86.Final.jar:4.1.86.Final]
> Sep 18 12:58:53 aws-instance ec2: at 
> io.netty.channel.AbstractChannelHandlerContext$3.run(AbstractChannelHandlerContext.java:243)
>  ~[netty-transport-4.1.86

[jira] [Commented] (JAMES-3943) Assemble a scaling SMTP server backed by pulsar (and PG)



[ 
https://issues.apache.org/jira/browse/JAMES-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851664#comment-17851664
 ] 

Benoit Tellier commented on JAMES-3943:
---

Can this work be closed or we can expect some further developments on this?

> Assemble a scaling SMTP server backed by pulsar (and PG)
> 
>
> Key: JAMES-3943
> URL: https://issues.apache.org/jira/browse/JAMES-3943
> Project: James Server
>  Issue Type: Sub-task
>  Components: app
>Reporter: Matthieu Baechler
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3946) Proposal: DropLists (akka blacklists)



[ 
https://issues.apache.org/jira/browse/JAMES-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851663#comment-17851663
 ] 

Benoit Tellier commented on JAMES-3946:
---

With the lastest PR delivering the feature as an optional "module chooser for 
the distributed-app" this work is feature complete enough IMO to land onto the 
master branch.

> Proposal: DropLists (akka blacklists)
> -
>
> Key: JAMES-3946
> URL: https://issues.apache.org/jira/browse/JAMES-3946
> Project: James Server
>  Issue Type: New Feature
>  Components: data, webadmin
>Reporter: Benoit Tellier
>Priority: Major
>  Time Spent: 11h
>  Remaining Estimate: 0h
>
> h3. What?
> Blacklist are a classical email related feature.
> Having a easy to activate core module to handle this feature would IMO be 
> nice.
> Ideally blacklist entries should be added globally, at the domain level, at 
> the user level and should concern individual addresses as well as entire 
> domains.
> h3. Disclaimer
> We identified this feature while working on TMail.
> I am convinced that this is generic enough to land on James. But might 
> consensus reject this, we could still make this a TMail module :-)
> Ideally I'd like to have this fully as an option, not activated by default.
> h3. How?
> Again, proposal here. My first shot was to think of RRTs but they do not take 
> sender into account (sd).
> Write in `/server/data/data-api` a `Droplist interfaces.
> A drop list entry is comprised of
>  - **ownerScope**: `global | domain | user`
>  - **owner**: String. 
> - For ownerScope global: this is always `ALL`.
> - For ownerScope domain: this is the domain, eg `domain.tld`
> - For ownerScope user, this is the users, eg `b...@domain.tld`
> - **deniedEntityType**: String. One of `address | domain`
> - **deniedEntity**: String. Either the domain or the address.
> {code:java}
> interface DropList {
> Mono add(DropListEntry entry);
> Mono remove(DropListEntry entry);
> Flux list(OwnerScope ownerSocpe, Owner owner);
>enum Status {
>ALLOWED,
>BLOCKED
>}
> Mono query(OwnerScope ownerSocpe, Owner owner, MailAddress 
> sender);
> }
> {code}
> And provide a memory + a Cassandra implementation of the DropList.
> Write a `IsInDropList` matcher: Given `attac...@evil.com` sends a mail to 
> `target@localhost`, the following queries are done:
>  - ownerScope all, owner All, deniedEntityType domain, deniedEntity evil.com
>  - ownerScope all, owner All, deniedEntityType address, deniedEntity 
> attac...@evil.com
>  - ownerScope domain, owner localhost, deniedEntityType domain, deniedEntity 
> evil.com
>  - ownerScope domain, owner localhost, deniedEntityType address, deniedEntity 
> attac...@evil.com
>  - ownerScope user, owner target@localhost, deniedEntityType domain, 
> deniedEntity evil.com
>  - ownerScope user, owner target@localhost, deniedEntityType address, 
> deniedEntity attac...@evil.com
> Manage to do only one set of queries at scope global. Manage to do one set of 
> queries at scope domain per domain!
> Webadmin APIs to manage the Drop List:
> {code:java}
> GET /droplist/global?deniedEntityType=null|domain|address
> [ "evil.com", "devil.com", "bad_...@crime.com", "hac...@murder.org" ]
> HEAD /droplist/global/evil.com
> HEAD /droplist/global/bad_...@murder.org
> 204 // 404
> PUT /droplist/global/evil.com
> PUT /droplist/global/bad_...@murder.org
> -> adds the entry into the droplist
> DELETE /droplist/global/evil.com
> DELETE /droplist/global/bad_...@murder.org
> -> removes the entry from the droplist
> 
> GET /droplist/domain/target.com?deniedEntityType=null|domain|address
> [ "evil.com", "devil.com", "bad_...@crime.com", "hac...@murder.org" ]
> HEAD /droplist/domain/target.com/evil.com
> HEAD /droplist/domain/target.com/bad_...@murder.org
> 204 // 404
> PUT /droplist/domain/target.com/evil.com
> PUT /droplist/domain/target.com/bad_...@murder.org
> -> adds the entry into the droplist
> DELETE /droplist/domain/target.com/evil.com
> DELETE /droplist/domain/target.com/bad_...@murder.org
> -> removes the entry from the droplist
> 
> GET /droplist/user/b...@target.com?deniedEntityType=null|domain|address
> [ "evil.com", "devil.com", "bad_...@crime.com", "hac...@murder.org" ]
> HEAD /droplist/user/b...@target.com/evil.com
> HEAD /droplist/user/b...@target.com/bad

[jira] [Commented] (JAMES-3955) James stops consuming sometimes RabbitMQ queue



[ 
https://issues.apache.org/jira/browse/JAMES-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851662#comment-17851662
 ] 

Benoit Tellier commented on JAMES-3955:
---

Looking at the code I have the impression the timeout only is applied on the 
queue (great!)

But the task never is wrapped onto a reactor timeout prior to the consumer 
timeout.

This means that if I submit a 2 day long task it would effectively KILL the 
consumer => bug.

A nicer behaviour would be:

to put a 10% margin onto the RabbitMQ consumer timeout
to specify .timeout(consumerTimeout) somewhere into the reactive chain
Catch the reactor timeout error and eplicitly CANCEL the task.

Thoughts?

> James stops consuming sometimes RabbitMQ queue
> --
>
> Key: JAMES-3955
> URL: https://issues.apache.org/jira/browse/JAMES-3955
> Project: James Server
>  Issue Type: Improvement
>  Components: rabbitmq
>Reporter: René Cordier
>Priority: Major
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> We sometimes had troubles with RabbitMQ in some production environments where 
> james would stop consuming some queues (like the mail queue) and we never 
> would understand really why, and we would just restart James in this case.
> Well recently I had similar issues but with TaskManagerWorkQueue. Except that 
> we managed to reproduce the problem manually. We have a task we play at night 
> that can take a long time to play. After had some other planned tasks as 
> well, we could observe the following pattern:
> While the heavy task is being executed by James, others are pilling up in the 
> TaskManagerWorkQueue. They getting unacked by James, meaning it's telling 
> RabbitMQ that it will consume them later (as James executes one task at a 
> time). Except that after 30 minutes after the first unacked item in the 
> queue, could see James stopping consuming the queue, and all items coming 
> back to the ready state.
> After looking around RabbitMQ configuration: 
> [https://www.rabbitmq.com/consumers.html#acknowledgement-timeout]
> RabbitMQ will close the channel with a `PRECONDITION_FAILED` channel 
> exception when detecting that an item here the first one being unacked) has 
> not been consumed within 30 minutes. Matching with what we observed actually.
> From this I guess we could deduce that when we had a similar issue with the 
> mail queue, maybe James failed to consume properly a message or failed at 
> acknowledging it for some reason and got the channel closed by RabbitMQ. 
> Which I guess is there to prevent having messages being stuck if the consumer 
> has issue to ack it correctly. 
> From there, there is some actions we can take to prevent this:
>  * adding error logs when we get the channel closed on such an exception
>  * trying to reconnect to the channel when such an exception occurs
>  * on at least important queues like task manager queue, mail queue, event bus
>  * potentially try to audit as well if in some cases we do not ack/nack the 
> message back
>  *  giving the possibility to increase the consumer timeout of the above 
> queue with the `x-consumer-timeout` queue argument (would require to run 
> rabbitmq 3.12 at least)
> For now we can as well increase that timeout in rabbitmq.conf to minimize the 
> problems.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Reopened] (JAMES-3955) James stops consuming sometimes RabbitMQ queue



 [ 
https://issues.apache.org/jira/browse/JAMES-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier reopened JAMES-3955:
---

> James stops consuming sometimes RabbitMQ queue
> --
>
> Key: JAMES-3955
> URL: https://issues.apache.org/jira/browse/JAMES-3955
> Project: James Server
>  Issue Type: Improvement
>  Components: rabbitmq
>Reporter: René Cordier
>Priority: Major
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> We sometimes had troubles with RabbitMQ in some production environments where 
> james would stop consuming some queues (like the mail queue) and we never 
> would understand really why, and we would just restart James in this case.
> Well recently I had similar issues but with TaskManagerWorkQueue. Except that 
> we managed to reproduce the problem manually. We have a task we play at night 
> that can take a long time to play. After had some other planned tasks as 
> well, we could observe the following pattern:
> While the heavy task is being executed by James, others are pilling up in the 
> TaskManagerWorkQueue. They getting unacked by James, meaning it's telling 
> RabbitMQ that it will consume them later (as James executes one task at a 
> time). Except that after 30 minutes after the first unacked item in the 
> queue, could see James stopping consuming the queue, and all items coming 
> back to the ready state.
> After looking around RabbitMQ configuration: 
> [https://www.rabbitmq.com/consumers.html#acknowledgement-timeout]
> RabbitMQ will close the channel with a `PRECONDITION_FAILED` channel 
> exception when detecting that an item here the first one being unacked) has 
> not been consumed within 30 minutes. Matching with what we observed actually.
> From this I guess we could deduce that when we had a similar issue with the 
> mail queue, maybe James failed to consume properly a message or failed at 
> acknowledging it for some reason and got the channel closed by RabbitMQ. 
> Which I guess is there to prevent having messages being stuck if the consumer 
> has issue to ack it correctly. 
> From there, there is some actions we can take to prevent this:
>  * adding error logs when we get the channel closed on such an exception
>  * trying to reconnect to the channel when such an exception occurs
>  * on at least important queues like task manager queue, mail queue, event bus
>  * potentially try to audit as well if in some cases we do not ack/nack the 
> message back
>  *  giving the possibility to increase the consumer timeout of the above 
> queue with the `x-consumer-timeout` queue argument (would require to run 
> rabbitmq 3.12 at least)
> For now we can as well increase that timeout in rabbitmq.conf to minimize the 
> problems.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Created] (JAMES-4042) Task manager: issues with long tasks

Benoit Tellier created JAMES-4042:
-

 Summary: Task manager: issues with long tasks
 Key: JAMES-4042
 URL: https://issues.apache.org/jira/browse/JAMES-4042
 Project: James Server
  Issue Type: Improvement
  Components: rabbitmq, task
Affects Versions: master, 3.8.2
Reporter: Benoit Tellier


See JAMES-3955

Today a tasks per default obey a 1 day consumer timeout (configurable!)

Yet, we might have some tasks like deduplication that might take longer than a 
day to complete...

As of today tihis means such a task would complete but very likely crash the 
consumer.

Likely, getting such long jobs in the first place might not be the best idea.

Things like deduplication could be packaged in separate binaries that one could 
run independently of James thus avoiding the issue altogether... We could allow 
this as an alternative way to run such long tasks.

Moreover, I think if needed we could write a Proof Of Concept of running the GC 
on top of something like Apache Spark to leverage parallele computations.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3955) James stops consuming sometimes RabbitMQ queue



 [ 
https://issues.apache.org/jira/browse/JAMES-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3955.
-
Resolution: Fixed

Related PRs merged.

We IMO might still need a solution for very long running tasks that are a pain.

> James stops consuming sometimes RabbitMQ queue
> --
>
> Key: JAMES-3955
> URL: https://issues.apache.org/jira/browse/JAMES-3955
> Project: James Server
>  Issue Type: Improvement
>  Components: rabbitmq
>Reporter: René Cordier
>Priority: Major
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> We sometimes had troubles with RabbitMQ in some production environments where 
> james would stop consuming some queues (like the mail queue) and we never 
> would understand really why, and we would just restart James in this case.
> Well recently I had similar issues but with TaskManagerWorkQueue. Except that 
> we managed to reproduce the problem manually. We have a task we play at night 
> that can take a long time to play. After had some other planned tasks as 
> well, we could observe the following pattern:
> While the heavy task is being executed by James, others are pilling up in the 
> TaskManagerWorkQueue. They getting unacked by James, meaning it's telling 
> RabbitMQ that it will consume them later (as James executes one task at a 
> time). Except that after 30 minutes after the first unacked item in the 
> queue, could see James stopping consuming the queue, and all items coming 
> back to the ready state.
> After looking around RabbitMQ configuration: 
> [https://www.rabbitmq.com/consumers.html#acknowledgement-timeout]
> RabbitMQ will close the channel with a `PRECONDITION_FAILED` channel 
> exception when detecting that an item here the first one being unacked) has 
> not been consumed within 30 minutes. Matching with what we observed actually.
> From this I guess we could deduce that when we had a similar issue with the 
> mail queue, maybe James failed to consume properly a message or failed at 
> acknowledging it for some reason and got the channel closed by RabbitMQ. 
> Which I guess is there to prevent having messages being stuck if the consumer 
> has issue to ack it correctly. 
> From there, there is some actions we can take to prevent this:
>  * adding error logs when we get the channel closed on such an exception
>  * trying to reconnect to the channel when such an exception occurs
>  * on at least important queues like task manager queue, mail queue, event bus
>  * potentially try to audit as well if in some cases we do not ack/nack the 
> message back
>  *  giving the possibility to increase the consumer timeout of the above 
> queue with the `x-consumer-timeout` queue argument (would require to run 
> rabbitmq 3.12 at least)
> For now we can as well increase that timeout in rabbitmq.conf to minimize the 
> problems.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3956) Redis Event bus



 [ 
https://issues.apache.org/jira/browse/JAMES-3956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3956.
-
Resolution: Duplicate

See https://issues.apache.org/jira/browse/JAMES-3996

> Redis Event bus
> ---
>
> Key: JAMES-3956
> URL: https://issues.apache.org/jira/browse/JAMES-3956
> Project: James Server
>  Issue Type: New Feature
>  Components: eventbus
>Reporter: Maksim Meliashchuk
>Priority: Major
>  Labels: features
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Enhance event bus capabilities by implementing an additional high-speed event 
> bus due efficient memory-based storage.
> *How?*
> Implement:
> [https://github.com/apache/james-project/tree/master/event-bus/api]
>  * Create sub maven module for Redis in backends-common
>  * Use a memory event bus for groups
>  * Use Redis for handling Pub/Sub
>  * Use [lettuce|https://github.com/lettuce-io/lettuce-core] for the 
> non-blocking Redis driver
>  * Update docs



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3970) How to disable the dependency on JPA persistence module(james-database.properties) for James version 3.8.0



 [ 
https://issues.apache.org/jira/browse/JAMES-3970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3970.
-
Resolution: Not A Problem

> How to disable the dependency on JPA persistence 
> module(james-database.properties) for James version 3.8.0
> --
>
> Key: JAMES-3970
> URL: https://issues.apache.org/jira/browse/JAMES-3970
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.0
>Reporter: Sachin Nilajkar
>Priority: Blocker
>
> How to disable the dependency on JPA persistence 
> module(james-database.properties) for James version 3.8.0
> We want James 3.8.0 to be just as an email relay without any email 
> persistence in filestore or database. 
> We are looking for an option to disable reference to  
> james-database.properties file during James server startup so that James can 
> be used only as relay server. Kindly suggest if this requirement is possible 
> with James version 3.8.0. 
>  
> We are using Java version :  Java 11.0.18
> Maven version: 3.6.3



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3969) Compilation issue for James 3.8.0 related to scala-maven-plugin



[ 
https://issues.apache.org/jira/browse/JAMES-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851653#comment-17851653
 ] 

Benoit Tellier commented on JAMES-3969:
---

Does this still happen?

> Compilation issue for James 3.8.0 related to scala-maven-plugin
> ---
>
> Key: JAMES-3969
> URL: https://issues.apache.org/jira/browse/JAMES-3969
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.0
> Environment: Development
>Reporter: Sachin Nilajkar
>Priority: Blocker
>
> We are compiling the James source file - 
> [james-project-3.8.0-source-release.zip|https://www.apache.org/dyn/closer.lua/james/server/3.8.0/james-project-3.8.0-source-release.zip]
>  but compilation getting failed with error as - 
> [ERROR]: Failed to execute goal 
> alchim31.maven:scala-maven-plugin:4.6.1:compile (scala-compile-first) on 
> project apache-james-backend-pulsar
>  
> which is the correct version of "scala-maven-plugin" to be use for this 
> compilation?
> We are using Java 11 and
> tried various versions of Maven such as 3.6.0 till 3.9.6 but still getting 
> error so kindly help to provide correct version of "scala-maven-plugin" to be 
> use for this compilation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3971) James server startup issue with error as org.apache.openjpa.persistence.PersistenceProductDerivation:java.lang.NoClassDefFoundError



 [ 
https://issues.apache.org/jira/browse/JAMES-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3971.
-
Resolution: Fixed

> James server startup issue with error as  
> org.apache.openjpa.persistence.PersistenceProductDerivation:java.lang.NoClassDefFoundError
> 
>
> Key: JAMES-3971
> URL: https://issues.apache.org/jira/browse/JAMES-3971
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.0
>Reporter: Sachin Nilajkar
>Priority: Blocker
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We downloaded the binary file - 
> [*https://dlcdn.apache.org/james/server/3.8.0/james-server-jpa-guice.zip*] 
> and follow the documentation for installation in link - 
> [https://james.apache.org/server/install/guice-jpa-smtp.html]
>  
> As part of this the last step to launch the james server we are getting error 
> as - 
> #java -classpath 
> '//james-server-jpa-guice/james-server-jpa-guice.jar:james-server-jpa-guice.lib/*'
>  
> -javaagent:'//james-server-jpa-guice/james-server-jpa-app.lib/openjpa-3.2.0.jar'
>  -Dlogback.configurationFile='conf/logback.xml'  
> org.apache.james.JPAJamesServerMain
> Some product derivations are being skipped. For information about product 
> derivation status, run:
> java org.apache.openjpa.lib.conf.ProductDerivations
> org.apache.openjpa.persistence.PersistenceProductDerivation:java.lang.NoClassDefFoundError:
>  javax/persistence/spi/PersistenceUnitInfo
> Error: Could not find or load main class org.apache.james.JPAJamesServerMain 
> Please help to check and let us know how to fix it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3972) Unable to download - james-jpa-smtp-guice-3.3.0.zip



 [ 
https://issues.apache.org/jira/browse/JAMES-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3972.
-
Resolution: Not A Problem

> Unable to download - james-jpa-smtp-guice-3.3.0.zip
> ---
>
> Key: JAMES-3972
> URL: https://issues.apache.org/jira/browse/JAMES-3972
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.3.0
>Reporter: Sachin Nilajkar
>Priority: Blocker
>
> As per documentation - 
> [https://james.apache.org/server/install/guice-jpa-smtp.html] 
> we need to download  james-jpa-smtp-guice-3.3.0.zip but unable to get the 
> source and hyperlink routed to another page on which mentioned zip file not 
> available.
>  
> Do we have version  james-jpa-smtp-guice-3.8.0.zip? if yes then please 
> provide source to download it.
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3975) James server log/data file reference



 [ 
https://issues.apache.org/jira/browse/JAMES-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3975.
-
Resolution: Fixed

closing because this ticket is inactive.

Please open a new ticket if you have newer elements.

> James server log/data file reference
> 
>
> Key: JAMES-3975
> URL: https://issues.apache.org/jira/browse/JAMES-3975
> Project: James Server
>  Issue Type: Wish
>  Components: James Core
>Affects Versions: 3.8.0
>Reporter: Sachin Nilajkar
>Priority: Major
>
> We are using Apache James Server version 3.8.0
> I want to know the jar file name used  for creation for given log/data files 
> such as -
> derby.log
> .../KahaDB/db-1.log
> .../KahaDB/db.redo
> /KahaDB/db.data



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4037) Errors during connect or sync through IMAP



 [ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4037.
-
Resolution: Fixed

Hi Ilya,

Thanks for all your hard work on this topic!

As identified problems are solved, I close the ticket.

Feel free to open a new one might you experience any other issue.

Best regards,

Benoit

> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.java:755)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.request(FluxIterable.java:620)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onSubscribe(MonoFlatMapMany.java:245)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:179) 
> ~[reactor-core-3.5.1.jar:3.5.1]
>   at reactor.core.publisher.FluxIterable.subscribe(FluxIter

[jira] [Commented] (JAMES-3994) replace Akka with Apache Pekko



[ 
https://issues.apache.org/jira/browse/JAMES-3994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851647#comment-17851647
 ] 

Benoit Tellier commented on JAMES-3994:
---

Sorry for missing this! Of course this non compliance shall be addressed.

I started a mailing list thread on this. I would expect some contributors to 
show up and heroically migrate this to Apache Pekko; 

In order to buy them some time to perform this change I downgraded Akka to the 
latest ASF v2 compatible release. 
https://github.com/apache/james-project/pull/2270

Thanks a lot for the report...

> replace Akka with Apache Pekko
> --
>
> Key: JAMES-3994
> URL: https://issues.apache.org/jira/browse/JAMES-3994
> Project: James Server
>  Issue Type: Task
>  Components: Build System
>Reporter: PJ Fanning
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Akka is no longer OSS. Apache Pekko is an Open Source fork.
> pulsar4s now supports Pekko streams.
> So you should be in a reasonable position to rewrite to use Pekko. It's 
> usually pretty straightforward.
> https://pekko.apache.org/docs/pekko/current/project/migration-guides.html



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-3996) POC: Move RabbitMQ Event bus user notifications to Redis?



[ 
https://issues.apache.org/jira/browse/JAMES-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851634#comment-17851634
 ] 

Benoit Tellier commented on JAMES-3996:
---

For the record this is the mailing list thread: 
https://www.mail-archive.com/server-dev@james.apache.org/msg73913.html

This POC was conclusive, Linagora is yet to be contributing this as an option 
to the ASF.

> POC: Move RabbitMQ Event bus user notifications to Redis?
> -
>
> Key: JAMES-3996
> URL: https://issues.apache.org/jira/browse/JAMES-3996
> Project: James Server
>  Issue Type: Improvement
>Reporter: René Cordier
>Priority: Major
>
> In a setup with a RabbitMQ cluster, and quorum queues, we observed that if 
> Cassandra was unavailable for some reason, heavy nacks causing very high 
> absurd traffic with heavy memory usage, creating overall issues with the 
> event bus in general.
> Here the goal would be to investigate if it's possible to use Redis instead 
> of RabbitMQ for delegating the user notifications. Might use Redis pub-sub or 
> streams to achieve this, by registering the event listeners on Redis instead 
> of RabbitMQ.
> Concerned code on RabbitMQ event bus:
> {code:java}
> default Publisher register(EventListener listener, 
> RegistrationKey key) \{
> return register(EventListener.wrapReactive(listener), key);
> }
> Publisher register(EventListener.ReactiveEventListener 
> listener, RegistrationKey key);
> default Publisher 
> register(EventListener.ReactiveEventListener listener, 
> Collection keys) \{
> return Flux.fromIterable(keys)
> .concatMap(key -> register(listener, key))
> .reduce((reg1, reg2) -> () -> Flux.merge(reg1.unregister(), 
> reg2.unregister()))
> .map(unRegistrationWithMergedFlux -> () -> 
> Mono.from(Flux.from(unRegistrationWithMergedFlux.unregister())
> .then()));
> }
> {code}
> It's just a POC to see first if it's achievable or not, with performance 
> tests. If results are positive, we might envisage to integrate it in a 
> cleaner way. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4000) JMAP should guess charset of text parts when missing



 [ 
https://issues.apache.org/jira/browse/JAMES-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4000.
-
Resolution: Won't Fix

In the end a configurable default charset was doing it for us.

So far I would consider guessing the charset overkill and not really needed.

As such, closing, but if you want this feel free to reopen.

> JMAP should guess charset of text parts when missing
> 
>
> Key: JAMES-4000
> URL: https://issues.apache.org/jira/browse/JAMES-4000
> Project: James Server
>  Issue Type: Improvement
>Affects Versions: 3.8.0, 3.8.1
>Reporter: René Cordier
>Priority: Major
> Fix For: 3.9.0
>
> Attachments: jmap-charset-missing.png
>
>
> Given an email with a text part without charset
> {code:java}
> =_1706602778-31669-7
> Content-Type: text/plain; name="avertissement.txt"
> Content-Disposition: inline; filename="avertissement.txt"
> Content-Transfer-Encoding: base64
> Q2UgbWVzc2FnZSDpbGVjdHJvbmlxdWUgZXQgdG91cyBsZXMgZmljaGllcnMgYXR0YWNo6XMgcXUn
> aWwgY29udGllbnQgc29udCBjb25maWRlbnRpZWxzIGV0IGRlc3RpbulzIGV4Y2x1c2l2ZW1lbnQg
> {code}
>  
> Today James fails at rendering this properly (cf attachment)
> This is because we uses `US_ASCII` as a default when no charset is specified.
> Suggestion: we should rather "guess" the charset when that's the case: 
> [https://github.com/albfernandez/juniversalchardet]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-3999) Listener: define priority and apply priority



 [ 
https://issues.apache.org/jira/browse/JAMES-3999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-3999.
-
Resolution: Won't Fix

Not need.

As exposed above this was more linked to a configuration issue.

> Listener: define priority and apply priority
> 
>
> Key: JAMES-3999
> URL: https://issues.apache.org/jira/browse/JAMES-3999
> Project: James Server
>  Issue Type: Improvement
>Affects Versions: 3.8.0, 3.8.1
>Reporter: René Cordier
>Priority: Major
> Fix For: 3.9.0
>
>
> Some listener should be executed quicklier than others.
> This is paramount to ensure best effort upon resynchronisation.
> The idea would be to prioritize in the execution order listeners that have a 
> higher priority to avoid causing delays.
> JMAP listener especially (EmailQueryView, MailboxChange) should be executed 
> with the highest priority possible.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4022) James server is getting crashed or stopped after 20 mins or with load



 [ 
https://issues.apache.org/jira/browse/JAMES-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4022.
-
Resolution: Invalid

Old version, no more input from the reporter.

I close but if you want to report similar issue on a modern james version 
please feelfree to open a ticket.

> James server is getting crashed or stopped after 20 mins or with load
> -
>
> Key: JAMES-4022
> URL: https://issues.apache.org/jira/browse/JAMES-4022
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.3.0
>Reporter: mohan
>Priority: Major
> Fix For: 3.3.0
>
>
> The below occured on blazemeter perf testing.For 1000 users with 1hour perf 
> test scheduled.it ran only 10 mins after that no requests went jamesserver 
> hosts because ,james are down.
> 2024-03-22 12:20:39,665 INFO o.a.j.t.JMeterThread: Thread started: Thread 
> Group-ThreadStarter 1-249
> 2024-03-22 12:20:39,665 INFO o.a.j.t.JMeterThread: Thread started: Thread 
> Group-ThreadStarter 1-250
> 2024-03-22 12:32:23,824 WARN o.a.j.p.s.s.SmtpSampler: 
> com.sun.mail.smtp.SMTPSendFailedException: [EOF]
>   at 
> com.sun.mail.smtp.SMTPTransport.issueSendCommand(SMTPTransport.java:2108) 
> ~[mail-1.5.0-b01.jar:1.5.0-b01]
>   at com.sun.mail.smtp.SMTPTransport.finishData(SMTPTransport.java:1889) 
> ~[mail-1.5.0-b01.jar:1.5.0-b01]
>   at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:1120) 
> ~[mail-1.5.0-b01.jar:1.5.0-b01]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4034) SMTP submission: validate FROM header



 [ 
https://issues.apache.org/jira/browse/JAMES-4034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4034.
-
Resolution: Fixed

> SMTP submission: validate FROM header
> -
>
> Key: JAMES-4034
> URL: https://issues.apache.org/jira/browse/JAMES-4034
> Project: James Server
>  Issue Type: Improvement
>  Components: SMTPServer
>        Reporter: Benoit Tellier
>Priority: Major
>  Labels: security
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> h3. Why?
> Prevent and limit email forgery by local users.
> As a James administrator I should have a way to configure SMTP submission to 
> reject emails spoofing somebody else mail adress.
> As of today:
>  - JMAP stacks controls both the transport envelope and the headers (From) as 
> mandated by the JMAP Mail RFC (RFC-8621)
> - However SMTP only controls the transport envelop and not the headers.
> This control is currently implemented into SenderAuthIdentifyVerificationHook 
> and is intendeed as a hook on MAIL FROM SMTP command.
> We shall also enforce a control of the From header upon submission in SMTP.  
> Such controls would be mandatory for use of Apache James in an environment 
> where local users could not be fully trusted, as for instance a SaaS offer.
> h3. How?
> Modify SenderAuthIdentifyVerificationHook in order that it also is a 
> JamesMessageHook called before enqueing and checks each and every From header 
> the same way it checks "MAIL FROM" command.
> Modifying SenderAuthIdentifyVerificationHook would enforce this only for 
> connected users IE mail submission and would not affect email relay. This 
> also ensure this behaviour will apply by default. 
> Refactoring of the underlying AbstractSenderAuthIdentifyVerificationHook 
> might be required.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4012) MDN parsing should be more lemient



 [ 
https://issues.apache.org/jira/browse/JAMES-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4012.
-
Resolution: Fixed

> MDN parsing should be more lemient
> --
>
> Key: JAMES-4012
> URL: https://issues.apache.org/jira/browse/JAMES-4012
> Project: James Server
>  Issue Type: Bug
>  Components: mdn
>        Reporter: Benoit Tellier
>Priority: Major
> Fix For: 3.9.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Some of the MDNs I manage do have a human readable part that do not comply 
> with `text/plain` resulting in the following WARNING logs:
> {code:java}
> javax.mail.MessagingException: MDN can't be parse
>   at 
> org.apache.james.jmap.mailet.ExtractMDNOriginalJMAPMessageId.service(ExtractMDNOriginalJMAPMessageId.ja
> {code}
> Relaxing a bit the constraints on the human readable parts will enable me to 
> resume automated processing on this.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4019) ReactiveThrottler: possible starvation



 [ 
https://issues.apache.org/jira/browse/JAMES-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4019.
-
Resolution: Fixed

> ReactiveThrottler: possible starvation
> --
>
> Key: JAMES-4019
> URL: https://issues.apache.org/jira/browse/JAMES-4019
> Project: James Server
>  Issue Type: Bug
>Affects Versions: 3.8.0, 3.8.1
>        Reporter: Benoit Tellier
>Priority: Major
> Fix For: 3.9.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In production I encountered logs saying there was too much concurrent IMAP 
> request.
> From a starting point in time, my IMAP server stopped accepting request and 
> could not continue progress. Complete dead lock.
> By careful unit testing of ReactiveThrottler I was able to diagnose it was 
> not handling "cancel" correctly and not propagating to items in the queue 
> which would cause the deadlock.
> Note: one can avoid this fully by disabling IMAP throttling.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4027) Make all queues on Rabbitmq quorum queue when option enabled



 [ 
https://issues.apache.org/jira/browse/JAMES-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4027.
-
Resolution: Fixed

> Make all queues on Rabbitmq quorum queue when option enabled
> 
>
> Key: JAMES-4027
> URL: https://issues.apache.org/jira/browse/JAMES-4027
> Project: James Server
>  Issue Type: Bug
>  Components: eventbus, Queue, rabbitmq
>Reporter: Tran Hong Quan
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Today, when the quorum option is enabled, only some queues are quorum queues, 
> not all (e.g. event bus notification queues and Task Manager's termination 
> queues).
> On a James deployment where we use quorum queues and RabbitMQ cluster 3 
> nodes, when a RabbitMQ node outages, James can not be fault tolerant against 
> it.
> I tried to reproduce what happens and here is my theory: 
> The RabbitMQ node that stores the notification queues is down 
> -> James can not publish messages to RabbitMQ and causes e.g. IMAP SELECT, 
> STORE, APPEND, UNSELECT ... commands to fail 
> -> James keeps retrying the publish failures (retry for Group registration 
> which seems to rely on the classic queue too) and queues other IMAP requests.
> -> The IMAP server queue is full and the exception `The IMAP server has 
> reached its maximum capacity` is thrown.
> -> James IMAP becomes a zombie and cascading failures.
> James needs to be more fault-tolerant in this case.
> I propose we apply quorum queues for all the queues when `
> quorum.queues.enable=true` so the queues are still available even when a 
> RabbitMQ node is down, and help James keep functions well.
> We did a POC [here  |https://github.com/apache/james-project/pull/2191]and 
> the full quorum queues helped James be more fault tolerant as expected.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4020) Fully comply with IMAP FETCH



 [ 
https://issues.apache.org/jira/browse/JAMES-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4020.
-
Resolution: Fixed

> Fully comply with IMAP FETCH
> 
>
> Key: JAMES-4020
> URL: https://issues.apache.org/jira/browse/JAMES-4020
> Project: James Server
>  Issue Type: Bug
>  Components: IMAPServer
>Affects Versions: 3.8.0, 3.8.1
>    Reporter: Benoit Tellier
>Priority: Major
> Fix For: 3.9.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> https://datatracker.ietf.org/doc/html/rfc3501#section-6.4.5
> {code:java}
>  Every message has at least one part number.  Non-[MIME-IMB]
>  messages, and non-multipart [MIME-IMB] messages with no
>  encapsulated message, only have a part 1.
> {code}
> But when attempting:
> {code:java}
> C: f1 FETCH 1 (BODY[1.MIME])
> {code}
> It fails with:
> {code:java}
> org.apache.james.mailbox.exception.MailboxException: Unable to parse message
>   at 
> org.apache.james.mailbox.store.ResultUtils.loadMessageResult(ResultUtils.java:177)
>   at 
> org.apache.james.mailbox.store.StoreMessageManager.lambda$getMessagesReactive$13(StoreMessageManager.java:777)
>   at 
> com.github.fge.lambdas.functions.FunctionChainer.lambda$sneakyThrow$49(FunctionChainer.java:74)
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.tryOnNext(FluxMapFuseable.java:313)
>   at 
> reactor.core.publisher.FluxPublishOn$PublishOnConditionalSubscriber.runAsync(FluxPublishOn.java:992)
>   at 
> reactor.core.publisher.FluxPublishOn$PublishOnConditionalSubscriber.run(FluxPublishOn.java:1067)
>   at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
>   at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
>   at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
>  Source)
>   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown 
> Source)
>   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown 
> Source)
>   at java.base/java.lang.Thread.run(Unknown Source)
> Caused by: 
> org.apache.james.mailbox.store.streaming.PartContentBuilder$UnexpectedEOFException:
>  Unexpected EOF
>   at 
> org.apache.james.mailbox.store.streaming.PartContentBuilder.getMimeHeaders(PartContentBuilder.java:235)
>   at 
> org.apache.james.mailbox.store.ResultUtils.addMimeHeaders(ResultUtils.java:257)
>   at 
> org.apache.james.mailbox.store.ResultUtils.addPartContent(ResultUtils.java:213)
>   at 
> org.apache.james.mailbox.store.ResultUtils.addPartContent(ResultUtils.java:191)
>   at 
> org.apache.james.mailbox.store.ResultUtils.loadMessageResult(ResultUtils.java:173)
>   ... 12 common frames omitted
> {code}
> So explicitly add support for "all message should have part 1"



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4033) Option for EventBus to not propagate error when already store failed event in Dead Letter



 [ 
https://issues.apache.org/jira/browse/JAMES-4033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4033.
-
Resolution: Fixed

> Option for EventBus to not propagate error when already store failed event in 
> Dead Letter
> -
>
> Key: JAMES-4033
> URL: https://issues.apache.org/jira/browse/JAMES-4033
> Project: James Server
>  Issue Type: Improvement
>  Components: eventbus, rabbitmq
>Reporter: Tran Hong Quan
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> ## Why
> Today given e.g. RabbitMQ is down, EventBus can not dispatch the event to 
> RabbitMQ, and it would persist the event into the dead letter (in the 
> database). 
> Today even when the event is stored in the dead letter, EventBus still 
> propagates the error back and fails the original caller IMAP APPEND or IMAP 
> STORE for example (while the mail is stored and just Added event has not been 
> processed yet while being in dead letters).
> Given today we are more or less robust on auto reprocessing the failed event 
> (from event dead letters), we could consider not propagating the error back 
> and failing the original call.
> ## How
> Add a boolean configuration entry in `rabbitmq.properties` to enable/disable 
> EventBus error propagation upon storing failed events in dead letters.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4029) Bounce and forward combined can lead to an infinite loop



 [ 
https://issues.apache.org/jira/browse/JAMES-4029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4029.
-
Resolution: Fixed

> Bounce and forward combined can lead to an infinite loop
> 
>
> Key: JAMES-4029
> URL: https://issues.apache.org/jira/browse/JAMES-4029
> Project: James Server
>  Issue Type: Bug
>  Components: Mailet Contributions, RRT
>        Reporter: Benoit Tellier
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> GIVEN all mails sent to b...@gmail.com bounces
> AND b...@avocat.fr forwards his mails to b...@gmail.com
> WHEN custo...@other.com sends an email of  to b...@avocat.fr
> THEN remote delivery for the forwarded message fails and generates a bounce 
> (that is big) is forwarded and bounced again.
> Recommendation:
>  - bounce/dsnBounce should not reset forwqard state
>  - Write integration test for bounce/dsnBounce forward interactions



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4030) Improve Bouncer error handling



 [ 
https://issues.apache.org/jira/browse/JAMES-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4030.
-
Resolution: Fixed

> Improve Bouncer error handling
> --
>
> Key: JAMES-4030
> URL: https://issues.apache.org/jira/browse/JAMES-4030
> Project: James Server
>  Issue Type: Bug
>  Components: Remote Delivery
>        Reporter: Benoit Tellier
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> While looking at RemoteDelivery bouncer code, if MessagingException was 
> thrown then basically the email would be discarded after a warning log.
> This would result in email loss without even a bounce being sent.
> Instead we should carry over the exception, resulting on a nack on the 
> underlying mailqueue and guarantying retries.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4031) James should be able to use multiple RabbitMQ nodes uris



 [ 
https://issues.apache.org/jira/browse/JAMES-4031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4031.
-
Resolution: Fixed

> James should be able to use multiple RabbitMQ nodes uris
> 
>
> Key: JAMES-4031
> URL: https://issues.apache.org/jira/browse/JAMES-4031
> Project: James Server
>  Issue Type: Improvement
>  Components: rabbitmq
>Reporter: Tran Hong Quan
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We are currently pointing to only 1 RabbitMQ service endpoint.
> It would be better to point the RabbitMQ URIs to e.g. list of 3 RabbitMQ 
> nodes (in a cluster) so
> - support directly RabbitMQ cluster usage without a RabbitMQ proxy/load 
> balancer needed.
> - the RabbitMQ driver can better acknowledge the cluster usage and recover 
> quicker during the RabbitMQ node outage.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Closed] (JAMES-4028) MailImpl derivation truncation might make diagnostic harder



 [ 
https://issues.apache.org/jira/browse/JAMES-4028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4028.
-
Resolution: Fixed

> MailImpl derivation truncation might make diagnostic harder
> ---
>
> Key: JAMES-4028
> URL: https://issues.apache.org/jira/browse/JAMES-4028
> Project: James Server
>  Issue Type: Improvement
>    Reporter: Benoit Tellier
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> I was investigating a forward loop ( :-( ) and to my unplaisure, because mail 
> names are limited to 86 characters, 
> This caused the lopped email name not to be prefixed by the name of the 
> original name:
> {code:java}
>  Mail1713173708159-16b6f285-e237-405f-800f-6a54926825e6-!5984740
>   
> 713173708159-16b6f285-e237-405f-800f-6a54926825e6-!5984740-!9588184-!2870433
> 
> 285-e237-405f-800f-6a54926825e6-!5984740-!9588184-!2870433-!9660160-!4179738
>   
> -6a54926825e6-!5984740-!9588184-!2870433-!9660160-!4179738-!0418990-!8605805
> {code}
> Current rule: 
>  - 4 char for 'Mail' prefix
>  - 15 char for current unix timestamp
>  - 1 char separator
>  - 38 char uuid
>  - up to 7 x 9 derivative suffix
> Which means that truncation happens after 4 derivates.
> By twikking slightly generation mechanisms we can get a more compact name 
> that wouldn't get truncated to fir the 86 char rule.
>  - 4 char for 'Mail' prefix
>  - 15 char for current unix timestamp - to be kept as some people might use 
> it as diagnostic information?
> - 1 separator
>  - Generate a 20 char random alphanumeric name. Entropy should be similar 
> with the uuid
>  - Rework each derivative suffix to be 5 char - use `!` as a demiiter and not 
> `-!` and rely on alpha numeric for a more compact representation
> That way truncation never happens, and I could filter easily by base email 
> name onto my favorite mailing system to see the whole history without 
> headache.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JSPF-110) Infinite loop in the handleError method



[ 
https://issues.apache.org/jira/browse/JSPF-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851543#comment-17851543
 ] 

Benoit Tellier commented on JSPF-110:
-

Hello [~adguardemployee]

Thanks for the feedback.

Ok then I propose to move forward with 
https://github.com/apache/james-jspf/pull/20 .

Regarding pushing more debug data into SPFSession, it is not clear to me what 
you exactly expect. Maybe you can share a piece of code and open a pull request 
regarding this?

> Infinite loop in the handleError method
> ---
>
> Key: JSPF-110
> URL: https://issues.apache.org/jira/browse/JSPF-110
> Project: James jSPF
>  Issue Type: Bug
>  Components: Core
>Affects Versions: master
> Environment: Java: 21
> Spring Boot: 3.2.4
>Reporter: aleksey
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Hello, my dear friends!
> I use your library to check SPF in my work.
> {code:java}
> 
> org.apache.james.jspf
> apache-jspf-resolver
> 1.0.3
>  {code}
> Here's an exact example of how I use it:
> {code:java}
> SPF spfChecker = new DefaultSPF();
> spfChecker.setUseBestGuess(true);
> SPFResult spfResult = spfChecker.checkSPF(ipAddress, from, hostname);
> return spfResult.getResult();{code}
> And I have encountered a problem with the AsynchronousSPFExecutor:
> {code:java}
> "Thread-145410" #2022286 [1208634] daemon prio=5 os_prio=0 cpu=1337200.37ms 
> elapsed=1431.06s tid=0x7f38240a8d20 nid=1208634 runnable  
> [0x7f38353e1000]
>java.lang.Thread.State: RUNNABLE
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.handleError(AsynchronousSPFExecutor.java:99)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.lambda$execute$0(AsynchronousSPFExecutor.java:69)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor$$Lambda/0x7f3838a5ea48.accept(Unknown
>  Source)
> at 
> java.util.concurrent.CompletableFuture$UniAccept.tryFire(java.base@21.0.1/CompletableFuture.java:718)
> at 
> java.util.concurrent.CompletableFuture.postComplete(java.base@21.0.1/CompletableFuture.java:510)
> at 
> java.util.concurrent.CompletableFuture.postFire(java.base@21.0.1/CompletableFuture.java:614)
> at 
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(java.base@21.0.1/CompletableFuture.java:1163)
> at 
> java.util.concurrent.CompletableFuture$Completion.run(java.base@21.0.1/CompletableFuture.java:482)
> at java.lang.Thread.runWith(java.base@21.0.1/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.1/Thread.java:1583) {code}
> This infinite loop is affecting the server.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JSPF-110) Infinite loop in the handleError method



[ 
https://issues.apache.org/jira/browse/JSPF-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851134#comment-17851134
 ] 

Benoit Tellier commented on JSPF-110:
-

My shot at it https://github.com/apache/james-jspf/pull/20

I did propagate eceptions not currently handled by the library.

Thought?

> Infinite loop in the handleError method
> ---
>
> Key: JSPF-110
> URL: https://issues.apache.org/jira/browse/JSPF-110
> Project: James jSPF
>  Issue Type: Bug
>  Components: Core
>Affects Versions: master
> Environment: Java: 21
> Spring Boot: 3.2.4
>Reporter: aleksey
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Hello, my dear friends!
> I use your library to check SPF in my work.
> {code:java}
> 
> org.apache.james.jspf
> apache-jspf-resolver
> 1.0.3
>  {code}
> Here's an exact example of how I use it:
> {code:java}
> SPF spfChecker = new DefaultSPF();
> spfChecker.setUseBestGuess(true);
> SPFResult spfResult = spfChecker.checkSPF(ipAddress, from, hostname);
> return spfResult.getResult();{code}
> And I have encountered a problem with the AsynchronousSPFExecutor:
> {code:java}
> "Thread-145410" #2022286 [1208634] daemon prio=5 os_prio=0 cpu=1337200.37ms 
> elapsed=1431.06s tid=0x7f38240a8d20 nid=1208634 runnable  
> [0x7f38353e1000]
>java.lang.Thread.State: RUNNABLE
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.handleError(AsynchronousSPFExecutor.java:99)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.lambda$execute$0(AsynchronousSPFExecutor.java:69)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor$$Lambda/0x7f3838a5ea48.accept(Unknown
>  Source)
> at 
> java.util.concurrent.CompletableFuture$UniAccept.tryFire(java.base@21.0.1/CompletableFuture.java:718)
> at 
> java.util.concurrent.CompletableFuture.postComplete(java.base@21.0.1/CompletableFuture.java:510)
> at 
> java.util.concurrent.CompletableFuture.postFire(java.base@21.0.1/CompletableFuture.java:614)
> at 
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(java.base@21.0.1/CompletableFuture.java:1163)
> at 
> java.util.concurrent.CompletableFuture$Completion.run(java.base@21.0.1/CompletableFuture.java:482)
> at java.lang.Thread.runWith(java.base@21.0.1/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.1/Thread.java:1583) {code}
> This infinite loop is affecting the server.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JSPF-110) Infinite loop in the handleError method



[ 
https://issues.apache.org/jira/browse/JSPF-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851130#comment-17851130
 ] 

Benoit Tellier commented on JSPF-110:
-

Hello,

I'm getting a further look at it.

AsynchronousSPFExecutor and SynchronousSPFExecutor indeed may be affected by an 
infinite loop:


{code:java}
while (e != null) {
while (checker == null || !(checker instanceof 
SPFCheckerExceptionCatcher)) {
checker = session.popChecker();
}
try {
((SPFCheckerExceptionCatcher) checker).onException(e, 
session);
e = null;
} catch (SPFResultException ex) {
e = ex;
} finally {
checker = null;
}
}
{code}

The code explicitly rely on a checker on the stack to be present.

The issue is that if no checker at all is present then the code would loop 
indefinitly and lead to the very issue you describe.

This issue might be caused by errors in some specific checkers (in a position 
not covered by error handling?) but fine for others.

The code in JSPF looks overly complicated to me for what it does, it is very 
hard to wrap my head around this execution flow...

I bet the safest way to solve this would be:

 - 1. Extract the "get me the first checker of XYZ class" onto a separate 
SPFSession method
 - 2. Define a safe enough default behaviour to do when no explicit exception 
handling is found. A review of checkers onException... method might be helpful 
for that.
 - 3 Apply that behaviour

Thought?

> Infinite loop in the handleError method
> ---
>
> Key: JSPF-110
> URL: https://issues.apache.org/jira/browse/JSPF-110
> Project: James jSPF
>  Issue Type: Bug
>  Components: Core
>Affects Versions: master
> Environment: Java: 21
> Spring Boot: 3.2.4
>Reporter: aleksey
>Priority: Major
>  Labels: pull-request-available
>
> Hello, my dear friends!
> I use your library to check SPF in my work.
> {code:java}
> 
> org.apache.james.jspf
> apache-jspf-resolver
> 1.0.3
>  {code}
> Here's an exact example of how I use it:
> {code:java}
> SPF spfChecker = new DefaultSPF();
> spfChecker.setUseBestGuess(true);
> SPFResult spfResult = spfChecker.checkSPF(ipAddress, from, hostname);
> return spfResult.getResult();{code}
> And I have encountered a problem with the AsynchronousSPFExecutor:
> {code:java}
> "Thread-145410" #2022286 [1208634] daemon prio=5 os_prio=0 cpu=1337200.37ms 
> elapsed=1431.06s tid=0x7f38240a8d20 nid=1208634 runnable  
> [0x7f38353e1000]
>java.lang.Thread.State: RUNNABLE
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.handleError(AsynchronousSPFExecutor.java:99)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.lambda$execute$0(AsynchronousSPFExecutor.java:69)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor$$Lambda/0x7f3838a5ea48.accept(Unknown
>  Source)
> at 
> java.util.concurrent.CompletableFuture$UniAccept.tryFire(java.base@21.0.1/CompletableFuture.java:718)
> at 
> java.util.concurrent.CompletableFuture.postComplete(java.base@21.0.1/CompletableFuture.java:510)
> at 
> java.util.concurrent.CompletableFuture.postFire(java.base@21.0.1/CompletableFuture.java:614)
> at 
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(java.base@21.0.1/CompletableFuture.java:1163)
> at 
> java.util.concurrent.CompletableFuture$Completion.run(java.base@21.0.1/CompletableFuture.java:482)
> at java.lang.Thread.runWith(java.base@21.0.1/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.1/Thread.java:1583) {code}
> This infinite loop is affecting the server.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Updated] (JAMES-4032) Improve external forward handling

[
https://issues.apache.org/jira/browse/JAMES-4032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benoit Tellier updated JAMES-4032:
--
Description:
Email deliverability is a tricky topic and yesterday I got one more example for
this...

(The trick is to de able to improve deliverability while still being secure)

h3. The issue

The overall scenario outline is:

{code:java}
GIVEN two local users al...@me.org and b...@me.org on a james based instalation
AND al...@external.org on another email supplier
AND al...@external.org forwards its mail to al...@me.org.
AND this server preserves and resend as is the SMTP transport envelope

WHEN b...@me.org sends an email to al...@external.org

THEN james server rejects the incoming mail from external.org as it requires
authentication
to use local addresses in MAIL FROM
AND a bounce would be generated by external.org for the original sender as the
email could not be forwarded.
{code}

Example of such "external mail providers": orange.fr SFR professional services.

See fwd1.png

!image-2024-04-26-12-04-10-017.png!

h3. Why authentication is currently required in such cases

In order to maximise delivrability, James accepts any email for local users on
port 25 (spam filtering may be applied as part of the reception chain)

However a common attack is email spoofing: a remote attacker with not
whatsoever access sends an email pretending to be a local user. Without further
checks he could submit email as a local user without being authenticated.

To prevent this behaviour James implements `verifySenderIdentity` configuration
parameter in smtpServer.xml configuration file. If using a local mail from the
current behaviour is to actualy mandate authentication.

Anybody else than the user could thus NOT be using this MAIL FROM.

This prevents the following schario: fwd2.png

!image-2024-04-26-12-03-58-932.png!

However this causes the issue discussed above.

h3. Related deliverability concerns

h4. Receiving external forwards orginating from external mail servers

b...@external1.org sends a mail to al...@external2.org which forwards mails as
described above to al...@me.org.

On the last hop james accepts the email as the MAIL FROM (b...@external1.org)
is a remote one: verifySenderIdentity is not triggered for external senders.

On the spam filtering side:
- SPF checks fails (as external2.org did send a mail in external1.org name
which it is not allowed to do
- But DKIM tests pass
- Which should be enough for the spam filtering stack not to flag the message
too violently...

This current behaviour is considered lenient enough. White listing could
further be added for trusted domains if need be.

h4. Handling correctly forwards emition

We tackled the very same issue on the emition side: the goal was to improve
james forwarded email deliverability.

GIVEN b...@me.org forwards its email to b...@external.org
WHEN t...@other.org sends a mail to b...@me.org
THEN me.org would rewrite the MAIL FROM field and use b...@me.org when
forwarding to b...@external.org

Thus SPF check passes.

See https://issues.apache.org/jira/browse/JAMES-3944

h3. Proposed solution

We could experiment turning off sender verification hop on MAIL FROM

Instead we would verify the DKIM signature of the email before spooling it.

That way:

- We would have signed legitimate traffic forwarded back to us and would
accept it
!image-2024-04-26-12-14-35-358.png!

- And we would still reject spoofing attempts as an attacker cant craft a DKIM
signature.

!image-2024-04-26-12-13-43-910.png!

GIVEN that dkim signature of outgoing email is not setted up by default this
behaviour shall be disabled by default.

h3. Proposed changes

Implement a VerifySenderIdentity SMTP message hook that:

GIVEN an unauthenticated sender

It performs a verification of non relaying (ie all rcpts are local)

And would verify the DKIM signature of the mail.

Path to the DKIM public key needs to be provided.

This shall be contributed as part of the DKIM extention.

h3. Additional concerns

Given the Mail Exchange service (MX) and Mail Submission Agent (MSA) being
collocated - the default james architecture,

Then a common misconfiguration is not to configure TLS for SMTP and just have
regular users just try to send mails unauthenticated through SMTP.

This behaviour is classical from MacOS native client.

Current `verifySenderIdentity` behavior rejects such misconfigured client from
the MAIL FROM. Which is convenient. Fail fast:

- The error is more explicit: AUthentication required
- No message payload had been transfered.

Ideally we would like to preserve this behaviour.

The way to achieve this would be to have a "relaxed" mode to
verifySenderIdentity instead of "strict" and "disabled"

This relaxed mode would enforce the current behaviour if it determise the other
party to be a MX

[jira] [Closed] (JAMES-4041) OOM upon IMAP copy



 [ 
https://issues.apache.org/jira/browse/JAMES-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier closed JAMES-4041.
-
Fix Version/s: 3.8.3
   Resolution: Fixed

> OOM upon IMAP copy
> --
>
> Key: JAMES-4041
> URL: https://issues.apache.org/jira/browse/JAMES-4041
> Project: James Server
>  Issue Type: Bug
>  Components: IMAPServer, mailbox
>Affects Versions: master, 3.7.5, 3.8.2
>    Reporter: Benoit Tellier
>Priority: Major
> Fix For: 3.8.3, master
>
> Attachments: Screenshot from 2024-05-29 15-34-44.png
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> I encountered this on production:
> {code:java}
> "java.lang.OutOfMemoryError: Java heap space\n\t
> at java.base/java.util.Arrays.copyOf(Unknown Source)\n\t
> at java.base/java.util.ArrayList.grow(Unknown Source)\n\t
> at java.base/java.util.ArrayList.grow(Unknown Source)\n\t
> at java.base/java.util.ArrayList.add(Unknown Source)\n\t
> at java.base/java.util.ArrayList.add(Unknown Source)\n\t
> at 
> org.apache.james.mailbox.model.MessageRange.split(MessageRange.java:247)\n\t
> at 
> org.apache.james.mailbox.store.MessageBatcher.batchMessagesReactive(MessageBatcher.java:70)\n\t
> at 
> org.apache.james.mailbox.store.StoreMailboxManager.lambda$copyMessagesReactive$48(StoreMailboxManager.java:713)\n\t
> at 
> org.apache.james.mailbox.store.StoreMailboxManager$$Lambda/0x7f12613caab8.apply(Unknown
>  Source)\n\t
> at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:163)\n\t
> at reactor.core.publisher.MonoZip$ZipCoordinator.signal(MonoZip.java:297)\n\t
> at reactor.core.publisher.MonoZip$ZipInner.onNext(MonoZip.java:478)\n\t
> at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)\n\t
> at 
> reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)\n\t
> at reactor.core.publisher.MonoZip$ZipCoordinator.signal(MonoZip.java:297)\n\t
> at reactor.core.publisher.MonoZip$ZipInner.onNext(MonoZip.java:478)\n\t
> at 
> reactor.core.publisher.MonoFlatMap$FlatMapMain.secondComplete(MonoFlatMap.java:245)\n\t
> at 
> reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:305)\n\t
> at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)\n\t
> at 
> reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571)\n\t
> at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)\n\t
> at 
> reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:291)\n\t
> at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)\n\t
> at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)\n\t
> at 
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)\n\t
> at 
> reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)\n\t
> at 
> reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)\n\t
> at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)\n\t
> at 
> reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:181)\n\t
> at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)\n\t
> at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)\n\t
> at java.base/java.util.concurrent.FutureTask.run(Unknown Source)\n"
> {code}
> Was able to reproduce: CF screenshot
> This was actually encountered with the following batchSizes:
> {code:java}
> copy=10
> move=10
> {code}
> And increasing aggressively the batch size was actually usefull as a work 
> around:
> {code:java}
> copy=20
> move=20
> {code}
> However I fear this means the overall batching process for MOVE and COPY 
> makes little sense...
> I do think this could be handle in a pure reactive way:
>  - Fetch all the messages in the range
>  - window them using the batch size
>  - perform the update one window at a time
>  - and finally aggregate the resulting MessageRange
>  
>  I will try to get a shot at it later this week.
>  
>  BTW do my great unpleasure it was not possible to disable batching...
>  
> {code:java}
> Caused by: java.lang.IllegalArgumentException: 'copyBatchSize' must be 
> greater than zero
>   at 
> com.google.common.base.Preconditions.checkArgument(Preconditions.ja

[jira] [Comment Edited] (JSPF-110) Infinite loop in the handleError method



[ 
https://issues.apache.org/jira/browse/JSPF-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850741#comment-17850741
 ] 

Benoit Tellier edited comment on JSPF-110 at 5/30/24 2:37 PM:
--

Hello,

Does this happen on the latest master?

Does this work with the Synchronous executor?


was (Author: btellier):
Hello,

Does this happen on the latest master?

> Infinite loop in the handleError method
> ---
>
> Key: JSPF-110
> URL: https://issues.apache.org/jira/browse/JSPF-110
> Project: James jSPF
>  Issue Type: Bug
>  Components: Core
>Affects Versions: master
> Environment: Java: 21
> Spring Boot: 3.2.4
>Reporter: aleksey
>Priority: Major
>  Labels: pull-request-available
>
> Hello, my dear friends!
> I use your library to check SPF in my work.
> {code:java}
> 
> org.apache.james.jspf
> apache-jspf-resolver
> 1.0.3
>  {code}
> Here's an exact example of how I use it:
> {code:java}
> SPF spfChecker = new DefaultSPF();
> spfChecker.setUseBestGuess(true);
> SPFResult spfResult = spfChecker.checkSPF(ipAddress, from, hostname);
> return spfResult.getResult();{code}
> And I have encountered a problem with the AsynchronousSPFExecutor:
> {code:java}
> "Thread-145410" #2022286 [1208634] daemon prio=5 os_prio=0 cpu=1337200.37ms 
> elapsed=1431.06s tid=0x7f38240a8d20 nid=1208634 runnable  
> [0x7f38353e1000]
>java.lang.Thread.State: RUNNABLE
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.handleError(AsynchronousSPFExecutor.java:99)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.lambda$execute$0(AsynchronousSPFExecutor.java:69)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor$$Lambda/0x7f3838a5ea48.accept(Unknown
>  Source)
> at 
> java.util.concurrent.CompletableFuture$UniAccept.tryFire(java.base@21.0.1/CompletableFuture.java:718)
> at 
> java.util.concurrent.CompletableFuture.postComplete(java.base@21.0.1/CompletableFuture.java:510)
> at 
> java.util.concurrent.CompletableFuture.postFire(java.base@21.0.1/CompletableFuture.java:614)
> at 
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(java.base@21.0.1/CompletableFuture.java:1163)
> at 
> java.util.concurrent.CompletableFuture$Completion.run(java.base@21.0.1/CompletableFuture.java:482)
> at java.lang.Thread.runWith(java.base@21.0.1/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.1/Thread.java:1583) {code}
> This infinite loop is affecting the server.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Comment Edited] (JSPF-110) Infinite loop in the handleError method



[ 
https://issues.apache.org/jira/browse/JSPF-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850741#comment-17850741
 ] 

Benoit Tellier edited comment on JSPF-110 at 5/30/24 2:37 PM:
--

Hello,

Does this happen on the latest master? 
https://github.com/apache/james-jspf/commit/e188ef5db6a76dc687faa0d464eabc06c4896339
 might help a lot!

Does this work with the Synchronous executor?


was (Author: btellier):
Hello,

Does this happen on the latest master?

Does this work with the Synchronous executor?

> Infinite loop in the handleError method
> ---
>
> Key: JSPF-110
> URL: https://issues.apache.org/jira/browse/JSPF-110
> Project: James jSPF
>  Issue Type: Bug
>  Components: Core
>Affects Versions: master
> Environment: Java: 21
> Spring Boot: 3.2.4
>Reporter: aleksey
>Priority: Major
>  Labels: pull-request-available
>
> Hello, my dear friends!
> I use your library to check SPF in my work.
> {code:java}
> 
> org.apache.james.jspf
> apache-jspf-resolver
> 1.0.3
>  {code}
> Here's an exact example of how I use it:
> {code:java}
> SPF spfChecker = new DefaultSPF();
> spfChecker.setUseBestGuess(true);
> SPFResult spfResult = spfChecker.checkSPF(ipAddress, from, hostname);
> return spfResult.getResult();{code}
> And I have encountered a problem with the AsynchronousSPFExecutor:
> {code:java}
> "Thread-145410" #2022286 [1208634] daemon prio=5 os_prio=0 cpu=1337200.37ms 
> elapsed=1431.06s tid=0x7f38240a8d20 nid=1208634 runnable  
> [0x7f38353e1000]
>java.lang.Thread.State: RUNNABLE
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.handleError(AsynchronousSPFExecutor.java:99)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.lambda$execute$0(AsynchronousSPFExecutor.java:69)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor$$Lambda/0x7f3838a5ea48.accept(Unknown
>  Source)
> at 
> java.util.concurrent.CompletableFuture$UniAccept.tryFire(java.base@21.0.1/CompletableFuture.java:718)
> at 
> java.util.concurrent.CompletableFuture.postComplete(java.base@21.0.1/CompletableFuture.java:510)
> at 
> java.util.concurrent.CompletableFuture.postFire(java.base@21.0.1/CompletableFuture.java:614)
> at 
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(java.base@21.0.1/CompletableFuture.java:1163)
> at 
> java.util.concurrent.CompletableFuture$Completion.run(java.base@21.0.1/CompletableFuture.java:482)
> at java.lang.Thread.runWith(java.base@21.0.1/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.1/Thread.java:1583) {code}
> This infinite loop is affecting the server.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JSPF-110) Infinite loop in the handleError method



[ 
https://issues.apache.org/jira/browse/JSPF-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850741#comment-17850741
 ] 

Benoit Tellier commented on JSPF-110:
-

Hello,

Does this happen on the latest master?

> Infinite loop in the handleError method
> ---
>
> Key: JSPF-110
> URL: https://issues.apache.org/jira/browse/JSPF-110
> Project: James jSPF
>  Issue Type: Bug
>  Components: Core
>Affects Versions: master
> Environment: Java: 21
> Spring Boot: 3.2.4
>Reporter: aleksey
>Priority: Major
>  Labels: pull-request-available
>
> Hello, my dear friends!
> I use your library to check SPF in my work.
> {code:java}
> 
> org.apache.james.jspf
> apache-jspf-resolver
> 1.0.3
>  {code}
> Here's an exact example of how I use it:
> {code:java}
> SPF spfChecker = new DefaultSPF();
> spfChecker.setUseBestGuess(true);
> SPFResult spfResult = spfChecker.checkSPF(ipAddress, from, hostname);
> return spfResult.getResult();{code}
> And I have encountered a problem with the AsynchronousSPFExecutor:
> {code:java}
> "Thread-145410" #2022286 [1208634] daemon prio=5 os_prio=0 cpu=1337200.37ms 
> elapsed=1431.06s tid=0x7f38240a8d20 nid=1208634 runnable  
> [0x7f38353e1000]
>java.lang.Thread.State: RUNNABLE
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.handleError(AsynchronousSPFExecutor.java:99)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor.lambda$execute$0(AsynchronousSPFExecutor.java:69)
> at 
> org.apache.james.jspf.executor.AsynchronousSPFExecutor$$Lambda/0x7f3838a5ea48.accept(Unknown
>  Source)
> at 
> java.util.concurrent.CompletableFuture$UniAccept.tryFire(java.base@21.0.1/CompletableFuture.java:718)
> at 
> java.util.concurrent.CompletableFuture.postComplete(java.base@21.0.1/CompletableFuture.java:510)
> at 
> java.util.concurrent.CompletableFuture.postFire(java.base@21.0.1/CompletableFuture.java:614)
> at 
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(java.base@21.0.1/CompletableFuture.java:1163)
> at 
> java.util.concurrent.CompletableFuture$Completion.run(java.base@21.0.1/CompletableFuture.java:482)
> at java.lang.Thread.runWith(java.base@21.0.1/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.1/Thread.java:1583) {code}
> This infinite loop is affecting the server.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Updated] (JAMES-4041) OOM upon IMAP copy



 [ 
https://issues.apache.org/jira/browse/JAMES-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier updated JAMES-4041:
--
Attachment: Screenshot from 2024-05-29 15-34-44.png

> OOM upon IMAP copy
> --
>
> Key: JAMES-4041
> URL: https://issues.apache.org/jira/browse/JAMES-4041
> Project: James Server
>  Issue Type: Bug
>  Components: IMAPServer, mailbox
>Affects Versions: master, 3.7.5, 3.8.2
>    Reporter: Benoit Tellier
>Priority: Major
> Fix For: master
>
> Attachments: Screenshot from 2024-05-29 15-34-44.png
>
>
> I encountered this on production:
> {code:java}
> "java.lang.OutOfMemoryError: Java heap space\n\t
> at java.base/java.util.Arrays.copyOf(Unknown Source)\n\t
> at java.base/java.util.ArrayList.grow(Unknown Source)\n\t
> at java.base/java.util.ArrayList.grow(Unknown Source)\n\t
> at java.base/java.util.ArrayList.add(Unknown Source)\n\t
> at java.base/java.util.ArrayList.add(Unknown Source)\n\t
> at 
> org.apache.james.mailbox.model.MessageRange.split(MessageRange.java:247)\n\t
> at 
> org.apache.james.mailbox.store.MessageBatcher.batchMessagesReactive(MessageBatcher.java:70)\n\t
> at 
> org.apache.james.mailbox.store.StoreMailboxManager.lambda$copyMessagesReactive$48(StoreMailboxManager.java:713)\n\t
> at 
> org.apache.james.mailbox.store.StoreMailboxManager$$Lambda/0x7f12613caab8.apply(Unknown
>  Source)\n\t
> at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:163)\n\t
> at reactor.core.publisher.MonoZip$ZipCoordinator.signal(MonoZip.java:297)\n\t
> at reactor.core.publisher.MonoZip$ZipInner.onNext(MonoZip.java:478)\n\t
> at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)\n\t
> at 
> reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)\n\t
> at reactor.core.publisher.MonoZip$ZipCoordinator.signal(MonoZip.java:297)\n\t
> at reactor.core.publisher.MonoZip$ZipInner.onNext(MonoZip.java:478)\n\t
> at 
> reactor.core.publisher.MonoFlatMap$FlatMapMain.secondComplete(MonoFlatMap.java:245)\n\t
> at 
> reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:305)\n\t
> at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)\n\t
> at 
> reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571)\n\t
> at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)\n\t
> at 
> reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:291)\n\t
> at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)\n\t
> at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)\n\t
> at 
> reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)\n\t
> at 
> reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)\n\t
> at 
> reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)\n\t
> at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)\n\t
> at 
> reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:181)\n\t
> at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)\n\t
> at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)\n\t
> at java.base/java.util.concurrent.FutureTask.run(Unknown Source)\n"
> {code}
> Was able to reproduce: CF screenshot
> This was actually encountered with the following batchSizes:
> {code:java}
> copy=10
> move=10
> {code}
> And increasing aggressively the batch size was actually usefull as a work 
> around:
> {code:java}
> copy=20
> move=20
> {code}
> However I fear this means the overall batching process for MOVE and COPY 
> makes little sense...
> I do think this could be handle in a pure reactive way:
>  - Fetch all the messages in the range
>  - window them using the batch size
>  - perform the update one window at a time
>  - and finally aggregate the resulting MessageRange
>  
>  I will try to get a shot at it later this week.
>  
>  BTW do my great unpleasure it was not possible to disable batching...
>  
> {code:java}
> Caused by: java.lang.IllegalArgumentException: 'copyBatchSize' must be 
> greater than zero
>   at 
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:143)
>   at 
> org.apache.james.mailbox.store.BatchSizes$Builder.copyBatchSize(BatchSizes.java:86)
>

[jira] [Created] (JAMES-4041) OOM upon IMAP copy

Benoit Tellier created JAMES-4041:
-

 Summary: OOM upon IMAP copy
 Key: JAMES-4041
 URL: https://issues.apache.org/jira/browse/JAMES-4041
 Project: James Server
  Issue Type: Bug
  Components: IMAPServer, mailbox
Affects Versions: master, 3.7.5, 3.8.2
Reporter: Benoit Tellier
 Fix For: master


I encountered this on production:


{code:java}
"java.lang.OutOfMemoryError: Java heap space\n\t
at java.base/java.util.Arrays.copyOf(Unknown Source)\n\t
at java.base/java.util.ArrayList.grow(Unknown Source)\n\t
at java.base/java.util.ArrayList.grow(Unknown Source)\n\t
at java.base/java.util.ArrayList.add(Unknown Source)\n\t
at java.base/java.util.ArrayList.add(Unknown Source)\n\t
at org.apache.james.mailbox.model.MessageRange.split(MessageRange.java:247)\n\t
at 
org.apache.james.mailbox.store.MessageBatcher.batchMessagesReactive(MessageBatcher.java:70)\n\t
at 
org.apache.james.mailbox.store.StoreMailboxManager.lambda$copyMessagesReactive$48(StoreMailboxManager.java:713)\n\t
at 
org.apache.james.mailbox.store.StoreMailboxManager$$Lambda/0x7f12613caab8.apply(Unknown
 Source)\n\t
at 
reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:163)\n\t
at reactor.core.publisher.MonoZip$ZipCoordinator.signal(MonoZip.java:297)\n\t
at reactor.core.publisher.MonoZip$ZipInner.onNext(MonoZip.java:478)\n\t
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)\n\t
at 
reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74)\n\t
at reactor.core.publisher.MonoZip$ZipCoordinator.signal(MonoZip.java:297)\n\t
at reactor.core.publisher.MonoZip$ZipInner.onNext(MonoZip.java:478)\n\t
at 
reactor.core.publisher.MonoFlatMap$FlatMapMain.secondComplete(MonoFlatMap.java:245)\n\t
at 
reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:305)\n\t
at 
reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129)\n\t
at 
reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571)\n\t
at 
reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)\n\t
at 
reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:291)\n\t
at 
reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)\n\t
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)\n\t
at 
reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:76)\n\t
at 
reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:165)\n\t
at 
reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79)\n\t
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)\n\t
at 
reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:181)\n\t
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)\n\t
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)\n\t
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)\n"
{code}

Was able to reproduce: CF screenshot

This was actually encountered with the following batchSizes:

{code:java}
copy=10
move=10
{code}

And increasing aggressively the batch size was actually usefull as a work 
around:

{code:java}
copy=20
move=20
{code}


However I fear this means the overall batching process for MOVE and COPY makes 
little sense...

I do think this could be handle in a pure reactive way:
 - Fetch all the messages in the range
 - window them using the batch size
 - perform the update one window at a time
 - and finally aggregate the resulting MessageRange
 
 I will try to get a shot at it later this week.
 
 BTW do my great unpleasure it was not possible to disable batching...
 
{code:java}
Caused by: java.lang.IllegalArgumentException: 'copyBatchSize' must be greater 
than zero
at 
com.google.common.base.Preconditions.checkArgument(Preconditions.java:143)
at 
org.apache.james.mailbox.store.BatchSizes$Builder.copyBatchSize(BatchSizes.java:86)
at 
org.apache.james.modules.mailbox.CassandraSessionModule.getBatchSizesConfiguration(CassandraSessionModule.java:109)
{code}


:-(



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-4038) EHLO Error while connect SMTP

2024-05-27 Thread Benoit Tellier (Jira)



[ 
https://issues.apache.org/jira/browse/JAMES-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849742#comment-17849742
 ] 

Benoit Tellier commented on JAMES-4038:
---

This SMTP bug is a change introduced after 3.8.x, thus only affects master.

> Hello and thanks, eM surely generate alot of errors for idk reason, thanks 
> for ur hard work, alot of changes in code. 

Normal! Interoperability is a must, and here I'm the one guilty for not getting 
one more layer down the normative (cryptic) syntax and missed that point...

Thanks for sticking with us, your work is really useful and appreciated too.

Also I published chibenwa/james:jpa-4038-4037 docker image so that we can get 
em client eventually up to speed on top of james!

> EHLO Error while connect SMTP
> -
>
> Key: JAMES-4038
> URL: https://issues.apache.org/jira/browse/JAMES-4038
> Project: James Server
>  Issue Type: Bug
>  Components: James Core, SMTPServer
>Affects Versions: 3.8.1
> Environment: debian 11, oracle jdk 21
>Reporter: ilya terskov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Hi there
> docker run -d --net=host chibenwa/james-jpa:JAMES-4037-2 --generate-keystore
> docker exec ID james-cli AddDomain domain.online
> docker exec ID james-cli AddUser it@domain.online PASS
> networkctl status
> ●   State: routable
> Address: 172.16.149.37 on eth0
>172.17.0.1 on docker0
> Gateway: 172.16.149.18 (Intel Corporate) on eth0
>DNS: 172.16.149.14
> 172.16.149.33
> While trying get smtp test on 587 with SSL, 465 with STARTTLS or 25 
> with/without SSL/STARTTLS for connect happens this
> 06:35:28.976 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.977 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.982 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.983 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.984 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.989 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.993 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.997 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.998 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.999 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:29.004 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:29.004 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:29.006 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> IPV6 turned off on client side too as well as on server



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-4038) EHLO Error while connect SMTP

2024-05-27 Thread Benoit Tellier (Jira)



[ 
https://issues.apache.org/jira/browse/JAMES-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849723#comment-17849723
 ] 

Benoit Tellier commented on JAMES-4038:
---

Hello,

The issue outlined in https://github.com/apache/james-project/pull/1979 was the 
lack of validation of the EHLO argument wich resulted in arbitrary header 
injections via the received header.

The idea of the PR was to try to stick to the RFC but clearly it did not 
account for https://datatracker.ietf.org/doc/html/rfc5321#section-4.1.3 
IPv6v4-full which em client uses.

I opened https://github.com/apache/james-project/pull/2262 for addressing this.

This bug only affects unreleased version... 

> EHLO Error while connect SMTP
> -
>
> Key: JAMES-4038
> URL: https://issues.apache.org/jira/browse/JAMES-4038
> Project: James Server
>  Issue Type: Bug
>  Components: James Core, SMTPServer
>Affects Versions: 3.8.1
> Environment: debian 11, oracle jdk 21
>Reporter: ilya terskov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Hi there
> docker run -d --net=host chibenwa/james-jpa:JAMES-4037-2 --generate-keystore
> docker exec ID james-cli AddDomain domain.online
> docker exec ID james-cli AddUser it@domain.online PASS
> networkctl status
> ●   State: routable
> Address: 172.16.149.37 on eth0
>172.17.0.1 on docker0
> Gateway: 172.16.149.18 (Intel Corporate) on eth0
>DNS: 172.16.149.14
> 172.16.149.33
> While trying get smtp test on 587 with SSL, 465 with STARTTLS or 25 
> with/without SSL/STARTTLS for connect happens this
> 06:35:28.976 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.977 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.982 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.983 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.984 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.989 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.993 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.997 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.998 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.999 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:29.004 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:29.004 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:29.006 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> IPV6 turned off on client side too as well as on server



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Created] (JAMES-4040) Mailbox role: spam => junk

2024-05-27 Thread Benoit Tellier (Jira)

Benoit Tellier created JAMES-4040:
-

 Summary: Mailbox role: spam => junk
 Key: JAMES-4040
 URL: https://issues.apache.org/jira/browse/JAMES-4040
 Project: James Server
  Issue Type: Bug
Reporter: Benoit Tellier


Reported on Twake mail frontend but that is a James issue...

https://github.com/linagora/tmail-flutter/issues/2894

JMAP spec states that mailboxes role must be one of IMAP Mailbox Name Attributes

According to this link, the attribute name for Junk/Spam folder is Junk and not 
Spam.

Reported by Jean-Philippe Garcia Ballester 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-4038) EHLO Error while connect SMTP

2024-05-23 Thread Benoit Tellier (Jira)



[ 
https://issues.apache.org/jira/browse/JAMES-4038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848832#comment-17848832
 ] 

Benoit Tellier commented on JAMES-4038:
---

What client was used?

> EHLO Error while connect SMTP
> -
>
> Key: JAMES-4038
> URL: https://issues.apache.org/jira/browse/JAMES-4038
> Project: James Server
>  Issue Type: Bug
>  Components: James Core, SMTPServer
>Affects Versions: 3.8.1
> Environment: debian 11, oracle jdk 21
>Reporter: ilya terskov
>Priority: Major
>
> Hi there
> docker run -d --net=host chibenwa/james-jpa:JAMES-4037-2 --generate-keystore
> docker exec ID james-cli AddDomain domain.online
> docker exec ID james-cli AddUser it@domain.online PASS
> networkctl status
> ●   State: routable
> Address: 172.16.149.37 on eth0
>172.17.0.1 on docker0
> Gateway: 172.16.149.18 (Intel Corporate) on eth0
>DNS: 172.16.149.14
> 172.16.149.33
> While trying get smtp test on 587 with SSL, 465 with STARTTLS or 25 
> with/without SSL/STARTTLS for connect happens this
> 06:35:28.976 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.977 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.982 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.983 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.984 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.989 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.993 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.997 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:28.998 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:28.999 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:29.004 [INFO ] o.a.j.p.n.BasicChannelInboundHandler - Connection 
> established from 172.16.149.220
> 06:35:29.004 [ERROR] o.a.j.p.s.c.e.EhloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> 06:35:29.006 [ERROR] o.a.j.p.s.c.HeloCmdHandler - Invalid EHLO argument 
> received: [IPv6::::172.16.149.220]. Must be a domain name or an IP 
> address.
> IPV6 turned off on client side too as well as on server



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-4037) Errors during connect or sync through IMAP



[ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848816#comment-17848816
 ] 

Benoit Tellier commented on JAMES-4037:
---

Here you are: 

{code:java}
docker pull chibenwa/james-jpa:JAMES-4037-2
{code}


> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.java:755)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.request(FluxIterable.java:620)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onSubscribe(MonoFlatMapMany.java:245)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:179) 
> ~[reactor-core-3.5.1.jar:3.5.1]
>   at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:83) 
> ~[reactor-core-3.5.1.jar:3.5.1]
>   at reactor.core.publisher.Flux.subscribe(Flux.java:8660) 
> ~[reacto

[jira] [Commented] (JAMES-4037) Errors during connect or sync through IMAP



[ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848771#comment-17848771
 ] 

Benoit Tellier commented on JAMES-4037:
---

I reproduced with an integration tests and provided fixes for master and 3.8.x

https://github.com/apache/james-project/pull/2257

https://github.com/apache/james-project/pull/2258

Thanks a lot for the report!

If you need help I can package you a docker image to continue your tests with 
those fixes included. Do not hesitate to ask!

Regards,

Benoit

> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.java:755)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.request(FluxIterable.java:620)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onSubscribe(MonoFlatMapMany.java:245)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at reactor.core.publisher.FluxIterable.su

[jira] [Commented] (JAMES-4037) Errors during connect or sync through IMAP



[ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848713#comment-17848713
 ] 

Benoit Tellier commented on JAMES-4037:
---

Problem comes from the fact that `StatusProcessor is used by ListProcessor.

But it is not binded as a singleton

This means that the regular StatusProcessor and the StatusProcessor used by 
LIST are two separate entity.

When we go through the chain of processors we only configure the main one and 
the other is unconfigured thus the error.

The test suite exercising these extension is written with MPT. Current MPT 
tests mostly exercise IMAP and storage logic but hard code the wiring and fake 
the network layer thus failed catching the issue,

> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.java:755)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.request(FluxIterable.java:620)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onSubscribe(MonoFlatMapMany.java:245)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>

[jira] [Comment Edited] (JAMES-4037) Errors during connect or sync through IMAP



[ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848698#comment-17848698
 ] 

Benoit Tellier edited comment on JAMES-4037 at 5/22/24 5:53 PM:


Ok I reproduced: it's a different bug!

{code:java}
t1 LIST "" % RETURN (STATUS (APPENDLIMIT))
* LIST (\HasNoChildren) "." "INBOX"
t1 NO LIST processing failed.
{code}

Note that APPENDLIMIT for status item works OK:

{code:java}
a1 STATUS INBOX (APPENDLIMIT)
* STATUS "INBOX" (APPENDLIMIT NIL)
a1 OK STATUS completed.
{code}

Thanks for the report. I will fix that tomorrow.

As of today you can work around those bugs by specifying the disabedCaps IMAP 
configuration setting to "hide" those buggy capabilities to your clients.

CF 
https://github.com/apache/james-project/blob/32d09f9ec590f5bbde46aff634fc73fec282cd55/server/apps/distributed-app/docs/modules/ROOT/pages/configure/imap.adoc?plain=1#L81



was (Author: btellier):
Ok I reproduced:

{code:java}
t1 LIST "" % RETURN (STATUS (APPENDLIMIT))
* LIST (\HasNoChildren) "." "INBOX"
t1 NO LIST processing failed.
{code}

Note that APPENDLIMIT for status item works OK:

{code:java}
a1 STATUS INBOX (APPENDLIMIT)
* STATUS "INBOX" (APPENDLIMIT NIL)
a1 OK STATUS completed.
{code}

Thanks for the report. I will fix that tomorrow.

As of today you can work around those bugs by specifying the disabedCaps IMAP 
configuration setting to "hide" those buggy capabilities to your clients.

CF 
https://github.com/apache/james-project/blob/32d09f9ec590f5bbde46aff634fc73fec282cd55/server/apps/distributed-app/docs/modules/ROOT/pages/configure/imap.adoc?plain=1#L81


> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.

[jira] [Commented] (JAMES-4037) Errors during connect or sync through IMAP



[ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848698#comment-17848698
 ] 

Benoit Tellier commented on JAMES-4037:
---

Ok I reproduced:

{code:java}
t1 LIST "" % RETURN (STATUS (APPENDLIMIT))
* LIST (\HasNoChildren) "." "INBOX"
t1 NO LIST processing failed.
{code}

Note that APPENDLIMIT for status item works OK:

{code:java}
a1 STATUS INBOX (APPENDLIMIT)
* STATUS "INBOX" (APPENDLIMIT NIL)
a1 OK STATUS completed.
{code}

Thanks for the report. I will fix that tomorrow.

As of today you can work around those bugs by specifying the disabedCaps IMAP 
configuration setting to "hide" those buggy capabilities to your clients.

CF 
https://github.com/apache/james-project/blob/32d09f9ec590f5bbde46aff634fc73fec282cd55/server/apps/distributed-app/docs/modules/ROOT/pages/configure/imap.adoc?plain=1#L81


> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.java:755)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.request(FluxIterable.java:620)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onSubscribe(MonoFlatMapMany.java:245)
>  ~[reactor-core-3.5.1.jar:3

[jira] [Commented] (JAMES-4037) Errors during connect or sync through IMAP



[ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848647#comment-17848647
 ] 

Benoit Tellier commented on JAMES-4037:
---

Hello,
I do not have an outlook at hand.
Do you have a way to have the exact IMAP exchange between client and server.

 1. give the exact james log trace that was generated connecting outlook to 
james jpa 3.8.1?

 2. if you have a way to wireshark and bypass the TLS encryption I take it!

 3. And if not then exact command details shall be in the MDC (logging 
context). Mounting the JSON appender to your logs will be enough to get it, 
then send the failure logs at the JSON format. 

A logback configuration to do so is available here 
https://james.staged.apache.org/james-distributed-app/3.8.1/operate/logging.html#_using_docker
 

Thanks!

> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.java:755)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.request(FluxIterable.java:620)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onSubscribe(MonoFlatMapMany.java:245)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.F

[jira] [Commented] (JAMES-4037) Errors during connect or sync through IMAP



[ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848605#comment-17848605
 ] 

Benoit Tellier commented on JAMES-4037:
---

Works great for JPA according to me

{code:java}
$ docker run -it apache/james:jpa-3.8.1 --generate-keystore
{code}

Other terminal:

{code:java}
$ docker ps
$ docker exec -ti 4ed6be2935a2 james-cli adduser bo@localhost 123456
$ docker inspect 4ed6be2935a2
$ openssl s_client -connect 172.17.0.2:993
* OK JAMES IMAP4rev1 Server 4ed6be2935a2 is ready.
a0 LOGIN bo@localhost 123456
a0 OK LOGIN completed.
t1 LIST "" "%" RETURN (SPECIAL-USE)
* LIST (\HasNoChildren) "." "INBOX"
* LIST (\HasNoChildren \Archive) "." "Archive"
* LIST (\HasNoChildren \Drafts) "." "Drafts"
* LIST (\HasNoChildren) "." "Outbox"
* LIST (\HasNoChildren \Sent) "." "Sent"
* LIST (\HasNoChildren \Junk) "." "Spam"
* LIST (\HasNoChildren \Trash) "." "Trash"
t1 OK LIST completed.
t3 LIST (SPECIAL-USE) "" "*"
* LIST (\HasNoChildren \Archive) "." "Archive"
* LIST (\HasNoChildren \Drafts) "." "Drafts"
* LIST (\HasNoChildren \Sent) "." "Sent"
* LIST (\HasNoChildren \Junk) "." "Spam"
* LIST (\HasNoChildren \Trash) "." "Trash"
t3 OK LIST completed.
{code}

Is it possible for you to either give me a MUA or the exact IMAP command 
issued? As well as the commands you used to run JPA Guice 3.8.1 ?

> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.java:755)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$Iterabl

[jira] [Commented] (JAMES-4037) Errors during connect or sync through IMAP



[ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848558#comment-17848558
 ] 

Benoit Tellier commented on JAMES-4037:
---

Backport for 3.8.x https://github.com/apache/james-project/pull/2255

> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.java:755)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.request(FluxIterable.java:620)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onSubscribe(MonoFlatMapMany.java:245)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:179) 
> ~[reactor-core-3.5.1.jar:3.5.1]
>   at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:83) 
> ~[reactor-core-3.5.1.jar:3.5.1]
>   at reactor.core.publisher.Flux.subscribe(Flux.java:8660) 
> ~[reacto

[jira] [Commented] (JAMES-4037) Errors during connect or sync through IMAP



[ 
https://issues.apache.org/jira/browse/JAMES-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848555#comment-17848555
 ] 

Benoit Tellier commented on JAMES-4037:
---

Hello,

I opened https://github.com/apache/james-project/pull/2254

Apparently mailboxTyper was badly wired with the Spring packaging and a null 
reference was passed to it.

Thus when the mail user agent tries to use IMAP special use with the LIST 
command it failed.

I fixed the wiring for Spring in the above PR.

Thanks for the report.

I will backport this in 3.8.x too as that release also is impacted.

> Errors during connect or sync through IMAP
> --
>
> Key: JAMES-4037
> URL: https://issues.apache.org/jira/browse/JAMES-4037
> Project: James Server
>  Issue Type: Bug
>  Components: James Core
>Affects Versions: 3.8.1
> Environment: Apache James Guice and Spring 3.8.1, both windows 10 and 
> debian 11, Oracle JDK 21
>Reporter: ilya terskov
>Priority: Major
>  Labels: AbstractMailboxProcessor, IMAP, James, MailboxTyper, 
> imap.processor, mailbox
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> 22-May-2024 03:56:58.938 ERROR [parallel-1] 
> org.apache.james.imap.processor.AbstractMailboxProcessor.lambda$doProcess$1:105
>  - Unexpected error during IMAP processing
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.james.imap.api.process.MailboxTyper.getMailboxType(org.apache.james.imap.api.process.ImapSession,
>  org.apache.james.mailbox.model.MailboxPath)" because "this.mailboxTyper" is 
> null
>   at 
> org.apache.james.imap.processor.ListProcessor.getMailboxType(ListProcessor.java:376)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> org.apache.james.imap.processor.ListProcessor.lambda$processWithoutSubscribed$9(ListProcessor.java:218)
>  ~[protocols-imap-3.8.1.jar:3.8.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:196)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.Operators$BaseFluxToMonoOperator.completePossiblyEmpty(Operators.java:2071)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:118)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.fastPath(FluxIterable.java:755)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxIterable$IterableSubscriptionConditional.request(FluxIterable.java:620)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onSubscribe(MonoFlatMapMany.java:245)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   at 
> reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
>  ~[reactor-core-3.5.1.jar:3.5.1]
>   a

[jira] [Commented] (JAMES-4036) MailSpooler shutdown not graceful



[ 
https://issues.apache.org/jira/browse/JAMES-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848502#comment-17848502
 ] 

Benoit Tellier commented on JAMES-4036:
---

To be in 3.8.2 this needs to be backported.

To get it today before a release, you need to build from source.

Note you can also safely ignore those logs for the time being.

> MailSpooler shutdown not graceful
> -
>
> Key: JAMES-4036
> URL: https://issues.apache.org/jira/browse/JAMES-4036
> Project: James Server
>  Issue Type: Bug
>  Components: Mailet Contributions, MailStore  MailRepository, 
> SpoolManager  Processors
>Affects Versions: 3.7.4, 3.8.1, 3.7.5
> Environment: windows 10 and debian 11, oracle jdk 17,21 openjdk 
> 17,21,22
>Reporter: ilya terskov
>Priority: Major
>  Labels: Apache, james, mail, spool
> Fix For: 3.8.0, 3.7.3
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> 13:45:48.520 ←[34m[INFO ]←[0;39m o.a.j.m.i.JamesMailSpooler - start dispose() 
> ...
> 13:45:48.525 ←[34m[INFO ]←[0;39m o.a.j.m.i.JamesMailSpooler - Cancel queue 
> consumption...
> 13:45:48.525 ←[34m[INFO ]←[0;39m o.a.j.m.i.JamesMailSpooler - Queue 
> consumption canceled, shutting down processor threads...
> 13:45:48.528 ←[1;31m[ERROR]←[0;39m r.c.p.Operators - Operator called default 
> onErrorDropped
> java.lang.InterruptedException: null
> at java.base/java.lang.Object.wait0(Native Method)
> at java.base/java.lang.Object.wait(Object.java:375)
> at java.base/java.lang.Object.wait(Object.java:348)
> at 
> org.apache.activemq.FifoMessageDispatchChannel.dequeue(FifoMessageDispatchChannel.java:72)
> at 
> org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:497)
> ... 14 common frames omitted
> Wrapped by: javax.jms.JMSException: java.lang.InterruptedException
> at 
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:80)
> at 
> org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:534)
> at 
> org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:662)
> at 
> org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:246)
> ... 12 common frames omitted
> Wrapped by: org.apache.james.queue.api.MailQueue$MailQueueException: Unable 
> to dequeue next message
> at 
> org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:261)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
> at 
> reactor.core.publisher.FluxRepeatPredicate$RepeatPredicateSubscriber.resubscribe(FluxRepeatPredicate.java:120)
> at 
> reactor.core.publisher.MonoRepeatPredicate.subscribeOrReturn(MonoRepeatPredicate.java:48)
> at 
> reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:55)
> at 
> reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194)
> at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
> at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1570)
> Wrapped by: reactor.core.Exceptions$ErrorCallbackNotImplemented: 
> org.apache.james.queue.api.MailQueue$MailQueueException: Unable to dequeue 
> next message;
>   nested exception is:
> javax.jms.JMSException: java.lang.InterruptedException
> 13:45:48.537 ←[34m[INFO ]←[0;39m o.a.j.m.i.JamesMailSpooler - Thread shutdown 
> completed. Turning off mail queue.
> 13:45:48.553 ←[34m[INFO ]←[0;39m o.a.j.GuiceJamesServer - JAMES server stopped



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-4036) MailSpooler shutdown not graceful

2024-05-21 Thread Benoit Tellier (Jira)



[ 
https://issues.apache.org/jira/browse/JAMES-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848373#comment-17848373
 ] 

Benoit Tellier commented on JAMES-4036:
---

Hello,

Those logs are harmeless but I do agree they are annoying.

Please see https://github.com/apache/james-project/pull/2251 for silenting them.

Feel free to open PRs to backport them to specific versions.

> MailSpooler shutdown not graceful
> -
>
> Key: JAMES-4036
> URL: https://issues.apache.org/jira/browse/JAMES-4036
> Project: James Server
>  Issue Type: Bug
>  Components: Mailet Contributions, MailStore  MailRepository, 
> SpoolManager  Processors
>Affects Versions: 3.7.4, 3.8.1, 3.7.5
> Environment: windows 10 and debian 11, oracle jdk 17,21 openjdk 
> 17,21,22
>Reporter: ilya terskov
>Priority: Major
>  Labels: Apache, james, mail, spool
> Fix For: 3.8.0, 3.7.3
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> 13:45:48.520 ←[34m[INFO ]←[0;39m o.a.j.m.i.JamesMailSpooler - start dispose() 
> ...
> 13:45:48.525 ←[34m[INFO ]←[0;39m o.a.j.m.i.JamesMailSpooler - Cancel queue 
> consumption...
> 13:45:48.525 ←[34m[INFO ]←[0;39m o.a.j.m.i.JamesMailSpooler - Queue 
> consumption canceled, shutting down processor threads...
> 13:45:48.528 ←[1;31m[ERROR]←[0;39m r.c.p.Operators - Operator called default 
> onErrorDropped
> java.lang.InterruptedException: null
> at java.base/java.lang.Object.wait0(Native Method)
> at java.base/java.lang.Object.wait(Object.java:375)
> at java.base/java.lang.Object.wait(Object.java:348)
> at 
> org.apache.activemq.FifoMessageDispatchChannel.dequeue(FifoMessageDispatchChannel.java:72)
> at 
> org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:497)
> ... 14 common frames omitted
> Wrapped by: javax.jms.JMSException: java.lang.InterruptedException
> at 
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:80)
> at 
> org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:534)
> at 
> org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:662)
> at 
> org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:246)
> ... 12 common frames omitted
> Wrapped by: org.apache.james.queue.api.MailQueue$MailQueueException: Unable 
> to dequeue next message
> at 
> org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:261)
> at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
> at 
> reactor.core.publisher.FluxRepeatPredicate$RepeatPredicateSubscriber.resubscribe(FluxRepeatPredicate.java:120)
> at 
> reactor.core.publisher.MonoRepeatPredicate.subscribeOrReturn(MonoRepeatPredicate.java:48)
> at 
> reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:55)
> at 
> reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194)
> at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
> at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1570)
> Wrapped by: reactor.core.Exceptions$ErrorCallbackNotImplemented: 
> org.apache.james.queue.api.MailQueue$MailQueueException: Unable to dequeue 
> next message;
>   nested exception is:
> javax.jms.JMSException: java.lang.InterruptedException
> 13:45:48.537 ←[34m[INFO ]←[0;39m o.a.j.m.i.JamesMailSpooler - Thread shutdown 
> completed. Turning off mail queue.
> 13:45:48.553 ←[34m[INFO ]←[0;39m o.a.j.GuiceJamesServer - JAMES server stopped



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-4035) ActiveMQ not deleting files

2024-05-17 Thread Benoit Tellier (Jira)



[ 
https://issues.apache.org/jira/browse/JAMES-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847347#comment-17847347
 ] 

Benoit Tellier commented on JAMES-4035:
---

4/ Which operating system are you using?


> ActiveMQ not deleting files
> ---
>
> Key: JAMES-4035
> URL: https://issues.apache.org/jira/browse/JAMES-4035
> Project: James Server
>  Issue Type: Bug
>  Components: Blob
>Affects Versions: 3.7.1
>Reporter: Yoram Halberstam
>Priority: Critical
>
> Hi,
> These are emails for an account I deleted on James 3.7.1 but it the blob have 
> not been deleted. Do you know how to force deletion?
> {code:java}
> james-server-spring-app-3.7.1/var# du -h
> 1.6G    ./store/activemq/brokers/KahaDB
> 1.6G    ./store/activemq/brokers
> 301M    ./store/activemq/blob-transfer/6
> 303M    ./store/activemq/blob-transfer/9
> 302M    ./store/activemq/blob-transfer/8
> 305M    ./store/activemq/blob-transfer/3
> 303M    ./store/activemq/blob-transfer/10
> 303M    ./store/activemq/blob-transfer/5
> 303M    ./store/activemq/blob-transfer/7
> 304M    ./store/activemq/blob-transfer/4
> 303M    ./store/activemq/blob-transfer/2
> 305M    ./store/activemq/blob-transfer/1
> 3.0G    ./store/activemq/blob-transfer
> {code}
> Regards
> Yoram
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-4035) ActiveMQ not deleting files

2024-05-17 Thread Benoit Tellier (Jira)



[ 
https://issues.apache.org/jira/browse/JAMES-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847346#comment-17847346
 ] 

Benoit Tellier commented on JAMES-4035:
---

Hello,

This looks weird to me...

Can I ask you few questions?

1/ Have you tried later releases in the 3.7.x branch? It looks like 
https://github.com/apache/james-project/blob/master/CHANGELOG.md#374---2023-03-20
 ?
2/ I do not exactly understand the meaning of "delete an account". Can you 
provide details?
3/ Any error in the logs?

Thanks.

> ActiveMQ not deleting files
> ---
>
> Key: JAMES-4035
> URL: https://issues.apache.org/jira/browse/JAMES-4035
> Project: James Server
>  Issue Type: Bug
>  Components: Blob
>Affects Versions: 3.7.1
>Reporter: Yoram Halberstam
>Priority: Critical
>
> Hi,
> These are emails for an account I deleted on James 3.7.1 but it the blob have 
> not been deleted. Do you know how to force deletion?
> {code:java}
> james-server-spring-app-3.7.1/var# du -h
> 1.6G    ./store/activemq/brokers/KahaDB
> 1.6G    ./store/activemq/brokers
> 301M    ./store/activemq/blob-transfer/6
> 303M    ./store/activemq/blob-transfer/9
> 302M    ./store/activemq/blob-transfer/8
> 305M    ./store/activemq/blob-transfer/3
> 303M    ./store/activemq/blob-transfer/10
> 303M    ./store/activemq/blob-transfer/5
> 303M    ./store/activemq/blob-transfer/7
> 304M    ./store/activemq/blob-transfer/4
> 303M    ./store/activemq/blob-transfer/2
> 305M    ./store/activemq/blob-transfer/1
> 3.0G    ./store/activemq/blob-transfer
> {code}
> Regards
> Yoram
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Created] (JAMES-4034) SMTP submission: validate FROM header

2024-05-14 Thread Benoit Tellier (Jira)

Benoit Tellier created JAMES-4034:
-

 Summary: SMTP submission: validate FROM header
 Key: JAMES-4034
 URL: https://issues.apache.org/jira/browse/JAMES-4034
 Project: James Server
  Issue Type: Improvement
  Components: SMTPServer
Reporter: Benoit Tellier


h3. Why?

Prevent and limit email forgery by local users.

As a James administrator I should have a way to configure SMTP submission to 
reject emails spoofing somebody else mail adress.

As of today:
 - JMAP stacks controls both the transport envelope and the headers (From) as 
mandated by the JMAP Mail RFC (RFC-8621)
- However SMTP only controls the transport envelop and not the headers.

This control is currently implemented into SenderAuthIdentifyVerificationHook 
and is intendeed as a hook on MAIL FROM SMTP command.

We shall also enforce a control of the From header upon submission in SMTP.  
Such controls would be mandatory for use of Apache James in an environment 
where local users could not be fully trusted, as for instance a SaaS offer.

h3. How?

Modify SenderAuthIdentifyVerificationHook in order that it also is a 
JamesMessageHook called before enqueing and checks each and every From header 
the same way it checks "MAIL FROM" command.

Modifying SenderAuthIdentifyVerificationHook would enforce this only for 
connected users IE mail submission and would not affect email relay. This also 
ensure this behaviour will apply by default. 

Refactoring of the underlying AbstractSenderAuthIdentifyVerificationHook might 
be required.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Created] (JAMES-4032) Improve external forward handling

2024-04-26 Thread Benoit Tellier (Jira)

Benoit Tellier created JAMES-4032:
-

 Summary: Improve external forward handling
 Key: JAMES-4032
 URL: https://issues.apache.org/jira/browse/JAMES-4032
 Project: James Server
  Issue Type: Improvement
  Components: RRT
Affects Versions: 3.8.1
Reporter: Benoit Tellier
 Fix For: 3.9.0
 Attachments: image-2024-04-26-12-03-58-932.png, 
image-2024-04-26-12-04-10-017.png, image-2024-04-26-12-13-43-910.png, 
image-2024-04-26-12-14-35-358.png

Email deliverability is a tricky topic and yesterday I got one more example for 
this...

(The trick is to de able to improve deliverability while still being secure)

h3. The issue

The overall scenario outline is:

{code:java}
GIVEN two local users al...@me.org and b...@me.org on a james based instalation
AND al...@external.org on another email supplier
AND al...@external.org forwards its mail to al...@me.org.
AND this server preserves and resend as is the SMTP transport envelope

WHEN b...@me.org sends an email to al...@external.org

THEN james server rejects the incoming mail from external.org as it requires 
authentication 
to use local addresses in MAIL FROM
AND a bounce would be generated by external.org for the original sender as the 
email could not be forwarded.
{code}

Example of such "external mail providers": orange.fr SFR professional services.

See fwd1.png 

!image-2024-04-26-12-04-10-017.png! 

h3 Why authentication is currently required in such cases

In order to maximise delivrability, James accepts any email for local users on 
port 25 (spam filtering may be applied as part of the reception chain)

However a common attack is email spoofing: a remote attacker with not 
whatsoever access sends an email pretending to be a local user. Without further 
checks he could submit email as a local user without being authenticated.

To prevent this behaviour James implements `verifySenderIdentity` configuration 
parameter in smtpServer.xml configuration file. If using a local mail from the 
current behaviour is to actualy mandate authentication.

Anybody else than the user could thus NOT be using this MAIL FROM.

This prevents the following schario: fwd2.png

!image-2024-04-26-12-03-58-932.png! 

However this causes the issue discussed above.

h3. Related deliverability concerns

h4. Receiving external forwards orginating from external mail servers

b...@external1.org sends a mail to al...@external2.org which forwards mails as 
described above to al...@me.org.

On the last hop james accepts the email as the MAIL FROM (b...@external1.org) 
is a remote one: verifySenderIdentity is not triggered for external senders.

On the spam filtering side:
 - SPF checks fails (as external2.org did send a mail in external1.org name 
which it is not allowed to do
 - But DKIM tests pass
 - Which should be enough for the spam filtering stack not to flag the message 
too violently...

This current behaviour is considered lenient enough. White listing could 
further be added for trusted domains if need be.

h4. Handling correctly forwards emition

We tackled the very same issue on the emition side: the goal was to improve 
james forwarded email deliverability.

GIVEN b...@me.org forwards its email to b...@external.org
WHEN t...@other.org sends a mail to b...@me.org
THEN me.org would rewrite the MAIL FROM field and  use b...@me.org when 
forwarding to b...@external.org

Thus SPF check passes.

See https://issues.apache.org/jira/browse/JAMES-3944

h3. Proposed solution

We could experiment turning off sender verification hop on MAIL FROM

Instead we would verify the DKIM signature of the email before spooling it.

That way:

 - We would have signed legitimate traffic forwarded back to us and would 
accept it
 !image-2024-04-26-12-14-35-358.png! 

 - And we would still reject spoofing attempts as an attacker cant craft a DKIM 
signature.

 !image-2024-04-26-12-13-43-910.png! 

GIVEN that dkim signature of outgoing email is not setted up by default this 
behaviour shall be disabled by default.

h3. Proposed changes

Implement a VerifySenderIdentity SMTP message hook that:

GIVEN an unauthenticated sender

It performs a verification of non relaying (ie all rcpts are local)

And would verify the DKIM signature of the mail.

Path to the DKIM public key needs to be provided.

This shall be contributed as part of the DKIM extention.

h3. Additional concerns

Given the Mail Exchange service (MX) and Mail Submission Agent (MSA) being 
collocated - the default james architecture, 

Then a common misconfiguration is not to configure TLS for SMTP and just have 
regular users just try to send mails unauthenticated through SMTP.

This behaviour is classical from MacOS native client.

Current `verifySenderIdentity` behavior rejects such misconfigured client from 
the MAIL FROM. Which is convenient. Fail fast:

 - The error is more explicit: AUth

[jira] [Commented] (JAMES-4030) Improve Bouncer error handling



[ 
https://issues.apache.org/jira/browse/JAMES-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837409#comment-17837409
 ] 

Benoit Tellier commented on JAMES-4030:
---

The rabbitmq mail queue throws unchecked exception and thus was unnafected when 
specifying a bounce processor.

The JMS mailqueue throws checked exception and thus was fully affected.

Ommitting the bounce processor in the RemoteDelivery configuration would cause 
the set up to be impacted regardless of the implementation.

> Improve Bouncer error handling
> --
>
> Key: JAMES-4030
> URL: https://issues.apache.org/jira/browse/JAMES-4030
> Project: James Server
>  Issue Type: Bug
>  Components: Remote Delivery
>        Reporter: Benoit Tellier
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> While looking at RemoteDelivery bouncer code, if MessagingException was 
> thrown then basically the email would be discarded after a warning log.
> This would result in email loss without even a bounce being sent.
> Instead we should carry over the exception, resulting on a nack on the 
> underlying mailqueue and guarantying retries.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Created] (JAMES-4030) Improve Bouncer error handling

Benoit Tellier created JAMES-4030:
-

 Summary: Improve Bouncer error handling
 Key: JAMES-4030
 URL: https://issues.apache.org/jira/browse/JAMES-4030
 Project: James Server
  Issue Type: Bug
  Components: Remote Delivery
Reporter: Benoit Tellier


While looking at RemoteDelivery bouncer code, if MessagingException was thrown 
then basically the email would be discarded after a warning log.

This would result in email loss without even a bounce being sent.

Instead we should carry over the exception, resulting on a nack on the 
underlying mailqueue and guarantying retries.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Created] (JAMES-4029) Bounce and forward combined can lead to an infinite loop

Benoit Tellier created JAMES-4029:
-

 Summary: Bounce and forward combined can lead to an infinite loop
 Key: JAMES-4029
 URL: https://issues.apache.org/jira/browse/JAMES-4029
 Project: James Server
  Issue Type: Bug
  Components: Mailet Contributions, RRT
Reporter: Benoit Tellier


GIVEN all mails sent to b...@gmail.com bounces
AND b...@avocat.fr forwards his mails to b...@gmail.com
WHEN custo...@other.com sends an email of  to b...@avocat.fr
THEN remote delivery for the forwarded message fails and generates a bounce 
(that is big) is forwarded and bounced again.

Recommendation:

 - bounce/dsnBounce should not reset forwqard state
 - Write integration test for bounce/dsnBounce forward interactions



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Created] (JAMES-4028) MailImpl derivation truncation might make diagnostic harder

Benoit Tellier created JAMES-4028:
-

 Summary: MailImpl derivation truncation might make diagnostic 
harder
 Key: JAMES-4028
 URL: https://issues.apache.org/jira/browse/JAMES-4028
 Project: James Server
  Issue Type: Improvement
Reporter: Benoit Tellier


I was investigating a forward loop ( :-( ) and to my unplaisure, because mail 
names are limited to 86 characters, 

This caused the lopped email name not to be prefixed by the name of the 
original name:

{code:java}
 Mail1713173708159-16b6f285-e237-405f-800f-6a54926825e6-!5984740
  
713173708159-16b6f285-e237-405f-800f-6a54926825e6-!5984740-!9588184-!2870433

285-e237-405f-800f-6a54926825e6-!5984740-!9588184-!2870433-!9660160-!4179738
  
-6a54926825e6-!5984740-!9588184-!2870433-!9660160-!4179738-!0418990-!8605805
{code}

Current rule: 
 - 4 char for 'Mail' prefix
 - 15 char for current unix timestamp
 - 1 char separator
 - 38 char uuid
 - up to 7 x 9 derivative suffix

Which means that truncation happens after 4 derivates.

By twikking slightly generation mechanisms we can get a more compact name that 
wouldn't get truncated to fir the 86 char rule.


 - 4 char for 'Mail' prefix
 - 15 char for current unix timestamp - to be kept as some people might use it 
as diagnostic information?
- 1 separator
 - Generate a 20 char random alphanumeric name. Entropy should be similar with 
the uuid
 - Rework each derivative suffix to be 5 char - use `!` as a demiiter and not 
`-!` and rely on alpha numeric for a more compact representation

That way truncation never happens, and I could filter easily by base email name 
onto my favorite mailing system to see the whole history without headache.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

Re: Making all queues on Rabbitmq quorum queue when option enabled

2024-04-15 Thread Benoit TELLIER


Hi Quan,

First thanks for the job done on this topic.

I know some members of the community (Karsten ?) already did significant 
work on the topic but more oriented toward the POP3 server.


This work is of course welcome as it would result in a higher 
reliability for the IMAP / JMAP components.


> What do you think about making all queues on Rabbitmq quorum queue 
when option enabled? On the principle, +1 In practice that is slightly 
harder for the event bus notification queue... - We can likely afford 
losing some of those pub sub message? - The queue is tied to a 
connection, thus if the node/connection goes done it can be recreated 
elsewhere? - We would need to come up with a cleanup strategy in order 
to eventually deletes queues haging around. - Also, how relevant is this 
RabbitMQ backend pub sub implementation when compared with the work done 
with Redis? IMO the eventbus notification was the main blocker in order 
to achieve decent HA with RabbitMQ. Do you share this point of view? 
Best regards, Benoit TELLIER


On 15/04/2024 09:53, Quan tran hong wrote:

Hi folks,

Recently we encountered a deployment issue that used a RabbitMQ Cluster
where a RabbitMQ node outage (for about 1 hour) forced James service more
or less to be down too.

I created a Jira ticket to report the issue:
https://issues.apache.org/jira/projects/JAMES/issues/JAMES-4027

More details below for one did not read the Jira ticket yet:

Today, when the quorum option is enabled, only some queues are quorum
queues, not all (e.g. event bus notification queues and Task Manager's
termination queues).

I tried to reproduce the issue and here is my theory:

The RabbitMQ node that stores the notification queues is down
-> James can not publish messages to RabbitMQ and causes e.g. IMAP SELECT,
STORE, APPEND, UNSELECT ... commands to fail
-> James keeps retrying the publish failures (retry for Group registration
which seems to rely on the classic queue too) and queues other IMAP
requests in the meantime.
-> The IMAP server queue becomes full and the exception `The IMAP server
has reached its maximum capacity` is thrown.
-> James IMAP becomes a zombie and cascading failures.

James needs to be more fault-tolerant in this case.

We think making all queues on Rabbitmq quorum queue when
`quorum.queues.enable=true` would help James be more fault tolerant on that
scenario.

We investigated a POC athttps://github.com/apache/james-project/pull/2191  and
the full quorum queues helped James be more fault tolerant as expected.

After full quorum queues are used, the James performance is a bit slower
but is still fine, and that cost is likely needed to make James more
reliable.

If we use Redis backed event bus notifications, the performance is better
than the RabbitMQ notification quorum queues.

What do you think about making all queues on Rabbitmq quorum queue when
option enabled? Feedback and review are very welcome.

Thanks for reading.

Quan

[jira] [Commented] (JAMES-4024) Add support for SNI (separate certificate per domain)

2024-04-01 Thread Benoit Tellier (Jira)



[ 
https://issues.apache.org/jira/browse/JAMES-4024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832937#comment-17832937
 ] 

Benoit Tellier commented on JAMES-4024:
---

Hello,

Technically it should be rather easy to overlad SSLHandler to achieve your 
needs I bet - the protocol framework is supposed to allow for that.

However an implementation of SNI could land in James source tree.

Would you be motivated to contribute such a thing?



> Add support for SNI (separate certificate per domain)
> -
>
> Key: JAMES-4024
> URL: https://issues.apache.org/jira/browse/JAMES-4024
> Project: James Server
>  Issue Type: New Feature
>  Components: protocols
>Affects Versions: 3.8.1
>Reporter: Amichai Rothman
>Priority: Major
>
> Currently it is only possible to configure one global certificate for all TLS 
> communication of the entire server. However, many SMTP servers nowadays can 
> be configured to validate that a certificate matches the (mx record) domain 
> name when connecting to another SMTP server, and thus many SMTP servers also 
> support SNI so they can serve up the proper certificate when receiving mail 
> messages for multiple domains.
> James should also support SNI so it can work properly with secure 
> configurations of all other SMTP servers, i.e. allow adding a separate 
> certificate per supported domain(s), and support SNI to select the correct 
> one per TLS connection. The existing global certificate can remain as 
> fallback and to avoid breaking existing configurations.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Updated] (JAMES-4023) SMTP server is blocking on the IO thread



 [ 
https://issues.apache.org/jira/browse/JAMES-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier updated JAMES-4023:
--
Attachment: Screenshot from 2024-03-29 16-30-53.png

> SMTP server is blocking on the IO thread
> 
>
> Key: JAMES-4023
> URL: https://issues.apache.org/jira/browse/JAMES-4023
> Project: James Server
>  Issue Type: Bug
>  Components: POP3Server, SMTPServer
>Affects Versions: master, 3.8.0, 3.8.1
>    Reporter: Benoit Tellier
>Priority: Major
> Fix For: 3.9.0, 3.8.2
>
> Attachments: Screenshot from 2024-03-29 16-30-53.png, 
> cnb-imap-29-03-2024.html
>
>
> While conducting a routine flame graph analysis [attached] I realized some 
> file calls where done on the event loop [screenshot attached]
> By printing the thread that performs the enqueuing I was able to confirm this:
> {code:java}
> ENQUEUING on smtpserver-io-1
> {code}
> Blocking on the netty eventloop blocks all other concurrent requests to that 
> thread and can cause significant delays by making the overall server 
> irresponsive. This shall be avoided at all cost!
> By reworking slightly the Netty pipeline, and relying on the 
> eventExecutorGroup for the core handler this had effectively been mitigated:
> {code:java}
> ENQUEUING on smtpserver-0
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Updated] (JAMES-4023) SMTP server is blocking on the IO thread



 [ 
https://issues.apache.org/jira/browse/JAMES-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benoit Tellier updated JAMES-4023:
--
Attachment: cnb-imap-29-03-2024.html

> SMTP server is blocking on the IO thread
> 
>
> Key: JAMES-4023
> URL: https://issues.apache.org/jira/browse/JAMES-4023
> Project: James Server
>  Issue Type: Bug
>  Components: POP3Server, SMTPServer
>Affects Versions: master, 3.8.0, 3.8.1
>    Reporter: Benoit Tellier
>Priority: Major
> Fix For: 3.9.0, 3.8.2
>
> Attachments: cnb-imap-29-03-2024.html
>
>
> While conducting a routine flame graph analysis [attached] I realized some 
> file calls where done on the event loop [screenshot attached]
> By printing the thread that performs the enqueuing I was able to confirm this:
> {code:java}
> ENQUEUING on smtpserver-io-1
> {code}
> Blocking on the netty eventloop blocks all other concurrent requests to that 
> thread and can cause significant delays by making the overall server 
> irresponsive. This shall be avoided at all cost!
> By reworking slightly the Netty pipeline, and relying on the 
> eventExecutorGroup for the core handler this had effectively been mitigated:
> {code:java}
> ENQUEUING on smtpserver-0
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Created] (JAMES-4023) SMTP server is blocking on the IO thread

Benoit Tellier created JAMES-4023:
-

 Summary: SMTP server is blocking on the IO thread
 Key: JAMES-4023
 URL: https://issues.apache.org/jira/browse/JAMES-4023
 Project: James Server
  Issue Type: Bug
  Components: POP3Server, SMTPServer
Affects Versions: master, 3.8.0, 3.8.1
Reporter: Benoit Tellier
 Fix For: 3.8.2, 3.9.0


While conducting a routine flame graph analysis [attached] I realized some file 
calls where done on the event loop [screenshot attached]

By printing the thread that performs the enqueuing I was able to confirm this:

{code:java}
ENQUEUING on smtpserver-io-1
{code}

Blocking on the netty eventloop blocks all other concurrent requests to that 
thread and can cause significant delays by making the overall server 
irresponsive. This shall be avoided at all cost!

By reworking slightly the Netty pipeline, and relying on the eventExecutorGroup 
for the core handler this had effectively been mitigated:

{code:java}
ENQUEUING on smtpserver-0
{code}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org

[jira] [Commented] (JAMES-4016) JMAP email keywords in mixed JMAP/IMAP usage