Benoit Tellier created JAMES-3891:
-------------------------------------
Summary: ActiveMQ: restarting james can block mail queue
Key: JAMES-3891
URL: https://issues.apache.org/jira/browse/JAMES-3891
Project: James Server
Issue Type: Improvement
Components: Queue
Affects Versions: master, 3.7.3
Reporter: Benoit Tellier
Fix For: 3.8.0, 3.7.4
Given a mail queue currently containing emails and being consumed, When I
restart James, Then after the restart the email consumptions sometime do not
resume.
The following log can be seen:
{code:java}
james1 | java.io.FileNotFoundException:
/root/var/store/activemq/blob-transfer/10/ID_james1.local-33677-1677150190846-5_1_1_1_1037
(No such file or directory)
james1 | at java.base/java.io.FileInputStream.open0(Native Method)
james1 | at java.base/java.io.FileInputStream.open(Unknown Source)
james1 | at java.base/java.io.FileInputStream.<init>(Unknown
Source)
james1 | at
org.apache.james.queue.activemq.FileSystemBlobStrategy.getInputStream(FileSystemBlobStrategy.java:101)
james1 | at
org.apache.activemq.blob.BlobDownloader.getInputStream(BlobDownloader.java:38)
james1 | at
org.apache.activemq.command.ActiveMQBlobMessage.getInputStream(ActiveMQBlobMessage.java:132)
james1 | at
org.apache.james.queue.activemq.MimeMessageBlobMessageSource.getInputStream(MimeMessageBlobMessageSource.java:45)
james1 | at
org.apache.james.server.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:307)
james1 | at
org.apache.james.server.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:296)
james1 | at
org.apache.james.server.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:292)
james1 | at
org.apache.james.server.core.MimeMessageWrapper.writeTo(MimeMessageWrapper.java:276)
james1 | at
org.apache.james.server.core.MimeMessageWrapper.<init>(MimeMessageWrapper.java:158)
james1 | ... 19 common frames omitted
james1 | Wrapped by: javax.mail.MessagingException: IOException while
copying message
james1 | at
org.apache.james.server.core.MimeMessageWrapper.<init>(MimeMessageWrapper.java:165)
james1 | at
org.apache.james.server.core.MailImpl.setMessage(MailImpl.java:505)
james1 | at
com.github.fge.lambdas.consumers.ConsumerChainer.lambda$sneakyThrow$9(ConsumerChainer.java:73)
james1 | at java.base/java.util.Optional.ifPresent(Unknown Source)
james1 | at
org.apache.james.server.core.MailImpl$Builder.build(MailImpl.java:279)
james1 | at
org.apache.james.queue.jms.JMSCacheableMailQueue.createMail(JMSCacheableMailQueue.java:391)
james1 | at
org.apache.james.queue.activemq.ActiveMQCacheableMailQueue.createMailQueueItem(ActiveMQCacheableMailQueue.java:230)
james1 | at
org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:249)
james1 | ... 12 common frames omitted
james1 | Wrapped by:
org.apache.james.queue.api.MailQueue$MailQueueException: Unable to dequeue next
message
james1 | at
org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:260)
james1 | at
reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
james1 | at
reactor.core.publisher.FluxRepeatPredicate$RepeatPredicateSubscriber.resubscribe(FluxRepeatPredicate.java:120)
james1 | at
reactor.core.publisher.MonoRepeatPredicate.subscribeOrReturn(MonoRepeatPredicate.java:48)
james1 | at
reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:55)
james1 | at
reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194)
james1 | at
reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
james1 | at
reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
james1 | at java.base/java.util.concurrent.FutureTask.run(Unknown
Source)
james1 | at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source)
james1 | at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
james1 | at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
james1 | at java.base/java.lang.Thread.run(Unknown Source)
james1 | Wrapped by:
reactor.core.Exceptions$ErrorCallbackNotImplemented:
org.apache.james.queue.api.MailQueue$MailQueueException: Unable to dequeue next
message;
james1 | nested exception is:
james1 | javax.mail.MessagingException: IOException while copying
message;
james1 | nested exception is:
james1 | java.io.FileNotFoundException:
/root/var/store/activemq/blob-transfer/10/ID_james1.local-33677-1677150190846-5_1_1_1_1037
(No such file or directory)
{code}
The issue is that blobs are handle via file so their deletion is not linked to
the JMS trasaction: the JMS transaction can be rolled back for a deleted blob,
thus leading to messages without blob.
Steps to reproduce:
- Send 5000 mails to james of 500 bytes each
{code:java}
smtp-source -A -C100 -r 1 -l 500 -m 5000 -s 20 -d -c -f a...@other.com -M
other.com -t bob@localhost 172.24.0.4:25
{code}
- Once all mails are queued, restart james several time during processing
- Increasing the count of spooler workers helps.
As a fix I propose to discard emails not backed by a blob.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org
