Hey,
we have improved our monitoring since the last crash.
There was the same crash again today and now we were able to exactly
pinpoint the lines in the log after which the queue was left in an
unreadable state.
Oct 21 17:17:01 postamt-v1-email 7e782f750655[608231]:
2026-10-21T17:17:01.230Z [WARN ] org.apache.activemq.artemis.core.client
- AMQ212051: Invalid concurrent session usage. Sessions are not supposed
to be used by more than one thread concurrently.
Oct 21 17:17:01 java.lang.Exception: trace
Oct 21 17:17:01 at
org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.startCall(ClientSessionImpl.java:1492)
Oct 21 17:17:01 at
org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:223)
Oct 21 17:17:01 at
org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)
Oct 21 17:17:01 at
org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:124)
Oct 21 17:17:01 at
org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.doSendx(ActiveMQMessageProducer.java:483)
Oct 21 17:17:01 at
org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:203)
Oct 21 17:17:01 at
org.apache.james.queue.jms.JMSCacheableMailQueue.produceMail(JMSCacheableMailQueue.java:401)
Oct 21 17:17:01 at
org.apache.james.queue.jms.JMSCacheableMailQueue.enQueue(JMSCacheableMailQueue.java:340)
Oct 21 17:17:01 at
org.apache.james.queue.api.MailQueue.enQueue(MailQueue.java:103)
Oct 21 17:17:01 at
org.apache.james.queue.jms.JMSCacheableMailQueue.enQueue(JMSCacheableMailQueue.java:369)
Oct 21 17:17:01 at
org.apache.james.smtpserver.SendMailHandler.lambda$onMessage$4(SendMailHandler.java:117)
Oct 21 17:17:01 at
com.github.fge.lambdas.runnable.RunnableChainer.doRun(RunnableChainer.java:18)
Oct 21 17:17:01 at
com.github.fge.lambdas.runnable.ThrowingRunnable.run(ThrowingRunnable.java:16)
Oct 21 17:17:01 at
java.base/java.util.Optional.ifPresentOrElse(Unknown Source)
Oct 21 17:17:01 at
org.apache.james.smtpserver.SendMailHandler.onMessage(SendMailHandler.java:89)
Oct 21 17:17:01 at
org.apache.james.smtpserver.DataLineJamesMessageHookHandler.executeJamesMessageHooks(DataLineJamesMessageHookHandler.java:216)
Oct 21 17:17:01 at
org.apache.james.smtpserver.DataLineJamesMessageHookHandler.processExtensions(DataLineJamesMessageHookHandler.java:198)
Oct 21 17:17:01 at
org.apache.james.smtpserver.DataLineJamesMessageHookHandler.onLine(DataLineJamesMessageHookHandler.java:115)
Oct 21 17:17:01 at
org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:87)
Oct 21 17:17:01 at
org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:75)
Oct 21 17:17:01 at
org.apache.james.protocols.smtp.core.SeparatingDataLineFilter.onBodyLine(SeparatingDataLineFilter.java:100)
Oct 21 17:17:01 at
org.apache.james.protocols.smtp.core.SeparatingDataLineFilter.onLine(SeparatingDataLineFilter.java:58)
Oct 21 17:17:01 at
org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:87)
Oct 21 17:17:01 at
org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:75)
Oct 21 17:17:01 at
org.apache.james.protocols.smtp.core.esmtp.MailSizeEsmtpExtension.onLine(MailSizeEsmtpExtension.java:158)
Oct 21 17:17:01 at
org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:87)
Oct 21 17:17:01 at
org.apache.james.protocols.smtp.core.DataCmdHandler$DataLineFilterWrapper.onLine(DataCmdHandler.java:75)
Oct 21 17:17:01 at
org.apache.james.protocols.netty.LineHandlerUpstreamHandler.channelRead(LineHandlerUpstreamHandler.java:52)
Oct 21 17:17:01 at
org.apache.james.protocols.netty.BasicChannelInboundHandler.channelRead(BasicChannelInboundHandler.java:182)
Oct 21 17:17:01 at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
Oct 21 17:17:01 at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
Oct 21 17:17:01 at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
Oct 21 17:17:01 at
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:361)
Oct 21 17:17:01 at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:325)
Oct 21 17:17:01 at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
Oct 21 17:17:01 at
io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
Oct 21 17:17:01 at
io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:425)
Oct 21 17:17:01 at
io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
Oct 21 17:17:01 at
io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66)
Oct 21 17:17:01 at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
Oct 21 17:17:01 at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
Oct 21 17:17:01 at java.base/java.lang.Thread.run(Unknown
Source)
Oct 21 17:17:01 2026-10-21T17:17:01.232Z [WARN ]
org.apache.activemq.artemis.core.server - AMQ222106: Replacing
incomplete LargeMessage with ID=23623536710
The last line is the only occurence of that warning in 2 days.
After this log line, James was not able to take anything out of the
queues anymore.
I guess that the analysis from the last email was incorrect and the
problem are these incomplete LargeMessages.
We exported the last email that caused the problem and imported it again
via the web API and the attachments (email body) seem to be incomplete.
I will create a Jira issue for this.
Best regards,
Felix
---
Gesellschaft für Interkulturelles Zusammenleben gGmbH (GIZ)
Felix Auringer (he/him)
Softwareentwickler
Reformationsplatz 2
13597 Berlin
Tel: -; Fax: 030/513010009
www.giz.berlin; [email protected]
Amtsgericht Charlottenburg HRB 200872 B
Geschäftsführerin: Dr. Britta Marschke
Am 2026-10-20 03:41, schrieb Felix Auringer:
Hey,
another update:
We are pretty sure that the problem is caused by Artemis' large message
feature.
By default, messages larger than 100KB are not persisted in the normal
journal but written to dedicated files.
When getting messages from the queue, the body is not populated if the
message is a large message.
This results in a return value of null which prevents the message from
being dequeued.
However, we were not able to find a way to fetch the complete body for
the ObjectMessage used in James.
Streaming the full body is only supported by BytesMessage or
StreamMessage.
As the body contains the raw email, we do not quite understand why the
ObjectMessage is used because it only contains the Java serialization
of a byte[].
Maybe somebody else has insight why it was built this way and how it
can be fixed for large messages.
We would appreciate it!
Best regards,
Felix
Oct 19, 2026 22:54:33 Felix Auringer <[email protected]>:
Hey,
we tried to debug the problem this afternoon/evening and have some
more information.
The problem appeared on our new production email server which was
running approximately for 24h before this problem.
It has delivered a few hundred emails before this and there were no
errors as far as we can tell.
At some point, new emails have stopped arriving at their destinations
but new emails were still accepted by the server.
If we stop the server and copy the persisted Artemis Email queue to
our staging environment and start the server, we get exactly the same
error. So there has to be a problem with the persisted state of
Artemis.
We have exported the queue data to an XML file using the Artemis CLI
with the command `data exp`. We manually looked at the emails and did
not see any obvious errors in the XML file (but we are also not sure
what we are looking for).
Our working hypothesis is that there is some weird email at the start
of the queue which can not be dequeued and thereby blocks all the
other emails in the queue.
However, we did not manage to remove the first email from the
persisted queue while still keeping the rest of the queue intact.
We tried to use the James web API but that returned the error from my
last email when trying to list the content of the spool queue.
Maybe somebody else has more experience with Artemis and could help us
with that?
Best regards,
Felix
Additionally, we also see this error in the logs but we are not sure
whether it is related / relevant:
2026-10-19T19:37:54.962Z [WARN ]
org.apache.activemq.artemis.core.client - AMQ212051: Invalid
concurrent session usage. Sessions are not supposed to be used by more
than one thread concurrently.
java.lang.Exception: trace
at
org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.startCall(ClientSessionImpl.java:1492)
at
org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:223)
at
org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)
at
org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:124)
at
org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.doSendx(ActiveMQMessageProducer.java:483)
at
org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:203)
at
org.apache.james.queue.jms.JMSCacheableMailQueue.produceMail(JMSCacheableMailQueue.java:401)
at
org.apache.james.queue.jms.JMSCacheableMailQueue.enQueue(JMSCacheableMailQueue.java:340)
at
org.apache.james.transport.mailets.remote.delivery.DeliveryRunnable.reAttemptDelivery(DeliveryRunnable.java:254)
at
org.apache.james.transport.mailets.remote.delivery.DeliveryRunnable.handleTemporaryFailure(DeliveryRunnable.java:226)
at
org.apache.james.transport.mailets.remote.delivery.DeliveryRunnable.attemptDelivery(DeliveryRunnable.java:192)
at
org.apache.james.transport.mailets.remote.delivery.DeliveryRunnable.lambda$processMail$0(DeliveryRunnable.java:129)
at
reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:61)
at reactor.core.publisher.Mono.subscribe(Mono.java:4576)
at
reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:126)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
at java.base/java.util.concurrent.FutureTask.run(Unknown
Source)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.base/java.lang.Thread.run(Unknown Source)
Oct 19, 2026 17:32:27 Felix Auringer <[email protected]>:
Hey,
I have a problem with the James Queue.
The Mail Spooler puts emails into the queue but they can not be
dequeued.
When James starts and loads the queue, I get the following error
```
2026-10-19T15:05:13.235Z ESC[1;31m[ERROR]ESC[0;39m
reactor.core.publisher.Operators - Operator called default
onErrorDropped
java.lang.NullPointerException: Cannot read the array length because
"<parameter1>" is null
at java.base/java.io.ByteArrayInputStream.<init>(Unknown
Source)
at
jakarta.mail.util.SharedByteArrayInputStream.<init>(SharedByteArrayInputStream.java:46)
at
org.apache.james.queue.jms.MimeMessageObjectMessageSource.<init>(MimeMessageObjectMessageSource.java:52)
at
org.apache.james.queue.jms.JMSCacheableMailQueue.mimeMessage(JMSCacheableMailQueue.java:466)
at
org.apache.james.queue.jms.JMSCacheableMailQueue.createMail(JMSCacheableMailQueue.java:453)
at
org.apache.james.queue.jms.JMSCacheableMailQueue.createMailQueueItem(JMSCacheableMailQueue.java:592)
at
org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:298)
... 12 common frames omitted
Wrapped by: org.apache.james.queue.api.MailQueue$MailQueueException:
Unable to dequeue next message;
nested exception is:
java.lang.NullPointerException: Cannot read the array length
because "<parameter1>" is null
at
org.apache.james.queue.jms.JMSCacheableMailQueue.deQueueOneItem(JMSCacheableMailQueue.java:312)
at
reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:45)
at
reactor.core.publisher.FluxRepeatPredicate$RepeatPredicateSubscriber.resubscribe(FluxRepeatPredicate.java:120)
at
reactor.core.publisher.MonoRepeatPredicate.subscribeOrReturn(MonoRepeatPredicate.java:48)
at
reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:55)
at
reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
at java.base/java.util.concurrent.FutureTask.run(Unknown
Source)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.base/java.lang.Thread.run(Unknown Source)
Wrapped by: reactor.core.Exceptions$ErrorCallbackNotImplemented:
org.apache.james.queue.api.MailQueue$MailQueueException: Unable to
dequeue next message;
nested exception is:
java.lang.NullPointerException: Cannot read the array length
because "<parameter1>" is null
```
This happens with James `2d7e913ff728ac1992d82c493aa9421e2abc0ea2`
(Tmail 1.0.22).
Has anybody else seen this error?
Is there any way to debug Emails in the queue?
Best regards,
Felix
Today from a different email address because my James server does not
deliver my emails :(
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
