Hi,
Why are MQTT last will messages authenticated with Artemis before being sent?
From my understanding of the MQTT last will feature is that authentication is a
separate step handled during the client's initial connection. The LWT itself is
a message prepared by a client and stored by the broker to be published only if
the client disconnects unexpectedly. The security of the LWT message is
therefore dependent on the security of the initial client connection, which
must be established through methods like username/password or TLS and not when
the LWT is delivered/published.
We have written a custom securityManager plugin that uses Json Web Tokens as
passwords for connecting MQTT clients. We use MQTT for server availability
which has a state of either up or down. When a server client connects with the
MQTT Broker it is authenticated with the broker using a JWT, registers a last
will, and then publishes a server up message on a MQTT topic. We have other
MQTT clients that also connect with the Artemis broker using JWTs for auth and
then subscribe to server state topics. The MQTT last will is used to publish
server down messages when the long running server dies for some reason. Some of
the servers publish “re-announce” messages as they complete certain steps of
processing that they publish on a different topic using the initial client
connection. This was initially developed using Artemis 2.28.0 and later used
in Artemis 2.34.0 and 2.40.0. Our securityManager plugin,
JwtJassSecurityManager, implements the ActiveMQSecurityManager5 interface.
During our initial development of the securityManager plugin we saw MQTT last
will messages failing authentication and put a workaround in place to check
that if there is a subject and a principal with the same user name (each MQTT
client has a unique user name) which means that this is the same session that
has previously been authenticated, we will validate the JWT, but ignore the
expiration time. We are now upgrading to Artemis 2.43.0 and would like to
remove the workaround. Our workaround for last will authentication breaks down
when the keys are rotated. It is about to become a real problem as we are
going to rotate the JWT key on a more frequent schedule. Here are the errors
produced:
2025-10-14 15:06:51,858 INFO [com.hpe.hpc.activemq.JwtJaasSecurityManager]
validation failed: username: x3000c0s9b0n0, details: auth is invalid
InvalidJwtException: JWT processing failed. Additional details: [[17] Unable to
process JOSE object (cause: org.jose4j.lang.UnresolvableKeyException: Unable to
find a suitable verification key for JWS w/ header
{"alg":"RS256","kid”:”previous KEY-A","typ":"JWT"} from JWKs
[org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-A, alg=RS256, n=CENSORED,
e=CENSORED}, org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-B, alg=RS256,
n=CENSORED, e=CENSORED}, org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-C,
alg=RS256, n=CENSORED, e=CENSORED}]):
JsonWebSignature{"alg":"RS256","kid”:”previous KEY-A","typ":”JWT”CENSORED
]
2025-10-14 15:06:51,861 INFO [com.hpe.hpc.activemq.JwtJaasSecurityManager]
Invalid authentication for user: x3000c0s9b0n0, subject: Subject:
Principal: pzxQ2XQN
Principal: admin
Principal: x3000c0s9b0n0
2025-10-14 15:06:51,863 WARN [org.apache.activemq.artemis.core.server]
AMQ222216: Security problem while authenticating: AMQ229031: Unable to validate
user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL certificate subject DN:
unavailable
2025-10-14 15:06:51,863 ERROR [org.apache.activemq.artemis.core.protocol.mqtt]
AMQ834002: Error processing control packet:
MqttPublishMessage[fixedHeader=MqttFixedHeader[messageType=PUBLISH,
isDup=false, qosLevel=AT_MOST_ONCE, isRetain=true, remainingLength=67],
variableHeader=MqttPublishVariableHeader[topicName=trusted/x3000c0s9b0n0/dvs/server/state,
packetId=-1], payload=PooledSlicedByteBuf(ridx: 0, widx: 27, cap: 27/27,
unwrapped: PooledUnsafeDirectByteBuf(ridx: 69, widx: 69, cap: 80))]
org.apache.activemq.artemis.api.core.ActiveMQSecurityException: AMQ229031:
Unable to validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL
certificate subject DN: unavailable
at
org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.authenticationFailed(SecurityStoreImpl.java:449)
at
org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.check(SecurityStoreImpl.java:341)
at
org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.securityCheck(ServerSessionImpl.java:527)
at
org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.doSend(ServerSessionImpl.java:2365)
at
org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1995)
at
org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1934)
at
org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendToQueue(MQTTPublishManager.java:242)
at
org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.handlePublish(MQTTProtocolHandler.java:322)
at
org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.act(MQTTProtocolHandler.java:164)
at org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:32)
at
org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:69)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at
org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:120)
2025-10-14 15:06:51,866 WARN [org.apache.activemq.artemis.core.server]
AMQ222216: Security problem while authenticating: AMQ229031: Unable to validate
user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL certificate subject DN:
unavailable
2025-10-14 15:06:51,866 ERROR [org.apache.activemq.artemis.core.protocol.mqtt]
AMQ834007: Authorization failure sending will message: AMQ229031: Unable to
validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL certificate
subject DN: unavailable
Regards,
Paul Shields
