This approach has always been a challenge for token-based authn systems, it isn’t JWT specific. A long running stateful connection is used for performance and lowering latency (usually queues, databases, etc).
Feels like back-end systems are an afterthought for JWT system design, or rather JWT presumes REST/API style connection-per-request behavior. Two-way SSL certificates tend to split the difference. -Matt > On Oct 28, 2025, at 2:12 PM, Shields, Paul <[email protected]> > wrote: > > Hi Matt, > > Thank you for the suggestion, But our backend s/w design is based around long > term connections, So open and closing the connection has consequences in our > backend. The JWT works well for the authenticate step of gaining access to > the broker and setting up a session. It’s our authorize method that needs > work. Does a session live as long as the MQTT client is connected to the > broker? Any docs on how sessions are used and how they interact with the > authorize method would be helpful. > > Regards, > Paul > > From: Matt Pavlovich <[email protected]> > Date: Tuesday, October 28, 2025 at 1:59 PM > To: [email protected] <[email protected]> > Subject: Re: MQTT Last Will not sent because denied authentication > > Hi Paul- > > The usage of JWT and LWT are competing features, since JWT expires and LWT is > intended to alert for unplanned disconnect of long-running connections. A > possible solution is to perform periodic close-open of the connections and > re-register the LWT send would always fall within the timeframe of the JWT > expiry. > > Matt Pavlovich > >> On Oct 28, 2025, at 11:36 AM, Shields, Paul <[email protected]> >> wrote: >> >> Hi Justin, >> >> I am struggling with this statement >> However, authorization for actually >> sending the will message is not performed at this point. Authorization is >> only performed when the LWT message is actually sent (e.g. when the >> client's connection fails). All sending operations are authorized at the >> time they occur whether that's via the normal PUBLISH packet from the >> client or for a LWT message sent on the client's behalf by the broker. >> If we were using a user name and password (even if that was stored in LDAP) >> that would be ok. But we are using the machine ID for the user name and a >> JWT is used in the password field of the MQTT connect. My problem is that >> the JWTs for authentication have a short expiration time (5min). This link >> explains the JWT concept we are using for authentication. >> https://urldefense.com/v3/__https://stytch.com/blog/understanding-jwks/__;!!NpxR!l0K48Qr3zBCaoFmyCZXVvGQOZMZY59F9n8cgHM4q5vNc5xz0PCpg7C4HOGannJ2cB0tn2U7ian9s7tNc7A$ >> but instead of using Stytch as the authorization server JWKS https >> endpoint we are using Spire with a plugin for the JWKS. The specific issue >> is that the JWT supplied with the connect to the broker has expired by the >> time the LWT is being delivered. We chose the MQTT protocol for its >> simplicity and the LWT feature for server heartbeat. >> >> Not sure we have implemented the authorize function of the securityManager >> class in alignment with how Artemis operates. We do not create users up >> front but on the fly as each machine connects to publish its status or >> scribe to another's status. We use the admin role for all of the machine >> users/IDs. One on of the key features for using the JWT for us is that we >> imbed the machine ID in the jwt payload and in the authorize function >> reject any publish (SEND) request if the target topic does not match the >> sender machine ID. Not sure how we would update the JWT for every connection >> every 5min so that the JWT would be valid at the time it will be sent? >> >> Is there a reference implementation of a pluggable securityManager that uses >> JWTs? >> >> Any other suggestions? >> >> Regards, >> Paul >> >> From: Justin Bertram <[email protected]> >> Date: Tuesday, October 28, 2025 at 9:34 AM >> To: [email protected] <[email protected]> >> Subject: Re: MQTT Last Will not sent because denied authentication >> >> Just following up to ensure my explanation made sense. Do you need anything >> further here? >> >> >> Justin >> >> On Fri, Oct 24, 2025 at 4:17 PM Justin Bertram <[email protected]> wrote: >> >>> An MQTT client connects to the broker via a CONNECT packet. Typically this >>> packet contains the client's credentials which are then authenticated by >>> the broker. >>> >>> Keep in mind that authentication and authorization are related but >>> separate things. A client may be authenticated but not authorized to >>> consume messages from or send messages to specific topics. This is a >>> fundamental concept of role-based access control. >>> >>> The CONNECT packet also contains all the details about the LWT message >>> (i.e. payload, properties, & topic). However, authorization for actually >>> sending the will message is not performed at this point. Authorization is >>> only performed when the LWT message is actually sent (e.g. when the >>> client's connection fails). All sending operations are authorized at the >>> time they occur whether that's via the normal PUBLISH packet from the >>> client or for a LWT message sent on the client's behalf by the broker. >>> >>> Authorization is important here because the destination topic for the LWT >>> message is arbitrary. If authorization was not performed then it would be >>> simple for a client to send a message to a topic which it would not >>> otherwise have authorization. >>> >>> If authorization is failing when the broker attempts to send the client's >>> LWT message and you're circumventing this so that the message is actually >>> sent then it seems you may be undermining the security of your environment. >>> You may inadvertently allow a clever, nefarious actor to send a message to >>> a topic to which they are not authorized. >>> >>> How are you currently re-authenticating your clients when their JWTs >>> expire? >>> >>> >>> Justin >>> >>> On Fri, Oct 24, 2025 at 3:19 PM Shields, Paul <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> Why are MQTT last will messages authenticated with Artemis before being >>>> sent? From my understanding of the MQTT last will feature is that >>>> authentication is a separate step handled during the client's initial >>>> connection. The LWT itself is a message prepared by a client and stored by >>>> the broker to be published only if the client disconnects unexpectedly. The >>>> security of the LWT message is therefore dependent on the security of the >>>> initial client connection, which must be established through methods like >>>> username/password or TLS and not when the LWT is delivered/published. >>>> >>>> We have written a custom securityManager plugin that uses Json Web >>>> Tokens as passwords for connecting MQTT clients. We use MQTT for server >>>> availability which has a state of either up or down. When a server client >>>> connects with the MQTT Broker it is authenticated with the broker using a >>>> JWT, registers a last will, and then publishes a server up message on a >>>> MQTT topic. We have other MQTT clients that also connect with the Artemis >>>> broker using JWTs for auth and then subscribe to server state topics. The >>>> MQTT last will is used to publish server down messages when the long >>>> running server dies for some reason. Some of the servers publish >>>> “re-announce” messages as they complete certain steps of processing that >>>> they publish on a different topic using the initial client connection. >>>> This was initially developed using Artemis 2.28.0 and later used in Artemis >>>> 2.34.0 and 2.40.0. Our securityManager plugin, JwtJassSecurityManager, >>>> implements the ActiveMQSecurityManager5 interface. During our initial >>>> development of the securityManager plugin we saw MQTT last will messages >>>> failing authentication and put a workaround in place to check that if there >>>> is a subject and a principal with the same user name (each MQTT client has >>>> a unique user name) which means that this is the same session that has >>>> previously been authenticated, we will validate the JWT, but ignore the >>>> expiration time. We are now upgrading to Artemis 2.43.0 and would like to >>>> remove the workaround. Our workaround for last will authentication breaks >>>> down when the keys are rotated. It is about to become a real problem as we >>>> are going to rotate the JWT key on a more frequent schedule. Here are the >>>> errors produced: >>>> >>>> 2025-10-14 15:06:51,858 INFO >>>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] validation failed: username: >>>> x3000c0s9b0n0, details: auth is invalid >>>> InvalidJwtException: JWT processing failed. Additional details: [[17] >>>> Unable to process JOSE object (cause: >>>> org.jose4j.lang.UnresolvableKeyException: Unable to find a suitable >>>> verification key for JWS w/ header {"alg":"RS256","kid”:”previous >>>> KEY-A","typ":"JWT"} from JWKs [org.jose4j.jwk.RsaJsonWebKey{kty=RSA, >>>> kid=KEY-A, alg=RS256, n=CENSORED, e=CENSORED}, >>>> org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-B, alg=RS256, n=CENSORED, >>>> e=CENSORED}, org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-C, alg=RS256, >>>> n=CENSORED, e=CENSORED}]): JsonWebSignature{"alg":"RS256","kid”:”previous >>>> KEY-A","typ":”JWT”CENSORED >>>> ] >>>> 2025-10-14 15:06:51,861 INFO >>>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] Invalid authentication for >>>> user: x3000c0s9b0n0, subject: Subject: >>>> Principal: pzxQ2XQN >>>> Principal: admin >>>> Principal: x3000c0s9b0n0 >>>> >>>> 2025-10-14 15:06:51,863 WARN [org.apache.activemq.artemis.core.server] >>>> AMQ222216: Security problem while authenticating: AMQ229031: Unable to >>>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL >>>> certificate subject DN: unavailable >>>> 2025-10-14 15:06:51,863 ERROR >>>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834002: Error >>>> processing control packet: >>>> MqttPublishMessage[fixedHeader=MqttFixedHeader[messageType=PUBLISH, >>>> isDup=false, qosLevel=AT_MOST_ONCE, isRetain=true, remainingLength=67], >>>> variableHeader=MqttPublishVariableHeader[topicName=trusted/x3000c0s9b0n0/dvs/server/state, >>>> packetId=-1], payload=PooledSlicedByteBuf(ridx: 0, widx: 27, cap: 27/27, >>>> unwrapped: PooledUnsafeDirectByteBuf(ridx: 69, widx: 69, cap: 80))] >>>> org.apache.activemq.artemis.api.core.ActiveMQSecurityException: >>>> AMQ229031: Unable to validate user from 127.0.0.6:49859. Username: >>>> x3000c0s9b0n0; SSL certificate subject DN: unavailable >>>> at >>>> org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.authenticationFailed(SecurityStoreImpl.java:449) >>>> at >>>> org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.check(SecurityStoreImpl.java:341) >>>> at >>>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.securityCheck(ServerSessionImpl.java:527) >>>> at >>>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.doSend(ServerSessionImpl.java:2365) >>>> at >>>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1995) >>>> at >>>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1934) >>>> at >>>> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendToQueue(MQTTPublishManager.java:242) >>>> at >>>> org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.handlePublish(MQTTProtocolHandler.java:322) >>>> at >>>> org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.act(MQTTProtocolHandler.java:164) >>>> at >>>> org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:32) >>>> at >>>> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:69) >>>> at >>>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) >>>> at >>>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) >>>> at >>>> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:120) >>>> 2025-10-14 15:06:51,866 WARN [org.apache.activemq.artemis.core.server] >>>> AMQ222216: Security problem while authenticating: AMQ229031: Unable to >>>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL >>>> certificate subject DN: unavailable >>>> 2025-10-14 15:06:51,866 ERROR >>>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834007: Authorization >>>> failure sending will message: AMQ229031: Unable to validate user from >>>> 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL certificate subject DN: >>>> unavailable >>>> >>>> Regards, >>>> Paul Shields >>>> >>> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > For further information, visit: > https://urldefense.com/v3/__https://activemq.apache.org/contact__;!!NpxR!l0K48Qr3zBCaoFmyCZXVvGQOZMZY59F9n8cgHM4q5vNc5xz0PCpg7C4HOGannJ2cB0tn2U7ian-lWM-jFw$ > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] For further information, visit: https://activemq.apache.org/contact
