I just granted your Jira account request.
Justin On Tue, Oct 28, 2025 at 2:53 PM Shields, Paul <[email protected]> wrote: > I don’t have a ASF Jira account. I am happy to request one to open a Jira > ticket for this. > > Paul > > From: Justin Bertram <[email protected]> > Date: Tuesday, October 28, 2025 at 2:20 PM > To: [email protected] <[email protected]> > Subject: Re: MQTT Last Will not sent because denied authentication > > I tend to agree with Matt here, but I think perhaps a feature could be > added to Artemis so that LWT are authorized on connect to avoid this kind > of problem. This behavior would be off by default so as not to impact > existing users. > > Do you have a Jira account? If so, would you mind opening an issue [1] for > this? > > > Justin > > [1] > https://urldefense.com/v3/__https://issues.apache.org/jira/browse/ARTEMIS__;!!NpxR!hdQbIYBgb_bBdoWH7TVo30l1yIKlErqg_yZOBlMZLt8SWQyPYex2pjuUSytWaI1zd5pSClzgpi0oxTF0wA$ > > On Tue, Oct 28, 2025 at 1:58 PM Matt Pavlovich <[email protected]> > wrote: > > > Hi Paul- > > > > The usage of JWT and LWT are competing features, since JWT expires and > LWT > > is intended to alert for unplanned disconnect of long-running > connections. > > A possible solution is to perform periodic close-open of the connections > > and re-register the LWT send would always fall within the timeframe of > the > > JWT expiry. > > > > Matt Pavlovich > > > > > On Oct 28, 2025, at 11:36 AM, Shields, Paul > <[email protected]> > > wrote: > > > > > > Hi Justin, > > > > > > I am struggling with this statement > > > However, authorization for actually > > > sending the will message is not performed at this point. Authorization > is > > > only performed when the LWT message is actually sent (e.g. when the > > > client's connection fails). All sending operations are authorized at > the > > > time they occur whether that's via the normal PUBLISH packet from the > > > client or for a LWT message sent on the client's behalf by the broker. > > > If we were using a user name and password (even if that was stored in > > LDAP) that would be ok. But we are using the machine ID for the user name > > and a JWT is used in the password field of the MQTT connect. My problem > is > > that the JWTs for authentication have a short expiration time (5min). > This > > link explains the JWT concept we are using for authentication. > > > https://urldefense.com/v3/__https://stytch.com/blog/understanding-jwks/__;!!NpxR!hdQbIYBgb_bBdoWH7TVo30l1yIKlErqg_yZOBlMZLt8SWQyPYex2pjuUSytWaI1zd5pSClzgpi1KjsUsCg$ > but instead of using Stytch > > as the authorization server JWKS https endpoint we are using Spire with a > > plugin for the JWKS. The specific issue is that the JWT supplied with the > > connect to the broker has expired by the time the LWT is being delivered. > > We chose the MQTT protocol for its simplicity and the LWT feature for > > server heartbeat. > > > > > > Not sure we have implemented the authorize function of the > > securityManager class in alignment with how Artemis operates. We do not > > create users up front but on the fly as each machine connects to publish > > its status or scribe to another's status. We use the admin role for all > of > > the machine users/IDs. One on of the key features for using the JWT for > us > > is that we imbed the machine ID in the jwt payload and in the authorize > > function reject any publish (SEND) request if the target topic does not > > match the sender machine ID. Not sure how we would update the JWT for > every > > connection every 5min so that the JWT would be valid at the time it will > be > > sent? > > > > > > Is there a reference implementation of a pluggable securityManager that > > uses JWTs? > > > > > > Any other suggestions? > > > > > > Regards, > > > Paul > > > > > > From: Justin Bertram <[email protected]> > > > Date: Tuesday, October 28, 2025 at 9:34 AM > > > To: [email protected] <[email protected]> > > > Subject: Re: MQTT Last Will not sent because denied authentication > > > > > > Just following up to ensure my explanation made sense. Do you need > > anything > > > further here? > > > > > > > > > Justin > > > > > > On Fri, Oct 24, 2025 at 4:17 PM Justin Bertram <[email protected]> > > wrote: > > > > > >> An MQTT client connects to the broker via a CONNECT packet. Typically > > this > > >> packet contains the client's credentials which are then authenticated > by > > >> the broker. > > >> > > >> Keep in mind that authentication and authorization are related but > > >> separate things. A client may be authenticated but not authorized to > > >> consume messages from or send messages to specific topics. This is a > > >> fundamental concept of role-based access control. > > >> > > >> The CONNECT packet also contains all the details about the LWT message > > >> (i.e. payload, properties, & topic). However, authorization for > actually > > >> sending the will message is not performed at this point. Authorization > > is > > >> only performed when the LWT message is actually sent (e.g. when the > > >> client's connection fails). All sending operations are authorized at > the > > >> time they occur whether that's via the normal PUBLISH packet from the > > >> client or for a LWT message sent on the client's behalf by the broker. > > >> > > >> Authorization is important here because the destination topic for the > > LWT > > >> message is arbitrary. If authorization was not performed then it would > > be > > >> simple for a client to send a message to a topic which it would not > > >> otherwise have authorization. > > >> > > >> If authorization is failing when the broker attempts to send the > > client's > > >> LWT message and you're circumventing this so that the message is > > actually > > >> sent then it seems you may be undermining the security of your > > environment. > > >> You may inadvertently allow a clever, nefarious actor to send a > message > > to > > >> a topic to which they are not authorized. > > >> > > >> How are you currently re-authenticating your clients when their JWTs > > >> expire? > > >> > > >> > > >> Justin > > >> > > >> On Fri, Oct 24, 2025 at 3:19 PM Shields, Paul > > <[email protected]> > > >> wrote: > > >> > > >>> Hi, > > >>> > > >>> Why are MQTT last will messages authenticated with Artemis before > being > > >>> sent? From my understanding of the MQTT last will feature is that > > >>> authentication is a separate step handled during the client's initial > > >>> connection. The LWT itself is a message prepared by a client and > > stored by > > >>> the broker to be published only if the client disconnects > > unexpectedly. The > > >>> security of the LWT message is therefore dependent on the security of > > the > > >>> initial client connection, which must be established through methods > > like > > >>> username/password or TLS and not when the LWT is delivered/published. > > >>> > > >>> We have written a custom securityManager plugin that uses Json Web > > >>> Tokens as passwords for connecting MQTT clients. We use MQTT for > server > > >>> availability which has a state of either up or down. When a server > > client > > >>> connects with the MQTT Broker it is authenticated with the broker > > using a > > >>> JWT, registers a last will, and then publishes a server up message > on a > > >>> MQTT topic. We have other MQTT clients that also connect with the > > Artemis > > >>> broker using JWTs for auth and then subscribe to server state topics. > > The > > >>> MQTT last will is used to publish server down messages when the long > > >>> running server dies for some reason. Some of the servers publish > > >>> “re-announce” messages as they complete certain steps of processing > > that > > >>> they publish on a different topic using the initial client > connection. > > >>> This was initially developed using Artemis 2.28.0 and later used in > > Artemis > > >>> 2.34.0 and 2.40.0. Our securityManager plugin, > JwtJassSecurityManager, > > >>> implements the ActiveMQSecurityManager5 interface. During our initial > > >>> development of the securityManager plugin we saw MQTT last will > > messages > > >>> failing authentication and put a workaround in place to check that if > > there > > >>> is a subject and a principal with the same user name (each MQTT > client > > has > > >>> a unique user name) which means that this is the same session that > has > > >>> previously been authenticated, we will validate the JWT, but ignore > the > > >>> expiration time. We are now upgrading to Artemis 2.43.0 and would > > like to > > >>> remove the workaround. Our workaround for last will authentication > > breaks > > >>> down when the keys are rotated. It is about to become a real problem > > as we > > >>> are going to rotate the JWT key on a more frequent schedule. Here are > > the > > >>> errors produced: > > >>> > > >>> 2025-10-14 15:06:51,858 INFO > > >>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] validation failed: > > username: > > >>> x3000c0s9b0n0, details: auth is invalid > > >>> InvalidJwtException: JWT processing failed. Additional details: [[17] > > >>> Unable to process JOSE object (cause: > > >>> org.jose4j.lang.UnresolvableKeyException: Unable to find a suitable > > >>> verification key for JWS w/ header {"alg":"RS256","kid”:”previous > > >>> KEY-A","typ":"JWT"} from JWKs [org.jose4j.jwk.RsaJsonWebKey{kty=RSA, > > >>> kid=KEY-A, alg=RS256, n=CENSORED, e=CENSORED}, > > >>> org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-B, alg=RS256, > n=CENSORED, > > >>> e=CENSORED}, org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-C, > > alg=RS256, > > >>> n=CENSORED, e=CENSORED}]): > > JsonWebSignature{"alg":"RS256","kid”:”previous > > >>> KEY-A","typ":”JWT”CENSORED > > >>> ] > > >>> 2025-10-14 15:06:51,861 INFO > > >>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] Invalid authentication > > for > > >>> user: x3000c0s9b0n0, subject: Subject: > > >>> Principal: pzxQ2XQN > > >>> Principal: admin > > >>> Principal: x3000c0s9b0n0 > > >>> > > >>> 2025-10-14 15:06:51,863 WARN > [org.apache.activemq.artemis.core.server] > > >>> AMQ222216: Security problem while authenticating: AMQ229031: Unable > to > > >>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL > > >>> certificate subject DN: unavailable > > >>> 2025-10-14 15:06:51,863 ERROR > > >>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834002: Error > > >>> processing control packet: > > >>> MqttPublishMessage[fixedHeader=MqttFixedHeader[messageType=PUBLISH, > > >>> isDup=false, qosLevel=AT_MOST_ONCE, isRetain=true, > remainingLength=67], > > >>> > > > variableHeader=MqttPublishVariableHeader[topicName=trusted/x3000c0s9b0n0/dvs/server/state, > > >>> packetId=-1], payload=PooledSlicedByteBuf(ridx: 0, widx: 27, cap: > > 27/27, > > >>> unwrapped: PooledUnsafeDirectByteBuf(ridx: 69, widx: 69, cap: 80))] > > >>> org.apache.activemq.artemis.api.core.ActiveMQSecurityException: > > >>> AMQ229031: Unable to validate user from 127.0.0.6:49859. Username: > > >>> x3000c0s9b0n0; SSL certificate subject DN: unavailable > > >>> at > > >>> > > > org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.authenticationFailed(SecurityStoreImpl.java:449) > > >>> at > > >>> > > > org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.check(SecurityStoreImpl.java:341) > > >>> at > > >>> > > > org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.securityCheck(ServerSessionImpl.java:527) > > >>> at > > >>> > > > org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.doSend(ServerSessionImpl.java:2365) > > >>> at > > >>> > > > org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1995) > > >>> at > > >>> > > > org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1934) > > >>> at > > >>> > > > org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendToQueue(MQTTPublishManager.java:242) > > >>> at > > >>> > > > org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.handlePublish(MQTTProtocolHandler.java:322) > > >>> at > > >>> > > > org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.act(MQTTProtocolHandler.java:164) > > >>> at > > >>> org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:32) > > >>> at > > >>> > > > org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:69) > > >>> at > > >>> > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) > > >>> at > > >>> > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) > > >>> at > > >>> > > > org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:120) > > >>> 2025-10-14 15:06:51,866 WARN > [org.apache.activemq.artemis.core.server] > > >>> AMQ222216: Security problem while authenticating: AMQ229031: Unable > to > > >>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL > > >>> certificate subject DN: unavailable > > >>> 2025-10-14 15:06:51,866 ERROR > > >>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834007: > > Authorization > > >>> failure sending will message: AMQ229031: Unable to validate user from > > >>> 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL certificate subject > DN: > > >>> unavailable > > >>> > > >>> Regards, > > >>> Paul Shields > > >>> > > >> > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > For further information, visit: > https://urldefense.com/v3/__https://activemq.apache.org/contact__;!!NpxR!hdQbIYBgb_bBdoWH7TVo30l1yIKlErqg_yZOBlMZLt8SWQyPYex2pjuUSytWaI1zd5pSClzgpi1wpKbyCQ$ > > > > > > >
