I'm not sure what would have changed in those versions to impact performance in the way you're observing. However, the fact that it's no longer a problem in the latest release means I'm not going to invest much time into further investigation.
That said, I wanted to clarify a few points: - If the SSL *handshake* is failing then the connection isn't even making it to the stage where the broker is verifying the certificate via the CertificateLoginModule or enforcing role-based security. Given that, the authentication and authorization cache which I mentioned in my previous email is not involved. Generally speaking, you can read more about this caching in the documentation [1]. - The broker delegates all the real SSL/TLS work to either the JDK or to OpenSSL based on the value of the sslProvider URL parameter on your acceptor [2]. By default, it delegates to the JDK. However, you would probably get better performance if you set sslProvider=OPENSSL and deployed the proper tcnative jar from Netty [3]. Justin [1] https://activemq.apache.org/components/artemis/documentation/latest/security.html [2] https://activemq.apache.org/components/artemis/documentation/latest/configuring-transports.html#configuring-netty-ssl [3] https://netty.io/wiki/forked-tomcat-native.html On Tue, Aug 22, 2023 at 4:37 AM Shiv Kumar Dixit <shivkumardi...@eurofins.com.invalid> wrote: > Hi Justin, > We are using certificate based authentication. > > Below is the details of issue we faced with Artemis 2.27.1. > 1. Host A has 10 client applications who use certificate based > authentication. Most of them are using OpenWire protocol. Some are AMQP and > core based as well. > 2. 8 of them are configured correctly to pull correct client certificate > and present to broker. These result in successful SSL handshake and > connections are made. They are able to consume messages without issue. > 3. Many times they disconnect due to time out or broker restart or > customer application restart etc. but connect back to broker and message > consumption continues without issue. > 4. 2 of them have not configured SSL correctly or have wrong/invalid > certificate and they try to connect to broker. > 5. Broker throws SSL handshake error and rejects connection. As these > listener keep running hence keep generating this connection issue. > 6. Broker memory consumption increases and it also slows down a bit. > 7. When any valid connection times out (from previous 8 good clients), > they are now not able to connect back to broker as invalid connection > attempts are too much and good clients get SSL handshake error with > connection timeout. > 8. Now this impacts those 8 good clients which were working fine before 2 > bad clients started creating issue. > 9. Similar behavior was not seen in Artemis 2.26.0 or Artemis 2.29.0+. > Here broker gets a hit due to invalid connection attempts, slows down a bit > but continue to support good connection. > > Do you have some suggestion as how to handle such case in certificate > based authentication? > > Also regarding the caching of good and bad connection, where is this > setting defined in broker configuration? > > Thanks > Shiv > -----Original Message----- > From: Justin Bertram <jbert...@apache.org> > Sent: Monday, August 21, 2023 7:40 PM > To: users@activemq.apache.org > Subject: Re: Pre-authentication with Broker > > > > CAUTION: EXTERNAL EMAIL - Sent from an email domain that is not formally > trusted by Eurofins. > > Do not click on links or open attachments unless you recognise the sender > and are certain that the content is safe. > > Can you elaborate on the nature of the bottleneck which is preventing new, > good connections, etc.? Is it related to CPU, network bandwidth, or > something internal to the code, etc.? > > By default the broker caches the result of both successful and failed > authentication in order to reduce strain on the broker and any underlying > systems (e.g. LDAP). Have you disabled this caching? > > You mention that "some broker versions (e.g. 2.27.1) are very sensitive." > Does that mean that other versions are *not* sensitive in the same way? If > so, which versions? > > Ultimately just about any software serving clients on a network is > susceptible to denial-of-service (i.e. DOS) issues like this. If you get > into a situation where a particular host is causing problems and impacting > other clients your only option may be to block that host even if it means > blocking well-behaving clients. > > Are there any logging improvements you'd like to see in this kind of > situation? We will probably implement authn/z metrics soon (via > ARTEMIS-4306 [1]) at which point it may be viable to reduce the logging > for authn failures. > > Ultimately we want to minimize the impact of clients like this as much as > possible, but it will never be 0. > > Lastly, I'm not aware of any simple way to "pre-authenticate" clients. Off > the top of my head the only thing I can conceive is that if the broker is > using a centralized resource like LDAP then the client could talk to LDAP > before talking to the broker to ensure it had valid credentials. That, of > course, would be a lot of extra work for the client and would pose a > potential problem for the LDAP server itself. > > > Justin > > [1] https://issues.apache.org/jira/browse/ARTEMIS-4306 > > On Mon, Jul 24, 2023 at 1:31 AM Shiv Kumar Dixit < > shivkumardi...@eurofins.com.invalid> wrote: > > > We would like to know if there is any mechanism to pre-authenticate > > broker users first (basic authentication or certificate > > authentication) and if the credentials/certificate is valid then only > > connection attempt is made on the broker. We are seeing a case where > > some users are either using invalid user-name password or invalid > > certificate (expired/missing private key or different cases of SSL > handshake failure) to connect to brokers. > > > > Since such applications keep running with invalid authentication and > > take lot of time to fix from client side, we are seeing too many > > connection attempts being made which subsequently failing on the > > broker. Broker logs also get filled very fast due to it. We can't just > > block those erring IP as same IP can host a good and a bad > > application. Blocking the IP will also block well behaving application. > > > > Some broker versions (e.g. 2.27.1) are very sensitive to such errors > > and it impacted normal broker operations where new good connections > > were denied or delayed, existing consumers were not able to pull > > messages or clustering and movement of messages across cluster was > impacted. > > > > We would like to explore any proxy or pre-authentication where such > > erring consumers are not allowed to make any connection attempt itself > > thus safeguarding the broker. Any input or lead will be very useful. > > > > Thanks > > Shiv > > >
