Re: AW: AW: Crypto Randomly Not Getting Initialized
Thomas, On 6/13/2023 10:45 AM, Thomas Hoffmann (Speed4Trade GmbH) wrote: Hello Jerry, -Ursprüngliche Nachricht- Von: Jerry Malcolm Gesendet: Dienstag, 13. Juni 2023 17:35 An: users@tomcat.apache.org Betreff: Re: AW: Crypto Randomly Not Getting Initialized Hi Thomas, On 6/13/2023 2:08 AM, Thomas Hoffmann (Speed4Trade GmbH) wrote: Hello Jerry, -Ursprüngliche Nachricht- Von: Jerry Malcolm Gesendet: Dienstag, 13. Juni 2023 08:50 An: users@tomcat.apache.org Betreff: Crypto Randomly Not Getting Initialized I am running Tomcat 9.0.56 in multiple AWS EC2 instances with Amazon Linux2 in a production environment. A couple of years ago, we started getting weird errors that the "Crypto Mechanism" failed to initialize. Through a lot of trial and error, and reasons I don't quite remember, we put a 2-min delay in rc.local before starting Tomcat, and the problem went away. I'm not a Linux nor a crypto guru. But we traced it to some crypto file that we assumed was not available until later in the Linux boot sequence. Anyway, the 2 minute delay made it go away, for over two years. Then all of a sudden in the last day or so, it's back with a vengeance. It fails with the same crypto error from 2 years ago in about 50% of the EC2 boot ups. I tried bumping the wait to 3 min, and no change. I need help. Our whole production environment is unstable now since every time an ASG brings a new instance online, I've got a 50-50 chance that tomcat is going to die (and the health check doesn't catch it, but that's a different issue). There are no errors in the Tomcat boot sequence logs. But the first time and every subsequent time I try to get a connection from the DataSource pool, I get the stack dump shown below. I figure it has to be a timing/race condition. But I have no clue what to do to fix it. I'm baffled that it worked for two years, and now fails every other time I start an instance. And every instance is running copies of the exact same Amazon Machine Image. The same EC2 will come up clean 50% of the time the next time it boots. Can somebody with Tomcat/Crypto/Linux knowledge unravel what's going on here? Thx java.lang.ExceptionInInitializerError at java.base/javax.crypto.Cipher.getInstance(Cipher.java:540) at java.base/sun.security.ssl.JsseJce.getCipher(JsseJce.java:190) at java.base/sun.security.ssl.SSLCipher.isTransformationAvailable(SSLCip her.jav a:509) at java.base/sun.security.ssl.SSLCipher.(SSLCipher.java:498) at java.base/sun.security.ssl.SSLCipher.(SSLCipher.java:81) at java.base/sun.security.ssl.CipherSuite.(CipherSuite.java:65) at java.base/sun.security.ssl.SSLContextImpl.getApplicableSupportedCiphe rSuit es(SSLContextImpl.java:348) at java.base/sun.security.ssl.SSLContextImpl$AbstractTLSContext. (SSLC ontextImpl.java:580) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:315) ... at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java: 948) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:448) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java: 198) at org.apache.tomcat.dbcp.dbcp2.DriverConnectionFactory.createConnection ( DriverConnectionFactory.java:52) at org.apache.tomcat.dbcp.dbcp2.PoolableConnectionFactory.makeObject(Po olableConnectionFactory.java:415) at org.apache.tomcat.dbcp.dbcp2.BasicDataSource.validateConnectionFactor y (BasicDataSource.java:111) at org.apache.tomcat.dbcp.dbcp2.BasicDataSource.createPoolableConnection Factory(BasicDataSource.java:649) at org.apache.tomcat.dbcp.dbcp2.BasicDataSource.createDataSource(BasicDa taSource.java:532) at org.apache.tomcat.dbcp.dbcp2.BasicDataSource.getConnection(BasicDataS ource.java:731) at jwm.db.DBData.getConnection(DBData.java:506) my call to get a db connection from connection pool ... Caused by: java.lang.SecurityException: Can not initialize cryptographic mechanism at java.base/javax.crypto.JceSecurity.(JceSecurity.java:120) ... 86 mo Caused by: java.lang.SecurityException: Can't read cryptographic policy directory: unlimited at java.base/javax.crypto.JceSecurity.setupJurisdictionPolicies(JceSecurity.java: 326) at java.base/javax.crypto.JceSecurity$1.run(JceSecurity.java:111) at java.base/javax.crypto.JceSecurity$1.run(JceSecurity.java:108) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.crypto.JceSecurity.(JceSecurity.java:107) .
AW: AW: Crypto Randomly Not Getting Initialized
Hello Jerry, > -Ursprüngliche Nachricht- > Von: Jerry Malcolm > Gesendet: Dienstag, 13. Juni 2023 17:35 > An: users@tomcat.apache.org > Betreff: Re: AW: Crypto Randomly Not Getting Initialized > > Hi Thomas, > > On 6/13/2023 2:08 AM, Thomas Hoffmann (Speed4Trade GmbH) wrote: > > Hello Jerry, > > > >> -Ursprüngliche Nachricht- > >> Von: Jerry Malcolm > >> Gesendet: Dienstag, 13. Juni 2023 08:50 > >> An: users@tomcat.apache.org > >> Betreff: Crypto Randomly Not Getting Initialized > >> > >> I am running Tomcat 9.0.56 in multiple AWS EC2 instances with Amazon > >> Linux2 in a production environment. A couple of years ago, we > >> started getting weird errors that the "Crypto Mechanism" failed to > >> initialize. Through a lot of trial and error, and reasons I don't > >> quite remember, we put a 2-min delay in rc.local before starting > >> Tomcat, and the problem went away. I'm not a Linux nor a crypto > >> guru. But we traced it to some crypto file that we assumed was not > >> available until later in the Linux boot sequence. Anyway, the 2 > >> minute delay made it go away, for over two years. Then all of a > >> sudden in the last day or so, it's back with a vengeance. It fails > >> with the same crypto error from 2 years ago in about 50% of the EC2 boot > ups. I tried bumping the wait to 3 min, and no change. > >> > >> I need help. Our whole production environment is unstable now since > >> every time an ASG brings a new instance online, I've got a 50-50 > >> chance that tomcat is going to die (and the health check doesn't > >> catch it, but that's a different issue). > >> > >> There are no errors in the Tomcat boot sequence logs. But the first > >> time and every subsequent time I try to get a connection from the > >> DataSource pool, I get the stack dump shown below. > >> > >> I figure it has to be a timing/race condition. But I have no clue > >> what to do to fix it. I'm baffled that it worked for two years, and > >> now fails every other time I start an instance. And every instance > >> is running copies of the exact same Amazon Machine Image. The same > >> EC2 will come up clean 50% of the time the next time it boots. > >> > >> Can somebody with Tomcat/Crypto/Linux knowledge unravel what's > going > >> on here? Thx > >> > >> java.lang.ExceptionInInitializerError > >> at java.base/javax.crypto.Cipher.getInstance(Cipher.java:540) > >> at java.base/sun.security.ssl.JsseJce.getCipher(JsseJce.java:190) > >> at > >> java.base/sun.security.ssl.SSLCipher.isTransformationAvailable(SSLCip > >> her.jav > >> a:509) > >> at > >> java.base/sun.security.ssl.SSLCipher.(SSLCipher.java:498) > >> at > >> java.base/sun.security.ssl.SSLCipher.(SSLCipher.java:81) > >> at > >> java.base/sun.security.ssl.CipherSuite.(CipherSuite.java:65) > >> at > >> java.base/sun.security.ssl.SSLContextImpl.getApplicableSupportedCiphe > >> rSuit > >> es(SSLContextImpl.java:348) > >> at > >> java.base/sun.security.ssl.SSLContextImpl$AbstractTLSContext. > >> (SSLC > >> ontextImpl.java:580) > >> at java.base/java.lang.Class.forName0(Native Method) > >> at java.base/java.lang.Class.forName(Class.java:315) > >> ... > >> > >> at > >> > com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java: > >> 948) > >> at > >> > com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818) > >> at > com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:448) > >> at > >> com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241) > >> at > >> > com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java: > >> 198) > >> at > >> > org.apache.tomcat.dbcp.dbcp2.DriverConnectionFactory.createConnection > >> ( > >> DriverConnectionFactory.java:52) > >> at > >> > org.apache.tomcat.dbcp.dbcp2.PoolableConnectionFactory.makeObject(Po > >> olableConnectionFactory.java:415) > >> at > >> > org.apache.tomcat.dbcp.dbcp2.BasicDataSource.validateConnectionFactor > >> y > >> (BasicDataSourc
Re: AW: Crypto Randomly Not Getting Initialized
Hi Thomas, On 6/13/2023 2:08 AM, Thomas Hoffmann (Speed4Trade GmbH) wrote: Hello Jerry, -Ursprüngliche Nachricht- Von: Jerry Malcolm Gesendet: Dienstag, 13. Juni 2023 08:50 An: users@tomcat.apache.org Betreff: Crypto Randomly Not Getting Initialized I am running Tomcat 9.0.56 in multiple AWS EC2 instances with Amazon Linux2 in a production environment. A couple of years ago, we started getting weird errors that the "Crypto Mechanism" failed to initialize. Through a lot of trial and error, and reasons I don't quite remember, we put a 2-min delay in rc.local before starting Tomcat, and the problem went away. I'm not a Linux nor a crypto guru. But we traced it to some crypto file that we assumed was not available until later in the Linux boot sequence. Anyway, the 2 minute delay made it go away, for over two years. Then all of a sudden in the last day or so, it's back with a vengeance. It fails with the same crypto error from 2 years ago in about 50% of the EC2 boot ups. I tried bumping the wait to 3 min, and no change. I need help. Our whole production environment is unstable now since every time an ASG brings a new instance online, I've got a 50-50 chance that tomcat is going to die (and the health check doesn't catch it, but that's a different issue). There are no errors in the Tomcat boot sequence logs. But the first time and every subsequent time I try to get a connection from the DataSource pool, I get the stack dump shown below. I figure it has to be a timing/race condition. But I have no clue what to do to fix it. I'm baffled that it worked for two years, and now fails every other time I start an instance. And every instance is running copies of the exact same Amazon Machine Image. The same EC2 will come up clean 50% of the time the next time it boots. Can somebody with Tomcat/Crypto/Linux knowledge unravel what's going on here? Thx java.lang.ExceptionInInitializerError at java.base/javax.crypto.Cipher.getInstance(Cipher.java:540) at java.base/sun.security.ssl.JsseJce.getCipher(JsseJce.java:190) at java.base/sun.security.ssl.SSLCipher.isTransformationAvailable(SSLCipher.jav a:509) at java.base/sun.security.ssl.SSLCipher.(SSLCipher.java:498) at java.base/sun.security.ssl.SSLCipher.(SSLCipher.java:81) at java.base/sun.security.ssl.CipherSuite.(CipherSuite.java:65) at java.base/sun.security.ssl.SSLContextImpl.getApplicableSupportedCipherSuit es(SSLContextImpl.java:348) at java.base/sun.security.ssl.SSLContextImpl$AbstractTLSContext.(SSLC ontextImpl.java:580) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:315) ... at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java: 948) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818) at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:448) at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241) at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java: 198) at org.apache.tomcat.dbcp.dbcp2.DriverConnectionFactory.createConnection( DriverConnectionFactory.java:52) at org.apache.tomcat.dbcp.dbcp2.PoolableConnectionFactory.makeObject(Po olableConnectionFactory.java:415) at org.apache.tomcat.dbcp.dbcp2.BasicDataSource.validateConnectionFactory (BasicDataSource.java:111) at org.apache.tomcat.dbcp.dbcp2.BasicDataSource.createPoolableConnection Factory(BasicDataSource.java:649) at org.apache.tomcat.dbcp.dbcp2.BasicDataSource.createDataSource(BasicDa taSource.java:532) at org.apache.tomcat.dbcp.dbcp2.BasicDataSource.getConnection(BasicDataS ource.java:731) at jwm.db.DBData.getConnection(DBData.java:506) my call to get a db connection from connection pool ... Caused by: java.lang.SecurityException: Can not initialize cryptographic mechanism at java.base/javax.crypto.JceSecurity.(JceSecurity.java:120) ... 86 mo Caused by: java.lang.SecurityException: Can't read cryptographic policy directory: unlimited at java.base/javax.crypto.JceSecurity.setupJurisdictionPolicies(JceSecurity.java: 326) at java.base/javax.crypto.JceSecurity$1.run(JceSecurity.java:111) at java.base/javax.crypto.JceSecurity$1.run(JceSecurity.java:108) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.crypto.JceSecurity.(JceSecurity.java:107) ... 86 more - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Could it be this issue? https://github.com/docker-library/openjdk/issues/101 Maybe you can add information about the used jdk and whether you are using
AW: Crypto Randomly Not Getting Initialized
Hello Jerry, > -Ursprüngliche Nachricht- > Von: Jerry Malcolm > Gesendet: Dienstag, 13. Juni 2023 08:50 > An: users@tomcat.apache.org > Betreff: Crypto Randomly Not Getting Initialized > > I am running Tomcat 9.0.56 in multiple AWS EC2 instances with Amazon > Linux2 in a production environment. A couple of years ago, we started > getting weird errors that the "Crypto Mechanism" failed to initialize. Through > a lot of trial and error, and reasons I don't quite remember, we put a 2-min > delay in rc.local before starting Tomcat, and the problem went away. I'm > not a Linux nor a crypto guru. But we traced it to some crypto file that we > assumed was not available until later in the Linux boot sequence. Anyway, > the 2 minute delay made it go away, for over two years. Then all of a sudden > in the last day or so, it's back with a vengeance. It fails with the same > crypto > error from 2 years ago in about 50% of the EC2 boot ups. I tried bumping the > wait to 3 min, and no change. > > I need help. Our whole production environment is unstable now since every > time an ASG brings a new instance online, I've got a 50-50 chance that > tomcat is going to die (and the health check doesn't catch it, but that's a > different issue). > > There are no errors in the Tomcat boot sequence logs. But the first time and > every subsequent time I try to get a connection from the DataSource pool, I > get the stack dump shown below. > > I figure it has to be a timing/race condition. But I have no clue what to do > to > fix it. I'm baffled that it worked for two years, and now fails every other > time I start an instance. And every instance is running copies of the exact > same Amazon Machine Image. The same EC2 will come up clean 50% of the > time the next time it boots. > > Can somebody with Tomcat/Crypto/Linux knowledge unravel what's going > on here? Thx > > java.lang.ExceptionInInitializerError > at java.base/javax.crypto.Cipher.getInstance(Cipher.java:540) > at java.base/sun.security.ssl.JsseJce.getCipher(JsseJce.java:190) > at > java.base/sun.security.ssl.SSLCipher.isTransformationAvailable(SSLCipher.jav > a:509) > at java.base/sun.security.ssl.SSLCipher.(SSLCipher.java:498) > at java.base/sun.security.ssl.SSLCipher.(SSLCipher.java:81) > at > java.base/sun.security.ssl.CipherSuite.(CipherSuite.java:65) > at > java.base/sun.security.ssl.SSLContextImpl.getApplicableSupportedCipherSuit > es(SSLContextImpl.java:348) > at > java.base/sun.security.ssl.SSLContextImpl$AbstractTLSContext.(SSLC > ontextImpl.java:580) > at java.base/java.lang.Class.forName0(Native Method) > at java.base/java.lang.Class.forName(Class.java:315) > ... > > at > com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java: > 948) > at > com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:818) > at com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:448) > at > com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:241) > at > com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java: > 198) > at > org.apache.tomcat.dbcp.dbcp2.DriverConnectionFactory.createConnection( > DriverConnectionFactory.java:52) > at > org.apache.tomcat.dbcp.dbcp2.PoolableConnectionFactory.makeObject(Po > olableConnectionFactory.java:415) > at > org.apache.tomcat.dbcp.dbcp2.BasicDataSource.validateConnectionFactory > (BasicDataSource.java:111) > at > org.apache.tomcat.dbcp.dbcp2.BasicDataSource.createPoolableConnection > Factory(BasicDataSource.java:649) > at > org.apache.tomcat.dbcp.dbcp2.BasicDataSource.createDataSource(BasicDa > taSource.java:532) > at > org.apache.tomcat.dbcp.dbcp2.BasicDataSource.getConnection(BasicDataS > ource.java:731) > at jwm.db.DBData.getConnection(DBData.java:506) my call to get > a db connection from connection pool > > ... > > Caused by: java.lang.SecurityException: Can not initialize cryptographic > mechanism > at > java.base/javax.crypto.JceSecurity.(JceSecurity.java:120) ... 86 mo > Caused by: java.lang.SecurityException: Can't read cryptographic policy > directory: unlimited > at > java.base/javax.crypto.JceSecurity.setupJurisdictionPolicies(JceSecurity.java: > 326) > at java.base/javax.crypto.JceSecurity$1.run(JceSecurity.java:111) > at java.base/javax.crypto.JceSecurity$1.run(JceSecurity.java:108) > at java.base/java.security.AccessController.doPrivileged(Native > Method) > at > java.base/javax.crypto.JceSecurity.(JceSecurity.java:107) > ... 86 more > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Could it be this issue?
