Hi, > Jerry, > > On 6/13/23 11:42, Jerry Malcolm wrote: >> Simon, >> >> On 6/13/2023 2:20 AM, Simon Matter wrote: >>> Hi, >>> >>>> I am running Tomcat 9.0.56 in multiple AWS EC2 instances with Amazon >>>> Linux2 in a production environment. A couple of years ago, we started >>>> getting weird errors that the "Crypto Mechanism" failed to initialize. >>>> Through a lot of trial and error, and reasons I don't quite remember, >>>> we >>>> put a 2-min delay in rc.local before starting Tomcat, and the problem >>>> went away. I'm not a Linux nor a crypto guru. But we traced it to >>>> some >>>> crypto file that we assumed was not available until later in the Linux >>>> boot sequence. Anyway, the 2 minute delay made it go away, for over >>>> two >>>> years. Then all of a sudden in the last day or so, it's back with a >>>> vengeance. It fails with the same crypto error from 2 years ago in >>>> about 50% of the EC2 boot ups. I tried bumping the wait to 3 min, and >>>> no change. >>>> >>>> I need help. Our whole production environment is unstable now since >>>> every time an ASG brings a new instance online, I've got a 50-50 >>>> chance >>>> that tomcat is going to die (and the health check doesn't catch it, >>>> but >>>> that's a different issue). >>>> >>>> There are no errors in the Tomcat boot sequence logs. But the first >>>> time and every subsequent time I try to get a connection from the >>>> DataSource pool, I get the stack dump shown below. >>>> >>>> I figure it has to be a timing/race condition. But I have no clue >>>> what >>>> to do to fix it. I'm baffled that it worked for two years, and now >>>> fails every other time I start an instance. And every instance is >>>> running copies of the exact same Amazon Machine Image. The same EC2 >>>> will come up clean 50% of the time the next time it boots. >>> Could it be that your hosts are running out of entropy on boot? >>> >>> Maybe there were RNG related changes in the kernel or rng-tools? >>> >>> Maybe monitor available entropy in >>> /proc/sys/kernel/random/entropy_avail, >>> it should not go below 100 or so. >>> >>> Regards, >>> Simon >>> >> I haven't done any Linux or other system updates in several weeks. I'll >> look into the entropy possibility. Would running out of entropy cause >> an exception stating that a crypto directory doesn't exist? I don't >> know much about Java entropy. Any ideas what would cause entropy to be >> good on one boot and bad on the next boot? Thx. > > This isn't about entropy. Focus on the actual error message that the > system can't find the unlimited policy file for some reason. > > -chris
Sorry, I didn't read the errors too carefully :) I don't know Amazon Linux2 but one question, does it have the directory /etc/crypto-policies? Apart from this, does systemd somehow fiddling with crypto stuff while booting? Since systemd parallelizes the whole boot process many things can (and sometimes do) break and it's difficult to find it out. Regards, Simon --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
