Simon,
On 6/13/2023 2:20 AM, Simon Matter wrote:
Hi,
I am running Tomcat 9.0.56 in multiple AWS EC2 instances with Amazon
Linux2 in a production environment. A couple of years ago, we started
getting weird errors that the "Crypto Mechanism" failed to initialize.
Through a lot of trial and error, and reasons I don't quite remember, we
put a 2-min delay in rc.local before starting Tomcat, and the problem
went away. I'm not a Linux nor a crypto guru. But we traced it to some
crypto file that we assumed was not available until later in the Linux
boot sequence. Anyway, the 2 minute delay made it go away, for over two
years. Then all of a sudden in the last day or so, it's back with a
vengeance. It fails with the same crypto error from 2 years ago in
about 50% of the EC2 boot ups. I tried bumping the wait to 3 min, and
no change.
I need help. Our whole production environment is unstable now since
every time an ASG brings a new instance online, I've got a 50-50 chance
that tomcat is going to die (and the health check doesn't catch it, but
that's a different issue).
There are no errors in the Tomcat boot sequence logs. But the first
time and every subsequent time I try to get a connection from the
DataSource pool, I get the stack dump shown below.
I figure it has to be a timing/race condition. But I have no clue what
to do to fix it. I'm baffled that it worked for two years, and now
fails every other time I start an instance. And every instance is
running copies of the exact same Amazon Machine Image. The same EC2
will come up clean 50% of the time the next time it boots.
Could it be that your hosts are running out of entropy on boot?
Maybe there were RNG related changes in the kernel or rng-tools?
Maybe monitor available entropy in /proc/sys/kernel/random/entropy_avail,
it should not go below 100 or so.
Regards,
Simon
I haven't done any Linux or other system updates in several weeks. I'll
look into the entropy possibility. Would running out of entropy cause
an exception stating that a crypto directory doesn't exist? I don't
know much about Java entropy. Any ideas what would cause entropy to be
good on one boot and bad on the next boot? Thx.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org