On Mon, May 20, 2019, 5:46 PM Louis Zipes <louis.zi...@kdrp.com wrote: > Can you refresh me on how to capture the stack dumps you suggested? > It's been a while.... > > >In Task Manager find the Tomcat process, right click on it and there is > an option to create a dump file. >
Well, that will create Windows memory dump. > > >Also, if you have access to jconsole or other JMX tool you can connect > to it and see what is going on. > Agreed. Alternatively you can use jstack to generate thread dumps. Exact command will be like: jstack -l PID_of_JVM I'll suggest to take thread dumps 1 sec apart. But still I am surprised to see that it happens only during entire server reboot and from your log, subsequent TC restart is not taking long time. Is that a right statement or I am entirely misreading it? If that is true, you can try 'atomatic delayed start' option. > > -----Original Message----- > From: André Warnier (tomcat) [mailto:a...@ice-sa.com] > Sent: Monday, May 20, 2019 4:44 AM > To: users@tomcat.apache.org > Subject: Re: Massive Startup Time after Server Reboot > > - - - external message, proceed with caution - - - > > > On 20.05.2019 00:09, Jerry Malcolm wrote: > > Just an FYI.... this server has been in production a little over a > year. This TC load > > problem only started 2-3 months ago. So 'something' changed to cause > this problem to > > manifest itself. It's been too long to try to correlate a specific > server change to the > > start of this problem. But I just wanted to point out that this problem > has NOT been > > around since day 1 on this server. > > Just in case, since this is a Windows server .. > 2-3 months ago, an update of a virus scanner ? > (of the intrusive kind which pre-checks each file that wants to be opened) > Checking may be easy or not, depending on your access : disable the virus > scanner just the > time to start tomcat and check. > > > > > Jerry > > > > On 5/19/2019 5:03 PM, Jerry Malcolm wrote: > >> Rainer, > >> > >> No change with the urandom parm. I am attaching a portion of the > Catalina log. The > >> first half shows between 8 and 15 seconds to deploy each app for a > single virtual host > >> (there are no war files, the app is already exploded in the appbase > dir). I have quite > >> a few virtual hosts with several apps each. That initial server start > took 21+ minutes. > >> I then restarted the TC service and got the ~.5 sec start per webapp > shown in the 2nd > >> half of the log below. > >> > >> Can you refresh me on how to capture the stack dumps you suggested? > It's been a while.... > >> > >> Thx. > >> > >> Jerry > >> > >> First start of Tomcat after server reboot > >> --------------------------------------------------------------------- > >> [C:\domains\<myDomain>.com\webapps\JSPWiki.war] has finished in > [8,579] ms > >> tory [C:\domains\<myDomain>.com\webapps\cis] > >> irectory [C:\domains\<myDomain>.com\webapps\cis] has finished in > [11,486] ms > >> tory [C:\domains\<myDomain>.com\webapps\gallery] > >> irectory [C:\domains\<myDomain>.com\webapps\gallery] has finished in > [9,204] ms > >> tory [C:\domains\<myDomain>.com\webapps\gl] > >> irectory [C:\domains\<myDomain>.com\webapps\gl] has finished in [8,469] > ms > >> tory [C:\domains\<myDomain>.com\webapps\idmanager] > >> irectory [C:\domains\<myDomain>.com\webapps\idmanager] has finished in > [8,689] ms > >> tory [C:\domains\<myDomain>.com\webapps\itemtrack] > >> irectory [C:\domains\<myDomain>.com\webapps\itemtrack] has finished in > [6,907] ms > >> tory [C:\domains\<myDomain>.com\webapps\malcolment] > >> irectory [C:\domains\<myDomain>.com\webapps\malcolment] has finished in > [8,469] ms > >> tory [C:\domains\<myDomain>.com\webapps\notebook] > >> irectory [C:\domains\<myDomain>.com\webapps\notebook] has finished in > [10,189] ms > >> tory [C:\domains\<myDomain>.com\webapps\order] > >> irectory [C:\domains\<myDomain>.com\webapps\order] has finished in > [8,501] ms > >> tory [C:\domains\<myDomain>.com\webapps\payment] > >> irectory [C:\domains\<myDomain>.com\webapps\payment] has finished in > [14,209] ms > >> tory [C:\domains\<myDomain>.com\webapps\projectmanager] > >> irectory [C:\domains\<myDomain>.com\webapps\projectmanager] has > finished in [9,018] ms > >> [C:\Tomcat 9.0\conf\Catalina\myridetx.net\manager.xml] > >> ptor [C:\Tomcat 9.0\conf\Catalina\myridetx.net\manager.xml] has > finished in [62] ms > >> > >> Restart of Tomcat > --------------------------------------------------------------------- > >> tory [C:\domains\<myDomain>.com\webapps\cis] > >> irectory [C:\domains\<myDomain>.com\webapps\cis] has finished in [594] > ms > >> tory [C:\domains\<myDomain>.com\webapps\gallery] > >> irectory [C:\domains\<myDomain>.com\webapps\gallery] has finished in > [547] ms > >> tory [C:\domains\<myDomain>.com\webapps\gl] > >> irectory [C:\domains\<myDomain>.com\webapps\gl] has finished in [562] ms > >> tory [C:\domains\<myDomain>.com\webapps\idmanager] > >> irectory [C:\domains\<myDomain>.com\webapps\idmanager] has finished in > [578] ms > >> tory [C:\domains\<myDomain>.com\webapps\itemtrack] > >> irectory [C:\domains\<myDomain>.com\webapps\itemtrack] has finished in > [547] ms > >> tory [C:\domains\<myDomain>.com\webapps\malcolment] > >> irectory [C:\domains\<myDomain>.com\webapps\malcolment] has finished in > [579] ms > >> tory [C:\domains\<myDomain>.com\webapps\notebook] > >> irectory [C:\domains\<myDomain>.com\webapps\notebook] has finished in > [547] ms > >> tory [C:\domains\<myDomain>.com\webapps\order] > >> irectory [C:\domains\<myDomain>.com\webapps\order] has finished in > [531] ms > >> tory [C:\domains\<myDomain>.com\webapps\payment] > >> irectory [C:\domains\<myDomain>.com\webapps\payment] has finished in > [562] ms > >> tory [C:\domains\<myDomain>.com\webapps\projectmanager] > >> irectory [C:\domains\<myDomain>.com\webapps\projectmanager] has > finished in [531] ms > >> > >> On 5/18/2019 3:18 PM, Rainer Jung wrote: > >>> Most likely it hangs waiting for enough entropy for random number > generator seeding. > >>> > >>> Try whether the problem goes away if you add > >>> > >>> -Djava.security.egd=file:/dev/urandom > >>> > >>> to you process flags. If you are using older Java than Java 8 (not > possible for Tomcat > >>> 9 but just in case you also have older software stacks running), then > it would be > >>> > >>> -Djava.security.egd=file:/dev/./urandom > >>> > >>> The cryptic /./ is not a typo, you would need it for Java before Java > 8. > >>> > >>> If this doesn't help, then you should try capturing a few stack dumps > (thread dumps) > >>> during the long startup time. People here can help interpret them. > >>> > >>> Regards, > >>> > >>> Rainer > >>> > >>> Am 18.05.2019 um 21:55 schrieb Jerry Malcolm: > >>>> This is a weird one. It started a few months ago. I have TC 9 > running on Windows > >>>> Server 16. After I reboot the entire server, Tomcat takes forever on > startup. It > >>>> normally starts in about 30 seconds. But after a server reboot it > takes up to 15 > >>>> minutes... chugging along at a snail's pace starting up all of the > apps on all of the > >>>> virtual hosts. It always finally gets there with everything > successfully running. > >>>> Other servers on the same box (Apache, JAMES, ISC BIND, MySQL) don't > have any problem > >>>> starting up. CPU, Disk, Memory, etc. usages are barely showing on the > performance > >>>> graphs. There's nothing in the Catalina log or system.err other than > showing a couple > >>>> of minute gap in the time stamp between each app as it starts up. If > I need to reboot > >>>> again later, it boots again in about 30 sec as expected. > >>>> > >>>> First question... any ideas off the top of your head that might be > causing this? > >>>> > >>>> Second question... is there any other logging I can turn on that > might explain what TC > >>>> is blocking on? > >>>> > >>>> Thanks. > >>>> > >>>> Jerry > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >>> For additional commands, e-mail: users-h...@tomcat.apache.org > >>> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > >> For additional commands, e-mail: users-h...@tomcat.apache.org > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > --------------------------------------- > CONFIDENTIALITY NOTICE: This message is for intended addressee(s) only and > may contain information that is confidential, proprietary or exempt from > disclosure. If you are not the intended recipient, please contact the > sender immediately. Unauthorized use or distribution is prohibited and may > be unlawful. >
