On Thu, Jul 23, 2020 at 3:10 PM Kushagra Bindal <bindal.kusha...@gmail.com>
wrote:
> Hi Martin,
>
> Due to our environment I was not able to use pastebin service. I have
> taken different Thread dump during shutdown and attaching the same with
> this email.
>
> Please review the same and let me know, what is the probable root cause of
> the problem and what could be the fix of the same.
>
You have many AMQP (RabbitMQ) listener threads which are not daemons.
It seems your application does not notify Spring Framework that it needs to
destroy its application context or the beans with
type org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer
are not notified to stop for some other
reason.
>
> On Thu, Jul 23, 2020 at 3:22 PM Martin Grigorov <mgrigo...@apache.org>
> wrote:
>
>> Hi,
>>
>> On Thu, Jul 23, 2020 at 6:35 AM Kushagra Bindal <
>> bindal.kusha...@gmail.com>
>> wrote:
>>
>> > Hi Martin,
>> >
>> > These are the only two behaviors right now which I am getting on a
>> > regular basis.
>> > 1. During startup of the application and then shutdown
>> > 2. Running application and then shutdown.
>> >
>> > Please let me know if anything specific is further needed from my end
>> which
>> > I can provide to have a better clarity.
>> >
>> > I have shared the server.xml and command which we are using in stopping
>> the
>> > tomcat.
>> >
>> >
>> > On Thu, Jul 23, 2020 at 2:49 AM Martin Grigorov <mgrigo...@apache.org>
>> > wrote:
>> >
>> > > On Wed, Jul 22, 2020, 15:55 Kushagra Bindal <
>> bindal.kusha...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi Christopher,
>> > > >
>> > > > Did you get a chance to look into this?
>> > > >
>> > > > Please help us in resolving this issue.
>> > > >
>> > > > On Sat, Jul 18, 2020 at 11:26 AM Kushagra Bindal <
>> > > > bindal.kusha...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi Chris,
>> > > > >
>> > > > > Additionally when trying to stop running application, we are
>> getting
>> > > > below
>> > > > > error.
>> > > > >
>> > > > > Sat Jul 18 05:49:40 UTC 2020
>> > > > > **************************
>> > > > > * Stopping the Web Server
>> > > > > **************************
>> > > > > Sat Jul 18 05:49:40 UTC 2020
>> > > > > ./wfc: line 28: /usr/local/nginx/nginx: No such file or directory
>> > > > > ./wfc: line 233: /usr/local/nginx/nginx: No such file or directory
>> > > > >
>> > > > > Sat Jul 18 05:49:40 UTC 2020
>> > > > > * Nginx has been stopped
>> > > > > **************************
>> > > > > * Shutdown the wfc Server gracefully
>> > > > > **************************
>> > > > > #
>> > *********************************************************************
>> > > > > # Tomcat shutdown with wait time: 30
>> > > > > #
>> > *********************************************************************
>> > > > > Using CATALINA_BASE: /usr/local/xyz/tomcat
>> > > > > Using CATALINA_HOME: /usr/local/xyz/tomcat
>> > > > > Using CATALINA_TMPDIR: /usr/local/xyz/tomcat/temp
>> > > > > Using JRE_HOME: /usr/local/xyz/jdk
>> > > > > Using CLASSPATH:
>> > > > >
>> > > >
>> > >
>> >
>> /usr/local/xyz/tomcat/bin/bootstrap.jar:/usr/local/xyz/tomcat/bin/tomcat-juli.jar
>> > > > > Using CATALINA_PID: /usr/local/xyz/tomcat/tomcat.pid
>> > > > > Tomcat did not stop in time.
>> > > > > To aid diagnostics a thread dump has been written to standard out.
>> > > >
>> > >
>> > > Anything interesting in this thread dump?
>> >
>>
>> Tomcat does not stop properly in time and a thread dump is written to the
>> stdout.
>> Please share the thread dump. Use some pastebin service!
>>
>>
>> > >
>> > >
>> > > > Killing Tomcat with the PID: 4280
>> > > > > The Tomcat process has been killed.
>> > > > > #
>> > *********************************************************************
>> > > > > # Tomcat shutdown result: 0
>> > > > > #
>> > *********************************************************************
>> > > > >
>> > > > >
>> > > > >
>> > > > > Whereas below error that I mentioned earlier is coming when
>> > application
>> > > > is
>> > > > > in INPROGRESS and not yet come into running state and when we stop
>> > the
>> > > > same
>> > > > > it is throwing below error.
>> > > > >
>> > > > > On Sat, Jul 18, 2020 at 10:17 AM Kushagra Bindal <
>> > > > > bindal.kusha...@gmail.com> wrote:
>> > > > >
>> > > > >> Hi Chris,
>> > > > >>
>> > > > >> To stop tomcat we are using the below command.
>> > > > >>
>> > > > >> bin/shutdown.sh -$sleeptime -force
>> > > > >>
>> > > > >> Where in our case sleeptime is set to 30.
>> > > > >>
>> > > > >> Please find the attached server.xml which we are using in our
>> > > > application.
>> > > > >>
>> > > > >> Also I have copy-paste the complete content below in this email
>> as
>> > > well.
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>> ----------------------------------------------------------------------------------------
>> > > > >>
>> > > > >> <?xml version='1.0' encoding='utf-8'?>
>> > > > >> <Server port="8005" shutdown="SHUTDOWN">
>> > > > >> <Listener
>> > > > className="org.apache.catalina.startup.VersionLoggerListener"
>> > > > >> />
>> > > > >> <Listener
>> > className="org.apache.catalina.core.AprLifecycleListener"
>> > > > >> SSLEngine="on" />
>> > > > >> <Listener
>> > > > >>
>> className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
>> > > />
>> > > > >> <Listener
>> > > > >>
>> > >
>> className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener"
>> > > > />
>> > > > >> <Listener
>> > > > >>
>> > className="org.apache.catalina.core.ThreadLocalLeakPreventionListener"
>> > > > />
>> > > > >> <GlobalNamingResources>
>> > > > >> <Resource name="UserDatabase" auth="Container"
>> > > > >> type="org.apache.catalina.UserDatabase"
>> > > > >> description="User database that can be updated and
>> > > saved"
>> > > > >>
>> > > > >> factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
>> > > > >> pathname="conf/tomcat-users.xml" />
>> > > > >> <Resource name="jdbc/edbDataSource" auth="Container"
>> > > > >> type="javax.sql.DataSource"
>> > > > >> username="db_username_stub"
>> > > > >> password="db_password_stub"
>> > > > >>
>> > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
>> > > > >> url="READ_WRITE_URL_STUB"
>> > > > >> driverClassName="com.edb.Driver"
>> > > > >> initialSize="5"
>> > > > >> maxWait="30000"
>> > > > >> maxActive="50"
>> > > > >> maxIdle="20"
>> > > > >> minIdle="5"
>> > > > >> maxAge="7200000"
>> > > > >> validationQuery="SELECT 1; COMMIT;"
>> > > > >> initSQL="ALTER SESSION SET statement_timeout=3600000; ALTER
>> SESSION
>> > > SET
>> > > > >> idle_in_transaction_session_timeout=3660000; COMMIT;"
>> > > > >> poolPreparedStatements="true"
>> > > > >> testWhileIdle="false"
>> > > > >> testOnBorrow="true"
>> > > > >> testOnReturn="false"
>> > > > >> validationInterval="120000"
>> > > > >> timeBetweenEvictionRunsMillis="15000"
>> > > > >> removeAbandonedTimeout="300"
>> > > > >> removeAbandoned="false"
>> > > > >> logAbandoned="false"
>> > > > >> minEvictableIdleTimeMillis="120000"
>> > > > >> jmxEnabled="true"
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>> jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
>> > > > >>
>> > > > >> org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer;
>> > > > >>
>> > > > >>
>> org.apache.tomcat.jdbc.pool.interceptor.StatementCache(max=4000)" />
>> > > > >> <Resource name="jdbc/edbDataSourceReadOnly" auth="Container"
>> > > > >> type="javax.sql.DataSource"
>> > > > >> username="db_username_stub"
>> > > > >> password="db_password_stub"
>> > > > >>
>> > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
>> > > > >> url="READ_ONLY_URL_STUB"
>> > > > >> driverClassName="com.edb.Driver"
>> > > > >> initialSize="5"
>> > > > >> maxWait="30000"
>> > > > >> maxActive="50"
>> > > > >> maxIdle="20"
>> > > > >> minIdle="5"
>> > > > >> defaultReadOnly="true"
>> > > > >> validationQuery="SELECT 1; COMMIT;"
>> > > > >> initSQL="ALTER SESSION SET
>> statement_timeout=3600000;
>> > > > ALTER
>> > > > >> SESSION SET idle_in_transaction_session_timeout=3660000; COMMIT;"
>> > > > >> poolPreparedStatements="true"
>> > > > >> testWhileIdle="false"
>> > > > >> testOnBorrow="true"
>> > > > >> testOnReturn="false"
>> > > > >> validationInterval="120000"
>> > > > >> timeBetweenEvictionRunsMillis="15000"
>> > > > >> removeAbandonedTimeout="300"
>> > > > >> removeAbandoned="false"
>> > > > >> logAbandoned="false"
>> > > > >> minEvictableIdleTimeMillis="120000"
>> > > > >> jmxEnabled="true"
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>> jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
>> > > > >>
>> > > > >> org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer;
>> > > > >>
>> > > > >>
>> org.apache.tomcat.jdbc.pool.interceptor.StatementCache(max=4000)" />
>> > > > >> </GlobalNamingResources>
>> > > > >> <Service name="Catalina">
>> > > > >> <Connector port="8080" protocol="HTTP/1.1"
>> > > > >> server=""
>> > > > >> connectionTimeout="20000"
>> > > > >> redirectPort="8443" maxHttpHeaderSize="16384"/>
>> > > > >> <Connector port="8085" protocol="HTTP/1.1"
>> > > > >> server=""
>> > > > >> connectionTimeout="20000"
>> > > > >> redirectPort="8443" maxHttpHeaderSize="16384"/>
>> > > > >>
>> > > > >> <Engine name="Catalina" defaultHost="localhost">
>> > > > >> <Realm className="org.apache.catalina.realm.LockOutRealm">
>> > > > >> <Realm
>> > className="org.apache.catalina.realm.UserDatabaseRealm"
>> > > > >> resourceName="UserDatabase"/>
>> > > > >> </Realm>
>> > > > >> <Host name="localhost" appBase="webapps"
>> > > > >> unpackWARs="true" autoDeploy="true">
>> > > > >> <Valve
>> className="org.apache.catalina.valves.AccessLogValve"
>> > > > >> directory="${log.dir}/tomcat"
>> > > > >> prefix="localhost_access_log" suffix=".txt"
>> > > > >> pattern="%h %l %u %t "%r" %s %b" />
>> > > > >> </Host>
>> > > > >> </Engine>
>> > > > >> </Service>
>> > > > >> </Server>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>> -----------------------------------------------------------------------------------------
>> > > > >>
>> > > > >> Please suggest what needs to be done to resolve this issue.
>> > > > >>
>> > > > >>
>> > > > >> On Sat, Jul 18, 2020 at 12:47 AM Christopher Schultz <
>> > > > >> ch...@christopherschultz.net> wrote:
>> > > > >>
>> > > > >>> -----BEGIN PGP SIGNED MESSAGE-----
>> > > > >>> Hash: SHA256
>> > > > >>>
>> > > > >>> Kushagra,
>> > > > >>>
>> > > > >>> On 7/17/20 11:47, Kushagra Bindal wrote:
>> > > > >>> > Similar issues we are also facing in our environment. What
>> could
>> > be
>> > > > >>> > the problem?
>> > > > >>> >
>> > > > >>> > On Fri, Jul 17, 2020 at 6:04 PM om tiwari <
>> omt121...@gmail.com>
>> > > > >>> > wrote:
>> > > > >>> >
>> > > > >>> >> Hi,
>> > > > >>> >>
>> > > > >>> >> We have upgraded our tomcat version from 8.5.24 to 8.5.53 in
>> > > > >>> >> application. After upgrading we are facing issue while
>> stopping
>> > > > >>> >> tomcat. Below is the stack-trace :
>> > > > >>> >>
>> > > > >>> >> Using CATALINA_BASE: /usr/local/kronos/tomcat Using
>> > > > >>> >> CATALINA_HOME: /usr/local/kronos/tomcat Using
>> CATALINA_TMPDIR:
>> > > > >>> >> /usr/local/kronos/tomcat/temp Using JRE_HOME:
>> > > > >>> >> /usr/local/kronos/jdk Using CLASSPATH:
>> > > > >>> >>
>> > > > >>> >>
>> > > >
>> /usr/local/kronos/tomcat/bin/bootstrap.jar:/usr/local/kronos/tomcat/b
>> > > > >>> in/tomcat-juli.jar
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> Using CATALINA_PID: /usr/local/kronos/tomcat/tomcat.pid
>> > > > >>> >> Jul 17, 2020 11:44:44 AM org.apache.catalina.startup.Catalina
>> > > > >>> >> stopServer SEVERE: Could not contact [localhost:8005]. Tomcat
>> > may
>> > > > >>> >> not be running. Jul 17, 2020 11:44:44 AM
>> > > > >>> >> org.apache.catalina.startup.Catalina stopServer SEVERE:
>> > > > >>> >> Catalina.stop: java.net.ConnectException: Connection refused
>> > > > >>> >> (Connection refused) at
>> > > > >>> >> java.net.PlainSocketImpl.socketConnect(Native Method) at
>> > > > >>> >> java.net
>> > > > >>> >>
>> > > .AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> at java.net
>> > > > >>> >>
>> > > >
>> .AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.jav
>> > > > >>> a:206)
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> at java.net
>> > > > >>> >>
>> > .AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>> > > > >>> >> at java.net.Socket.connect(Socket.java:589) at
>> > > > >>> >> java.net.Socket.connect(Socket.java:538) at
>> > > > >>> >> java.net.Socket.<init>(Socket.java:434) at
>> > > > >>> >> java.net.Socket.<init>(Socket.java:211) at
>> > > > >>> >>
>> > org.apache.catalina.startup.Catalina.stopServer(Catalina.java:504)
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > > > >>> >> at
>> > > > >>> >>
>> > > >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
>> > > > >>> java:62)
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> at
>> > > > >>> >>
>> > > >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
>> > > > >>> sorImpl.java:43)
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> at java.lang.reflect.Method.invoke(Method.java:498)
>> > > > >>> >> at
>> > > > >>> >>
>> > > org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:389)
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> at
>> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:479)
>> > > > >>> >>
>> > > > >>> >> Tomcat did not stop in time. To aid diagnostics a thread dump
>> > has
>> > > > >>> >> been written to standard out. Killing Tomcat with the PID:
>> 21210
>> > > > >>> >> The Tomcat process has been killed.
>> > > > >>> >>
>> > > > >>> >>
>> > > >
>> ---------------------------------------------------------------------
>> > > > >>> - ---------------------------------
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> >>
>> > > > >>> At last the tomcat is stopped but we are not able to understand
>> > this
>> > > > >>> >> connection refused stack trace in this stop process.
>> > > > >>> >>
>> > > > >>> >> It also takes time in stopping the tomcat server now since
>> > > > >>> >> upgrade `10-15 sec.
>> > > > >>> >>
>> > > > >>> >> Can anyone help us in resolving this issue?
>> > > > >>>
>> > > > >>> Can you post your <Server> element from conf/server.xml?
>> > > > >>>
>> > > > >>> How do you stop Tomcat -- looks like using "catalina.sh stop" or
>> > > > similar
>> > > > >>> ?
>> > > > >>>
>> > > > >>> - -chris
>> > > > >>> -----BEGIN PGP SIGNATURE-----
>> > > > >>> Comment: Using GnuPG with Thunderbird -
>> https://www.enigmail.net/
>> > > > >>>
>> > > > >>> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl8R9N4ACgkQHPApP6U8
>> > > > >>> pFga8RAAws1xIkDJJQptsdTu/2T/jssbnnDVewrI4LXguNoavHBxncGzw8r3Jczw
>> > > > >>> Qkx7rJcpoP9KcBmvDjipLDimVMS/uqErAgzb1kS52qUD44gdL2HdKfTyvVmUnOGU
>> > > > >>> xqlVD6JrJjLhATxhHhomDp5omcy8dn+j5XHJD0bGbISjKMsH5SPNUHNI0x2NIWNX
>> > > > >>> u3qcQxNlKiCN5iZpniAPhjVOPA+EFW91W83H37LZkrA7YFV9q4Z/FuHbHit0DL+J
>> > > > >>> sJN7oW9afPeVco5Q80IKiT2EDQEhcLH2zUNRwYqGfktewsgOM5cWN8NSXmqqsZDz
>> > > > >>> B2Zq7OqhAGx3b/OV9OpVSO0v+Pv6G2GMmhdv35eFq/JTkJBwMhd8H9PEc8yDEbTF
>> > > > >>> 0VaKmzjWuxs4ALPvD7yQtyidZE0I8H1MQBg4E1ROC1vMNfEkQXT47ChVq4zvJJDv
>> > > > >>> J1h+6ZHCmY6uLkZLltOZQtUnkFUAS0qn6AY80Dm0FWmMcorBUY0XRQSSfY0Xzpmv
>> > > > >>> CafkAgx6TZKiFGTbmXH90JrSrHQBchBvm4GmOKDNp9W3U/ZV8QQoBH4JLsGu/2e4
>> > > > >>> Cgq/TwBmc7VrjW4gYVg/fQ+mkF/4z2K1o9h2vAD/Di50TpARAIrRZbangqwvQxKN
>> > > > >>> b6IFIT965wW0MuhiaIO1leWc3nqLXzII8EEcg5xsduFx/aqrnKI=
>> > > > >>> =2Ey1
>> > > > >>> -----END PGP SIGNATURE-----
>> > > > >>>
>> > > > >>>
>> > ---------------------------------------------------------------------
>> > > > >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> > > > >>> For additional commands, e-mail: users-h...@tomcat.apache.org
>> > > > >>>
>> > > > >>>
>> > > > >>
>> > > > >> --
>> > > > >> Regards,
>> > > > >> Kushagra
>> > > > >>
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Regards,
>> > > > > Kushagra
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Regards,
>> > > > Kushagra
>> > > >
>> > >
>> >
>> >
>> > --
>> > Regards,
>> > Kushagra
>> >
>>
>
>
> --
> Regards,
> Kushagra
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
