Hi Chris,
> Are there no logs or anything like that?
Our service stops quickly.
Nothing in logs though we redirect all tomcat logs into logs of our application.
> When Java encounters an error, it usually produces an hs_pid file which may
> be in an unusual location given the unusual CWDs of system processes. It's
> also possible that it CANNOT write its own crash report due to (lack of) file
> permissions.
No files related to JVM crash or something (hs_pid) after the stopping.
And we do have the files in case of issues with JVM, therefore I think the
files are not generated in the case.
> Can you run Tomcat NOT-as-a-service, then terminate it, and see if you get
> more information? One of the best ways to do that is to run
> "CATALINA_BASE/bin/catalina.bat run" from CMD/PowerShell and you'll get all
> logs on the console. Any crach-dump would be placed into the directory where
> you launched catalina.bat.
1. We keep 'server port ="-1"' in server.xml, therefore I set it before the test
2. I started our service with catalina.bat run and closed it later with
catalina.bat stop (please check Appendix A with log output)
3. echo %ERRORLEVEL% returns 0
4. The test looks like no issues at all.
5. Then I checked the service with the port set, but stopping as service -
still got the 1067 error.
If we revert/change tcnative-1.dll, tomcat9.exe and tomcat9w.exe files (only
these 3 files) to version 9.0.87/89 - no more 1067 error at stopping.
We disabled all interceptors on stop events of tomcat - the issue remains.
I appreciate any suggestion and/or advice.
Appendix A - Log file after stop signal has been issued
... well-known and expected output ...
13:21:29,534 INFO [StandardServer] A valid shutdown command was received via
the shutdown port. Stopping the Server instance.
Nov 24, 2024 1:21:29 PM org.apache.catalina.core.StandardServer await
INFO: A valid shutdown command was received via the shutdown port. Stopping the
Server instance.
Kostyantyn
-----Original Message-----
From: Christopher Schultz <ch...@christopherschultz.net>
Sent: Thursday, November 21, 2024 2:58 PM
To: users@tomcat.apache.org
Subject: [External] Re: Shutting Down Tomcat (9.0.97) Service Throws System
Error 1067
Kostyantyn,
On 11/21/24 4:22 AM, Kostyantyn Krakovych wrote:
>> Is Tomcat behind a front-end such as httpd or IIS?
> We use Apache HTTP Server.
>
>> Are you using tcnative-1.dll?
> Yes, we use OpenSSL.
>
>> Is there any native code (other than the JVM) deployed with the Tomcat
>> application?
> No
>
>> Was a JVM dump file produced when the process terminated unexpectedly?
> No
>
>> Can you post your connector configuration(s) - masking out any sensitive
>> information (eg, passwords)?
>
> <Connector acceptCount="150" address="127.0.0.1" connectionTimeout="20000"
> maxHeaderCount="-1" maxHttpHeaderSize="65536" maxKeepAliveRequests="100"
> port="9884" protocol="HTTP/1.1" redirectPort="1444"
> server="<app-name>"/>
>
> <Connector SSLEnabled="true" acceptCount="150" address="127.0.0.1"
> clientAuth="false" keystoreFile="conf/<app-name>.keystore"
> keystorePass="${keyStorePassword}" keystoreType="${keyStoreType}"
> maxKeepAliveRequests="100" maxThreads="150" port="1444"
> protocol="org.apache.coyote.http11.Http11NioProtocol" scheme="https"
> secure="true" server="<app-name>" sslProtocol="TLS"
> truststoreFile="conf/<app-name>.truststore"
> truststorePass="${trustStorePassword}"
> truststoreType="${trustStoreType}"/>
>
> <Connector address="127.0.0.1" port="19884" protocol="AJP/1.3"
> redirectPort="-1" secretRequired="false"/>
Interestingly, System Error 1067 is "The process terminated unexpectedly"
though you are intentionally stopping it :)
Are there no logs or anything like that? If you are using Windows, you are
probably using tomcat9.exe to launch it, which is re-named procrun.exe. That
binary usually writes to a log file somewhere. Can you locate that file and see
if it contains any relevant information? IIRC, the file should be called
catalina.err or something similar.
When Java encounters an error, it usually produces an hs_pid file which may be
in an unusual location given the unusual CWDs of system processes. It's also
possible that it CANNOT write its own crash report due to (lack of) file
permissions.
Can you run Tomcat NOT-as-a-service, then terminate it, and see if you get more
information? One of the best ways to do that is to run
"CATALINA_BASE/bin/catalina.bat run" from CMD/PowerShell and you'll get all
logs on the console. Any crach-dump would be placed into the directory where
you launched catalina.bat.
-chris
> -----Original Message-----
> From: Chuck Caldarale <n82...@gmail.com>
> Sent: Wednesday, November 20, 2024 4:10 AM
> To: Tomcat Users List <users@tomcat.apache.org>
> Subject: [External] Re: Shutting Down Tomcat (9.0.97) Service Throws
> System Error 1067
>
> [You don't often get email from n82...@gmail.com. Learn why this is
> important at https://aka.ms/LearnAboutSenderIdentification ]
>
> WARNING: This message was sent by someone outside Securitas. Please do not
> reply, click links or open attachments from senders that you do not trust.
> When in doubt, use the Report Message button.
>
>> On Nov 19, 2024, at 08:16, Kostyantyn Krakovych
>> <kostyantyn.krakov...@securitashealthcare.com> wrote:
>>
>> Hi Team,
>>
>> Could you please suggest or advice regarding the following?
>>
>> 1. We use Tomcat 9.0.87 as a windows service. It works fine with our
>> application.
>> 2. To fix security vulnerabilities we are upgrading Tomcat to version 9.0.97.
>> 3. The upgraded service works fine except when shutting down/stopping it.
>> When we stop the service via either console or Services app, we get system
>> error 1067 (please check the Appendix A).
>> 4. Event Viewer does not contain too many details about the issue (please
>> check the Appendix B).
>> 5. Nothing in logs though we redirect Tomcat logs to logs of our application.
>> 6. We checked a set of versions, and it looks like the issue started to
>> appear since Tomcat version 9.0.90.
>>
>> Any suggestions or advice how to find a cause of the issue after the upgrade
>> is welcome.
>
>
> Need a bit more information here. Is Tomcat behind a front-end such as
> httpd or IIS? Are you using tcnative-1.dll? Is there any native code
> (other than the JVM) deployed with the Tomcat application? Was a JVM
> dump file produced when the process terminated unexpectedly? (Sorry, I
> don't know where to look for that on Windows.)
>
> Can you post your connector configuration(s) - masking out any sensitive
> information (eg, passwords)?
>
> - Chuck
>
>
>> Appendix A - Output in console when stooping the service
>>
>>> net stop <service-name>
>> The <service-name> service is stopping.
>> A system error has occurred.
>> System error 1067 has occurred.
>> The process terminated unexpectedly.
>> The <service-name> service was stopped successfully.
>>
>> Appendix B - Event Viewer details about the issue
>>
>> <Event
>> xmlns="https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F
>> %
>> 2Fschemas.microsoft.com%2Fwin%2F2004%2F08%2Fevents%2Fevent&data=05%7C
>> 0
>> 2%7Ckostyantyn.krakovych%40securitashealthcare.com%7C67476498d717401d
>> 2
>> d8c08dd09088a93%7Cfb0af7d18a8348bc8d2dcd92034c79d4%7C0%7C0%7C63867665
>> 4
>> 461827410%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjA
>> u
>> MDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C
>> & sdata=e8RHcGFcdZEC6ai9CUNE3h%2FnOwNjidC0ENwCjQeowSg%3D&reserved=0">
>> - <System>
>> <Provider Name="Service Control Manager"
>> Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}"
>> EventSourceName="Service Control Manager" /> <EventID
>> Qualifiers="49152">7034</EventID> <Version>0</Version>
>> <Level>2</Level> <Task>0</Task> <Opcode>0</Opcode>
>> <Keywords>0x8080000000000000</Keywords>
>> <TimeCreated SystemTime="2024-11-19T07:36:27.2825786Z" />
>> <EventRecordID>1577729</EventRecordID>
>> <Correlation />
>> <Execution ProcessID="1196" ThreadID="19180" />
>> <Channel>System</Channel> <Computer><computer-name></Computer>
>> <Security />
>> </System>
>> - <EventData>
>> <Data Name="param1"><service-name></Data>
>> <Data Name="param2">140</Data>
>> <Binary><service-name-binary-data></Binary>
>> </EventData>
>> </Event>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
