As you suggested.,
I have added my annotations in Catalina_opts ., and I am not able to see any
Java Process with the JMX, neither the port is being listened.
It came back to previous stage where Annotations are not on the JVM .,
Came back to previous stage, where connection refused error., as follows.
Could not initialize instance: tomcat-10.75.12.136-49151:
java.util.concurrent.ExecutionException: java.io.IOException: Failed to
retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root
exception is java.rmi.ConnectException: Connection refused to host:
10.75.12.136; nested exception is:
java.net.ConnectException: Connection refused (Connection refused)]
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at org.datadog.jmxfetch.App.processRecoveryResults(App.java:953)
at org.datadog.jmxfetch.App$5.invoke(App.java:927)
at
org.datadog.jmxfetch.tasks.TaskProcessor.processTasks(TaskProcessor.java:63)
at org.datadog.jmxfetch.App.init(App.java:917)
at org.datadog.jmxfetch.App.start(App.java:413)
at org.datadog.jmxfetch.App.run(App.java:173)
at org.datadog.jmxfetch.JmxFetch.main(JmxFetch.java:57)
at org.datadog.jmxfetch.App.main(App.java:91)
Caused by: java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: 10.75.12.136; nested
exception is:
java.net.ConnectException: Connection refused (Connection refused)]
at
java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:370)
at
java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
at org.datadog.jmxfetch.Connection.createConnection(Connection.java:63)
at
org.datadog.jmxfetch.RemoteConnection.<init>(RemoteConnection.java:101)
at
org.datadog.jmxfetch.ConnectionFactory.createConnection(ConnectionFactory.java:38)
at org.datadog.jmxfetch.Instance.getConnection(Instance.java:405)
at org.datadog.jmxfetch.Instance.init(Instance.java:418)
at
org.datadog.jmxfetch.InstanceInitializingTask.call(InstanceInitializingTask.java:15)
at
org.datadog.jmxfetch.InstanceInitializingTask.call(InstanceInitializingTask.java:3)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: 10.75.12.136; nested
exception is:
java.net.ConnectException: Connection refused (Connection refused)]
at
jdk.naming.rmi/com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:137)
at
java.naming/com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:220)
at
java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
at
java.management.rmi/javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1839)
at
java.management.rmi/javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1813)
at
java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:302)
... 12 more
Caused by: java.rmi.ConnectException: Connection refused to host: 10.75.12.136;
nested exception is:
java.net.ConnectException: Connection refused (Connection refused)
at
java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:623)
at
java.rmi/sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:209)
at
java.rmi/sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:196)
at java.rmi/sun.rmi.server.UnicastRef.newCall(UnicastRef.java:343)
at
java.rmi/sun.rmi.registry.RegistryImpl_Stub.lookup(RegistryImpl_Stub.java:116)
at
jdk.naming.rmi/com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:133)
... 17 more
Here we were not able to find any process , to confirm that atleast port is
being listened to.
Thanks & Regards,
--
SAI VAMSI .B
Senior DevOps Engineer
________________________________
From: Christopher Schultz <ch...@christopherschultz.net>
Sent: Tuesday, December 5, 2023 19:19
To: users@tomcat.apache.org <users@tomcat.apache.org>
Subject: Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.
Sai Vamsa,
On 12/5/23 01:25, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote:
> Thanks for the response..
>
> as you suggested., I tried adding my annotations in CATALINA_OPTS., Instead
> of Java_Opts and Java_tool_options.,
>
> But its not even working out., Java process is not getting created., and port
> is not bound to any process.
> Let me know if I am missing anything.
Okay. Can you please summarize the current situation? I think you've
made a few changes, etc. so it would help to know the current state.
Stop Tomcat and any other related services, delete all appropriate log
files, etc. and start Tomcat (or attempt to). Then post any output that
is relevant, and if you get things like "address in use" then please
post the details about which process id owns that port and the full
command-line that launched that process.
Thanks,
-chris
> ________________________________
> From: Christopher Schultz <ch...@christopherschultz.net>
> Sent: Monday, December 4, 2023 23:22
> To: users@tomcat.apache.org <users@tomcat.apache.org>
> Subject: Re: [EXT] Re: Datadog _ JMX Integration facing connection issues.
>
> Sai Vamsi,
>
> On 12/4/23 03:53, Bodavula, Sai Vamsi Mohan Krishna (TR Technology) wrote:
>> Firstly thanks for adding a point me in asking me to check, if the
>> annotations are reflecting in the Java process, which opened me a door to
>> add the concerned annotations in correct place., by adding in
>> java_tool_options in stead of Java_opts.
>
> You will probably want to use CATALINA_OPTS instead of any of the other
> ones. JAVA_TOOL_OPTS isn't an environment variable regognized by Tomcat.
> You certainly don't want to use JAVA_OPTS, because Tomcat uses JAVA_OPTS
> any time it invokes a JVM. For example, running bind/digest.sh doesn't
> need to have the JMX subsystem starting-up and trying to grab a port.
>
> JAVA_TOOL_OPTS is an environment variable used by JVM-launching
> processes, like jps for example...
>
>> yeah they are reflecting and creating a Java Process.
>> but I am facing a problem here., while i am checking JSP, Thats : the port
>> i am using here to enable JMX is been opening a process with the mentioned
>> port and at the same time shows port is being used.:
>>
>> root@lab1workflow4scalsvc2zus1-deployment-577d856494-ftb22:/# jps
>> Picked up JAVA_TOOL_OPTIONS: -Xms2048M -Xmx10240M
>> -XX:+UseStringDeduplication -XX:+UseContainerSupport
>> -Dcom.sun.management.jmxremote
>> -Dcom.sun.management.jmxremote.authenticate=false
>> -Dcom.sun.management.jmxremote.ssl=false
>> -Dcom.sun.management.jmxremote.local.only=false
>> -Dcom.sun.management.jmxremote.port=49151
>> -Djava.rmi.server.hostname=tomcat.default.svc.cluster.local
>> -javaagent:/datadog-lib/dd-java-agent.jar
>> -XX:OnError=/datadog-lib/continuousprofiler/tmp/dd_crash_uploader.sh
>> -XX:ErrorFile=/datadog-lib/continuousprofiler/tmp/hs_err_pid_%p.log
>> Error: Exception thrown by the agent : java.rmi.server.ExportException: Port
>> already in use: 49151; nested exception is:
>> java.net.BindException: Address already in use (Bind failed)
>
> Yes: you have set JAVA_TOOL_OPTS and then run jps. jps is trying to bind
> to your port which is aready bound by your Tomcat process.
>
> The solution is to use only CATALINA_OPTS to set these options.
>
> -chris
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
