Dear all, some time ago, I also need to pass JXM through a tunneled connection (using STunnel). The Problem with JMX via RMI is that here is more than one connection and involved in the handshake is an IP address that is different on both ends of the tunnel. A solution was to use an alternative transport layer named JMXMP because this need only one connection.
You may use my notes on https://github.com/gjaekel/jmxmp-lifecycle-listener as a starting point. One have to provide additionals JARS, on Tomcat to $CATALINA_HOME/lib. Then one have to compile and install an additional listener <Listener className="javax.management.remote.extension.JMXMPLifecycleListener" port="5555" /> The client also must be "undergrid" with an additional library, e.g. visualvm --cp:a jmxremote_optional.jar and the connection URL must use the alternative protocol service:jmx:jmxmp://<remote_server>:<port> with greetings Guido -----Ursprüngliche Nachricht----- Von: Christopher Schultz [mailto:ch...@christopherschultz.net] Gesendet: Donnerstag, 12. Dezember 2019 16:05 An: users@tomcat.apache.org Betreff: Re: remote jmx monitoring through ssh tunnel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Chris, On 12/11/19 15:52, Chris Cheshire wrote: > On Wed, Dec 11, 2019 at 12:24 PM Christopher Schultz > <ch...@christopherschultz.net> wrote: >> >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 >> >> >> >> On 12/10/19 12:59, Chris Cheshire wrote: >>> On Tue, Dec 10, 2019 at 11:58 AM Chris Cheshire >>> <yahoono...@gmail.com> wrote: >>>> >>>> On Tue, Dec 10, 2019 at 9:42 AM Christopher Schultz >>>> <ch...@christopherschultz.net> wrote: >>>>> >>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 >>>>> >>>>> Chris, >>>>> >>>>> On 12/9/19 17:10, Chris Cheshire wrote: >>>>>> In CATALINA_BASE/bin/setenv.sh I have the following : >>>>>> >>>>>> CATALINA_OPTS="-Dcom.sun.management.jmxremote >>>>>> -Dcom.sun.management.jmxremote.ssl=false >>>>>> -Dcom.sun.management.jmxremote.authenticate=false" >>>>> >>>>> Okay. >>>>> >>>>>> In CATALINA_BASE/conf/server.xml I have a listener configured : >>>>>> >>>>>> <Listener >>>>>> className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener" >>>>>> >>>>>> >> >>>>>> rmiRegistryPortPlatform="10001" rmiServerPortPlatform="10002" >>>>>> useLocalPorts="true" /> >>>>>> >>>>>> >>>>>> Upon startup I see in logs : INFO [main] >>>>>> org.apache.catalina.mbeans.JmxRemoteLifecycleListener.createServe r >>>>>> >>>>>> >> >>>>>> The JMX Remote Listener has configured the registry on port >>>>>> [10001] and the server on port [10002] for the [Platform] server >>>>>> >>>>>> >>>>>> $ netstat -an | grep 10001 tcp4 0 0 >>>>>> 127.0.0.1.10001 *.* LISTEN tcp6 >>>>>> 0 0 ::1.10001 *.* LISTEN >>>>>> >>>>>> On my local machine I have a tunnel set up as follows : >>>>>> ssh -N -L10001:localhost:10001 -L10002:localhost:10002 >>>>>> user@remotehost >>>>>> >>>>>> (where user is the user tomcat is running under) >>>>>> >>>>>> When I try to add a remote JMX connection in VisualVM on my >>>>>> client machine to localhost:10001 I get an error dialog after a >>>>>> brief delay with the message "Cannot connect to localhost:10001 >>>>>> using service:jmx:rmi:///jndi/rmi://localhost:10001/jmxrmi". If I >>>>>> change it to port 10002 I get the same error. On the server at >>>>>> this time : $ netstat -an | grep 10001 tcp4 >>>>>> 0 0 127.0.0.1.10001 *.* LISTEN >>>>>> tcp6 0 0 ::1.10001 *.* LISTEN >>>>>> tcp4 0 0 127.0.0.1.62637 127.0.0.1.10001 >>>>>> TIME_WAIT >>>>>> >>>>>> >>>>>> If I try to use jconsole connecting to port 10001 I get the error >>>>>> "Connection failed: non-JRMP server at remote endpoint". >>>>>> Connecting to port 10002 I get the error "Connection failed: no >>>>>> such object in table" >>>>> >>>>> You should be using the port defined by rmiRegistryPortPlatform, >>>>> so 10001 is the correct port to use. >>>>> >>>>>> I've been through the tomcat configuration documentation a couple >>>>>> times but I can't see what else I need to configure. >>>>> >>>>> What you have looks good to me without reproducing it myself. Can >>>>> you do : >>>>> >>>>> $ netstat -an | grep 1000[0-9] >>>>> >>>>> ? >>>>> >>>>> Just to be sure about both ports? >>>>> >>>> >>>> $ netstat -an | grep 1000[0-9] tcp6 0 0 :::10001 >>>> :::* LISTEN tcp6 0 0 :::10002 >>>> :::* LISTEN >>>> >>>> >>>> Hmmmm. Tomcat is only listening on ipv6 ports, but my tunnel is >>>> using ipv4. After digging around [1], I added this to CATALINA_OPTS >>>> in setenv.sh >>>> >>>> -Djava.net.preferIPv4Stack=true >>>> -Djava.net.preferIPv4Addresses=true >>>> >>>> $ netstat -an | grep 1000[0-9] tcp 0 0 >>>> 0.0.0.0:10001 0.0.0.0:* LISTEN tcp 0 >>>> 0 0.0.0.0:10002 0.0.0.0:* LISTEN >>>> >>>> When I try to connect with jconsole I get the same error (non-JRMP >>>> server at remote endpoint), with the server showing >>>> >>>> tcp 0 0 0.0.0.0:10001 0.0.0.0:* LISTEN >>>> tcp 0 0 0.0.0.0:10002 0.0.0.0:* LISTEN >>>> tcp 0 0 127.0.0.1:10001 127.0.0.1:43803 >>>> TIME_WAIT tcp 0 0 127.0.0.1:10001 >>>> 127.0.0.1:43815 TIME_WAIT >>>> >>>> >>>> I have also updated sshd_config with >>>> >>>> PermitTunnel yes >>>> >>>> and restarted that. Still no change. >>>> >>>> Chris >>>> >>>> >>>> [1] >>>> https://serverfault.com/questions/390840/how-does-one-get-tomcat-to - -b >> >>>> ind-to-ipv4-address >>> >>>> >>> >>> As a followup to take the tunnel out of the equation I downloaded >>> jmxterm [1] on the server and tried to connect >>> >>> >>> $ java -jar jmxterm-1.0.0-uber.jar Welcome to JMX terminal. >>> Type "help" for available commands. $>open localhost:10001 >>> #RuntimeIOException: Runtime IO exception: Failed to retrieve >>> RMIServer stub: javax.naming.CommunicationException [Root exception >>> is java.rmi.ConnectIOException: non-JRMP server at remote endpoint] >>> $> >>> >>> >>> Back to the tomcat documentation, I added this to CATALINA_OPTS >>> (based on listener config and assumed defaults) >>> >>> -Dcom.sun.management.jmxremote.registry.ssl=false >>> >>> and now I get a different error : $>open localhost:10001 >>> #RuntimeIOException: Runtime IO exception: Failed to retrieve >>> RMIServer stub: javax.naming.CommunicationException [Root exception >>> is java.rmi.UnmarshalException: error unmarshalling return; nested >>> exception is: java.lang.ClassNotFoundException: >>> org/apache/catalina/mbeans/JmxRemoteLifecycleListener$RmiClientLocal ho >> >>> stSocketFactory >>> >>> >> (no security manager: RMI class loader disabled)] >>> >>> >>> So I enabled the security manager by adding to CATALINA_OPTS >>> >>> -Djava.security.manager >>> -Djava.security.policy=$CATALINA_BASE/conf/catalina.policy >>> >>> And got a reminder why I turned it off in the first place. Now I >>> have to figure out how to allow the mysql drivers to work (and >>> probably everything else about the web app) so tomcat will start :/ >>> >>> Uggh. >>> >>> Chris >> >> There's always the JMXProxyServlet. >> >> JMX is such an ugly protocol. Why not use HTTP(S) which is much >> easier to configure and connect to? It also means you don't need a >> Java client :) >> >> - -chris > > I went this route because I thought it would be the quickest way to > start poking around within the exposed mbeans without writing code to > query them myself. > > So if tomcat is not jconsole/visualvm compatible, how do I access the > exposed JMX mbeans? Oh, Tomcat most definitely is jconsole/visualvm compatible. I can connect without any problems on any local environment. I've never bothered to set it up remotely, because frankly Java clients are too wasteful IMO to deploy. I use Perl and/or Python-based clients which query the JMXProxyServlet. Have a look at http://tomcat.apache.org/presentations.html#latest-monitoring-with-jmx to see how you cann use the JMXProxyServlet with ... any client you'd like. There are examples using curl in that presentation. You can also have a look at: https://github.com/ChristopherSchultz/check-jmxproxy or: https://github.com/ChristopherSchultz/apache-tomcat-stuff/tree/master/bi n/nagios (I have forgotten which of those is more up-to-date... looks like the latest commit was on the latter.) - -chris -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3yVxcACgkQHPApP6U8 pFhqrA/+Ptt/CKlDw66uTJkHW3OPix9Cx5VZw5O9T2zrxZ6JMMpzmLWQTQtgdvFV +4wWGu2nGjAj3MqIl8j7wF+Scm10YnBLITzfXYA1zUHgWk99/ZvK07t3YZ2y+nT4 Yl02Zc5KJ8dIMkcLwAcwZ8jRL+4uaA48W5zjfHMOaG3LRKU4ONRHO4Qkxc3YQgvu TPWdUypdO1La1hGV11D2ZiUbf61ybogOFpIbtYMj1Nqm8gGY4HhXMTc2EtwmX+DV TVO+4SUoTU6ZqNK5QSnmIu9rR91gp6nQ3V9nKA4bV019qVKmHGeVuQtdzIFVMJJO 16BImqh8G4gKckH2gArmKYfzpwGUl8Th+QHueJ5OTbDwS17zvp8jmeuosWIQutux gqKFPWTeoGtERgO57IT9xfiW92FLUNqzmTiGJZHDpiaQbffZzYsTmz3GQwGft95E F6/dDRSvHu2ZEd/5WG94+7DpYItkTxBiS77bDKDzPsI1c1UEJhdRcJOnxdODzWNw G7NQfXria3yCqhmG+Qz2dh3F9CKjHrhUyXcwMgnqyttXeetWyVDZ8Y2v8wG9VF8q h53eCYw88aOyJntKnQMlZ2OvBzOaXZXiz17YEBrlgH6X2/vsc3uXvmCiUn9byTdm Xb4WWNWU4/YXnfvYLJweOircgjVCIkqacrjkanq1GkJDzIt1iiU= =hRok -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
