Github user sjcorbett commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/762#discussion_r35202260
--- Diff: docs/guide/dev/tips/troubleshooting-exceptions.md ---
@@ -0,0 +1,487 @@
+---
+layout: website-normal
+title: Troubleshooting Exceptions and Node Failure
+toc: /guide/toc.json
+---
+
+Whether you're customizing out-of-the-box blueprints, or developing your
own custom blueprints, you will
+inevitably have to deal with node failure, or exceptions being thrown by
your node. Thankfully Brooklyn
+provides plenty of information to help you locate and resolve any issues
you may encounter.
+
+This guide looks at three common failure scenarios and describes the steps
that can be taken to
+identify the issue.
+
+## Script failure
+Many blueprints run bash scripts as part of the installation. This section
highlights how to identify a problem with
+a bash script.
+
+First let's take a look at the `customize()` method of the Tomcat server
blueprint:
+
+{% highlight java %}
+ @Override
+ public void customize() {
+ newScript(CUSTOMIZING)
+ .body.append("mkdir -p conf logs webapps temp")
+ .failOnNonZeroResultCode()
+ .execute();
+
+ copyTemplate(entity.getConfig(TomcatServer.SERVER_XML_RESOURCE),
Os.mergePaths(getRunDir(), "conf", "server.xml"));
+ copyTemplate(entity.getConfig(TomcatServer.WEB_XML_RESOURCE),
Os.mergePaths(getRunDir(), "conf", "web.xml"));
+
+ // Deduplicate same code in JBoss
+ if (isProtocolEnabled("HTTPS")) {
+ String keystoreUrl =
Preconditions.checkNotNull(getSslKeystoreUrl(), "keystore URL must be specified
if using HTTPS for " + entity);
+ String destinationSslKeystoreFile = getHttpsSslKeystoreFile();
+ InputStream keystoreStream =
resource.getResourceFromUrl(keystoreUrl);
+ getMachine().copyTo(keystoreStream, destinationSslKeystoreFile);
+ }
+
+ getEntity().deployInitialWars();
+ }
+{% endhighlight %}
+
+Here we can see that it's running a script to create four directories
before continuing with the customization. Let's
+introduce an error by changing `mkdir` to `mkrid`:
+
+{% highlight java %}
+ newScript(CUSTOMIZING)
+ .body.append("mkrid -p conf logs webapps temp") // `mkdir`
changed to `mkrid`
+ .failOnNonZeroResultCode()
+ .execute();
+{% endhighlight %}
+
+Now let's try deploying this using the following YAML:
+
+{% highlight yaml %}
+
+name: Tomcat failure test
+location: localhost
+services:
+- type: brooklyn.entity.webapp.tomcat.TomcatServer
+
+{% endhighlight %}
+
+Shortly after deployment, the entity fails with the following error:
+
+`Failure running task ssh: customizing TomcatServerImpl{id=e1HP2s8x}
(HmyPAozV):
+Execution failed, invalid result 127 for customizing
TomcatServerImpl{id=e1HP2s8x}`
+
+[](images/script-failure-large.png)
+
+By selecting the `Activities` tab and drilling down into the tasks, we
eventually get to the task that failed:
+
+[](images/failed-task-large.png)
+
+By clicking on the `stderr` link, we can see the script failed with the
following error:
+
+{% highlight console %}
+/tmp/brooklyn-20150721-132251052-l4b9-customizing_TomcatServerImpl_i.sh:
line 10: mkrid: command not found
+{% endhighlight %}
+
+This tells us *what* went wrong, but doesn't tell us *where*. In order to
find that, we'll need to look at the
+stack trace that was logged when the exception was thrown.
+
+It's always worth looking at the Detailed Status section as sometimes this
will give you the information you need.
+In this case, the stack trace is limited to the thread that was used to
execute the task that ran the script:
+
+{% highlight console %}
+Failed after 40ms
+
+STDERR
+/tmp/brooklyn-20150721-132251052-l4b9-customizing_TomcatServerImpl_i.sh:
line 10: mkrid: command not found
+
+
+STDOUT
+Executed
/tmp/brooklyn-20150721-132251052-l4b9-customizing_TomcatServerImpl_i.sh, result
127: Execution failed, invalid result 127 for customizing
TomcatServerImpl{id=e1HP2s8x}
+
+java.lang.IllegalStateException: Execution failed, invalid result 127 for
customizing TomcatServerImpl{id=e1HP2s8x}
+ at
brooklyn.entity.basic.lifecycle.ScriptHelper.logWithDetailsAndThrow(ScriptHelper.java:390)
+ at
brooklyn.entity.basic.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:379)
+ at
brooklyn.entity.basic.lifecycle.ScriptHelper$8.call(ScriptHelper.java:289)
+ at
brooklyn.entity.basic.lifecycle.ScriptHelper$8.call(ScriptHelper.java:287)
+ at
brooklyn.util.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:343)
+ at
brooklyn.util.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:469)
+ at java.util.concurrent.FutureTask.run(FutureTask.java:262)
+ at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
+ at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
+at java.lang.Thread.run(Thread.java:745)
+{% endhighlight %}
+
+In order to find the exception, we'll need to look in Brooklyn's debug log
file. By default, the debug log file
+is named `brooklyn.debug.log`. Usually the easiest way to navigate the log
file is to use `less`, e.g.
+`less brooklyn.debug.log`. We can quickly find find the stack trace by
first navigating to the end of the log file
+with `Shift-G`, then performing a reverse-lookup by typing `?Tomcat` and
pressing `Enter`. If searching for the
+blueprint type (in this case Tomcat) simply matches tasks unrelated to the
exception, you can also search for
+the text of the error message, in this case `? invalid result 127`. You
can make the search case-insensitivy by
+typing `-i` before performing the search. To skip the current match and
move to the next on (i.e. 'up' as we're
+performing a reverse-lookup), simply press `n`
+
+In this case, the `?Tomcat` search takes us directly to the full stack
trace (Only the last part of the trace
+is shown here):
+
+{% highlight console %}
+
+at
com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63)
~[guava-17.0.jar:na]
+at brooklyn.util.task.BasicTask.get(BasicTask.java:343) ~[classes/:na]
+at brooklyn.util.task.BasicTask.getUnchecked(BasicTask.java:352)
~[classes/:na]
+... 9 common frames omitted
+Caused by: brooklyn.util.exceptions.PropagatedRuntimeException:
+at brooklyn.util.exceptions.Exceptions.propagate(Exceptions.java:97)
~[classes/:na]
+at brooklyn.util.task.BasicTask.getUnchecked(BasicTask.java:354)
~[classes/:na]
+at
brooklyn.entity.basic.lifecycle.ScriptHelper.execute(ScriptHelper.java:339)
~[classes/:na]
+at
brooklyn.entity.webapp.tomcat.TomcatSshDriver.customize(TomcatSshDriver.java:72)
~[classes/:na]
+at
brooklyn.entity.basic.AbstractSoftwareProcessDriver$8.run(AbstractSoftwareProcessDriver.java:150)
~[classes/:na]
+at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_71]
+at
brooklyn.util.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:343)
~[classes/:na]
+... 5 common frames omitted
+Caused by: java.util.concurrent.ExecutionException:
java.lang.IllegalStateException: Execution failed, invalid result 127 for
customizing TomcatServerImpl{id=e1HP2s8x}
+at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[na:1.7.0_71]
+at java.util.concurrent.FutureTask.get(FutureTask.java:188) [na:1.7.0_71]
+at
com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63)
~[guava-17.0.jar:na]
+at brooklyn.util.task.BasicTask.get(BasicTask.java:343) ~[classes/:na]
+at brooklyn.util.task.BasicTask.getUnchecked(BasicTask.java:352)
~[classes/:na]
+... 10 common frames omitted
+Caused by: java.lang.IllegalStateException: Execution failed, invalid
result 127 for customizing TomcatServerImpl{id=e1HP2s8x}
+at
brooklyn.entity.basic.lifecycle.ScriptHelper.logWithDetailsAndThrow(ScriptHelper.java:390)
~[classes/:na]
+at
brooklyn.entity.basic.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:379)
~[classes/:na]
+at
brooklyn.entity.basic.lifecycle.ScriptHelper$8.call(ScriptHelper.java:289)
~[classes/:na]
+at
brooklyn.entity.basic.lifecycle.ScriptHelper$8.call(ScriptHelper.java:287)
~[classes/:na]
+... 6 common frames omitted
+
+{% endhighlight %}
+
+Brooklyn's use of tasks and helper classes can make the stack trace a
little harder than usual to follow, but a good
+place to start is to look through the stack trace for the node's
implementation or ssh driver classes (usually
+named `FooNodeImpl` or `FooSshDriver`). In this case we can see the
following:
+
+{% highlight console %}
+at
brooklyn.entity.webapp.tomcat.TomcatSshDriver.customize(TomcatSshDriver.java:72)
~[classes/:na]
+{% endhighlight %}
+
+Combining this with the error message of `mkrid: command not found` we can
see that indeed `mkdir` has been
+misspelled `mkrid` on line 72 of `TomcatSshDriver.java`.
+
+
+## Non-Script failure
+The section above gives an example of a failure that occurs when a script
is run. In this section we will look at
+a failure in a non-script related part of the code. We'll use the
`customize()` method of the Tomcat server again,
+but this time, we'll correct the spelling of 'mkdir' and add a line that
attempts to copy a nonexistent resource
+to the remote server:
+
+{% highlight java %}
+
+newScript(CUSTOMIZING)
+ .body.append("mkdir -p conf logs webapps temp")
+ .failOnNonZeroResultCode()
+ .execute();
+
+copyTemplate(entity.getConfig(TomcatServer.SERVER_XML_RESOURCE),
Os.mergePaths(getRunDir(), "conf", "server.xml"));
+copyTemplate(entity.getConfig(TomcatServer.WEB_XML_RESOURCE),
Os.mergePaths(getRunDir(), "conf", "web.xml"));
+copyTemplate("classpath://nonexistent.xml", Os.mergePaths(getRunDir(),
"conf", "nonexistent.xml")); // Resource does not exist!
+
+{% endhighlight %}
+
+Let's deploy this using the same YAML from above. Here's the resulting
error in the Brooklyn debug console:
+
+[](images/resource-exception-large.png)
+
+Again, this tells us *what* the error is, but we need to find *where* the
code is that attempts to copy this file. In
+this case it's shown in the Detailed Status section, and we don't need to
go to the log file:
+
+{% highlight console %}
+
+Failed after 221ms: Error getting resource 'classpath://nonexistent.xml'
for TomcatServerImpl{id=PVZxDKU1}: java.io.IOException: Error accessing
classpath://nonexistent.xml: java.io.IOException: nonexistent.xml not found on
classpath
+
+java.lang.RuntimeException: Error getting resource
'classpath://nonexistent.xml' for TomcatServerImpl{id=PVZxDKU1}:
java.io.IOException: Error accessing classpath://nonexistent.xml:
java.io.IOException: nonexistent.xml not found on classpath
+ at
brooklyn.util.ResourceUtils.getResourceFromUrl(ResourceUtils.java:297)
+ at
brooklyn.util.ResourceUtils.getResourceAsString(ResourceUtils.java:475)
+ at
brooklyn.entity.basic.AbstractSoftwareProcessDriver.getResourceAsString(AbstractSoftwareProcessDriver.java:447)
+ at
brooklyn.entity.basic.AbstractSoftwareProcessDriver.processTemplate(AbstractSoftwareProcessDriver.java:469)
+ at
brooklyn.entity.basic.AbstractSoftwareProcessDriver.copyTemplate(AbstractSoftwareProcessDriver.java:390)
+ at
brooklyn.entity.basic.AbstractSoftwareProcessDriver.copyTemplate(AbstractSoftwareProcessDriver.java:379)
+ at
brooklyn.entity.webapp.tomcat.TomcatSshDriver.customize(TomcatSshDriver.java:79)
+ at
brooklyn.entity.basic.AbstractSoftwareProcessDriver$8.run(AbstractSoftwareProcessDriver.java:150)
+ at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
+ at
brooklyn.util.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:343)
+ at
brooklyn.util.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:469)
+ at java.util.concurrent.FutureTask.run(FutureTask.java:262)
+ at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
+ at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
+at java.lang.Thread.run(Thread.java:745)
+ Caused by: java.io.IOException: Error accessing
classpath://nonexistent.xml: java.io.IOException: nonexistent.xml not found on
classpath
+at brooklyn.util.ResourceUtils.getResourceFromUrl(ResourceUtils.java:233)
+ ... 14 more
+ Caused by: java.io.IOException: nonexistent.xml not found on classpath
+ at
brooklyn.util.ResourceUtils.getResourceViaClasspath(ResourceUtils.java:372)
+at brooklyn.util.ResourceUtils.getResourceFromUrl(ResourceUtils.java:230)
+ ... 14 more
+
+{% endhighlight %}
+
+Looking for `Tomcat` in the stack trace, we can see in this case the
problem lies at line 79 of `TomcatSshDriver.java`
+
+
+## External failure
+
+Sometimes an entity will fail outside the direct commands issues by
Brooklyn. When installing and launching an entity,
+Brooklyn will check the return code of scripts that were run to ensure
that they completed successfully (i.e. the
+return code of the script is zero). It is possible, for example, that a
launch script completes successfully, but
+the entity fails to start.
+
+We can simulate this type of failure by launching Tomcat with an invalid
configuration file. As seen in the previous
+examples, Brooklyn copies two xml configuration files to the server:
`server.xml` and `web.xml`
+
+The first few non-comment lines of `server.xml` are as follows (you can
see the full file [here]
+(https://github.com/apache/incubator-brooklyn/blob/master/software/webapp/src/main/resources/brooklyn/entity/webapp/tomcat/server.xml)):
+
+{% highlight xml %}
+
+<Server port="${driver.shutdownPort?c}" shutdown="SHUTDOWN">
+ <Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
+ <Listener className="org.apache.catalina.core.JasperListener" />
+
+{% endhighlight%}
+
+Let's add an unmatched XML element, which will make this XML file invalid:
+
+{% highlight xml %}
+
+<Server port="${driver.shutdownPort?c}" shutdown="SHUTDOWN">
+ <unmatched-element> <!-- This is invalid XML as we won't add
</unmatched-element> -->
+ <Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
+ <Listener className="org.apache.catalina.core.JasperListener" />
+
+{% endhighlight%}
+
+As Brooklyn doesn't know how these types of resources are used, they're
not validated as they're copied to the remote machine.
+As far as Brooklyn is concerned, the file will have copied successfully.
+
+Let's deploy Tomcat again, using the same YAML as before. This time, the
deployment runs for a few minutes before failing
+with `Timeout waiting for SERVICE_UP`:
+
+[](images/external-error-large.png)
+
+If we drill down into the tasks in the `Activities` tab, we can see that
all of the installation and launch tasks
+completed successfully, and stdout of the `launch` script is as follows:
+
+{% highlight console %}
+
+Executed
/tmp/brooklyn-20150721-153049139-fK2U-launching_TomcatServerImpl_id_.sh, result 0
+
+{% endhighlight %}
+
+The task that failed was the `post-start` task, and the stack trace from
the Detailed Status section is as follows:
+
+{% highlight console %}
+
+Failed after 5m 1s: Timeout waiting for SERVICE_UP from
TomcatServerImpl{id=BUHgQeOs}
+
+java.lang.IllegalStateException: Timeout waiting for SERVICE_UP from
TomcatServerImpl{id=BUHgQeOs}
+ at brooklyn.entity.basic.Entities.waitForServiceUp(Entities.java:1073)
+ at
brooklyn.entity.basic.SoftwareProcessImpl.waitForServiceUp(SoftwareProcessImpl.java:388)
+ at
brooklyn.entity.basic.SoftwareProcessImpl.waitForServiceUp(SoftwareProcessImpl.java:385)
+ at
brooklyn.entity.basic.SoftwareProcessDriverLifecycleEffectorTasks.postStartCustom(SoftwareProcessDriverLifecycleEffectorTasks.java:164)
+ at
brooklyn.entity.software.MachineLifecycleEffectorTasks$7.run(MachineLifecycleEffectorTasks.java:433)
+ at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
+ at
brooklyn.util.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:343)
+ at
brooklyn.util.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:469)
+ at java.util.concurrent.FutureTask.run(FutureTask.java:262)
+ at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
+ at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
+at java.lang.Thread.run(Thread.java:745)
+
+{% endhighlight %}
+
+This doesn't really tell us what we need to know, and looking in the
`brooklyn.debug.log` file yields no further
+clues. The key here is the error message `Timeout waiting for SERVICE_UP`.
After running the installation and
+launch scripts, assuming all scripts completed successfully, Brooklyn will
periodically check the health of the node
+and will set the node on fire if the health check does not pass within a
pre-prescribed period (the default is
+two minutes, and can be configured using the `start.timeout` config key).
The periodic health check also continues
+after the successful launch in order to check continued operation of the
node, but in this case it fails to pass
+at all.
+
+The first thing we need to do is to find out how Brooklyn determines the
health of the node. The health-check is
+often implemented in the `isRunning()` method in the entity's ssh driver.
Tomcat's implementation of `isRunning()`
+is as follows:
+
+{% highlight java %}
+@Override
+public boolean isRunning() {
+ return newScript(MutableMap.of(USE_PID_FILE, "pid.txt"),
CHECK_RUNNING).execute() == 0;
+}
+{% endhighlight %}
+
+The `newScript` method has conveniences for default scripts to check if a
process is running based on its PID. In this
+case, it will look for Tomcat's PID in the `pid.txt` file and check if the
PID is the PID of a running process
+
+It's worth a quick sanity check at this point to check if the PID file
exists, and if the process is running.
+By default, the pid file is located in the run directory of the entity.
You can find the location of the entity's run
+directory by looking at the `run.dir` sensor. In this case it is
`/tmp/brooklyn-martin/apps/jIzIHXtP/entities/TomcatServer_BUHgQeOs`.
+To find the pid, you simply cat the pid.txt file in this directory:
+
+{% highlight console %}
+$ cat
/tmp/brooklyn-martin/apps/jIzIHXtP/entities/TomcatServer_BUHgQeOs/pid.txt
+73714
+{% endhighlight %}
+
+In this case, the PID in the file is 73714. You can then check if the
process is running using `ps`. You can also
+pipe the output to `fold` so the full launch command is visible:
+
+{% highlight console %}
+
+$ ps -p 73714 | fold -w 120
+PID TTY TIME CMD
+73714 ?? 0:08.03
/Library/Java/JavaVirtualMachines/jdk1.8.0_51.jdk/Contents/Home/bin/java -Dnop
-Djava.util.logg
+ing.manager=org.apache.juli.ClassLoaderLogManager
-javaagent:/tmp/brooklyn-martin/apps/jIzIHXtP/entities/TomcatServer_BU
+HgQeOs/brooklyn-jmxmp-agent-shaded-0.8.0-SNAPSHOT.jar -Xms200m -Xmx800m
-XX:MaxPermSize=400m -Dcom.sun.management.jmxrem
+ote -Dbrooklyn.jmxmp.rmi-port=1099 -Dbrooklyn.jmxmp.port=31001
-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.manage
+ment.jmxremote.authenticate=false
-Djava.endorsed.dirs=/tmp/brooklyn-martin/installs/TomcatServer_7.0.56/apache-tomcat-7
+.0.56/endorsed -classpath
/tmp/brooklyn-martin/installs/TomcatServer_7.0.56/apache-tomcat-7.0.56/bin/bootstrap.jar:/tmp/
+brooklyn-martin/installs/TomcatServer_7.0.56/apache-tomcat-7.0.56/bin/tomcat-juli.jar
-Dcatalina.base=/tmp/brooklyn-mart
+in/apps/jIzIHXtP/entities/TomcatServer_BUHgQeOs
-Dcatalina.home=/tmp/brooklyn-martin/installs/TomcatServer_7.0.56/apache
+-tomcat-7.0.56
-Djava.io.tmpdir=/tmp/brooklyn-martin/apps/jIzIHXtP/entities/TomcatServer_BUHgQeOs/temp
org.apache.catali
+na.startup.Bootstrap start
+
+{% endhighlight %}
+
+This confirms that the process is running. The next thing we can look at
is the `service.notUp.indicators` sensor. This
+reads as follows:
+
+{% highlight json %}
+
+{"service.process.isRunning":"The software process for this entity does
not appear to be running"}
+
+{% endhighlight %}
+
+This confirms that the problem is indeed due to the
`service.process.isRunning` sensor. We assumed earlier that this was
+set by the `isRunning()` method in `TomcatSshDriver.java`, but this isn't
always the case. The `service.process.isRunning`
+sensor is wired up by the `connectSensors()` method in the node's
implementation class, in this case
+`TomcatServerImpl.java`. Tomcat's implementation of `connectSensors()` is
as follows:
+
+{% highlight java %}
+
+@Override
+public void connectSensors() {
+ super.connectSensors();
+
+ if (getDriver().isJmxEnabled()) {
+ String requestProcessorMbeanName =
"Catalina:type=GlobalRequestProcessor,name=\"http-*\"";
+
+ Integer port = isHttpsEnabled() ? getAttribute(HTTPS_PORT) :
getAttribute(HTTP_PORT);
+ String connectorMbeanName =
format("Catalina:type=Connector,port=%s", port);
+
+ jmxWebFeed = JmxFeed.builder()
+ .entity(this)
+ .period(3000, TimeUnit.MILLISECONDS)
+ .pollAttribute(new JmxAttributePollConfig<Integer>(ERROR_COUNT)
+ .objectName(requestProcessorMbeanName)
+ .attributeName("errorCount"))
+ .pollAttribute(new
JmxAttributePollConfig<Integer>(REQUEST_COUNT)
+ .objectName(requestProcessorMbeanName)
+ .attributeName("requestCount"))
+ .pollAttribute(new
JmxAttributePollConfig<Integer>(TOTAL_PROCESSING_TIME)
+ .objectName(requestProcessorMbeanName)
+ .attributeName("processingTime"))
+ .pollAttribute(new
JmxAttributePollConfig<String>(CONNECTOR_STATUS)
+ .objectName(connectorMbeanName)
+ .attributeName("stateName"))
+ .pollAttribute(new
JmxAttributePollConfig<Boolean>(SERVICE_PROCESS_IS_RUNNING)
+ .objectName(connectorMbeanName)
+ .attributeName("stateName")
+
.onSuccess(Functions.forPredicate(Predicates.<Object>equalTo("STARTED")))
+ .setOnFailureOrException(false))
+ .build();
+
+ jmxAppFeed = JavaAppUtils.connectMXBeanSensors(this);
+ } else {
+ // if not using JMX
+ LOG.warn("Tomcat running without JMX monitoring; limited
visibility of service available");
+ connectServiceUpIsRunning();
+ }
+}
+
+{% endhighlight %}
+
+We can see here that if jmx is not enabled, the method will call
`connectServiceUpIsRunning()` which will use the
+default PID-based method of determining if a process is running. However,
as JMX *is* running, the `service.process.isRunning`
+sensor (denoted here by the `SERVICE_PROCESS_IS_RUNNING` variable) is set
to true if and only if the
+`stateName` JMX attribute equals `STARTED`. We can see from the previous
call to `.pollAttribute` that this
+attribute is also published to the `CONNECTOR_STATUS` sensor. The
`CONNECTOR_STATUS` sensor is defined as follows:
+
+{% highlight java %}
+
+AttributeSensor<String> CONNECTOR_STATUS =
+ new BasicAttributeSensor<String>(String.class,
"webapp.tomcat.connectorStatus", "Catalina connector state name");
+
+{% endhighlight %}
+
+Let's go back to the Brooklyn debug console and look for the
`webapp.tomcat.connectorStatus`:
+
+[](images/jmx-sensors-large.png)
+
+As the sensor is not shown, it's likely that it's simply null or not set.
We can check this by clicking
+the `Show/hide empty records` icon (highlighted in yellow above):
--- End diff --
I genuinely didn't know this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
