[
https://issues.apache.org/jira/browse/BROOKLYN-201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Corbett updated BROOKLYN-201:
---------------------------------
Description:
I observed a strange problem in which a Tomcat server appeared to be missing
most sensor values.
Context:
Two-member dynamic cluster of Tomcat servers with identical member specs.
Member 1: GZKUwB3x
Member 2: l1LEYWIA
Each member has a mirror entity:
Member #1 mirror: cn7VfAGi
Member #2 mirror: nwVhOYzr
#1 deployed normally.
#2 failed to start:
{code}
Failed after 4ms: Execution failed, invocation error for JMX_SETUP_PREINSTALL
Tomcat8ServerImpl{id=l1LEYWIA}: jmx port must not be null for
Tomcat8ServerImpl{id=l1LEYWIA}: java.lang.NullPointerException: jmx port must
not be null for Tomcat8ServerImpl{id=l1LEYWIA}
java.lang.IllegalStateException: Execution failed, invocation error for
JMX_SETUP_PREINSTALL Tomcat8ServerImpl{id=l1LEYWIA}: jmx port must not be null
for Tomcat8ServerImpl{id=l1LEYWIA}
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.logWithDetailsAndThrow(ScriptHelper.java:387)
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:370)
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper$8.call(ScriptHelper.java:287)
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper$8.call(ScriptHelper.java:285)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:342)
at
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:468)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: jmx port must not be null for
Tomcat8ServerImpl{id=l1LEYWIA}
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:253)
at
org.apache.brooklyn.entity.java.JmxSupport.applyJmxJavaSystemProperties(JmxSupport.java:275)
at
org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJmxJavaSystemProperties(JavaSoftwareProcessSshDriver.java:258)
at
org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJavaSystemProperties(JavaSoftwareProcessSshDriver.java:205)
at
org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJavaOpts(JavaSoftwareProcessSshDriver.java:125)
at
org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getShellEnvironment(JavaSoftwareProcessSshDriver.java:105)
at
org.apache.brooklyn.entity.webapp.tomcat.TomcatSshDriver.getShellEnvironment(TomcatSshDriver.java:139)
at
org.apache.brooklyn.entity.software.base.AbstractSoftwareProcessSshDriver.execute(AbstractSoftwareProcessSshDriver.java:261)
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:366)
... 8 more
{code}
This is the command that failed:
{code}
Integer jmxPort = Preconditions.checkNotNull(entity.getAttribute(JMX_PORT),
"jmx port must not be null for %s", entity);
{code}
So for some reason the Tomcat entity with id l1LEYWIA has no jmx.port sensor.
In the debug log, before the exception is thrown:
{code}
2015-12-08 15:23:15,994 DEBUG o.a.b.c.s.PortAttributeSensorAndConfigKey
[brooklyn-execmanager-C52UZvVL-147269]: Tomcat8ServerImpl{id=l1LEYWIA}:
choosing port 31001 for jmx.direct.port
2015-12-08 15:23:16,116 DEBUG o.a.b.c.entity.AbstractEntity
[brooklyn-execmanager-C52UZvVL-147272]: Emitting sensor notification
entity.sensor.added value Sensor: jmx.direct.port (java.lang.Integer) on
DelegateEntityImpl{id=nwVhOYzr}
2015-12-08 15:23:19,537 DEBUG o.a.b.c.s.PortAttributeSensorAndConfigKey
[brooklyn-execmanager-C52UZvVL-147228]: Tomcat8ServerImpl{id=GZKUwB3x}:
choosing port 31001 for jmx.direct.port
2015-12-08 15:23:19,537 DEBUG o.a.b.c.entity.AbstractEntity
[brooklyn-execmanager-C52UZvVL-147136]: Emitting sensor notification
entity.sensor.added value Sensor: jmx.direct.port (java.lang.Integer) on
DelegateEntityImpl{id=cn7VfAGi}
{code}
But the attribute _had_ been set on the entity.
To make things more confusing, refer to the images attached to this issue.
Image 1 shows the senors for the running entity (GZKUwB3x)
Image 2 shows all sensors for the failed entity (l1LEYWIA, including those with
no value)
Image 3 shows only the sensors on the failed entity (l1LEYWIA) that have a
value set
Image 4 shows only the sensors on the failed entity's delegate (nwVhOYzr) that
have a value set.
Image 5 shows the failed task.
2 and 4 are the most interesting. The real entity has no sensors set but its
delegate does?! Note that the delegate entity has a value for jmx.direct.port
and has none of the Clocker attributes like docker.host or
docker.infrastructure.
As far as I can tell from the logs the sensor value for jmx.direct.port was not
subequently unset. No sensor notifications were emitted for it after it was
first set.
was:
I observed a strange problem in which a Tomcat server appeared to be missing
most sensor values.
Context:
Two-member dynamic cluster of Tomcat servers with identical member specs.
Member 1: GZKUwB3x
Member 2: l1LEYWIA
Each member has a mirror entity:
Member #1 mirror: cn7VfAGi
Member #2 mirror: nwVhOYzr
#1 deployed normally.
#2 failed to start:
{code}
Failed after 4ms: Execution failed, invocation error for JMX_SETUP_PREINSTALL
Tomcat8ServerImpl{id=l1LEYWIA}: jmx port must not be null for
Tomcat8ServerImpl{id=l1LEYWIA}: java.lang.NullPointerException: jmx port must
not be null for Tomcat8ServerImpl{id=l1LEYWIA}
java.lang.IllegalStateException: Execution failed, invocation error for
JMX_SETUP_PREINSTALL Tomcat8ServerImpl{id=l1LEYWIA}: jmx port must not be null
for Tomcat8ServerImpl{id=l1LEYWIA}
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.logWithDetailsAndThrow(ScriptHelper.java:387)
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:370)
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper$8.call(ScriptHelper.java:287)
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper$8.call(ScriptHelper.java:285)
at
org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:342)
at
org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:468)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: jmx port must not be null for
Tomcat8ServerImpl{id=l1LEYWIA}
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:253)
at
org.apache.brooklyn.entity.java.JmxSupport.applyJmxJavaSystemProperties(JmxSupport.java:275)
at
org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJmxJavaSystemProperties(JavaSoftwareProcessSshDriver.java:258)
at
org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJavaSystemProperties(JavaSoftwareProcessSshDriver.java:205)
at
org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJavaOpts(JavaSoftwareProcessSshDriver.java:125)
at
org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getShellEnvironment(JavaSoftwareProcessSshDriver.java:105)
at
org.apache.brooklyn.entity.webapp.tomcat.TomcatSshDriver.getShellEnvironment(TomcatSshDriver.java:139)
at
org.apache.brooklyn.entity.software.base.AbstractSoftwareProcessSshDriver.execute(AbstractSoftwareProcessSshDriver.java:261)
at
org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:366)
... 8 more
{code}
This is the command that failed:
{code}
Integer jmxPort = Preconditions.checkNotNull(entity.getAttribute(JMX_PORT),
"jmx port must not be null for %s", entity);
{code}
So for some reason the Tomcat entity with id l1LEYWIA has no jmx.port sensor.
In the debug log, before the exception is thrown:
{code}
2015-12-08 15:23:15,994 DEBUG o.a.b.c.s.PortAttributeSensorAndConfigKey
[brooklyn-execmanager-C52UZvVL-147269]: Tomcat8ServerImpl{id=l1LEYWIA}:
choosing port 31001 for jmx.direct.port
2015-12-08 15:23:16,116 DEBUG o.a.b.c.entity.AbstractEntity
[brooklyn-execmanager-C52UZvVL-147272]: Emitting sensor notification
entity.sensor.added value Sensor: jmx.direct.port (java.lang.Integer) on
DelegateEntityImpl{id=nwVhOYzr}
2015-12-08 15:23:19,537 DEBUG o.a.b.c.s.PortAttributeSensorAndConfigKey
[brooklyn-execmanager-C52UZvVL-147228]: Tomcat8ServerImpl{id=GZKUwB3x}:
choosing port 31001 for jmx.direct.port
2015-12-08 15:23:19,537 DEBUG o.a.b.c.entity.AbstractEntity
[brooklyn-execmanager-C52UZvVL-147136]: Emitting sensor notification
entity.sensor.added value Sensor: jmx.direct.port (java.lang.Integer) on
DelegateEntityImpl{id=cn7VfAGi}
{code}
But the attribute _had_ been set on the entity.
To make things more confusing, refer to the images attached to this issue.
Image 1 shows the senors for the running entity (GZKUwB3x)
Image 2 shows all sensors for the failed entity (l1LEYWIA, including those with
no value)
Image 3 shows only the sensors on the failed entity (l1LEYWIA) that have a
value set
Image 4 shows only the sensors on the failed entity's delegate (nwVhOYzr) that
have a value set.
Image 5 shows the failed task.
3 and 4 are the most interesting. The real entity has no sensors set but its
delegate does?! Note that the delegate entity has a value for jmx.direct.port
and has none of the Clocker attributes like docker.host or
docker.infrastructure.
As far as I can tell from the logs the sensor value for jmx.direct.port was not
subequently unset. No sensor notifications were emitted for it after it was
first set.
> Failure retrieving port sensor value for TomcatServer
> -----------------------------------------------------
>
> Key: BROOKLYN-201
> URL: https://issues.apache.org/jira/browse/BROOKLYN-201
> Project: Brooklyn
> Issue Type: Bug
> Reporter: Sam Corbett
> Attachments: 1-running-entity-sensors.png,
> 2-failed-entity-all-sensors.png, 3-failed-entity-sensors-with-value.png,
> 4-failed-entity-sensors-with-value-under-docker-app.png, 5-task-error.png
>
>
> I observed a strange problem in which a Tomcat server appeared to be missing
> most sensor values.
> Context:
> Two-member dynamic cluster of Tomcat servers with identical member specs.
> Member 1: GZKUwB3x
> Member 2: l1LEYWIA
> Each member has a mirror entity:
> Member #1 mirror: cn7VfAGi
> Member #2 mirror: nwVhOYzr
> #1 deployed normally.
> #2 failed to start:
> {code}
> Failed after 4ms: Execution failed, invocation error for JMX_SETUP_PREINSTALL
> Tomcat8ServerImpl{id=l1LEYWIA}: jmx port must not be null for
> Tomcat8ServerImpl{id=l1LEYWIA}: java.lang.NullPointerException: jmx port must
> not be null for Tomcat8ServerImpl{id=l1LEYWIA}
> java.lang.IllegalStateException: Execution failed, invocation error for
> JMX_SETUP_PREINSTALL Tomcat8ServerImpl{id=l1LEYWIA}: jmx port must not be
> null for Tomcat8ServerImpl{id=l1LEYWIA}
> at
> org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.logWithDetailsAndThrow(ScriptHelper.java:387)
> at
> org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:370)
> at
> org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper$8.call(ScriptHelper.java:287)
> at
> org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper$8.call(ScriptHelper.java:285)
> at
> org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:342)
> at
> org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:468)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException: jmx port must not be null for
> Tomcat8ServerImpl{id=l1LEYWIA}
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:253)
> at
> org.apache.brooklyn.entity.java.JmxSupport.applyJmxJavaSystemProperties(JmxSupport.java:275)
> at
> org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJmxJavaSystemProperties(JavaSoftwareProcessSshDriver.java:258)
> at
> org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJavaSystemProperties(JavaSoftwareProcessSshDriver.java:205)
> at
> org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getJavaOpts(JavaSoftwareProcessSshDriver.java:125)
> at
> org.apache.brooklyn.entity.java.JavaSoftwareProcessSshDriver.getShellEnvironment(JavaSoftwareProcessSshDriver.java:105)
> at
> org.apache.brooklyn.entity.webapp.tomcat.TomcatSshDriver.getShellEnvironment(TomcatSshDriver.java:139)
> at
> org.apache.brooklyn.entity.software.base.AbstractSoftwareProcessSshDriver.execute(AbstractSoftwareProcessSshDriver.java:261)
> at
> org.apache.brooklyn.entity.software.base.lifecycle.ScriptHelper.executeInternal(ScriptHelper.java:366)
> ... 8 more
> {code}
> This is the command that failed:
> {code}
> Integer jmxPort = Preconditions.checkNotNull(entity.getAttribute(JMX_PORT),
> "jmx port must not be null for %s", entity);
> {code}
> So for some reason the Tomcat entity with id l1LEYWIA has no jmx.port sensor.
> In the debug log, before the exception is thrown:
> {code}
> 2015-12-08 15:23:15,994 DEBUG o.a.b.c.s.PortAttributeSensorAndConfigKey
> [brooklyn-execmanager-C52UZvVL-147269]: Tomcat8ServerImpl{id=l1LEYWIA}:
> choosing port 31001 for jmx.direct.port
> 2015-12-08 15:23:16,116 DEBUG o.a.b.c.entity.AbstractEntity
> [brooklyn-execmanager-C52UZvVL-147272]: Emitting sensor notification
> entity.sensor.added value Sensor: jmx.direct.port (java.lang.Integer) on
> DelegateEntityImpl{id=nwVhOYzr}
> 2015-12-08 15:23:19,537 DEBUG o.a.b.c.s.PortAttributeSensorAndConfigKey
> [brooklyn-execmanager-C52UZvVL-147228]: Tomcat8ServerImpl{id=GZKUwB3x}:
> choosing port 31001 for jmx.direct.port
> 2015-12-08 15:23:19,537 DEBUG o.a.b.c.entity.AbstractEntity
> [brooklyn-execmanager-C52UZvVL-147136]: Emitting sensor notification
> entity.sensor.added value Sensor: jmx.direct.port (java.lang.Integer) on
> DelegateEntityImpl{id=cn7VfAGi}
> {code}
> But the attribute _had_ been set on the entity.
> To make things more confusing, refer to the images attached to this issue.
> Image 1 shows the senors for the running entity (GZKUwB3x)
> Image 2 shows all sensors for the failed entity (l1LEYWIA, including those
> with no value)
> Image 3 shows only the sensors on the failed entity (l1LEYWIA) that have a
> value set
> Image 4 shows only the sensors on the failed entity's delegate (nwVhOYzr)
> that have a value set.
> Image 5 shows the failed task.
> 2 and 4 are the most interesting. The real entity has no sensors set but its
> delegate does?! Note that the delegate entity has a value for jmx.direct.port
> and has none of the Clocker attributes like docker.host or
> docker.infrastructure.
> As far as I can tell from the logs the sensor value for jmx.direct.port was
> not subequently unset. No sensor notifications were emitted for it after it
> was first set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
