[jira] [Commented] (SOLR-13712) JMX MBeans are not exposed because of race condition between creating platform mbean server and registering mbeans
[
https://issues.apache.org/jira/browse/SOLR-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915477#comment-16915477
]
Shalin Shekhar Mangar commented on SOLR-13712:
--
bq. It sounds like a workaround may be to configure log4j2 to register it's
mbeans earlier.
Yes, that should workaround this bug. I'll be interested to know what you find.
Thanks!
> JMX MBeans are not exposed because of race condition between creating
> platform mbean server and registering mbeans
> --
>
> Key: SOLR-13712
> URL: https://issues.apache.org/jira/browse/SOLR-13712
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: JMX
>Affects Versions: 6.6, 6.6.2, 6.6.5, 7.7.2, 8.2, 8.1.1
>Reporter: Shalin Shekhar Mangar
>Priority: Major
> Attachments: SOLR-13712.patch
>
>
> This is quite easy to reproduce.
> {code}
> wget https://archive.apache.org/dist/lucene/solr/6.6.0/solr-6.6.0.tgz
> tar xvf solr-6.6.0.tgz
> cd solr-6.6.0
> {code}
> Enable jmx reporting by editing the server/solr/solr.xml and adding the
> following under the "" tag:
> {code}
>
> class="org.apache.solr.metrics.reporters.SolrJmxReporter" />
>
> {code}
> Start solr with:
> {code}
> ./bin/solr start -e cloud -noprompt
> {code}
> Open jconsole and inspect mbeans for solr nodes running on port 8983 and
> 7574. You'll find that all mbeans (node, jvm, jetty, solr) are present for
> the solr on port 8983 but completely absent for the solr node running on port
> 7574.
> Same behavior is on 6.6.2 and 6.6.6. However, Solr 7.x and 8.x seem to be
> fine.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13712) JMX MBeans are not exposed because of race condition between creating platform mbean server and registering mbeans
[
https://issues.apache.org/jira/browse/SOLR-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915170#comment-16915170
]
Lucene/Solr QA commented on SOLR-13712:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 49m 58s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.HttpPartitionTest |
| | solr.handler.admin.TestApiFramework |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13712 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978516/SOLR-13712.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 225fd3d |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| unit |
https://builds.apache.org/job/PreCommit-SOLR-Build/539/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/539/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/539/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> JMX MBeans are not exposed because of race condition between creating
> platform mbean server and registering mbeans
> --
>
> Key: SOLR-13712
> URL: https://issues.apache.org/jira/browse/SOLR-13712
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: JMX
>Affects Versions: 6.6, 6.6.2, 6.6.5, 7.7.2, 8.2, 8.1.1
>Reporter: Shalin Shekhar Mangar
>Priority: Major
> Attachments: SOLR-13712.patch
>
>
> This is quite easy to reproduce.
> {code}
> wget https://archive.apache.org/dist/lucene/solr/6.6.0/solr-6.6.0.tgz
> tar xvf solr-6.6.0.tgz
> cd solr-6.6.0
> {code}
> Enable jmx reporting by editing the server/solr/solr.xml and adding the
> following under the "" tag:
> {code}
>
> class="org.apache.solr.metrics.reporters.SolrJmxReporter" />
>
> {code}
> Start solr with:
> {code}
> ./bin/solr start -e cloud -noprompt
> {code}
> Open jconsole and inspect mbeans for solr nodes running on port 8983 and
> 7574. You'll find that all mbeans (node, jvm, jetty, solr) are present for
> the solr on port 8983 but completely absent for the solr node running on port
> 7574.
> Same behavior is on 6.6.2 and 6.6.6. However, Solr 7.x and 8.x seem to be
> fine.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13712) JMX MBeans are not exposed because of race condition between creating platform mbean server and registering mbeans
[
https://issues.apache.org/jira/browse/SOLR-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915126#comment-16915126
]
Shalin Shekhar Mangar commented on SOLR-13712:
--
Patch for master which returns {{ManagementFactory.getPlatformMBeanServer()}}
inside JmxUtils.findFirstMBeanServer() if no mbean servers exist.
> JMX MBeans are not exposed because of race condition between creating
> platform mbean server and registering mbeans
> --
>
> Key: SOLR-13712
> URL: https://issues.apache.org/jira/browse/SOLR-13712
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: JMX
>Affects Versions: 6.6, 6.6.2, 6.6.5, 7.7.2, 8.2, 8.1.1
>Reporter: Shalin Shekhar Mangar
>Priority: Major
> Attachments: SOLR-13712.patch
>
>
> This is quite easy to reproduce.
> {code}
> wget https://archive.apache.org/dist/lucene/solr/6.6.0/solr-6.6.0.tgz
> tar xvf solr-6.6.0.tgz
> cd solr-6.6.0
> {code}
> Enable jmx reporting by editing the server/solr/solr.xml and adding the
> following under the "" tag:
> {code}
>
> class="org.apache.solr.metrics.reporters.SolrJmxReporter" />
>
> {code}
> Start solr with:
> {code}
> ./bin/solr start -e cloud -noprompt
> {code}
> Open jconsole and inspect mbeans for solr nodes running on port 8983 and
> 7574. You'll find that all mbeans (node, jvm, jetty, solr) are present for
> the solr on port 8983 but completely absent for the solr node running on port
> 7574.
> Same behavior is on 6.6.2 and 6.6.6. However, Solr 7.x and 8.x seem to be
> fine.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13712) JMX MBeans are not exposed because of race condition between creating platform mbean server and registering mbeans
[
https://issues.apache.org/jira/browse/SOLR-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914380#comment-16914380
]
Jeremy Branham commented on SOLR-13712:
---
Thanks for tracking this down -
It sounds like a workaround may be to configure log4j2 to register it's mbeans
earlier.
I'm not sure if this can be done through configuration, but I'll do some tests
and post my results.
> JMX MBeans are not exposed because of race condition between creating
> platform mbean server and registering mbeans
> --
>
> Key: SOLR-13712
> URL: https://issues.apache.org/jira/browse/SOLR-13712
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: JMX
>Affects Versions: 6.6, 6.6.2, 6.6.5, 7.7.2, 8.2, 8.1.1
>Reporter: Shalin Shekhar Mangar
>Priority: Major
>
> This is quite easy to reproduce.
> {code}
> wget https://archive.apache.org/dist/lucene/solr/6.6.0/solr-6.6.0.tgz
> tar xvf solr-6.6.0.tgz
> cd solr-6.6.0
> {code}
> Enable jmx reporting by editing the server/solr/solr.xml and adding the
> following under the "" tag:
> {code}
>
> class="org.apache.solr.metrics.reporters.SolrJmxReporter" />
>
> {code}
> Start solr with:
> {code}
> ./bin/solr start -e cloud -noprompt
> {code}
> Open jconsole and inspect mbeans for solr nodes running on port 8983 and
> 7574. You'll find that all mbeans (node, jvm, jetty, solr) are present for
> the solr on port 8983 but completely absent for the solr node running on port
> 7574.
> Same behavior is on 6.6.2 and 6.6.6. However, Solr 7.x and 8.x seem to be
> fine.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13712) JMX MBeans are not exposed because of race condition between creating platform mbean server and registering mbeans
[
https://issues.apache.org/jira/browse/SOLR-13712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913947#comment-16913947
]
Shalin Shekhar Mangar commented on SOLR-13712:
--
I tracked down the problem to SolrJmxReporter.
In the absence of any agentId or serviceUrl, the platform mbean server is
supposed to be used. The SolrJmxReporter.init (or doInit() method in later
versions) first tries to get the first mbean server and registers mbeans to it.
However, it exits early if no mbean server is found. Attaching a debugger to
the solr processes I found that the platform mbean server is always found for
the solr node running on port 8983 but not for solr on port 7574.
The problem is that the platform mbean server is created on the first
invocation of {{ManagementFactory.getPlatformMBeanServer()}} and
SolrJmxReporter calls this method in JmxMetricsReporter.build() which happens
after the mbean server is looked up. If no mbean server is found then the call
to build() never happens and therefore no mbeans are registered with the mbean
server.
So why does the mbean server exist on 8983? It is because of the embedded
zookeeper which calls {{ManagementFactory.getPlatformMBeanServer()}} before
solr tries to initialize SolrJmxReporter.
The code is virtually unchanged in Solr 7.x and 8.x. Then why do those versions
seem fine? It is because of log4j2 which registers its mbeans with the platform
mbean server before Solr initializes.
> JMX MBeans are not exposed because of race condition between creating
> platform mbean server and registering mbeans
> --
>
> Key: SOLR-13712
> URL: https://issues.apache.org/jira/browse/SOLR-13712
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: JMX
>Affects Versions: 6.6, 6.6.2, 6.6.5, 7.7.2, 8.2, 8.1.1
>Reporter: Shalin Shekhar Mangar
>Priority: Major
>
> This is quite easy to reproduce.
> {code}
> wget https://archive.apache.org/dist/lucene/solr/6.6.0/solr-6.6.0.tgz
> tar xvf solr-6.6.0.tgz
> cd solr-6.6.0
> {code}
> Enable jmx reporting by editing the server/solr/solr.xml and adding the
> following under the "" tag:
> {code}
>
> class="org.apache.solr.metrics.reporters.SolrJmxReporter" />
>
> {code}
> Start solr with:
> {code}
> ./bin/solr start -e cloud -noprompt
> {code}
> Open jconsole and inspect mbeans for solr nodes running on port 8983 and
> 7574. You'll find that all mbeans (node, jvm, jetty, solr) are present for
> the solr on port 8983 but completely absent for the solr node running on port
> 7574.
> Same behavior is on 6.6.2 and 6.6.6. However, Solr 7.x and 8.x seem to be
> fine.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
