[
https://issues.apache.org/jira/browse/CASSANDRA-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759738#comment-17759738
]
Stefan Miklosovic commented on CASSANDRA-18572:
-----------------------------------------------
I started to implement this after CASSANDRA-18752 is in but .... I wanted to
fix this from 3.11 to trunk, 3.11 seems OK but 4.0 is where the problem starts.
To explain the issue, I prepared this commit on top of the 4.0 PR (1).
Without (2), HintedHandoffNodetoolTest was failing. I prepared another test,
SimpleTest, which isolates this problem and it is easier to debug.
The reason it fails without it is that as we are starting Instance in startup
method, we are actually not _starting_ Cassandra, it is a fake startup. In that
startup method, we are dealing with StorageService only (or, mostly), so that
means that its static "instance" field is initialized as well which in turn
means that it is registered as an MBean.
However, there is no such interaction hence no initialization of StorageProxy
so StorageProxyMBean is not registered either which means that as we want to
call a command via NodeTool, NodeProbe which creates MBean proxies in its
"connect" method will not be able to do that - because nothing registered such
MBean on the server side.
The solution seems to be to eagerly register all MBeans as part of startup of
Instance. This likely means that we also need to harden this initialization
because some implementations of MBeans are not checking if such MBean is
already registered or not so if we register it eagerly, it might be registered
for the second time as the startup method logic progresses which might throw
errors as well.
After a closer look, I also see there are some strange interdependencies among
these MBeans. For example StorageProxy in its constructor (actually, in a
static block), registers not only StorageProxy itself but also HintsService
(???). I can imagine there will be a lot of cases like this over the codebase
and we should be very careful what we do here.
Due to complexity of this issue I think we should just focus on 5.0 and trunk
and leave other branches untouched. It does not make sense to do any big
refactoring across the codebase like this.
(1) https://github.com/apache/cassandra/pull/2631/commits
(2)
https://github.com/apache/cassandra/pull/2631/commits/793aa07899b7fa0ea1409beb37663ef2b4d567c8
> Instance.nodetoolResult should connect to JMX if there is such feature
> enabled in its config
> --------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-18572
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18572
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
> Reporter: Stefan Miklosovic
> Assignee: Stefan Miklosovic
> Priority: Normal
> Fix For: 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: fix-jmx-issue-on-shutdown.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Instance.nodetoolResult is not using JMX internally but it throws
> UnsupportedOperationException in InternalNodeProbe for some nodetool comamnds.
> This makes calls to instance.nodetoolResult("info"), for example, impossible,
> because these commands are using JMX / calling methods which are throwing
> exception and it makes the testing painful.
> There is no reason to not support this if Instance has JMX feature as it was
> added recently.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
