Julian Chandra Sutadi created ZOOKEEPER-5110:
------------------------------------------------
Summary: AdminServer "monitor" command fails with HTTP 500
(NullPointerException) while the peer is not serving
Key: ZOOKEEPER-5110
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-5110
Project: ZooKeeper
Issue Type: Bug
Reporter: Julian Chandra Sutadi
ZOOKEEPER-3353 made the "monitor" admin command server-optional, so that it can
run even when no ZooKeeperServer exists (e.g. while a peer is in leader
election), and guarded the server-specific metrics with a null check:
{code:java}
// ZOOKEEPER-3353: MonitorCommand.run(...)
response.put("version", Version.getFullVersion());
...
if (zkServer != null) {
// server-specific metrics
...
}
{code}
The refactor done by ZOOKEEPER-3366 kept the monitor admin command
server-optional (i.e. can be run even when no ZooKeeperServer exists). However,
now it invokes the zkServer.dumpMonitorValues(...) without checking if
(zkServer != null), which was the guard added by ZOOKEEPER-3353.
{code:java}
// Commands.MonitorCommand (current master)
public MonitorCommand() {
super(Arrays.asList("monitor", "mntr"), false); // false =
server-optional
}
@Override
public CommandResponse runGet(ZooKeeperServer zkServer, Map<String, String>
kwargs) {
CommandResponse response = initializeResponse();
zkServer.dumpMonitorValues(response::put); // NPE when zkServer ==
null
ServerMetrics.getMetrics().getMetricsProvider().dump(response::put);
return response;
}
{code}
Because the command is server-optional, Commands.runCommand skips its "not
currently serving" check and passes a null server to it:
{code:java}
private static CommandResponse runCommand(...) {
if (command.isServerRequired() && (zkServer == null ||
!zkServer.isRunning())) { // skipped because command.isServerRequired() is
false
// set the status code to 200 to keep the current behavior of existing
commands
LOG.warn("This ZooKeeper instance is not currently serving requests for
command");
return new CommandResponse(cmdName, "This ZooKeeper instance is not
currently serving requests", HttpServletResponse.SC_OK);
}
...
return isGet ? command.runGet(zkServer, kwargs) : command.runPost(zkServer,
inputStream);
}
{code}
*Steps to reproduce*
1. Configure a zookeeper ensemble
2. Start one server and leave the others stopped. This simulatse a peer in a
running ensemble after it loses quorum and has no ZooKeeperServer object.
3. Run {{curl http://<host>:8080/commands/monitor}}. It returns HTTP 500:
{noformat}
java.lang.NullPointerException: Cannot invoke
"org.apache.zookeeper.server.ZooKeeperServer.dumpMonitorValues(java.util.function.BiConsumer)"
because "zkServer" is null
{noformat}
4. Run {{curl http://<host>:8080/commands/configuration}}, which is a
server-required command. As a comparison, it returns HTTP 200 with {{"error" :
"This ZooKeeper instance is not currently serving requests"}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
