This is an automated email from the ASF dual-hosted git repository.
FrankChen021 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new a8ea1a4f9ed refactor!: remove deprecated SysMonitor (#20200)
a8ea1a4f9ed is described below
commit a8ea1a4f9edf4ce8a14739f70bad28080936f229
Author: Frank Chen <[email protected]>
AuthorDate: Tue Sep 8 09:57:28 2026 +0800
refactor!: remove deprecated SysMonitor (#20200)
* refactor!: remove deprecated SysMonitor
* refactor: remove remaining Sigar usage
* docs: remove upgrade notes entry
* refactor: remove Sigar PID discoverer
* fix: preserve JVM CPU baselines on process lookup failure
* docs: explain JVM process lookup
---
docs/configuration/index.md | 1 -
docs/operations/metrics.md | 28 +-
licenses.yaml | 129 -----
pom.xml | 9 -
processing/pom.xml | 38 --
.../druid/java/util/metrics/JvmCpuMonitor.java | 68 ++-
.../druid/java/util/metrics/JvmPidDiscoverer.java | 35 +-
.../druid/java/util/metrics/NoopSysMonitor.java | 36 --
.../druid/java/util/metrics/OshiSysMonitor.java | 2 +-
.../java/util/metrics/SigarPidDiscoverer.java | 44 --
.../apache/druid/java/util/metrics/SigarUtil.java | 84 ---
.../apache/druid/java/util/metrics/SysMonitor.java | 577 ---------------------
.../druid/java/util/metrics/JvmCpuMonitorTest.java | 98 ++++
.../java/util/metrics/NoopSysMonitorTest.java | 42 --
.../druid/java/util/metrics/SigarLoadTest.java | 45 --
.../java/util/metrics/SigarPidDiscovererTest.java | 38 --
.../druid/server/metrics/DruidSysMonitor.java | 47 --
.../apache/druid/server/metrics/MetricsModule.java | 13 -
.../druid/server/metrics/MetricsModuleTest.java | 36 --
19 files changed, 146 insertions(+), 1224 deletions(-)
diff --git a/docs/configuration/index.md b/docs/configuration/index.md
index 0bbc7fc7958..c4def258739 100644
--- a/docs/configuration/index.md
+++ b/docs/configuration/index.md
@@ -105,7 +105,6 @@ There are four JVM parameters that we set on all of our
services:
* The temp directory should not be volatile tmpfs.
* This directory should also have good read and write speed.
* Avoid NFS mount.
- * The `org.apache.druid.java.util.metrics.SysMonitor` requires execute
privileges on files in `java.io.tmpdir`. If you are using the system monitor,
do not set `java.io.tmpdir` to `noexec`.
* `-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager` This
allows log4j2 to handle logs for non-log4j2 components (like jetty) which use
standard java logging.
### Extensions
diff --git a/docs/operations/metrics.md b/docs/operations/metrics.md
index f478889f19b..8cad874d1a4 100644
--- a/docs/operations/metrics.md
+++ b/docs/operations/metrics.md
@@ -137,7 +137,7 @@ Most metric values reset each emission period, as specified
in `druid.monitoring
### Real-time
:::info
-Monitors on peons that previously emitted the `id` dimension from
`JettyMonitor`, `OshiSysMonitor`, `JvmMonitor`, `JvmCpuMonitor`,
`JvmThreadsMonitor` and `SysMonitor`
+Monitors on peons that previously emitted the `id` dimension from
`JettyMonitor`, `OshiSysMonitor`, `JvmMonitor`, `JvmCpuMonitor` and
`JvmThreadsMonitor`
to represent the task ID are deprecated and will be removed in a future
release. Use the `taskId` dimension instead.
:::
@@ -574,32 +574,6 @@ These metrics are available only when
`druid.zk.service.enabled = true`.
|`zk/connected`|Indicator of connection status. `1` for connected, `0` for
disconnected. Emitted once per monitor period.|None|1|
|`zk/reconnect/time`|Amount of time, in milliseconds, that a server was
disconnected from ZooKeeper before reconnecting. Emitted on reconnection. Not
emitted if connection to ZooKeeper is permanently lost, because in this case,
there is no reconnection.|None|Not present|
-## Sys [Deprecated]
-
-> SysMonitor is now deprecated and will be removed in future releases.
-> Instead, use the new OSHI monitor called [OshiSysMonitor](#oshisysmonitor).
The new monitor has a wider support for different machine architectures
including ARM instances.
-
-These metrics are only available if the `SysMonitor` module is included.
-
-|Metric|Description|Dimensions|Normal value|
-|------|-----------|----------|------------|
-|`sys/swap/free`|Free swap||Varies|
-|`sys/swap/max`|Max swap||Varies|
-|`sys/swap/pageIn`|Paged in swap||Varies|
-|`sys/swap/pageOut`|Paged out swap||Varies|
-|`sys/disk/write/count`|Writes to disk|`fsDevName`, `fsDirName`, `fsTypeName`,
`fsSysTypeName`, `fsOptions`|Varies|
-|`sys/disk/read/count`|Reads from disk|`fsDevName`, `fsDirName`, `fsTypeName`,
`fsSysTypeName`, `fsOptions`|Varies|
-|`sys/disk/write/size`|Bytes written to disk. One indicator of the amount of
paging occurring for segments.|`fsDevName`,`fsDirName`,`fsTypeName`,
`fsSysTypeName`, `fsOptions`|Varies|
-|`sys/disk/read/size`|Bytes read from disk. One indicator of the amount of
paging occurring for segments.|`fsDevName`,`fsDirName`, `fsTypeName`,
`fsSysTypeName`, `fsOptions`|Varies|
-|`sys/net/write/size`|Bytes written to the network|`netName`, `netAddress`,
`netHwaddr`|Varies|
-|`sys/net/read/size`|Bytes read from the network|`netName`, `netAddress`,
`netHwaddr`|Varies|
-|`sys/fs/used`|Filesystem bytes used|`fsDevName`, `fsDirName`, `fsTypeName`,
`fsSysTypeName`, `fsOptions`|< max|
-|`sys/fs/max`|Filesystem bytes max|`fsDevName`, `fsDirName`, `fsTypeName`,
`fsSysTypeName`, `fsOptions`|Varies|
-|`sys/mem/used`|Memory used||< max|
-|`sys/mem/max`|Memory max||Varies|
-|`sys/storage/used`|Disk space used|`fsDirName`|Varies|
-|`sys/cpu`|CPU used|`cpuName`, `cpuTime`|Varies|
-
## OshiSysMonitor
These metrics are only available if the `OshiSysMonitor` module is included.
diff --git a/licenses.yaml b/licenses.yaml
index 6bc163d5ffe..bd7bd7c74f4 100644
--- a/licenses.yaml
+++ b/licenses.yaml
@@ -2602,135 +2602,6 @@ libraries:
---
-name: SIGAR
-license_category: binary
-module: java-core
-license_name: Apache License version 2.0
-version: 1.6.5.132-6
-libraries:
- - org.hyperic: sigar
-notices:
- - sigar: |
- Copyright (c) 2004-2011 VMware, Inc.
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-
- ADDITIONAL LICENSE INFORMATION:
-
- Hyperic SIGAR includes some third-party open source components
- in its distribution. The list below identifies the community or
- organization and links to their appropriate license terms.
-
- The Hyperic team would like to thank all the communities
- of the projects listed below for their contributions.
-
- ----------------------------------------------------------
- Components under the Apache License 2.0:
- ----------------------------------------------------------
-
- The following components are included without modification:
-
- - log4j -
- Information: http://logging.apache.org/
- License: http://www.apache.org/licenses/LICENSE-2.0
-
- The following components are included with modification:
-
- - cpptasks -
- Information: http://ant-contrib.sourceforge.net/
- License: http://www.apache.org/licenses/LICENSE-2.0
-
- - (portions of) APR -
- Information: http://apr.apache.org/
- License: http://www.apache.org/licenses/LICENSE-2.0
-
- ----------------------------------------------------------
- Components under BSD/MIT Style Licenses:
- ----------------------------------------------------------
-
- The following components are included with modification:
-
- - solaris get_mib2 -
- Information: ftp://vic.cc.purdue.edu/pub/tools/unix/solaris/get_mib2/
- License: within src/os/solaris/get_mib2.[ch]
-
- Copyright 1995 Purdue Research Foundation, West Lafayette, Indiana
- 47907. All rights reserved.
-
- Written by Victor A. Abell <[email protected]>
-
- This software is not subject to any license of the American Telephone
- and Telegraph Company or the Regents of the University of California.
-
- Permission is granted to anyone to use this software for any purpose on
- any computer system, and to alter it and redistribute it freely, subject
- to the following restrictions:
-
- 1. Neither Victor A Abell nor Purdue University are responsible for
- any consequences of the use of this software.
-
- 2. The origin of this software must not be misrepresented, either by
- explicit claim or by omission. Credit to Victor A. Abell and Purdue
- University must appear in documentation and sources.
-
- 3. Altered versions must be plainly marked as such, and must not be
- misrepresented as being the original software.
-
- 4. This notice may not be removed or altered.
-
- - getline by Chris Thewalt -
- Information: http://tinyurl.com/r438r
- License: within src/sigar_getline.c
-
- Copyright (C) 1991, 1992 by Chris Thewalt ([email protected])
-
- Permission to use, copy, modify, and distribute this software
- for any purpose and without fee is hereby granted, provided
- that the above copyright notices appear in all copies and that both the
- copyright notice and this permission notice appear in supporting
- documentation. This software is provided "as is" without express or
- implied warranty.
-
- - PrintfFormat.java -
- Information:
http://java.sun.com/developer/technicalArticles/Programming/sprintf/PrintfFormat.java
- License: within
bindings/java/src/org/hyperic/sigar/util/PrintfFormat.java
-
- (c) 2000 Sun Microsystems, Inc.
- ALL RIGHTS RESERVED
-
- License Grant-
-
- Permission to use, copy, modify, and distribute this Software and its
- documentation for NON-COMMERCIAL or COMMERCIAL purposes and without fee
is
- hereby granted.
-
- This Software is provided "AS IS". All express warranties, including any
- implied warranty of merchantability, satisfactory quality, fitness for a
- particular purpose, or non-infringement, are disclaimed, except to the
extent
- that such disclaimers are held to be legally invalid.
-
- You acknowledge that Software is not designed, licensed or intended for
use in
- the design, construction, operation or maintenance of any nuclear
facility
- ("High Risk Activities"). Sun disclaims any express or implied warranty
of
- fitness for such uses.
-
- Please refer to the file http://www.sun.com/policies/trademarks/ for
further
- important trademark information and to
- http://java.sun.com/nav/business/index.html for further important
licensing
- information for the Java Technology.
-
----
-
name: OSHI
license_category: binary
module: java-core
diff --git a/pom.xml b/pom.xml
index 130fe1600fd..354881be817 100644
--- a/pom.xml
+++ b/pom.xml
@@ -291,15 +291,6 @@
</snapshots>
</repository>
- <!-- Only used by core, but moved to root for parallel build
dependency resolution -->
- <repository>
- <id>sigar</id>
-
<url>https://repository.mulesoft.org/nexus/content/repositories/public</url>
- <snapshots>
- <enabled>false</enabled>
- </snapshots>
- </repository>
-
<repository>
<id>cflt-public</id>
<url>https://packages.confluent.io/maven/</url>
diff --git a/processing/pom.xml b/processing/pom.xml
index 26a278c7e26..03afcfda21b 100644
--- a/processing/pom.xml
+++ b/processing/pom.xml
@@ -34,8 +34,6 @@
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- <sigar.base.version>1.6.5</sigar.base.version>
- <sigar.version>1.6.5.132-6</sigar.version>
<oshi.version>7.6.1</oshi.version>
<mariadb.legacy.version>2.7.3</mariadb.legacy.version>
</properties>
@@ -321,21 +319,6 @@
<artifactId>async-http-client</artifactId>
</dependency>
- <!-- Extra dependencies for server-metrics -->
- <dependency>
- <groupId>org.hyperic</groupId>
- <artifactId>sigar</artifactId>
- <version>${sigar.version}</version>
- </dependency>
- <dependency>
- <groupId>org.hyperic</groupId>
- <artifactId>sigar-dist</artifactId>
- <version>${sigar.version}</version>
- <type>zip</type>
- <!-- "Provided" because this dependency is used only during the build
itself: some files from this
- dependency are copied as resources. See maven-dependency-plugin
configuration and <resources> below. -->
- <scope>provided</scope>
- </dependency>
<dependency>
<groupId>com.github.oshi</groupId>
<artifactId>oshi-core</artifactId>
@@ -495,22 +478,6 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
- <execution>
- <id>copy-sigar-lib-to-resources</id>
- <phase>generate-resources</phase>
- <goals>
- <goal>unpack-dependencies</goal>
- </goals>
- <configuration>
- <includeGroupIds>org.hyperic</includeGroupIds>
- <includeArtifactIds>sigar-dist</includeArtifactIds>
- <includes>**/sigar-bin/lib/*</includes>
- <excludes>**/sigar-bin/lib/*jar</excludes>
- <outputDirectory>
- ${project.build.directory}
- </outputDirectory>
- </configuration>
- </execution>
<execution>
<id>copy-mariadb-legacy-test-driver</id>
<phase>generate-test-resources</phase>
@@ -566,11 +533,6 @@
<include>loggingEmitterAllowedMetrics.json</include>
</includes>
</resource>
- <resource>
- <directory>
-
${project.build.directory}/hyperic-sigar-${sigar.base.version}/sigar-bin/lib
- </directory>
- </resource>
</resources>
</build>
diff --git
a/processing/src/main/java/org/apache/druid/java/util/metrics/JvmCpuMonitor.java
b/processing/src/main/java/org/apache/druid/java/util/metrics/JvmCpuMonitor.java
index 606be1daba9..994fd4737be 100644
---
a/processing/src/main/java/org/apache/druid/java/util/metrics/JvmCpuMonitor.java
+++
b/processing/src/main/java/org/apache/druid/java/util/metrics/JvmCpuMonitor.java
@@ -19,13 +19,14 @@
package org.apache.druid.java.util.metrics;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableMap;
import org.apache.druid.java.util.common.logger.Logger;
import org.apache.druid.java.util.emitter.service.ServiceEmitter;
import org.apache.druid.java.util.emitter.service.ServiceMetricEvent;
-import org.hyperic.sigar.ProcCpu;
-import org.hyperic.sigar.Sigar;
-import org.hyperic.sigar.SigarException;
+import oshi.SystemInfo;
+import oshi.software.os.OSProcess;
+import oshi.software.os.OperatingSystem;
import java.util.Map;
@@ -33,10 +34,10 @@ public class JvmCpuMonitor extends FeedDefiningMonitor
{
private static final Logger log = new Logger(JvmCpuMonitor.class);
- private final Sigar sigar = SigarUtil.getSigar();
- private final long currentProcessId = sigar.getPid();
-
+ private final OperatingSystem operatingSystem;
+ private final int currentProcessId;
private final KeyedDiff diff = new KeyedDiff();
+ private OSProcess previousProcess;
public JvmCpuMonitor()
{
@@ -44,35 +45,52 @@ public class JvmCpuMonitor extends FeedDefiningMonitor
}
public JvmCpuMonitor(String feed)
+ {
+ this(feed, new SystemInfo().getOperatingSystem());
+ }
+
+ @VisibleForTesting
+ JvmCpuMonitor(String feed, OperatingSystem operatingSystem)
{
super(feed);
+ this.operatingSystem = operatingSystem;
+ this.currentProcessId = operatingSystem.getProcessId();
+ this.previousProcess = operatingSystem.getProcess(currentProcessId);
}
@Override
public boolean doMonitor(ServiceEmitter emitter)
{
- // process CPU
- try {
- ProcCpu procCpu = sigar.getProcCpu(currentProcessId);
- final ServiceMetricEvent.Builder builder = builder();
- // delta for total, sys, user
- Map<String, Long> procDiff = diff.to(
- "proc/cpu", ImmutableMap.of(
- "jvm/cpu/total", procCpu.getTotal(),
- "jvm/cpu/sys", procCpu.getSys(),
- "jvm/cpu/user", procCpu.getUser()
- )
- );
- if (procDiff != null) {
- for (Map.Entry<String, Long> entry : procDiff.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(), entry.getValue()));
- }
+ // Use getProcess() instead of getCurrentProcess() so lookup failures
return null rather than a zero-valued
+ // CurrentProcessStub, which would corrupt the CPU deltas and
previous-process baseline.
+ final OSProcess currentProcess =
operatingSystem.getProcess(currentProcessId);
+ if (currentProcess == null) {
+ log.error("Unable to get current process CPU metrics");
+ return true;
+ }
+
+ final ServiceMetricEvent.Builder builder = builder();
+ final long userTime = currentProcess.getUserTime();
+ final long sysTime = currentProcess.getKernelTime();
+ final Map<String, Long> procDiff = diff.to(
+ "proc/cpu", ImmutableMap.of(
+ "jvm/cpu/total", userTime + sysTime,
+ "jvm/cpu/sys", sysTime,
+ "jvm/cpu/user", userTime
+ )
+ );
+ if (procDiff != null) {
+ for (Map.Entry<String, Long> entry : procDiff.entrySet()) {
+ emitter.emit(builder.setMetric(entry.getKey(), entry.getValue()));
}
- emitter.emit(builder.setMetric("jvm/cpu/percent", procCpu.getPercent()));
}
- catch (SigarException e) {
- log.error(e, "Failed to get ProcCpu");
+
+ final double cpuLoad =
currentProcess.getProcessCpuLoadBetweenTicks(previousProcess);
+ if (cpuLoad >= 0) {
+ emitter.emit(builder.setMetric("jvm/cpu/percent", cpuLoad * 100));
}
+
+ previousProcess = currentProcess;
return true;
}
}
diff --git
a/processing/src/main/java/org/apache/druid/java/util/metrics/JvmPidDiscoverer.java
b/processing/src/main/java/org/apache/druid/java/util/metrics/JvmPidDiscoverer.java
index dedcfea1168..b726cf67661 100644
---
a/processing/src/main/java/org/apache/druid/java/util/metrics/JvmPidDiscoverer.java
+++
b/processing/src/main/java/org/apache/druid/java/util/metrics/JvmPidDiscoverer.java
@@ -19,14 +19,8 @@
package org.apache.druid.java.util.metrics;
-import org.apache.druid.java.util.common.RE;
-
-import java.lang.management.ManagementFactory;
-import java.util.regex.Pattern;
-
/**
- * For systems that for whatever reason cannot use Sigar (through
org.apache.druid.java.util.metrics.SigarPidDiscoverer ),
- * this attempts to get the PID from the JVM "name".
+ * Discovers the PID of the current JVM.
*/
public class JvmPidDiscoverer implements PidDiscoverer
{
@@ -45,36 +39,13 @@ public class JvmPidDiscoverer implements PidDiscoverer
}
/**
- * Returns the PID as a best guess. This uses methods that are not
guaranteed to actually be the PID.
- * <p>
- * TODO: switch to ProcessHandle.current().getPid() for java9 potentially
+ * Returns the PID of the current JVM.
*
* @return the PID of the current jvm if available
- *
- * @throws RuntimeException if the pid cannot be determined
*/
@Override
public long getPid()
{
- return Inner.PID;
- }
-
- private static class Inner
- {
- private static final long PID;
-
- static {
- final String jvmName = ManagementFactory.getRuntimeMXBean().getName();
- final String[] nameSplits = jvmName.split(Pattern.quote("@"));
- if (nameSplits.length != 2) {
- throw new RE("Unable to determine pid from [%s]", jvmName);
- }
- try {
- PID = Long.parseLong(nameSplits[0]);
- }
- catch (NumberFormatException nfe) {
- throw new RE(nfe, "Unable to determine pid from [%s]", jvmName);
- }
- }
+ return ProcessHandle.current().pid();
}
}
diff --git
a/processing/src/main/java/org/apache/druid/java/util/metrics/NoopSysMonitor.java
b/processing/src/main/java/org/apache/druid/java/util/metrics/NoopSysMonitor.java
deleted file mode 100644
index 82ee742db91..00000000000
---
a/processing/src/main/java/org/apache/druid/java/util/metrics/NoopSysMonitor.java
+++ /dev/null
@@ -1,36 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.druid.java.util.metrics;
-
-import org.apache.druid.java.util.emitter.service.ServiceEmitter;
-
-public class NoopSysMonitor extends SysMonitor
-{
- public NoopSysMonitor()
- {
- super();
- }
-
- @Override
- public boolean doMonitor(ServiceEmitter emitter)
- {
- return false;
- }
-}
diff --git
a/processing/src/main/java/org/apache/druid/java/util/metrics/OshiSysMonitor.java
b/processing/src/main/java/org/apache/druid/java/util/metrics/OshiSysMonitor.java
index f60ae8e963c..7701c90b6af 100644
---
a/processing/src/main/java/org/apache/druid/java/util/metrics/OshiSysMonitor.java
+++
b/processing/src/main/java/org/apache/druid/java/util/metrics/OshiSysMonitor.java
@@ -42,7 +42,7 @@ import java.util.Map;
import java.util.function.Consumer;
/**
- * SysMonitor implemented using {@link oshi}
+ * System monitor implemented using {@link oshi}
* <p>
* Following stats are emitted:
* <ul>
diff --git
a/processing/src/main/java/org/apache/druid/java/util/metrics/SigarPidDiscoverer.java
b/processing/src/main/java/org/apache/druid/java/util/metrics/SigarPidDiscoverer.java
deleted file mode 100644
index e129cfbb810..00000000000
---
a/processing/src/main/java/org/apache/druid/java/util/metrics/SigarPidDiscoverer.java
+++ /dev/null
@@ -1,44 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.druid.java.util.metrics;
-
-public class SigarPidDiscoverer implements PidDiscoverer
-{
- private static final SigarPidDiscoverer INSTANCE = new SigarPidDiscoverer();
-
- public static SigarPidDiscoverer instance()
- {
- return INSTANCE;
- }
-
- /**
- * use {SigarPidDiscoverer.instance()}
- */
- private SigarPidDiscoverer()
- {
-
- }
-
- @Override
- public long getPid()
- {
- return SigarUtil.getCurrentProcessId();
- }
-}
diff --git
a/processing/src/main/java/org/apache/druid/java/util/metrics/SigarUtil.java
b/processing/src/main/java/org/apache/druid/java/util/metrics/SigarUtil.java
deleted file mode 100644
index 333ff622224..00000000000
--- a/processing/src/main/java/org/apache/druid/java/util/metrics/SigarUtil.java
+++ /dev/null
@@ -1,84 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.druid.java.util.metrics;
-
-import org.apache.druid.java.util.common.FileUtils;
-import org.apache.druid.java.util.common.StreamUtils;
-import org.apache.druid.java.util.common.logger.Logger;
-import org.hyperic.jni.ArchLoaderException;
-import org.hyperic.jni.ArchNotSupportedException;
-import org.hyperic.sigar.Sigar;
-import org.hyperic.sigar.SigarLoader;
-
-import java.io.File;
-import java.io.IOException;
-import java.net.URL;
-
-public class SigarUtil
-{
- private static final Logger log = new Logger(SigarUtil.class);
-
- // Note: this is required to load the sigar native lib.
- static {
- SigarLoader loader = new SigarLoader(Sigar.class);
- try {
- String libName = loader.getLibraryName();
-
- final URL url = SysMonitor.class.getResource("/" + libName);
- if (url != null) {
- final File tmpDir = FileUtils.createTempDir("sigar");
- // As per java.io.DeleteOnExitHook.runHooks() deletion order is
reversed from registration order
- tmpDir.deleteOnExit();
- final File nativeLibTmpFile = new File(tmpDir, libName);
- nativeLibTmpFile.deleteOnExit();
- StreamUtils.copyToFileAndClose(url.openStream(), nativeLibTmpFile);
- log.info("Loading sigar native lib at tmpPath[%s]", nativeLibTmpFile);
- loader.load(nativeLibTmpFile.getParent());
- } else {
- log.info("No native libs found in jar, letting the normal load
mechanisms figger it out.");
- }
- }
- catch (ArchNotSupportedException | ArchLoaderException | IOException e) {
- throw new RuntimeException(e);
- }
- }
-
- public static Sigar getSigar()
- {
- return new Sigar();
- }
-
- /**
- * CurrentProcessIdHolder class is initialized after SigarUtil, that
guarantees that new Sigar() is executed after
- * static block (which loads the library) of SigarUtil is executed. This is
anyway guaranteed by JLS if the static
- * field goes below the static block in textual order, but fragile e. g. if
someone applies automatic reformatting and
- * the static field is moved above the static block.
- */
- private static class CurrentProcessIdHolder
- {
- private static final long CURRENT_PROCESS_ID = new Sigar().getPid();
- }
-
- public static long getCurrentProcessId()
- {
- return CurrentProcessIdHolder.CURRENT_PROCESS_ID;
- }
-
-}
diff --git
a/processing/src/main/java/org/apache/druid/java/util/metrics/SysMonitor.java
b/processing/src/main/java/org/apache/druid/java/util/metrics/SysMonitor.java
deleted file mode 100644
index 031574d5371..00000000000
---
a/processing/src/main/java/org/apache/druid/java/util/metrics/SysMonitor.java
+++ /dev/null
@@ -1,577 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.druid.java.util.metrics;
-
-import com.google.common.base.Joiner;
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.ImmutableMap;
-import org.apache.druid.java.util.common.logger.Logger;
-import org.apache.druid.java.util.emitter.service.ServiceEmitter;
-import org.apache.druid.java.util.emitter.service.ServiceMetricEvent;
-import org.hyperic.sigar.Cpu;
-import org.hyperic.sigar.DirUsage;
-import org.hyperic.sigar.DiskUsage;
-import org.hyperic.sigar.FileSystem;
-import org.hyperic.sigar.FileSystemUsage;
-import org.hyperic.sigar.Mem;
-import org.hyperic.sigar.NetInterfaceConfig;
-import org.hyperic.sigar.NetInterfaceStat;
-import org.hyperic.sigar.NetStat;
-import org.hyperic.sigar.Sigar;
-import org.hyperic.sigar.SigarException;
-import org.hyperic.sigar.Swap;
-import org.hyperic.sigar.Tcp;
-import org.hyperic.sigar.Uptime;
-
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.List;
-import java.util.Map;
-
-
-/**
- * Deprecated, SysMonitor will now be maintained in {@link OshiSysMonitor}
- *
- * Sys monitor was implemented using @link org.hyperic.sigar which is no
longer maintained.
- * {@link oshi} based SysMonitor will be maintained and used from now on, and
is implemented in org.apache.druid.java.util.metrics.OshiSysMonitor
- */
-@Deprecated
-public class SysMonitor extends FeedDefiningMonitor
-{
- private static final Logger log = new Logger(SysMonitor.class);
-
- private final Sigar sigar = SigarUtil.getSigar();
-
- private final List<String> fsTypeWhitelist = ImmutableList.of("local");
- private final List<String> netAddressBlacklist = ImmutableList.of("0.0.0.0",
"127.0.0.1");
-
- private final List<Stats> statsList;
-
- public SysMonitor()
- {
- this(DEFAULT_METRICS_FEED);
- }
-
- public SysMonitor(String feed)
- {
- super(feed);
-
- sigar.enableLogging(true);
-
- this.statsList = new ArrayList<>();
- this.statsList.addAll(
- Arrays.asList(
- new MemStats(),
- new FsStats(),
- new DiskStats(),
- new NetStats(),
- new CpuStats(),
- new SwapStats(),
- new SysStats(),
- new TcpStats()
- )
- );
- }
-
- public void addDirectoriesToMonitor(String[] dirList)
- {
- for (int i = 0; i < dirList.length; i++) {
- dirList[i] = dirList[i].trim();
- }
- statsList.add(new DirStats(dirList));
- }
-
- @Override
- public boolean doMonitor(ServiceEmitter emitter)
- {
- for (Stats stats : statsList) {
- stats.emit(emitter);
- }
- return true;
- }
-
- private interface Stats
- {
- void emit(ServiceEmitter emitter);
- }
-
- private class MemStats implements Stats
- {
- @Override
- public void emit(ServiceEmitter emitter)
- {
- Mem mem = null;
- try {
- mem = sigar.getMem();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get Mem");
- }
- if (mem != null) {
- final Map<String, Long> stats = ImmutableMap.of(
- "sys/mem/max", mem.getTotal(),
- "sys/mem/used", mem.getUsed(),
- "sys/mem/actual/used", mem.getActualUsed(),
- "sys/mem/actual/free", mem.getActualFree()
- );
- final ServiceMetricEvent.Builder builder = builder();
- for (Map.Entry<String, Long> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(), entry.getValue()));
- }
- }
- }
- }
-
- /**
- * Gets the swap stats from sigar and emits the periodic pages in & pages
out of memory
- * along with the max swap and free swap memory.
- */
- private class SwapStats implements Stats
- {
- private long prevPageIn = 0;
- private long prevPageOut = 0;
-
- private SwapStats()
- {
- try {
- Swap swap = sigar.getSwap();
- this.prevPageIn = swap.getPageIn();
- this.prevPageOut = swap.getPageOut();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get Swap");
- }
- }
-
- @Override
- public void emit(ServiceEmitter emitter)
- {
- Swap swap = null;
- try {
- swap = sigar.getSwap();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get Swap");
- }
- if (swap != null) {
- long currPageIn = swap.getPageIn();
- long currPageOut = swap.getPageOut();
-
- final Map<String, Long> stats = ImmutableMap.of(
- "sys/swap/pageIn", (currPageIn - prevPageIn),
- "sys/swap/pageOut", (currPageOut - prevPageOut),
- "sys/swap/max", swap.getTotal(),
- "sys/swap/free", swap.getFree()
- );
-
- final ServiceMetricEvent.Builder builder = builder();
- for (Map.Entry<String, Long> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(), entry.getValue()));
- }
-
- this.prevPageIn = currPageIn;
- this.prevPageOut = currPageOut;
- }
- }
- }
-
- /**
- * Gets the disk usage of a particular directory.
- */
- private class DirStats implements Stats
- {
- private final String[] dirList;
-
- private DirStats(String[] dirList)
- {
- this.dirList = dirList;
- }
-
- @Override
- public void emit(ServiceEmitter emitter)
- {
- for (String dir : dirList) {
- DirUsage du = null;
- try {
- du = sigar.getDirUsage(dir);
- }
- catch (SigarException e) {
- log.error("Failed to get DiskUsage for [%s] due to [%s]", dir,
e.getMessage());
- }
- if (du != null) {
- final Map<String, Long> stats = ImmutableMap.of(
- "sys/storage/used", du.getDiskUsage()
- );
- final ServiceMetricEvent.Builder builder = builder()
- .setDimension("fsDirName", dir); // fsDirName because FsStats
uses fsDirName
- for (Map.Entry<String, Long> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(), entry.getValue()));
- }
- }
- }
- }
- }
-
- private class FsStats implements Stats
- {
- @Override
- public void emit(ServiceEmitter emitter)
- {
- FileSystem[] fss = null;
- try {
- fss = sigar.getFileSystemList();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get FileSystem list");
- }
- if (fss != null) {
- log.debug("Found FileSystem list: [%s]", Joiner.on(", ").join(fss));
- for (FileSystem fs : fss) {
- final String name = fs.getDirName(); // (fs.getDevName() does
something wonky here!)
- if (fsTypeWhitelist.contains(fs.getTypeName())) {
- FileSystemUsage fsu = null;
- try {
- fsu = sigar.getFileSystemUsage(name);
- }
- catch (SigarException e) {
- log.error(e, "Failed to get FileSystemUsage[%s]", name);
- }
- if (fsu != null) {
- final Map<String, Long> stats = ImmutableMap.<String,
Long>builder()
- .put("sys/fs/max", fsu.getTotal() * 1024)
- .put("sys/fs/used", fsu.getUsed() * 1024)
- .put("sys/fs/files/count", fsu.getFiles())
- .put("sys/fs/files/free", fsu.getFreeFiles())
- .build();
- final ServiceMetricEvent.Builder builder = builder()
- .setDimension("fsDevName", fs.getDevName())
- .setDimension("fsDirName", fs.getDirName())
- .setDimension("fsTypeName", fs.getTypeName())
- .setDimension("fsSysTypeName", fs.getSysTypeName())
- .setDimension("fsOptions", fs.getOptions().split(","));
- for (Map.Entry<String, Long> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(),
entry.getValue()));
- }
- }
- } else {
- log.debug("Not monitoring fs stats for name[%s] with
typeName[%s]", name, fs.getTypeName());
- }
- }
- }
- }
- }
-
- private class DiskStats implements Stats
- {
- private final KeyedDiff diff = new KeyedDiff();
-
- @Override
- public void emit(ServiceEmitter emitter)
- {
- FileSystem[] fss = null;
- try {
- fss = sigar.getFileSystemList();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get FileSystem list");
- }
- if (fss != null) {
- log.debug("Found FileSystem list: [%s]", Joiner.on(", ").join(fss));
- for (FileSystem fs : fss) {
- // fs.getDevName() appears to give the same results here, but on
some nodes results for one disc were substituted by another
- // LOG: Sigar - /proc/diskstats /dev/xvdj -> /dev/xvdb [202,16]
- final String name = fs.getDirName();
- if (fsTypeWhitelist.contains(fs.getTypeName())) {
- DiskUsage du = null;
- try {
- du = sigar.getDiskUsage(name);
- }
- catch (SigarException e) {
- log.error(e, "Failed to get DiskUsage[%s]", name);
- }
- if (du != null) {
- final Map<String, Long> stats = diff.to(
- name,
- ImmutableMap.<String, Long>builder()
- .put("sys/disk/read/size", du.getReadBytes())
- .put("sys/disk/read/count", du.getReads())
- .put("sys/disk/write/size", du.getWriteBytes())
- .put("sys/disk/write/count", du.getWrites())
- .put("sys/disk/queue",
Double.valueOf(du.getQueue()).longValue())
- .put("sys/disk/serviceTime",
Double.valueOf(du.getServiceTime()).longValue())
- .build()
- );
- log.debug("DiskUsage diff for [%s]: %s", name, stats);
- if (stats != null) {
- final ServiceMetricEvent.Builder builder = builder()
- .setDimension("fsDevName", fs.getDevName())
- .setDimension("fsDirName", fs.getDirName())
- .setDimension("fsTypeName", fs.getTypeName())
- .setDimension("fsSysTypeName", fs.getSysTypeName())
- .setDimension("fsOptions", fs.getOptions().split(","));
- for (Map.Entry<String, Long> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(),
entry.getValue()));
- }
- }
- }
- } else {
- log.debug("Not monitoring disk stats for name[%s] with
typeName[%s]", name, fs.getTypeName());
- }
- }
- }
- }
- }
-
- private class NetStats implements Stats
- {
- private final KeyedDiff diff = new KeyedDiff();
-
- @Override
- public void emit(ServiceEmitter emitter)
- {
- String[] ifaces = null;
- try {
- ifaces = sigar.getNetInterfaceList();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get NetInterface list");
- }
- if (ifaces != null) {
- log.debug("Found NetInterface list: [%s]", Joiner.on(",
").join(ifaces));
- for (String name : ifaces) {
- NetInterfaceConfig netconf = null;
- try {
- netconf = sigar.getNetInterfaceConfig(name);
- }
- catch (SigarException e) {
- log.error(e, "Failed to get NetInterfaceConfig[%s]", name);
- }
- if (netconf != null) {
- if (!(netAddressBlacklist.contains(netconf.getAddress()))) {
- NetInterfaceStat netstat = null;
- try {
- netstat = sigar.getNetInterfaceStat(name);
- }
- catch (SigarException e) {
- log.error(e, "Failed to get NetInterfaceStat[%s]", name);
- }
- if (netstat != null) {
- final Map<String, Long> stats = diff.to(
- name,
- ImmutableMap.<String, Long>builder()
- .put("sys/net/read/size", netstat.getRxBytes())
- .put("sys/net/read/packets", netstat.getRxPackets())
- .put("sys/net/read/errors", netstat.getRxErrors())
- .put("sys/net/read/dropped", netstat.getRxDropped())
- .put("sys/net/read/overruns", netstat.getRxOverruns())
- .put("sys/net/read/frame", netstat.getRxFrame())
- .put("sys/net/write/size", netstat.getTxBytes())
- .put("sys/net/write/packets", netstat.getTxPackets())
- .put("sys/net/write/errors", netstat.getTxErrors())
- .put("sys/net/write/dropped", netstat.getTxDropped())
- .put("sys/net/write/collisions",
netstat.getTxCollisions())
- .put("sys/net/write/overruns", netstat.getTxOverruns())
- .build()
- );
- if (stats != null) {
- final ServiceMetricEvent.Builder builder = builder()
- .setDimension("netName", netconf.getName())
- .setDimension("netAddress", netconf.getAddress())
- .setDimension("netHwaddr", netconf.getHwaddr());
- for (Map.Entry<String, Long> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(),
entry.getValue()));
- }
- }
- }
- } else {
- log.debug("Not monitoring net stats for name[%s] with
address[%s]", name, netconf.getAddress());
- }
- }
- }
- }
- }
- }
-
- private class CpuStats implements Stats
- {
- private final KeyedDiff diff = new KeyedDiff();
-
- @Override
- public void emit(ServiceEmitter emitter)
- {
- Cpu[] cpus = null;
- try {
- cpus = sigar.getCpuList();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get Cpu list");
- }
- if (cpus != null) {
- log.debug("Found Cpu list: [%s]", Joiner.on(", ").join(cpus));
- for (int i = 0; i < cpus.length; ++i) {
- final Cpu cpu = cpus[i];
- final String name = Integer.toString(i);
- final Map<String, Long> stats = diff.to(
- name,
- ImmutableMap.<String, Long>builder()
- .put("user", cpu.getUser()) // user = Δuser / Δtotal
- .put("sys", cpu.getSys()) // sys = Δsys / Δtotal
- .put("nice", cpu.getNice()) // nice = Δnice / Δtotal
- .put("wait", cpu.getWait()) // wait = Δwait / Δtotal
- .put("irq", cpu.getIrq()) // irq = Δirq / Δtotal
- .put("softIrq", cpu.getSoftIrq()) // softIrq = ΔsoftIrq /
Δtotal
- .put("stolen", cpu.getStolen()) // stolen = Δstolen / Δtotal
- .put("_total", cpu.getTotal()) // (not reported)
- .build()
- );
- if (stats != null) {
- final long total = stats.remove("_total");
- for (Map.Entry<String, Long> entry : stats.entrySet()) {
- final ServiceMetricEvent.Builder builder = builder()
- .setDimension("cpuName", name)
- .setDimension("cpuTime", entry.getKey());
- emitter.emit(builder.setMetric("sys/cpu", entry.getValue() * 100
/ total)); // [0,100]
- }
- }
- }
- }
- }
- }
-
- private class SysStats implements Stats
- {
- @Override
- public void emit(ServiceEmitter emitter)
- {
- final ServiceMetricEvent.Builder builder = builder();
-
- Uptime uptime = null;
- try {
- uptime = sigar.getUptime();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get Uptime");
- }
-
- double[] la = null;
- try {
- la = sigar.getLoadAverage();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get Load Average");
- }
-
- if (uptime != null) {
- final Map<String, Number> stats = ImmutableMap.of(
- "sys/uptime", Double.valueOf(uptime.getUptime()).longValue()
- );
- for (Map.Entry<String, Number> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(), entry.getValue()));
- }
- }
-
- if (la != null) {
- final Map<String, Number> stats = ImmutableMap.of(
- "sys/la/1", la[0],
- "sys/la/5", la[1],
- "sys/la/15", la[2]
- );
- for (Map.Entry<String, Number> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(), entry.getValue()));
- }
- }
- }
- }
-
- private class TcpStats implements Stats
- {
- private final KeyedDiff diff = new KeyedDiff();
-
- @Override
- public void emit(ServiceEmitter emitter)
- {
- final ServiceMetricEvent.Builder builder = builder();
-
- Tcp tcp = null;
- try {
- tcp = sigar.getTcp();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get Tcp");
- }
-
- if (tcp != null) {
- final Map<String, Long> stats = diff.to(
- "tcp", ImmutableMap.<String, Long>builder()
- .put("sys/tcp/activeOpens", tcp.getActiveOpens())
- .put("sys/tcp/passiveOpens", tcp.getPassiveOpens())
- .put("sys/tcp/attemptFails", tcp.getAttemptFails())
- .put("sys/tcp/estabResets", tcp.getEstabResets())
- .put("sys/tcp/in/segs", tcp.getInSegs())
- .put("sys/tcp/in/errs", tcp.getInErrs())
- .put("sys/tcp/out/segs", tcp.getOutSegs())
- .put("sys/tcp/out/rsts", tcp.getOutRsts())
- .put("sys/tcp/retrans/segs", tcp.getRetransSegs())
- .build()
- );
- if (stats != null) {
- for (Map.Entry<String, Long> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(), entry.getValue()));
- }
- }
- }
-
- NetStat netStat = null;
- try {
- netStat = sigar.getNetStat();
- }
- catch (SigarException e) {
- log.error(e, "Failed to get NetStat");
- }
- if (netStat != null) {
- final Map<String, Long> stats = ImmutableMap.<String, Long>builder()
- .put("sys/net/inbound", (long) netStat.getAllInboundTotal())
- .put("sys/net/outbound", (long) netStat.getAllOutboundTotal())
- .put("sys/tcp/inbound", (long) netStat.getTcpInboundTotal())
- .put("sys/tcp/outbound", (long) netStat.getTcpOutboundTotal())
- .put(
- "sys/tcp/state/established",
- (long) netStat.getTcpEstablished()
- )
- .put("sys/tcp/state/synSent", (long) netStat.getTcpSynSent())
- .put("sys/tcp/state/synRecv", (long) netStat.getTcpSynRecv())
- .put("sys/tcp/state/finWait1", (long) netStat.getTcpFinWait1())
- .put("sys/tcp/state/finWait2", (long) netStat.getTcpFinWait2())
- .put("sys/tcp/state/timeWait", (long) netStat.getTcpTimeWait())
- .put("sys/tcp/state/close", (long) netStat.getTcpClose())
- .put("sys/tcp/state/closeWait", (long) netStat.getTcpCloseWait())
- .put("sys/tcp/state/lastAck", (long) netStat.getTcpLastAck())
- .put("sys/tcp/state/listen", (long) netStat.getTcpListen())
- .put("sys/tcp/state/closing", (long) netStat.getTcpClosing())
- .put("sys/tcp/state/idle", (long) netStat.getTcpIdle())
- .put("sys/tcp/state/bound", (long) netStat.getTcpBound())
- .build();
- for (Map.Entry<String, Long> entry : stats.entrySet()) {
- emitter.emit(builder.setMetric(entry.getKey(), entry.getValue()));
- }
- }
- }
- }
-}
diff --git
a/processing/src/test/java/org/apache/druid/java/util/metrics/JvmCpuMonitorTest.java
b/processing/src/test/java/org/apache/druid/java/util/metrics/JvmCpuMonitorTest.java
new file mode 100644
index 00000000000..a72a94c6782
--- /dev/null
+++
b/processing/src/test/java/org/apache/druid/java/util/metrics/JvmCpuMonitorTest.java
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.java.util.metrics;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.mockito.Mockito;
+import oshi.software.os.OSProcess;
+import oshi.software.os.OperatingSystem;
+
+import java.util.List;
+
+public class JvmCpuMonitorTest
+{
+ @Test
+ public void testDoMonitor()
+ {
+ final OperatingSystem operatingSystem =
Mockito.mock(OperatingSystem.class);
+ final OSProcess initialProcess = Mockito.mock(OSProcess.class);
+ final OSProcess firstProcess = Mockito.mock(OSProcess.class);
+ final OSProcess secondProcess = Mockito.mock(OSProcess.class);
+ Mockito.when(operatingSystem.getProcessId()).thenReturn(123);
+ Mockito.when(operatingSystem.getProcess(123))
+ .thenReturn(initialProcess, firstProcess, secondProcess);
+ Mockito.when(firstProcess.getKernelTime()).thenReturn(200L);
+ Mockito.when(firstProcess.getUserTime()).thenReturn(300L);
+ Mockito.when(secondProcess.getKernelTime()).thenReturn(250L);
+ Mockito.when(secondProcess.getUserTime()).thenReturn(450L);
+
Mockito.when(firstProcess.getProcessCpuLoadBetweenTicks(initialProcess)).thenReturn(0.25);
+
Mockito.when(secondProcess.getProcessCpuLoadBetweenTicks(firstProcess)).thenReturn(0.5);
+
+ final JvmCpuMonitor monitor = new JvmCpuMonitor("test", operatingSystem);
+ final StubServiceEmitter emitter = new StubServiceEmitter();
+
+ Assertions.assertTrue(monitor.doMonitor(emitter));
+ Assertions.assertTrue(monitor.doMonitor(emitter));
+
+ emitter.verifyValue("jvm/cpu/total", 200L);
+ emitter.verifyValue("jvm/cpu/sys", 50L);
+ emitter.verifyValue("jvm/cpu/user", 150L);
+ Assertions.assertEquals(
+ List.of(25.0, 50.0),
+ emitter.getMetricValues("jvm/cpu/percent", null)
+ );
+ }
+
+ @Test
+ public void testFailedProcessLookupDoesNotAdvanceBaselines()
+ {
+ final OperatingSystem operatingSystem =
Mockito.mock(OperatingSystem.class);
+ final OSProcess initialProcess = Mockito.mock(OSProcess.class);
+ final OSProcess firstProcess = Mockito.mock(OSProcess.class);
+ final OSProcess secondProcess = Mockito.mock(OSProcess.class);
+ Mockito.when(operatingSystem.getProcessId()).thenReturn(123);
+ Mockito.when(operatingSystem.getProcess(123))
+ .thenReturn(initialProcess, firstProcess)
+ .thenReturn(null)
+ .thenReturn(secondProcess);
+ Mockito.when(firstProcess.getKernelTime()).thenReturn(200L);
+ Mockito.when(firstProcess.getUserTime()).thenReturn(300L);
+ Mockito.when(secondProcess.getKernelTime()).thenReturn(250L);
+ Mockito.when(secondProcess.getUserTime()).thenReturn(450L);
+
Mockito.when(firstProcess.getProcessCpuLoadBetweenTicks(initialProcess)).thenReturn(0.25);
+
Mockito.when(secondProcess.getProcessCpuLoadBetweenTicks(firstProcess)).thenReturn(0.5);
+
+ final JvmCpuMonitor monitor = new JvmCpuMonitor("test", operatingSystem);
+ final StubServiceEmitter emitter = new StubServiceEmitter();
+
+ Assertions.assertTrue(monitor.doMonitor(emitter));
+ Assertions.assertTrue(monitor.doMonitor(emitter));
+ Assertions.assertTrue(monitor.doMonitor(emitter));
+
+ emitter.verifyValue("jvm/cpu/total", 200L);
+ emitter.verifyValue("jvm/cpu/sys", 50L);
+ emitter.verifyValue("jvm/cpu/user", 150L);
+ Assertions.assertEquals(
+ List.of(25.0, 50.0),
+ emitter.getMetricValues("jvm/cpu/percent", null)
+ );
+ }
+}
diff --git
a/processing/src/test/java/org/apache/druid/java/util/metrics/NoopSysMonitorTest.java
b/processing/src/test/java/org/apache/druid/java/util/metrics/NoopSysMonitorTest.java
deleted file mode 100644
index 130a98ab9be..00000000000
---
a/processing/src/test/java/org/apache/druid/java/util/metrics/NoopSysMonitorTest.java
+++ /dev/null
@@ -1,42 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.druid.java.util.metrics;
-
-import org.apache.druid.java.util.emitter.service.ServiceEmitter;
-import org.junit.jupiter.api.Assertions;
-import org.junit.jupiter.api.Assumptions;
-import org.junit.jupiter.api.Test;
-import org.mockito.Mockito;
-
-public class NoopSysMonitorTest
-{
- private static final String CPU_ARCH = System.getProperty("os.arch");
-
- @Test
- public void testDoMonitor()
- {
- Assumptions.assumeFalse("aarch64".equals(CPU_ARCH));
-
- ServiceEmitter serviceEmitter = Mockito.mock(ServiceEmitter.class);
- NoopSysMonitor noopSysMonitor = new NoopSysMonitor();
-
- Assertions.assertFalse(noopSysMonitor.doMonitor(serviceEmitter));
- }
-}
diff --git
a/processing/src/test/java/org/apache/druid/java/util/metrics/SigarLoadTest.java
b/processing/src/test/java/org/apache/druid/java/util/metrics/SigarLoadTest.java
deleted file mode 100644
index 5f5d659b7ec..00000000000
---
a/processing/src/test/java/org/apache/druid/java/util/metrics/SigarLoadTest.java
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.druid.java.util.metrics;
-
-import org.hyperic.sigar.Sigar;
-import org.junit.jupiter.api.Assertions;
-import org.junit.jupiter.api.Assumptions;
-import org.junit.jupiter.api.BeforeEach;
-import org.junit.jupiter.api.Test;
-
-public class SigarLoadTest
-{
- private static final String CPU_ARCH = System.getProperty("os.arch");
-
- @BeforeEach
- public void before()
- {
- // Do not run the tests on ARM64. Sigar library has no binaries for ARM64
- Assumptions.assumeFalse("aarch64".equals(CPU_ARCH));
- }
-
- @Test
- public void testSigarLoad()
- {
- Sigar sigar = SigarUtil.getSigar();
- Assertions.assertTrue(sigar.getPid() > 0);
- }
-}
diff --git
a/processing/src/test/java/org/apache/druid/java/util/metrics/SigarPidDiscovererTest.java
b/processing/src/test/java/org/apache/druid/java/util/metrics/SigarPidDiscovererTest.java
deleted file mode 100644
index 40fa9db5ae8..00000000000
---
a/processing/src/test/java/org/apache/druid/java/util/metrics/SigarPidDiscovererTest.java
+++ /dev/null
@@ -1,38 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.druid.java.util.metrics;
-
-import org.junit.jupiter.api.Assumptions;
-import org.junit.jupiter.api.Test;
-
-public class SigarPidDiscovererTest
-{
- private static final String CPU_ARCH = System.getProperty("os.arch");
-
- @Test
- public void simpleTest()
- {
- // Do not run the tests on ARM64. Sigar library has no binaries for ARM64
- Assumptions.assumeFalse("aarch64".equals(CPU_ARCH));
-
- // Just make sure we don't crash
- SigarPidDiscoverer.instance().getPid();
- }
-}
diff --git
a/server/src/main/java/org/apache/druid/server/metrics/DruidSysMonitor.java
b/server/src/main/java/org/apache/druid/server/metrics/DruidSysMonitor.java
deleted file mode 100644
index c62ce44fcee..00000000000
--- a/server/src/main/java/org/apache/druid/server/metrics/DruidSysMonitor.java
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.druid.server.metrics;
-
-import com.google.common.collect.Lists;
-import com.google.inject.Inject;
-import org.apache.druid.java.util.metrics.SysMonitor;
-import org.apache.druid.segment.loading.SegmentLoaderConfig;
-import org.apache.druid.segment.loading.StorageLocationConfig;
-
-import java.util.List;
-
-/**
- */
-public class DruidSysMonitor extends SysMonitor
-{
- @Inject
- public DruidSysMonitor(
- SegmentLoaderConfig config
- )
- {
- final List<StorageLocationConfig> locs = config.getLocations();
- List<String> dirs = Lists.newArrayListWithExpectedSize(locs.size());
- for (StorageLocationConfig loc : locs) {
- dirs.add(loc.getPath().toString());
- }
-
- addDirectoriesToMonitor(dirs.toArray(new String[0]));
- }
-}
diff --git
a/server/src/main/java/org/apache/druid/server/metrics/MetricsModule.java
b/server/src/main/java/org/apache/druid/server/metrics/MetricsModule.java
index 7935c1b0a3e..5333de4b662 100644
--- a/server/src/main/java/org/apache/druid/server/metrics/MetricsModule.java
+++ b/server/src/main/java/org/apache/druid/server/metrics/MetricsModule.java
@@ -48,10 +48,8 @@ import org.apache.druid.java.util.metrics.JvmThreadsMonitor;
import org.apache.druid.java.util.metrics.Monitor;
import org.apache.druid.java.util.metrics.MonitorScheduler;
import org.apache.druid.java.util.metrics.NoopOshiSysMonitor;
-import org.apache.druid.java.util.metrics.NoopSysMonitor;
import org.apache.druid.java.util.metrics.OshiSysMonitor;
import org.apache.druid.java.util.metrics.OshiSysMonitorConfig;
-import org.apache.druid.java.util.metrics.SysMonitor;
import org.apache.druid.query.ExecutorServiceMonitor;
import java.time.Duration;
@@ -170,17 +168,6 @@ public class MetricsModule implements Module
return new JvmThreadsMonitor();
}
- @Provides
- @ManageLifecycle
- public SysMonitor getSysMonitor(@Self Set<NodeRole> nodeRoles, Properties
properties)
- {
- if (nodeRoles.contains(NodeRole.PEON) && isManagedPeon(properties)) {
- return new NoopSysMonitor();
- } else {
- return new SysMonitor();
- }
- }
-
@Provides
@ManageLifecycle
public OshiSysMonitor getOshiSysMonitor(
diff --git
a/server/src/test/java/org/apache/druid/server/metrics/MetricsModuleTest.java
b/server/src/test/java/org/apache/druid/server/metrics/MetricsModuleTest.java
index c5d7cb4d3c6..dcbb97e7036 100644
---
a/server/src/test/java/org/apache/druid/server/metrics/MetricsModuleTest.java
+++
b/server/src/test/java/org/apache/druid/server/metrics/MetricsModuleTest.java
@@ -50,16 +50,13 @@ import
org.apache.druid.java.util.metrics.ClockDriftSafeMonitorScheduler;
import org.apache.druid.java.util.metrics.Monitor;
import org.apache.druid.java.util.metrics.MonitorScheduler;
import org.apache.druid.java.util.metrics.NoopOshiSysMonitor;
-import org.apache.druid.java.util.metrics.NoopSysMonitor;
import org.apache.druid.java.util.metrics.NoopTaskHolder;
import org.apache.druid.java.util.metrics.OshiSysMonitor;
import org.apache.druid.java.util.metrics.OshiSysMonitorConfig;
-import org.apache.druid.java.util.metrics.SysMonitor;
import org.apache.druid.java.util.metrics.TaskHolder;
import org.apache.druid.query.DruidMetrics;
import org.apache.druid.server.DruidNode;
import org.junit.jupiter.api.Assertions;
-import org.junit.jupiter.api.Assumptions;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentMatchers;
import org.mockito.Mockito;
@@ -70,8 +67,6 @@ import java.util.Set;
public class MetricsModuleTest
{
- private static final String CPU_ARCH = System.getProperty("os.arch");
-
@Test
public void testSimpleInjection()
{
@@ -233,37 +228,6 @@ public class MetricsModuleTest
Assertions.assertTrue(exception.getMessage().contains("Unknown monitor
scheduler[UnknownScheduler]"));
}
- @Test
- public void testGetSysMonitorViaInjector()
- {
- // Do not run the tests on ARM64. Sigar library has no binaries for ARM64
- Assumptions.assumeFalse("aarch64".equals(CPU_ARCH));
-
- final Properties properties = new Properties();
- properties.setProperty(MetricsModule.PROPERTY_PEON_MANAGED, "true");
- final Injector injector = createInjector(properties,
ImmutableSet.of(NodeRole.PEON));
- final SysMonitor sysMonitor = injector.getInstance(SysMonitor.class);
- final ServiceEmitter emitter = Mockito.mock(ServiceEmitter.class);
- sysMonitor.doMonitor(emitter);
-
- Assertions.assertTrue(sysMonitor instanceof NoopSysMonitor);
- Mockito.verify(emitter,
Mockito.never()).emit(ArgumentMatchers.any(ServiceEventBuilder.class));
- }
-
- @Test
- public void testGetSysMonitorWhenNull()
- {
- // Do not run the tests on ARM64. Sigar library has no binaries for ARM64
- Assumptions.assumeFalse("aarch64".equals(CPU_ARCH));
-
- Injector injector = createInjector(new Properties(), ImmutableSet.of());
- final SysMonitor sysMonitor = injector.getInstance(SysMonitor.class);
- final ServiceEmitter emitter = Mockito.mock(ServiceEmitter.class);
- sysMonitor.doMonitor(emitter);
-
- Assertions.assertFalse(sysMonitor instanceof NoopSysMonitor);
- Mockito.verify(emitter,
Mockito.atLeastOnce()).emit(ArgumentMatchers.any(ServiceEventBuilder.class));
- }
@Test
public void testGetOshiSysMonitorViaInjector()
{
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
