YoavNordmann opened a new issue #10615:
URL: https://github.com/apache/druid/issues/10615
We can see many instances of `org.hyperic.sigar.SigarFileNotFoundException:
No such file or directory` in the log file after adding
`org.apache.druid.java.util.metrics.SysMonitor` to the list of monitors.
### Affected Version
We are using the official helm chart "incubator/druid" version "0.2.18
### Description
Please include as much detailed information about the problem as possible.
We are using the helm chart as is while adding some monitors such as the
`org.apache.druid.java.util.metrics.SysMonitor`.
- Cluster size: Default
- Configurations in use: Additional Monitors
- Steps to reproduce the problem: Just run the Helm chart and run some
ingestion. We specifically use 'kafka_ingestion"
Some log file excerpts:
`2020-10-18T12:16:57,518 ERROR [MonitorScheduler-0]
org.apache.druid.java.util.metrics.SysMonitor - Failed to get
DiskUsage[/opt/apache-druid-0.19.0/var]
org.hyperic.sigar.SigarFileNotFoundException: No such file or directory
at org.hyperic.sigar.DiskUsage.gather(Native Method)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.DiskUsage.fetch(DiskUsage.java:30)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.Sigar.getDiskUsage(Sigar.java:680)
~[sigar-1.6.5.132.jar:?]
at
org.apache.druid.java.util.metrics.SysMonitor$DiskStats.emit(SysMonitor.java:310)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.SysMonitor.doMonitor(SysMonitor.java:107)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.AbstractMonitor.monitor(AbstractMonitor.java:47)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:124)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:119)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.common.concurrent.ScheduledExecutors$4.run(ScheduledExecutors.java:158)
[druid-core-0.19.0.jar:0.19.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_252]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
2020-10-18T12:17:57,517 ERROR [MonitorScheduler-0]
org.apache.druid.java.util.metrics.SysMonitor - Failed to get
DiskUsage[/var/druid]
org.hyperic.sigar.SigarFileNotFoundException: No such file or directory
at org.hyperic.sigar.DiskUsage.gather(Native Method)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.DiskUsage.fetch(DiskUsage.java:30)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.Sigar.getDiskUsage(Sigar.java:680)
~[sigar-1.6.5.132.jar:?]
at
org.apache.druid.java.util.metrics.SysMonitor$DiskStats.emit(SysMonitor.java:310)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.SysMonitor.doMonitor(SysMonitor.java:107)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.AbstractMonitor.monitor(AbstractMonitor.java:47)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:124)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:119)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.common.concurrent.ScheduledExecutors$4.run(ScheduledExecutors.java:158)
[druid-core-0.19.0.jar:0.19.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_252]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
2020-10-18T12:17:57,518 ERROR [MonitorScheduler-0]
org.apache.druid.java.util.metrics.SysMonitor - Failed to get
DiskUsage[/dev/termination-log]
org.hyperic.sigar.SigarFileNotFoundException: No such file or directory
at org.hyperic.sigar.DiskUsage.gather(Native Method)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.DiskUsage.fetch(DiskUsage.java:30)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.Sigar.getDiskUsage(Sigar.java:680)
~[sigar-1.6.5.132.jar:?]
at
org.apache.druid.java.util.metrics.SysMonitor$DiskStats.emit(SysMonitor.java:310)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.SysMonitor.doMonitor(SysMonitor.java:107)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.AbstractMonitor.monitor(AbstractMonitor.java:47)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:124)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:119)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.common.concurrent.ScheduledExecutors$4.run(ScheduledExecutors.java:158)
[druid-core-0.19.0.jar:0.19.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_252]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
2020-10-18T12:17:57,518 ERROR [MonitorScheduler-0]
org.apache.druid.java.util.metrics.SysMonitor - Failed to get
DiskUsage[/etc/resolv.conf]
org.hyperic.sigar.SigarFileNotFoundException: No such file or directory
at org.hyperic.sigar.DiskUsage.gather(Native Method)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.DiskUsage.fetch(DiskUsage.java:30)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.Sigar.getDiskUsage(Sigar.java:680)
~[sigar-1.6.5.132.jar:?]
at
org.apache.druid.java.util.metrics.SysMonitor$DiskStats.emit(SysMonitor.java:310)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.SysMonitor.doMonitor(SysMonitor.java:107)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.AbstractMonitor.monitor(AbstractMonitor.java:47)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:124)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:119)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.common.concurrent.ScheduledExecutors$4.run(ScheduledExecutors.java:158)
[druid-core-0.19.0.jar:0.19.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_252]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
2020-10-18T12:17:57,518 ERROR [MonitorScheduler-0]
org.apache.druid.java.util.metrics.SysMonitor - Failed to get
DiskUsage[/etc/hostname]
org.hyperic.sigar.SigarFileNotFoundException: No such file or directory
at org.hyperic.sigar.DiskUsage.gather(Native Method)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.DiskUsage.fetch(DiskUsage.java:30)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.Sigar.getDiskUsage(Sigar.java:680)
~[sigar-1.6.5.132.jar:?]
at
org.apache.druid.java.util.metrics.SysMonitor$DiskStats.emit(SysMonitor.java:310)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.SysMonitor.doMonitor(SysMonitor.java:107)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.AbstractMonitor.monitor(AbstractMonitor.java:47)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:124)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:119)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.common.concurrent.ScheduledExecutors$4.run(ScheduledExecutors.java:158)
[druid-core-0.19.0.jar:0.19.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_252]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
2020-10-18T12:17:57,518 ERROR [MonitorScheduler-0]
org.apache.druid.java.util.metrics.SysMonitor - Failed to get
DiskUsage[/etc/hosts]
org.hyperic.sigar.SigarFileNotFoundException: No such file or directory
at org.hyperic.sigar.DiskUsage.gather(Native Method)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.DiskUsage.fetch(DiskUsage.java:30)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.Sigar.getDiskUsage(Sigar.java:680)
~[sigar-1.6.5.132.jar:?]
at
org.apache.druid.java.util.metrics.SysMonitor$DiskStats.emit(SysMonitor.java:310)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.SysMonitor.doMonitor(SysMonitor.java:107)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.AbstractMonitor.monitor(AbstractMonitor.java:47)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:124)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:119)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.common.concurrent.ScheduledExecutors$4.run(ScheduledExecutors.java:158)
[druid-core-0.19.0.jar:0.19.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_252]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
2020-10-18T12:17:57,518 ERROR [MonitorScheduler-0]
org.apache.druid.java.util.metrics.SysMonitor - Failed to get
DiskUsage[/opt/apache-druid-0.19.0/var]
org.hyperic.sigar.SigarFileNotFoundException: No such file or directory
at org.hyperic.sigar.DiskUsage.gather(Native Method)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.DiskUsage.fetch(DiskUsage.java:30)
~[sigar-1.6.5.132.jar:?]
at org.hyperic.sigar.Sigar.getDiskUsage(Sigar.java:680)
~[sigar-1.6.5.132.jar:?]
at
org.apache.druid.java.util.metrics.SysMonitor$DiskStats.emit(SysMonitor.java:310)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.SysMonitor.doMonitor(SysMonitor.java:107)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.AbstractMonitor.monitor(AbstractMonitor.java:47)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:124)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.metrics.MonitorScheduler$1.call(MonitorScheduler.java:119)
[druid-core-0.19.0.jar:0.19.0]
at
org.apache.druid.java.util.common.concurrent.ScheduledExecutors$4.run(ScheduledExecutors.java:158)
[druid-core-0.19.0.jar:0.19.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_252]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_252]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_252]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]`
A possible solution might be hidden in this issue:
https://itectec.com/ubuntu/ubuntu-install-cgconfig-in-ubuntu-16-04/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
