Re: [PR] STORM-4054 -Add Worker CPU Metric (storm)

[via GitHub]

agresch commented on code in PR #3639:
URL: https://github.com/apache/storm/pull/3639#discussion_r1566440996


##
storm-client/src/jvm/org/apache/storm/metric/SystemBolt.java:
##
@@ -67,12 +68,56 @@ public Long getValue() {
 
 
 context.registerGauge("newWorkerEvent", new NewWorkerGauge());
+context.registerGauge("workerCpuUsage", new WorkerCpuMetric());
 
 int bucketSize = 
ObjectReader.getInt(topoConf.get(Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS));
 registerMetrics(context, (Map) 
topoConf.get(Config.WORKER_METRICS), bucketSize, topoConf);
 registerMetrics(context, (Map) 
topoConf.get(Config.TOPOLOGY_WORKER_METRICS), bucketSize, topoConf);
 }
 
+private class WorkerCpuMetric implements Gauge {
+private long lastCalculationTimeNsec;
+private long previousCpuTotal;
+private double cpuUsage;
+
+WorkerCpuMetric() {
+lastCalculationTimeNsec = System.nanoTime();
+previousCpuTotal = getTotalCpuUsage();
+cpuUsage = 0.0d;
+}
+
+private long getTotalCpuUsage() {
+long totalCpuNsecs = 0L;
+ThreadMXBean threadMxBean = ManagementFactory.getThreadMXBean();
+for (Long threadId : threadMxBean.getAllThreadIds()) {
+long threadCpu = threadMxBean.getThreadCpuTime(threadId);
+if (threadCpu > 0L) {
+totalCpuNsecs += threadCpu;
+}
+}
+return totalCpuNsecs;
+}
+
+private void updateCalculation() {
+// we could have multiple reporters calling getValue() one right
+// after another, with inaccurate reporting due to the small time 
difference.
+long elapsed = System.nanoTime() - this.lastCalculationTimeNsec;
+if (elapsed >= 10L) {

Review Comment:
   updated



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] STORM-4054 -Add Worker CPU Metric (storm)

[via GitHub]

rzo1 commented on code in PR #3639:
URL: https://github.com/apache/storm/pull/3639#discussion_r1566353967


##
storm-client/src/jvm/org/apache/storm/metric/SystemBolt.java:
##
@@ -67,12 +68,56 @@ public Long getValue() {
 
 
 context.registerGauge("newWorkerEvent", new NewWorkerGauge());
+context.registerGauge("workerCpuUsage", new WorkerCpuMetric());
 
 int bucketSize = 
ObjectReader.getInt(topoConf.get(Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS));
 registerMetrics(context, (Map) 
topoConf.get(Config.WORKER_METRICS), bucketSize, topoConf);
 registerMetrics(context, (Map) 
topoConf.get(Config.TOPOLOGY_WORKER_METRICS), bucketSize, topoConf);
 }
 
+private class WorkerCpuMetric implements Gauge {
+private long lastCalculationTimeNsec;
+private long previousCpuTotal;
+private double cpuUsage;
+
+WorkerCpuMetric() {
+lastCalculationTimeNsec = System.nanoTime();
+previousCpuTotal = getTotalCpuUsage();
+cpuUsage = 0.0d;
+}
+
+private long getTotalCpuUsage() {
+long totalCpuNsecs = 0L;
+ThreadMXBean threadMxBean = ManagementFactory.getThreadMXBean();
+for (Long threadId : threadMxBean.getAllThreadIds()) {
+long threadCpu = threadMxBean.getThreadCpuTime(threadId);
+if (threadCpu > 0L) {
+totalCpuNsecs += threadCpu;
+}
+}
+return totalCpuNsecs;
+}
+
+private void updateCalculation() {
+// we could have multiple reporters calling getValue() one right
+// after another, with inaccurate reporting due to the small time 
difference.
+long elapsed = System.nanoTime() - this.lastCalculationTimeNsec;
+if (elapsed >= 10L) {

Review Comment:
   Maybe introduce a constant here? I think, that a constant instead of this 
magic number  would be more readable here. Something like `private static final 
long THRESHOLD = TimeUnit.NANOSECONDS.convert(1, TimeUnit.SECONDS);` , wdyt?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] STORM-4053 - Include Client API / Runtime for HDFS (storm)

[via GitHub]

jnioche merged PR #3638:
URL: https://github.com/apache/storm/pull/3638


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[PR] STORM-4053 - Include Client API / Runtime for HDFS (storm)

[via GitHub]

rzo1 opened a new pull request, #3638:
URL: https://github.com/apache/storm/pull/3638

   Followed advice from Hadoop Devs: 
https://lists.apache.org/thread/f47s6bxrtslkxbc8s2gybwrxps8vk63x


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@storm.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org