Repository: incubator-fluo Updated Branches: refs/heads/master 49b3abb64 -> 0d3acf85c
Fixes #534 - Enable application-specific metrics * Fluo users can report their own application-specific metrics from observers, loaders and clients using new MetricsReporter class. * Updated Grafana documentation to include new InfluxDB configuration for handling application metrics. * Added basic test to ObserverConfigIT to test creating reporter in an observer * Improved metrics documentation and naming Project: http://git-wip-us.apache.org/repos/asf/incubator-fluo/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-fluo/commit/0d3acf85 Tree: http://git-wip-us.apache.org/repos/asf/incubator-fluo/tree/0d3acf85 Diff: http://git-wip-us.apache.org/repos/asf/incubator-fluo/diff/0d3acf85 Branch: refs/heads/master Commit: 0d3acf85c996f692b474aab6958d860d0bb6199c Parents: 49b3abb Author: Mike Walch <mwa...@apache.org> Authored: Tue Sep 13 13:22:59 2016 -0400 Committer: Mike Walch <mwa...@apache.org> Committed: Mon Sep 19 12:00:00 2016 -0400 ---------------------------------------------------------------------- docs/grafana.md | 12 +- docs/metrics.md | 148 ++++++++++++------- .../org/apache/fluo/api/client/FluoClient.java | 7 +- .../java/org/apache/fluo/api/client/Loader.java | 6 + .../org/apache/fluo/api/metrics/Counter.java | 43 ++++++ .../org/apache/fluo/api/metrics/Histogram.java | 28 ++++ .../java/org/apache/fluo/api/metrics/Meter.java | 32 ++++ .../fluo/api/metrics/MetricsReporter.java | 48 ++++++ .../java/org/apache/fluo/api/metrics/Timer.java | 33 +++++ .../org/apache/fluo/api/observer/Observer.java | 5 + .../fluo/cluster/runnable/OracleRunnable.java | 5 +- .../fluo/cluster/runnable/WorkerRunnable.java | 5 +- .../apache/fluo/core/client/FluoClientImpl.java | 6 + .../core/client/LoaderExecutorAsyncImpl.java | 6 + .../org/apache/fluo/core/impl/Environment.java | 28 +++- .../java/org/apache/fluo/core/impl/TxStats.java | 20 +-- .../fluo/core/metrics/DummyMetricsReporter.java | 99 +++++++++++++ .../apache/fluo/core/metrics/MetricNames.java | 103 +++++++------ .../fluo/core/metrics/MetricsReporterImpl.java | 78 ++++++++++ .../fluo/core/metrics/types/CounterImpl.java | 47 ++++++ .../fluo/core/metrics/types/HistogramImpl.java | 32 ++++ .../fluo/core/metrics/types/MeterImpl.java | 37 +++++ .../fluo/core/metrics/types/TimerImpl.java | 34 +++++ .../fluo/core/worker/ObserverContext.java | 10 ++ .../fluo/integration/impl/ObserverConfigIT.java | 10 ++ 25 files changed, 754 insertions(+), 128 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/docs/grafana.md ---------------------------------------------------------------------- diff --git a/docs/grafana.md b/docs/grafana.md index cee0a04..e7310f6 100644 --- a/docs/grafana.md +++ b/docs/grafana.md @@ -1,8 +1,5 @@ # Fluo metrics in Grafana/InfluxDB -Fluo is instrumented using [dropwizard metrics][1] which allows Fluo to be configured to send -metrics to multiple metrics tools (such as Graphite, Ganglia, etc). - This document describes how to send Fluo metrics to [InfluxDB], a time series database, and make them viewable in [Grafana], a visualization tool. If you want general information on metrics, see the [Fluo metrics][2] documentation. @@ -30,9 +27,12 @@ Follow the instructions below to setup InfluxDB and Grafana. batch-pending = 5 batch-timeout = "1s" templates = [ - "fluo.*.*.tx.*.*.* .app.host.measurement.measurement.observer.field", - "fluo.*.*.*.*.* .app.host.measurement.measurement.field", - "fluo.*.*.*.* .app.host.measurement.measurement", + "fluo.class.*.*.*.*.* ..app.host.measurement.observer.field", + "fluo.class.*.*.*.* ..app.host.measurement.observer", + "fluo.system.*.*.*.* ..app.host.measurement.field", + "fluo.system.*.*.* ..app.host.measurement", + "fluo.app.*.*.* ..host.measurement.field", + "fluo.app.*.* ..host.measurement", ] ``` http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/docs/metrics.md ---------------------------------------------------------------------- diff --git a/docs/metrics.md b/docs/metrics.md index fd396bb..35319e4 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -1,69 +1,113 @@ # Fluo Metrics -Fluo core is instrumented using [dropwizard metrics][1]. This allows fluo users to easily gather -information about Fluo by configuring different reporters. While dropwizard can be configured to -report Fluo metrics to many different tools, below are some tools that have been used with Fluo. +A Fluo application can be configured (in [fluo.properties]) to report metrics. When metrics are +configured, Fluo will report some 'default' metrics about an application that help users monitor its +performance. Users can also write code to report 'application-specific' metrics from their +applications. Both 'application-specific' and 'default' metrics share the same reporter configured +by [fluo.properties] and are described in detail below. -1. [Grafana/InfluxDB][3] - Fluo has [documentation][3] for sending metrics to InfluxDB and viewing - them in Grafana. +## Configuring reporters -2. JMX - Fluo can be configured to reports metrics via JMX which can be viewed in jconsole or - jvisualvm. +Fluo metrics are not published by default. To publish metrics, configure a reporter in the 'metrics' +section of [fluo.properties]. There are several different reporter types (i.e Console, CSV, +Graphite, JMX, SLF4J) that are implemented using [Dropwizard]. The choice of which reporter to use +depends on the visualization tool used. If you are not currently using a visualization tool, there +is [documentation][grafana] for reporting Fluo metrics to Grafana/InfluxDB. -3. CSV - Fluo can be configured to output metrics as CSV to a specified directory. +## Metrics names -## Configuring Reporters +When Fluo metrics are reported, they are published using a naming scheme that encodes additional +information. This additional information is represented using all caps variables (i.e `METRIC`) +below. -In order to configure metrics reporters, look at the metrics section in an applications -`fluo.properties` file. This sections has a lot of commented out options for configuring reporters. +Default metrics start with `fluo.class` or `fluo.system` and have following naming schemes: - fluo.metrics.reporter.console.enable=false - fluo.metrics.reporter.console.frequency=30 + fluo.class.APPLICATION.REPORTER_ID.METRIC.CLASS + fluo.system.APPLICATION.REPORTER_ID.METRIC + +Application metrics start with `fluo.app` and have following scheme: + + fluo.app.REPORTER_ID.METRIC + +The variables below describe the additional information that is encoded in metrics names. -The frequency is in seconds for all reporters. +1. `APPLICATION` - Fluo application name +2. `REPORTER_ID` - Unique ID of the Fluo oracle, worker, or client that is reporting the metric. + When running in YARN, this ID is of the format `worker-INSTANCE_ID` or `oracle-INSTANCE_ID` + where `INSTANCE_ID` corresponds to instance number. When not running in YARN, this ID consists + of a hostname and a base36 long that is unique across all fluo processes. +3. `METRIC` - Name of the metric. For 'default' metrics, this is set by Fluo. For 'application' + metrics, this is set by user. Name should be unique and avoid using period '.' in name. +4. `CLASS` - Name of Fluo observer or loader class that produced metric. This allows things like + transaction collisions to be tracked per class. + +## Application-specific metrics -## Metrics reported by Fluo +Application metrics are implemented by retrieving a [MetricsReporter] from an [Observer], [Loader], +or [FluoClient]. These metrics are named using the format `fluo.app.REPORTER_ID.METRIC`. -All metrics reported by Fluo have the prefix `fluo.<APP>.<PID>.` which is denoted by `<prefix>` in -the table below. In the prefix, `<APP>` represents the Fluo application name and `<PID>` is the -process ID of the Fluo oracle or worker that is reporting the metric. When running in yarn, this id -is of the format `worker-<instance id>` or `oracle-<instance id>`. When not running from yarn, this -id consist of a hostname and a base36 long that is unique across all fluo processes. +## Default metrics -Some of the metrics reported have the class name as the suffix. This classname is the observer or -load task that executed the transactions. This should allow things like transaction collisions to -be tracked per class. In the table below this is denoted with `<cn>`. +Default metrics report for a particular Observer/Loader class or system-wide. -|Metric | Type | Description | -|---------------------------------------|----------------|-------------------------------------| -|\<prefix\>.tx.lock_wait_time.\<cn\> | [Timer][T] | *WHEN:* After each transaction. *COND:* > 0 *WHAT:* Time transaction spent waiting on locks held by other transactions. | -|\<prefix\>.tx.execution_time.\<cn\> | [Timer][T] | *WHEN:* After each transaction. *WHAT:* Time transaction took to execute. Updated for failed and successful transactions. This does not include commit time, only the time from start until commit is called. | -|\<prefix\>.tx.with_collision.\<cn\> | [Meter][M] | *WHEN:* After each transaction. *WHAT:* Rate of transactions with collisions. | -|\<prefix\>.tx.collisions.\<cn\> | [Meter][M] | *WHEN:* After each transaction. *WHAT:* Rate of collisions. | -|\<prefix\>.tx.entries_set.\<cn\> | [Meter][H] | *WHEN:* After each transaction. *WHAT:* Rate of row/columns set by transaction | -|\<prefix\>.tx.entries_read.\<cn\> | [Meter][H] | *WHEN:* After each transaction. *WHAT:* Rate of row/columns read by transaction that existed. There is currently no count of all reads (including non-existent data) | -|\<prefix\>.tx.locks_timedout.\<cn\> | [Meter][M] | *WHEN:* After each transaction. *WHAT:* Rate of timedout locks rolled back by transaction. These are locks that are held for very long periods by another transaction that appears to be alive based on zookeeper. | -|\<prefix\>.tx.locks_dead.\<cn\> | [Meter][M] | *WHEN:* After each transaction. *WHAT:* Rate of dead locks rolled by a transaction. These are locks held by a process that appears to be dead according to zookeeper. | -|\<prefix\>.tx.status_\<status\>.\<cn\> | [Meter][M] | *WHEN:* After each transaction. *WHAT:* Rate of different ways a transaction can terminate | -|\<prefix\>.oracle.response_time | [Timer][T] | *WHEN:* For each request for stamps to the server. *WHAT:* Time RPC call to oracle took | -|\<prefix\>.oracle.client_stamps | [Histogram][H] | *WHEN:* For each request for stamps to the server. *WHAT:* The number of stamps requested. | -|\<prefix\>.oracle.server_stamps | [Histogram][H] | *WHEN:* For each request for stamps from a client. *WHAT:* The number of stamps requested. | -|\<prefix\>.worker.notifications_queued | [Gauge][G] | *WHAT:* The current number of notifications queued for processing. | -|\<prefix\>.transactor.committing | [Gauge][G] | *WHAT:* The current number of transactions that are working their way through the commit steps. | +Below are metrics that are reported from each Observer/Loader class that is configured in a Fluo +application. These metrics are reported after each transaction and named using the format +`fluo.class.APPLICATION.REPORTER_ID.METRIC.CLASS`. -The table above outlines when a particular metric is updated and whats updated. The use of *COND* -indicates that the metric is not always updated. For example `i.f.<pid>.tx.lockWait.<cn>` is only -updated for transactions that had a non zero lock wait time. +* tx_lock_wait_time - [Timer] + - Time transaction spent waiting on locks held by other transactions. + - Only updated for transactions that have non-zero lock time. +* tx_execution_time - [Timer] + - Time transaction took to execute. + - Updated for failed and successful transactions. + - This does not include commit time, only the time from start until commit is called. +* tx_with_collision - [Meter] + - Rate of transactions with collisions. +* tx_collisions - [Meter] + - Rate of collisions. +* tx_entries_set - [Meter] + - Rate of row/columns set by transaction +* tx_entries_read - [Meter] + - Rate of row/columns read by transaction that existed. + - There is currently no count of all reads (including non-existent data) +* tx_locks_timedout - [Meter] + - Rate of timedout locks rolled back by transaction. + - These are locks that are held for very long periods by another transaction that appears to be + alive based on zookeeper. +* tx_locks_dead - [Meter] + - Rate of dead locks rolled by a transaction. + - These are locks held by a process that appears to be dead according to zookeeper. +* tx_status_`<STATUS>` - [Meter] + - Rate of different ways (i.e `<STATUS>`) a transaction can terminate + +Below are system-wide metrics that are reported for the entire Fluo application. These metrics are +named using the format `fluo.system.APPLICATION.REPORTER_ID.METRIC`. + +* oracle_response_time - [Timer] + - Time each RPC call to oracle for stamps took +* oracle_client_stamps - [Histogram] + - Number of stamps requested for each request for stamps to the server +* oracle_server_stamps - [Histogram] + - Number of stamps requested for each request for stamps from a client +* worker_notifications_queued - [Gauge] + - The current number of notifications queued for processing. +* transactor_committing - [Gauge] + - The current number of transactions that are working their way through the commit steps. Histograms and Timers have a counter. In the case of a histogram, the counter is the number of times the metric was updated and not a sum of the updates. For example if a request for 5 timestamps was -made to the oracle followed by a request for 3 timestamps, then the count for -`i.f.<pid>.oracle.server.stamps` would be 2 and the mean would be (5+3)/2. - -[1]: https://dropwizard.github.io/metrics/3.1.0/ -[3]: grafana.md -[T]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#timers -[C]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#counters -[H]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#histograms -[G]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#gauges -[M]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#meters +made to the oracle followed by a request for 3 timestamps, then the count for `oracle_server_stamps` +would be 2 and the mean would be (5+3)/2. + +[fluo.properties]: ../modules/distribution/src/main/config/fluo.properties +[Dropwizard]: https://dropwizard.github.io/metrics/3.1.0/ +[grafana]: grafana.md +[MetricsReporter]: ../modules/api/src/main/java/org/apache/fluo/api/metrics/MetricsReporter.java +[Observer]: ../modules/api/src/main/java/org/apache/fluo/api/observer/Observer.java +[Loader]: ../modules/api/src/main/java/org/apache/fluo/api/client/Loader.java +[FluoClient]: ../modules/api/src/main/java/org/apache/fluo/api/client/FluoClient.java +[Timer]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#timers +[Counter]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#counters +[Histogram]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#histograms +[Gauge]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#gauges +[Meter]: https://dropwizard.github.io/metrics/3.1.0/getting-started/#meters http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/api/src/main/java/org/apache/fluo/api/client/FluoClient.java ---------------------------------------------------------------------- diff --git a/modules/api/src/main/java/org/apache/fluo/api/client/FluoClient.java b/modules/api/src/main/java/org/apache/fluo/api/client/FluoClient.java index 4f64a16..140351e 100644 --- a/modules/api/src/main/java/org/apache/fluo/api/client/FluoClient.java +++ b/modules/api/src/main/java/org/apache/fluo/api/client/FluoClient.java @@ -16,6 +16,7 @@ package org.apache.fluo.api.client; import org.apache.fluo.api.config.SimpleConfiguration; +import org.apache.fluo.api.metrics.MetricsReporter; /** * Client interface for Fluo. Fluo clients will have shared resources used by all objects created by @@ -63,10 +64,14 @@ public interface FluoClient extends AutoCloseable { * {@link FluoAdmin#updateSharedConfig()}. Changes made to the returned Configuration will * not update Zookeeper. */ - SimpleConfiguration getAppConfiguration(); /** + * @return A {@link MetricsReporter} that is used to report application metrics + */ + MetricsReporter getMetricsReporter(); + + /** * Closes client resources */ @Override http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/api/src/main/java/org/apache/fluo/api/client/Loader.java ---------------------------------------------------------------------- diff --git a/modules/api/src/main/java/org/apache/fluo/api/client/Loader.java b/modules/api/src/main/java/org/apache/fluo/api/client/Loader.java index 1557112..903e2f6 100644 --- a/modules/api/src/main/java/org/apache/fluo/api/client/Loader.java +++ b/modules/api/src/main/java/org/apache/fluo/api/client/Loader.java @@ -16,6 +16,7 @@ package org.apache.fluo.api.client; import org.apache.fluo.api.config.SimpleConfiguration; +import org.apache.fluo.api.metrics.MetricsReporter; /** * Interface that is implemented by users to load data into Fluo. Loader classes are executed by a @@ -34,6 +35,11 @@ public interface Loader { * {@link FluoClient#getAppConfiguration()} */ SimpleConfiguration getAppConfiguration(); + + /** + * @return A {@link MetricsReporter} to report application metrics from this observer + */ + MetricsReporter getMetricsReporter(); } /** http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/api/src/main/java/org/apache/fluo/api/metrics/Counter.java ---------------------------------------------------------------------- diff --git a/modules/api/src/main/java/org/apache/fluo/api/metrics/Counter.java b/modules/api/src/main/java/org/apache/fluo/api/metrics/Counter.java new file mode 100644 index 0000000..41dfb99 --- /dev/null +++ b/modules/api/src/main/java/org/apache/fluo/api/metrics/Counter.java @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.api.metrics; + +/** + * Metrics Counter. See http://metrics.dropwizard.io/3.1.0/getting-started/#counters + */ +public interface Counter { + + /** + * Increments counter by 1 + */ + void inc(); + + /** + * Increments counter by value + */ + void inc(long value); + + /** + * Decrements counter by 1 + */ + void dec(); + + /** + * Decrements counter by value + */ + void dec(long value); + +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/api/src/main/java/org/apache/fluo/api/metrics/Histogram.java ---------------------------------------------------------------------- diff --git a/modules/api/src/main/java/org/apache/fluo/api/metrics/Histogram.java b/modules/api/src/main/java/org/apache/fluo/api/metrics/Histogram.java new file mode 100644 index 0000000..c561061 --- /dev/null +++ b/modules/api/src/main/java/org/apache/fluo/api/metrics/Histogram.java @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.api.metrics; + +/** + * Metrics Histogram. See http://metrics.dropwizard.io/3.1.0/getting-started/#histograms + */ +public interface Histogram { + + /** + * Adds recorded value + */ + void update(long value); + +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/api/src/main/java/org/apache/fluo/api/metrics/Meter.java ---------------------------------------------------------------------- diff --git a/modules/api/src/main/java/org/apache/fluo/api/metrics/Meter.java b/modules/api/src/main/java/org/apache/fluo/api/metrics/Meter.java new file mode 100644 index 0000000..03fa6b8 --- /dev/null +++ b/modules/api/src/main/java/org/apache/fluo/api/metrics/Meter.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.api.metrics; + +/** + * Metrics Meter. See http://metrics.dropwizard.io/3.1.0/getting-started/#meters + */ +public interface Meter { + + /** + * Mark the occurrence of event + */ + void mark(); + + /** + * Mark the occurrence of numEvents + */ + void mark(long numEvents); +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/api/src/main/java/org/apache/fluo/api/metrics/MetricsReporter.java ---------------------------------------------------------------------- diff --git a/modules/api/src/main/java/org/apache/fluo/api/metrics/MetricsReporter.java b/modules/api/src/main/java/org/apache/fluo/api/metrics/MetricsReporter.java new file mode 100644 index 0000000..12a1bc7 --- /dev/null +++ b/modules/api/src/main/java/org/apache/fluo/api/metrics/MetricsReporter.java @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.api.metrics; + +/** + * Reports application metrics using Fluo metrics reporters configured by 'fluo.metrics.reporter.*' + * properties. Several types of metrics are supported which are described by Dropwizard docs + * (http://metrics.dropwizard.io/3.1.0/getting-started/). Metrics should be identified by a unique + * names to avoid conflicts. Periods "." should not be used in metric names. + * + * @since 1.0.0 + */ +public interface MetricsReporter { + + /** + * @return Metrics {@link Counter} identified by metricName + */ + Counter counter(String metricName); + + /** + * @return Metrics {@link Histogram} identified by metricName + */ + Histogram histogram(String metricName); + + /** + * @return Metrics {@link Meter} identified by metricName + */ + Meter meter(String metricName); + + /** + * @return Metrics {@link Timer} identified by metricName + */ + Timer timer(String metricName); + +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/api/src/main/java/org/apache/fluo/api/metrics/Timer.java ---------------------------------------------------------------------- diff --git a/modules/api/src/main/java/org/apache/fluo/api/metrics/Timer.java b/modules/api/src/main/java/org/apache/fluo/api/metrics/Timer.java new file mode 100644 index 0000000..e4f64ef --- /dev/null +++ b/modules/api/src/main/java/org/apache/fluo/api/metrics/Timer.java @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.api.metrics; + +import java.util.concurrent.TimeUnit; + +/** + * Metrics Timer. See http://metrics.dropwizard.io/3.1.0/getting-started/#timers + */ +public interface Timer { + + /** + * Adds recorded duration + * + * @param duration Duration with scale of unit + * @param unit Time unit of duration + */ + void update(long duration, TimeUnit unit); + +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/api/src/main/java/org/apache/fluo/api/observer/Observer.java ---------------------------------------------------------------------- diff --git a/modules/api/src/main/java/org/apache/fluo/api/observer/Observer.java b/modules/api/src/main/java/org/apache/fluo/api/observer/Observer.java index 6835b99..381784a 100644 --- a/modules/api/src/main/java/org/apache/fluo/api/observer/Observer.java +++ b/modules/api/src/main/java/org/apache/fluo/api/observer/Observer.java @@ -20,6 +20,7 @@ import org.apache.fluo.api.client.TransactionBase; import org.apache.fluo.api.config.SimpleConfiguration; import org.apache.fluo.api.data.Bytes; import org.apache.fluo.api.data.Column; +import org.apache.fluo.api.metrics.MetricsReporter; /** * Implemented by users to a watch a {@link Column} and be notified of changes to the Column via the @@ -77,6 +78,10 @@ public interface Observer { */ SimpleConfiguration getObserverConfiguration(); + /** + * @return A {@link MetricsReporter} to report application metrics from this observer + */ + MetricsReporter getMetricsReporter(); } /** http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/OracleRunnable.java ---------------------------------------------------------------------- diff --git a/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/OracleRunnable.java b/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/OracleRunnable.java index 4bdc6d4..277f7a8 100644 --- a/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/OracleRunnable.java +++ b/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/OracleRunnable.java @@ -72,8 +72,9 @@ public class OracleRunnable extends AbstractTwillRunnable { FluoConfiguration config = new FluoConfiguration(propsFile); TwillContext context = getContext(); - if (context != null && System.getProperty(MetricNames.METRICS_ID_PROP) == null) { - System.setProperty(MetricNames.METRICS_ID_PROP, "oracle-" + context.getInstanceId()); + if (context != null && System.getProperty(MetricNames.METRICS_REPORTER_ID_PROP) == null) { + System.setProperty(MetricNames.METRICS_REPORTER_ID_PROP, + "oracle-" + context.getInstanceId()); } // FluoFactory cannot be used to create FluoOracle as Twill will not load its dependencies http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/WorkerRunnable.java ---------------------------------------------------------------------- diff --git a/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/WorkerRunnable.java b/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/WorkerRunnable.java index f50dd7a..a1bf9b6 100644 --- a/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/WorkerRunnable.java +++ b/modules/cluster/src/main/java/org/apache/fluo/cluster/runnable/WorkerRunnable.java @@ -86,8 +86,9 @@ public class WorkerRunnable extends AbstractTwillRunnable { } TwillContext context = getContext(); - if (context != null && System.getProperty(MetricNames.METRICS_ID_PROP) == null) { - System.setProperty(MetricNames.METRICS_ID_PROP, "worker-" + context.getInstanceId()); + if (context != null && System.getProperty(MetricNames.METRICS_REPORTER_ID_PROP) == null) { + System.setProperty(MetricNames.METRICS_REPORTER_ID_PROP, + "worker-" + context.getInstanceId()); } // FluoFactory cannot be used to create FluoWorker as Twill will not load its dependencies http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/client/FluoClientImpl.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/client/FluoClientImpl.java b/modules/core/src/main/java/org/apache/fluo/core/client/FluoClientImpl.java index 662ef0d..11b99e1 100644 --- a/modules/core/src/main/java/org/apache/fluo/core/client/FluoClientImpl.java +++ b/modules/core/src/main/java/org/apache/fluo/core/client/FluoClientImpl.java @@ -23,6 +23,7 @@ import org.apache.fluo.api.client.Snapshot; import org.apache.fluo.api.client.Transaction; import org.apache.fluo.api.config.FluoConfiguration; import org.apache.fluo.api.config.SimpleConfiguration; +import org.apache.fluo.api.metrics.MetricsReporter; import org.apache.fluo.core.impl.Environment; import org.apache.fluo.core.impl.TransactionImpl; import org.apache.fluo.core.log.TracingTransaction; @@ -105,6 +106,11 @@ public class FluoClientImpl implements FluoClient { } @Override + public MetricsReporter getMetricsReporter() { + return env.getMetricsReporter(); + } + + @Override public void close() { env.close(); try { http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/client/LoaderExecutorAsyncImpl.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/client/LoaderExecutorAsyncImpl.java b/modules/core/src/main/java/org/apache/fluo/core/client/LoaderExecutorAsyncImpl.java index b05467a..be375c6 100644 --- a/modules/core/src/main/java/org/apache/fluo/core/client/LoaderExecutorAsyncImpl.java +++ b/modules/core/src/main/java/org/apache/fluo/core/client/LoaderExecutorAsyncImpl.java @@ -27,6 +27,7 @@ import org.apache.fluo.api.client.Loader; import org.apache.fluo.api.client.LoaderExecutor; import org.apache.fluo.api.config.FluoConfiguration; import org.apache.fluo.api.config.SimpleConfiguration; +import org.apache.fluo.api.metrics.MetricsReporter; import org.apache.fluo.core.async.AsyncCommitObserver; import org.apache.fluo.core.async.AsyncTransaction; import org.apache.fluo.core.impl.Environment; @@ -118,6 +119,11 @@ public class LoaderExecutorAsyncImpl implements LoaderExecutor { public SimpleConfiguration getAppConfiguration() { return env.getAppConfiguration(); } + + @Override + public MetricsReporter getMetricsReporter() { + return env.getMetricsReporter(); + } }; try { http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/impl/Environment.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/impl/Environment.java b/modules/core/src/main/java/org/apache/fluo/core/impl/Environment.java index e1bf8fc..30b398d 100644 --- a/modules/core/src/main/java/org/apache/fluo/core/impl/Environment.java +++ b/modules/core/src/main/java/org/apache/fluo/core/impl/Environment.java @@ -38,7 +38,9 @@ import org.apache.fluo.api.config.FluoConfiguration; import org.apache.fluo.api.config.ObserverSpecification; import org.apache.fluo.api.config.SimpleConfiguration; import org.apache.fluo.api.data.Column; +import org.apache.fluo.api.metrics.MetricsReporter; import org.apache.fluo.core.metrics.MetricNames; +import org.apache.fluo.core.metrics.MetricsReporterImpl; import org.apache.fluo.core.util.AccumuloUtil; import org.apache.fluo.core.util.ColumnUtil; import org.apache.fluo.core.util.CuratorUtil; @@ -62,6 +64,7 @@ public class Environment implements AutoCloseable { private SharedResources resources; private MetricNames metricNames; private SimpleConfiguration appConfig; + private String metricsReporterID; /** * Constructs an environment from given FluoConfiguration @@ -237,9 +240,9 @@ public class Environment implements AutoCloseable { return config; } - public synchronized MetricNames getMetricNames() { - if (metricNames == null) { - String mid = System.getProperty(MetricNames.METRICS_ID_PROP); + public synchronized String getMetricsReporterID() { + if (metricsReporterID == null) { + String mid = System.getProperty(MetricNames.METRICS_REPORTER_ID_PROP); if (mid == null) { try { String hostname = InetAddress.getLocalHost().getHostName(); @@ -252,14 +255,27 @@ public class Environment implements AutoCloseable { throw new RuntimeException(e); } } + metricsReporterID = mid.replace('.', '_'); + } + return metricsReporterID; + } + + public String getMetricsAppName() { + return config.getApplicationName().replace('.', '_'); + } - mid = mid.replace('.', '_'); - String appName = config.getApplicationName().replace('.', '_'); - metricNames = new MetricNames(mid, appName); + public synchronized MetricNames getMetricNames() { + if (metricNames == null) { + metricNames = new MetricNames(getMetricsReporterID(), getMetricsAppName()); } return metricNames; } + public MetricsReporter getMetricsReporter() { + return new MetricsReporterImpl(getConfiguration(), getSharedResources().getMetricRegistry(), + getMetricsReporterID()); + } + public SimpleConfiguration getAppConfiguration() { // TODO create immutable wrapper return new SimpleConfiguration(appConfig); http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/impl/TxStats.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/impl/TxStats.java b/modules/core/src/main/java/org/apache/fluo/core/impl/TxStats.java index a70516d..f1ae480 100644 --- a/modules/core/src/main/java/org/apache/fluo/core/impl/TxStats.java +++ b/modules/core/src/main/java/org/apache/fluo/core/impl/TxStats.java @@ -147,24 +147,24 @@ public class TxStats { MetricRegistry registry = env.getSharedResources().getMetricRegistry(); String sn = execClass.getSimpleName(); if (getLockWaitTime() > 0) { - MetricsUtil.getTimer(env.getConfiguration(), registry, names.getTxLockWaitTime() + sn) - .update(getLockWaitTime(), TimeUnit.MILLISECONDS); + MetricsUtil.getTimer(env.getConfiguration(), registry, names.getTxLockWaitTime(sn)).update( + getLockWaitTime(), TimeUnit.MILLISECONDS); } - MetricsUtil.getTimer(env.getConfiguration(), registry, names.getTxExecTime() + sn).update( + MetricsUtil.getTimer(env.getConfiguration(), registry, names.getTxExecTime(sn)).update( getReadTime(), TimeUnit.MILLISECONDS); if (getCollisions() > 0) { - registry.meter(names.getTxWithCollision() + sn).mark(); - registry.meter(names.getTxCollisions() + sn).mark(getCollisions()); + registry.meter(names.getTxWithCollision(sn)).mark(); + registry.meter(names.getTxCollisions(sn)).mark(getCollisions()); } - registry.meter(names.getTxEntriesSet() + sn).mark(getEntriesSet()); - registry.meter(names.getTxEntriesRead() + sn).mark(getEntriesReturned()); + registry.meter(names.getTxEntriesSet(sn)).mark(getEntriesSet()); + registry.meter(names.getTxEntriesRead(sn)).mark(getEntriesReturned()); if (getTimedOutLocks() > 0) { - registry.meter(names.getTxLocksTimedout() + sn).mark(getTimedOutLocks()); + registry.meter(names.getTxLocksTimedout(sn)).mark(getTimedOutLocks()); } if (getDeadLocks() > 0) { - registry.meter(names.getTxLocksDead() + sn).mark(getDeadLocks()); + registry.meter(names.getTxLocksDead(sn)).mark(getDeadLocks()); } - registry.meter(names.getTxStatus() + status.toLowerCase() + "." + sn).mark(); + registry.meter(names.getTxStatus(status.toLowerCase(), sn)).mark(); } public void setCommitBeginTime(long t) { http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/metrics/DummyMetricsReporter.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/metrics/DummyMetricsReporter.java b/modules/core/src/main/java/org/apache/fluo/core/metrics/DummyMetricsReporter.java new file mode 100644 index 0000000..2989d55 --- /dev/null +++ b/modules/core/src/main/java/org/apache/fluo/core/metrics/DummyMetricsReporter.java @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.core.metrics; + +import java.util.concurrent.TimeUnit; + +import org.apache.fluo.api.metrics.Counter; +import org.apache.fluo.api.metrics.Histogram; +import org.apache.fluo.api.metrics.Meter; +import org.apache.fluo.api.metrics.MetricsReporter; +import org.apache.fluo.api.metrics.Timer; + +public class DummyMetricsReporter implements MetricsReporter { + + @Override + public Counter counter(String name) { + return new DummyCounter(); + } + + @Override + public Histogram histogram(String name) { + return new DummyHistogram(); + } + + @Override + public Meter meter(String name) { + return new DummyMeter(); + } + + @Override + public Timer timer(String name) { + return new DummyTimer(); + } + + class DummyCounter implements Counter { + + @Override + public void inc() { + throw new UnsupportedOperationException(); + } + + @Override + public void inc(long value) { + throw new UnsupportedOperationException(); + } + + @Override + public void dec() { + throw new UnsupportedOperationException(); + } + + @Override + public void dec(long value) { + throw new UnsupportedOperationException(); + } + } + + class DummyHistogram implements Histogram { + + @Override + public void update(long value) { + throw new UnsupportedOperationException(); + } + } + + class DummyMeter implements Meter { + + @Override + public void mark() { + throw new UnsupportedOperationException(); + } + + @Override + public void mark(long numEvents) { + throw new UnsupportedOperationException(); + } + } + + class DummyTimer implements Timer { + + @Override + public void update(long duration, TimeUnit unit) { + throw new UnsupportedOperationException(); + } + } +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/metrics/MetricNames.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/metrics/MetricNames.java b/modules/core/src/main/java/org/apache/fluo/core/metrics/MetricNames.java index c7939ec..3842a84 100644 --- a/modules/core/src/main/java/org/apache/fluo/core/metrics/MetricNames.java +++ b/modules/core/src/main/java/org/apache/fluo/core/metrics/MetricNames.java @@ -20,6 +20,16 @@ import org.apache.fluo.api.config.FluoConfiguration; public class MetricNames { + public static final String METRICS_REPORTER_ID_PROP = FluoConfiguration.FLUO_PREFIX + + ".metrics.reporter.id"; + + // Metrics prefixes for 'default' metrics + public static final String CLASS_PREFIX = FluoConfiguration.FLUO_PREFIX + ".class"; + public static final String SYSTEM_PREFIX = FluoConfiguration.FLUO_PREFIX + ".system"; + + // Metrics prefix for 'application' metrics + public static final String APPLICATION_PREFIX = FluoConfiguration.FLUO_PREFIX + ".app"; + private final String txLockWaitTime; private final String txExecTime; private final String txWithCollision; @@ -28,7 +38,7 @@ public class MetricNames { private final String txEntriesRead; private final String txLocksTimedOut; private final String txLocksDead; - private final String txStatus; + private final String txStatusPrefix; private final String txCommitting; private final String notificationsQueued; @@ -37,74 +47,69 @@ public class MetricNames { private final String oracleClientStamps; private final String oracleServerStamps; - public static final String METRICS_ID_PROP = FluoConfiguration.FLUO_PREFIX + ".metrics.id"; - - public MetricNames(String hostId, String appName) { + public MetricNames(String metricsReporterId, String appName) { Preconditions.checkArgument(!appName.contains("."), "Fluo App name should not contain '.': " + appName); - Preconditions.checkArgument(!hostId.contains("."), "Host ID should not contain '.': " + hostId); - - // All metrics start with prefix "fluo.APP.HOST." - final String metricsPrefix = FluoConfiguration.FLUO_PREFIX + "." + appName + "." + hostId + "."; - - // Transaction metrics: fluo.APP.HOST.tx.METRIC.OBSERVER - final String txPrefix = metricsPrefix + "tx."; - txLockWaitTime = txPrefix + "lock_wait_time."; - txExecTime = txPrefix + "execution_time."; - txWithCollision = txPrefix + "with_collision."; - txCollisions = txPrefix + "collisions."; - txEntriesSet = txPrefix + "entries_set."; - txEntriesRead = txPrefix + "entries_read."; - txLocksTimedOut = txPrefix + "locks_timedout."; - txLocksDead = txPrefix + "locks_dead."; - txStatus = txPrefix + "status_"; - - txCommitting = metricsPrefix + "transactor.committing"; - - // Worker metrics: fluo.APP.HOST.worker.METRIC - notificationsQueued = metricsPrefix + "worker.notifications_queued"; - - // Oracle metrics: fluo.APP.HOST.oracle.METRIC - final String oraclePrefix = metricsPrefix + "oracle."; - oracleResponseTime = oraclePrefix + "response_time"; - oracleClientStamps = oraclePrefix + "client_stamps"; - oracleServerStamps = oraclePrefix + "server_stamps"; + Preconditions.checkArgument(!metricsReporterId.contains("."), + "Metrics Reporter ID should not contain '.': " + metricsReporterId); + + // Metrics reported for a specific class + // FORMAT: fluo.class.APPLICATION.REPORTER_ID.METRIC.CLASS + final String classMetric = CLASS_PREFIX + "." + appName + "." + metricsReporterId + "."; + txLockWaitTime = classMetric + "tx_lock_wait_time"; + txExecTime = classMetric + "tx_execution_time"; + txWithCollision = classMetric + "tx_with_collision"; + txCollisions = classMetric + "tx_collisions"; + txEntriesSet = classMetric + "tx_entries_set"; + txEntriesRead = classMetric + "tx_entries_read"; + txLocksTimedOut = classMetric + "tx_locks_timedout"; + txLocksDead = classMetric + "tx_locks_dead"; + txStatusPrefix = classMetric + "tx_status_"; // status appended to metric name + + // System-wide metrics + // FORMAT: fluo.system.APPLICATION.REPORTER_ID.METRIC + final String systemMetric = SYSTEM_PREFIX + "." + appName + "." + metricsReporterId + "."; + txCommitting = systemMetric + "transactor_committing"; + notificationsQueued = systemMetric + "worker_notifications_queued"; + oracleResponseTime = systemMetric + "oracle_response_time"; + oracleClientStamps = systemMetric + "oracle_client_stamps"; + oracleServerStamps = systemMetric + "oracle_server_stamps"; } - public String getTxLockWaitTime() { - return txLockWaitTime; + public String getTxLockWaitTime(String className) { + return txLockWaitTime + "." + className; } - public String getTxExecTime() { - return txExecTime; + public String getTxExecTime(String className) { + return txExecTime + "." + className; } - public String getTxWithCollision() { - return txWithCollision; + public String getTxWithCollision(String className) { + return txWithCollision + "." + className; } - public String getTxCollisions() { - return txCollisions; + public String getTxCollisions(String className) { + return txCollisions + "." + className; } - public String getTxEntriesSet() { - return txEntriesSet; + public String getTxEntriesSet(String className) { + return txEntriesSet + "." + className; } - public String getTxEntriesRead() { - return txEntriesRead; + public String getTxEntriesRead(String className) { + return txEntriesRead + "." + className; } - public String getTxLocksTimedout() { - return txLocksTimedOut; + public String getTxLocksTimedout(String className) { + return txLocksTimedOut + "." + className; } - public String getTxLocksDead() { - return txLocksDead; + public String getTxLocksDead(String className) { + return txLocksDead + "." + className; } - public String getTxStatus() { - return txStatus; + public String getTxStatus(String status, String className) { + return txStatusPrefix + status + "." + className; } public String getNotificationQueued() { http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/metrics/MetricsReporterImpl.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/metrics/MetricsReporterImpl.java b/modules/core/src/main/java/org/apache/fluo/core/metrics/MetricsReporterImpl.java new file mode 100644 index 0000000..d29c9cb --- /dev/null +++ b/modules/core/src/main/java/org/apache/fluo/core/metrics/MetricsReporterImpl.java @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.core.metrics; + +import java.util.Objects; + +import com.codahale.metrics.MetricRegistry; +import com.google.common.base.Preconditions; +import org.apache.fluo.api.config.FluoConfiguration; +import org.apache.fluo.api.metrics.Counter; +import org.apache.fluo.api.metrics.Histogram; +import org.apache.fluo.api.metrics.Meter; +import org.apache.fluo.api.metrics.MetricsReporter; +import org.apache.fluo.api.metrics.Timer; +import org.apache.fluo.core.metrics.types.CounterImpl; +import org.apache.fluo.core.metrics.types.HistogramImpl; +import org.apache.fluo.core.metrics.types.MeterImpl; +import org.apache.fluo.core.metrics.types.TimerImpl; + +/** + * Implementation of {@link MetricsReporter} that reports application metrics using Fluo metrics + */ +public class MetricsReporterImpl implements MetricsReporter { + + private final FluoConfiguration config; + private final MetricRegistry registry; + private final String prefix; + + public MetricsReporterImpl(FluoConfiguration config, MetricRegistry registry, + String metricsReporterID) { + this.config = config; + this.registry = registry; + this.prefix = MetricNames.APPLICATION_PREFIX + "." + metricsReporterID + "."; + } + + @Override + public Counter counter(String metricName) { + validateName(metricName); + return new CounterImpl(registry.counter(prefix + metricName)); + } + + @Override + public Histogram histogram(String metricName) { + validateName(metricName); + return new HistogramImpl(MetricsUtil.getHistogram(config, registry, prefix + metricName)); + } + + @Override + public Meter meter(String metricName) { + validateName(metricName); + return new MeterImpl(registry.meter(prefix + metricName)); + } + + @Override + public Timer timer(String metricName) { + validateName(metricName); + return new TimerImpl(MetricsUtil.getTimer(config, registry, prefix + metricName)); + } + + private static void validateName(String metricName) { + Objects.requireNonNull(metricName); + Preconditions.checkArgument(!metricName.contains("."), "Metric name " + metricName + + " should not contain a period '.'"); + } +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/metrics/types/CounterImpl.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/metrics/types/CounterImpl.java b/modules/core/src/main/java/org/apache/fluo/core/metrics/types/CounterImpl.java new file mode 100644 index 0000000..95aae81 --- /dev/null +++ b/modules/core/src/main/java/org/apache/fluo/core/metrics/types/CounterImpl.java @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.core.metrics.types; + +import org.apache.fluo.api.metrics.Counter; + +public class CounterImpl implements Counter { + + com.codahale.metrics.Counter counter; + + public CounterImpl(com.codahale.metrics.Counter counter) { + this.counter = counter; + } + + @Override + public void inc() { + counter.inc(); + } + + @Override + public void inc(long value) { + counter.inc(value); + } + + @Override + public void dec() { + counter.dec(); + } + + @Override + public void dec(long value) { + counter.dec(value); + } +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/metrics/types/HistogramImpl.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/metrics/types/HistogramImpl.java b/modules/core/src/main/java/org/apache/fluo/core/metrics/types/HistogramImpl.java new file mode 100644 index 0000000..2a15bab --- /dev/null +++ b/modules/core/src/main/java/org/apache/fluo/core/metrics/types/HistogramImpl.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.core.metrics.types; + +import org.apache.fluo.api.metrics.Histogram; + +public class HistogramImpl implements Histogram { + + com.codahale.metrics.Histogram histogram; + + public HistogramImpl(com.codahale.metrics.Histogram histogram) { + this.histogram = histogram; + } + + @Override + public void update(long value) { + histogram.update(value); + } +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/metrics/types/MeterImpl.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/metrics/types/MeterImpl.java b/modules/core/src/main/java/org/apache/fluo/core/metrics/types/MeterImpl.java new file mode 100644 index 0000000..54ebf6f --- /dev/null +++ b/modules/core/src/main/java/org/apache/fluo/core/metrics/types/MeterImpl.java @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.core.metrics.types; + +import org.apache.fluo.api.metrics.Meter; + +public class MeterImpl implements Meter { + + com.codahale.metrics.Meter meter; + + public MeterImpl(com.codahale.metrics.Meter meter) { + this.meter = meter; + } + + @Override + public void mark() { + meter.mark(); + } + + @Override + public void mark(long numEvents) { + meter.mark(numEvents); + } +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/metrics/types/TimerImpl.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/metrics/types/TimerImpl.java b/modules/core/src/main/java/org/apache/fluo/core/metrics/types/TimerImpl.java new file mode 100644 index 0000000..cad0b8b --- /dev/null +++ b/modules/core/src/main/java/org/apache/fluo/core/metrics/types/TimerImpl.java @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license + * agreements. See the NOTICE file distributed with this work for additional information regarding + * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. You may obtain a + * copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + */ + +package org.apache.fluo.core.metrics.types; + +import java.util.concurrent.TimeUnit; + +import org.apache.fluo.api.metrics.Timer; + +public class TimerImpl implements Timer { + + com.codahale.metrics.Timer timer; + + public TimerImpl(com.codahale.metrics.Timer timer) { + this.timer = timer; + } + + @Override + public void update(long duration, TimeUnit unit) { + timer.update(duration, unit); + } +} http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/core/src/main/java/org/apache/fluo/core/worker/ObserverContext.java ---------------------------------------------------------------------- diff --git a/modules/core/src/main/java/org/apache/fluo/core/worker/ObserverContext.java b/modules/core/src/main/java/org/apache/fluo/core/worker/ObserverContext.java index ab0eedc..fadea3e 100644 --- a/modules/core/src/main/java/org/apache/fluo/core/worker/ObserverContext.java +++ b/modules/core/src/main/java/org/apache/fluo/core/worker/ObserverContext.java @@ -16,8 +16,11 @@ package org.apache.fluo.core.worker; import org.apache.fluo.api.config.SimpleConfiguration; +import org.apache.fluo.api.metrics.MetricsReporter; import org.apache.fluo.api.observer.Observer; import org.apache.fluo.core.impl.Environment; +import org.apache.fluo.core.metrics.DummyMetricsReporter; +import org.apache.fluo.core.metrics.MetricsReporterImpl; public class ObserverContext implements Observer.Context { @@ -50,4 +53,11 @@ public class ObserverContext implements Observer.Context { return observerConfig; } + @Override + public MetricsReporter getMetricsReporter() { + if (env == null) { + return new DummyMetricsReporter(); + } + return env.getMetricsReporter(); + } } http://git-wip-us.apache.org/repos/asf/incubator-fluo/blob/0d3acf85/modules/integration/src/test/java/org/apache/fluo/integration/impl/ObserverConfigIT.java ---------------------------------------------------------------------- diff --git a/modules/integration/src/test/java/org/apache/fluo/integration/impl/ObserverConfigIT.java b/modules/integration/src/test/java/org/apache/fluo/integration/impl/ObserverConfigIT.java index c97d000..a2c624f 100644 --- a/modules/integration/src/test/java/org/apache/fluo/integration/impl/ObserverConfigIT.java +++ b/modules/integration/src/test/java/org/apache/fluo/integration/impl/ObserverConfigIT.java @@ -27,7 +27,10 @@ import org.apache.fluo.api.config.ObserverSpecification; import org.apache.fluo.api.config.SimpleConfiguration; import org.apache.fluo.api.data.Bytes; import org.apache.fluo.api.data.Column; +import org.apache.fluo.api.metrics.Counter; +import org.apache.fluo.api.metrics.Meter; import org.apache.fluo.api.observer.AbstractObserver; +import org.apache.fluo.api.metrics.MetricsReporter; import org.apache.fluo.api.observer.Observer.NotificationType; import org.apache.fluo.integration.ITBaseMini; import org.junit.Assert; @@ -40,6 +43,8 @@ public class ObserverConfigIT extends ITBaseMini { private ObservedColumn observedColumn; private Bytes outputCQ; private boolean setWeakNotification = false; + private Meter meter; + private Counter counter; @Override public void init(Context context) { @@ -54,11 +59,16 @@ public class ObserverConfigIT extends ITBaseMini { if (swn.equals("true")) { setWeakNotification = true; } + meter = context.getMetricsReporter().meter("test_meter"); + counter = context.getMetricsReporter().counter("test_counter"); } @Override public void process(TransactionBase tx, Bytes row, Column col) throws Exception { + Assert.assertNotNull(meter); + Assert.assertNotNull(counter); + Bytes in = tx.get(row, col); tx.delete(row, col);