This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new ae31d77 [HUDI-836] [BLOG] Monitor Hudi metrics with Datadog (#1672)
ae31d77 is described below
commit ae31d77f16b5b78ffe387ed8a86509333f4cc15b
Author: Raymond Xu <[email protected]>
AuthorDate: Sat May 30 10:58:29 2020 -0700
[HUDI-836] [BLOG] Monitor Hudi metrics with Datadog (#1672)
Add a new blog post introducing Datadog metrics reporter.
---
...0-05-28-monitoring-hudi-metrics-with-datadog.md | 65 +++++++++++++++++++++
.../blog/2020-05-28-datadog-metrics-demo.png | Bin 0 -> 105729 bytes
2 files changed, 65 insertions(+)
diff --git a/docs/_posts/2020-05-28-monitoring-hudi-metrics-with-datadog.md
b/docs/_posts/2020-05-28-monitoring-hudi-metrics-with-datadog.md
new file mode 100644
index 0000000..8cc4fe4
--- /dev/null
+++ b/docs/_posts/2020-05-28-monitoring-hudi-metrics-with-datadog.md
@@ -0,0 +1,65 @@
+---
+title: "Monitor Hudi metrics with Datadog"
+excerpt: "Introducing the feature of reporting Hudi metrics via Datadog HTTP
API"
+author: rxu
+category: blog
+---
+
+## Availability
+
+**0.6.0 (unreleased)**
+
+## Introduction
+
+[Datadog](https://www.datadoghq.com/) is a popular monitoring service. In the
upcoming `0.6.0` release of Apache Hudi, we will introduce the feature of
reporting Hudi metrics via Datadog HTTP API, in addition to the current
reporter types: Graphite and JMX.
+
+## Configurations
+
+Similar to other supported reporters, turning on Datadog reporter requires
these 2 properties.
+
+```properties
+hoodie.metrics.on=true
+hoodie.metrics.reporter.type=DATADOG
+```
+
+The following property sets the Datadog API site. It determines whether the
requests will be sent to `api.datadoghq.eu` (EU) or `api.datadoghq.com` (US).
Set this according to your Datadog account settings.
+
+```properties
+hoodie.metrics.datadog.api.site=EU # or US
+```
+
+The property `hoodie.metrics.datadog.api.key` allows you to set the api key
directly from the configuration.
+
+```properties
+hoodie.metrics.datadog.api.key=<your api key>
+hoodie.metrics.datadog.api.key.supplier=<your api key supplier>
+```
+
+Due to security consideration in some cases, you may prefer to return the api
key at runtime. To go with this approach, implement
`java.util.function.Supplier<String>` and set the implementation's FQCN to
`hoodie.metrics.datadog.api.key.supplier`, and make sure
`hoodie.metrics.datadog.api.key` is _not_ set since it will take higher
precedence.
+
+The following property helps segregate metrics by setting different prefixes
for different jobs.
+
+```properties
+hoodie.metrics.datadog.metric.prefix=<your metrics prefix>
+```
+
+Note that it will use `.` to delimit the prefix and the metric name. For
example, if the prefix is set to `foo`, then `foo.` will be prepended to the
metric name.
+
+There are other optional properties, which are explained in the configuration
reference page.
+
+## Demo
+
+In this demo, we ran a `HoodieDeltaStreamer` job with metrics turn on and
other configurations set properly.
+
+
+
+As shown above, we were able to collect Hudi's action-related metrics like
+
+- `<prefix>.<table name>.commit.totalScanTime`
+- `<prefix>.<table name>.clean.duration`
+- `<prefix>.<table name>.index.lookup.duration`
+
+as well as `HoodieDeltaStreamer`-specific metrics
+
+- `<prefix>.<table name>.deltastreamer.duration`
+- `<prefix>.<table name>.deltastreamer.hiveSyncDuration`
diff --git a/docs/assets/images/blog/2020-05-28-datadog-metrics-demo.png
b/docs/assets/images/blog/2020-05-28-datadog-metrics-demo.png
new file mode 100644
index 0000000..8752ba6
Binary files /dev/null and
b/docs/assets/images/blog/2020-05-28-datadog-metrics-demo.png differ
