This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push: new 7b12147 [DOC][HUDI-1123] add doc for user defined metrics reporter (#1879) 7b12147 is described below commit 7b12147f7b2cb35598ee20ecd8caa2f25de5db47 Author: zherenyu831 <52404525+zherenyu...@users.noreply.github.com> AuthorDate: Tue Jul 28 21:33:13 2020 +0900 [DOC][HUDI-1123] add doc for user defined metrics reporter (#1879) --- docs/_docs/2_4_configurations.cn.md | 10 +++++++ docs/_docs/2_4_configurations.md | 11 ++++++++ docs/_docs/2_8_metrics.md | 53 +++++++++++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+) diff --git a/docs/_docs/2_4_configurations.cn.md b/docs/_docs/2_4_configurations.cn.md index a75419e..b577990 100644 --- a/docs/_docs/2_4_configurations.cn.md +++ b/docs/_docs/2_4_configurations.cn.md @@ -519,6 +519,16 @@ Hudi提供了一个选项,可以通过将对该分区中的插入作为对现 属性: `hoodie.metrics.datadog.metric.tags` <br/> <span style="color:grey">Datadog指标标签(逗号分隔),将和指标数据一并发送。</span> +#### 用户自定义发送器 + +##### on(metricsOn = false) {#on} +属性: `hoodie.metrics.on` <br/> +<span style="color:grey">打开或关闭发送指标。默认情况下处于关闭状态。</span> + +##### withReporterClass(className = "") {#withReporterClass} +属性: `hoodie.metrics.reporter.class` <br/> +<span style="color:grey">用于处理发送指标的用户自定义类,必须是AbstractUserDefinedMetricsReporter类的子类.</span> + ### 内存配置 控制由Hudi内部执行的压缩和合并的内存使用情况 [withMemoryConfig](#withMemoryConfig) (HoodieMemoryConfig) <br/> diff --git a/docs/_docs/2_4_configurations.md b/docs/_docs/2_4_configurations.md index 6fc1049..627d148 100644 --- a/docs/_docs/2_4_configurations.md +++ b/docs/_docs/2_4_configurations.md @@ -483,6 +483,17 @@ Property: `hoodie.metrics.datadog.metric.host` <br/> Property: `hoodie.metrics.datadog.metric.tags` <br/> <span style="color:grey">Datadog metric tags (comma-delimited) to be sent along with metrics data.</span> +#### USER DEFINED REPORTER + +##### on(metricsOn = false) {#on} +`hoodie.metrics.on` <br/> +<span style="color:grey">Turn on/off metrics reporting. off by default.</span> + +##### withReporterClass(className = "") {#withReporterClass} +Property: `hoodie.metrics.reporter.class` <br/> +<span style="color:grey">User-defined class used to report metrics, must be a subclass of AbstractUserDefinedMetricsReporter.</span> + + ### Memory configs Controls memory usage for compaction and merges, performed internally by Hudi [withMemoryConfig](#withMemoryConfig) (HoodieMemoryConfig) <br/> diff --git a/docs/_docs/2_8_metrics.md b/docs/_docs/2_8_metrics.md index e5043af..287053c 100644 --- a/docs/_docs/2_8_metrics.md +++ b/docs/_docs/2_8_metrics.md @@ -90,6 +90,59 @@ In this demo, we ran a `HoodieDeltaStreamer` job with `HoodieMetrics` turned on * `<prefix>.<table name>.deltastreamer.duration` * `<prefix>.<table name>.deltastreamer.hiveSyncDuration` + +### UserDefinedMetricsReporter + +Allows users to define a custom metrics reporter. + +#### Configurations +The following is an example of `UserDefinedMetricsReporter`. More detailed configurations can be referenced [here](configurations.html#user-defined-reporter). + +```properties +hoodie.metrics.on=true +hoodie.metrics.reporter.class=test.TestUserDefinedMetricsReporter +``` + +#### Demo +In this simple demo, TestMetricsReporter will print all gauges every 10 seconds + +```java +public static class TestUserDefinedMetricsReporter + extends AbstractUserDefinedMetricsReporter { + private static final Logger log = LogManager.getLogger(DummyMetricsReporter.class); + + private ScheduledExecutorService exec = Executors.newScheduledThreadPool(1, r -> { + Thread t = Executors.defaultThreadFactory().newThread(r); + t.setDaemon(true); + return t; + }); + + public TestUserDefinedMetricsReporter(Properties props, MetricRegistry registry) { + super(props, registry); + } + + @Override + public void start() { + exec.schedule(this::report, 10, TimeUnit.SECONDS); + } + + @Override + public void report() { + this.getRegistry().getGauges().forEach((key, value) -> + log.info("key: " + key + " value: " + value.getValue().toString())); + } + + @Override + public Closeable getReporter() { + return null; + } + + @Override + public void stop() { + exec.shutdown(); + } +} +``` ## HoodieMetrics