This is an automated email from the ASF dual-hosted git repository.
davidzollo pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 04b25acbc0 [Docs][Connector-V2] Improve Prometheus connector
documentation (#11253)
04b25acbc0 is described below
commit 04b25acbc027faa43a40cdf9dd6746fae4747c3d
Author: Daniel Carter <[email protected]>
AuthorDate: Wed Jul 1 22:26:19 2026 +0800
[Docs][Connector-V2] Improve Prometheus connector documentation (#11253)
Co-authored-by: DanielCarter-stack
<[email protected]>
---
docs/en/connectors/sink/Prometheus.md | 107 +++++++++++++------
docs/en/connectors/source/Prometheus.md | 181 ++++++++++++++++---------------
docs/zh/connectors/sink/Prometheus.md | 106 ++++++++++++------
docs/zh/connectors/source/Prometheus.md | 183 +++++++++++++++++---------------
4 files changed, 339 insertions(+), 238 deletions(-)
diff --git a/docs/en/connectors/sink/Prometheus.md
b/docs/en/connectors/sink/Prometheus.md
index ecfc0bc3c2..7a0a0d132d 100644
--- a/docs/en/connectors/sink/Prometheus.md
+++ b/docs/en/connectors/sink/Prometheus.md
@@ -15,46 +15,69 @@ import ChangeLog from
'../changelog/connector-prometheus.md';
- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md)
- [ ] [cdc](../../introduction/concepts/connector-v2-features.md)
- [x] [support multiple table
write](../../introduction/concepts/connector-v2-features.md)
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+- [x] [timer flush](../../introduction/concepts/connector-v2-features.md)
## Description
-Used to launch web hooks using data.
+Writes metric samples to a Prometheus-compatible remote write endpoint.
-> For example, if the data from upstream is [`label: {"__name__": "test1"},
value: 1.2.3,time:2024-08-15T17:00:00`], the body content is the following:
`{"label":{"__name__": "test1"}, "value":"1.23","time":"2024-08-15T17:00:00"}`
+The sink converts each SeaTunnel row into one Prometheus sample, serializes
the data as a remote write request, compresses it with Snappy, and sends it by
HTTP POST. Use a remote write endpoint such as
`http://prometheus:9090/api/v1/write` or
`http://victoria-metrics:8428/api/v1/write`.
-**Tips: Prometheus sink only support `post json` webhook and the data from
source will be treated as body content in web hook.And does not support passing
past data**
+Prometheus may reject samples that are too old for the target server's
retention and remote write rules.
## Supported DataSource Info
-In order to use the Http connector, the following dependencies are required.
-They can be downloaded via install-plugin.sh or from the Maven central
repository.
+In order to use the Prometheus connector, the following dependency is required.
+It can be downloaded via `install-plugin.sh` or from the Maven central
repository.
-| Datasource | Supported Versions |
Dependency |
-|------------|--------------------|------------------------------------------------------------------------------------------------------------------|
-| Http | universal |
[Download](https://mvnrepository.com/artifact/org.apache.seatunnel/seatunnel-connectors-v2/connector-prometheus)
|
+| Datasource | Supported Versions | Dependency |
+|------------|--------------------|------------|
+| Prometheus | universal |
[Download](https://mvnrepository.com/artifact/org.apache.seatunnel/seatunnel-connectors-v2/connector-prometheus)
|
## Sink Options
-| Name | Type | Required | Default | Description
|
-|-----------------------------|--------|----------|---------|-------------------------------------------------------------------------------------------------------------|
-| url | String | Yes | - | Http request url
|
-| headers | Map | No | - | Http headers
|
-| retry | Int | No | - | The max retry
times if request http return to `IOException`
|
-| retry_backoff_multiplier_ms | Int | No | 100 | The
retry-backoff times(millis) multiplier if request http failed
|
-| retry_backoff_max_ms | Int | No | 10000 | The maximum
retry-backoff times(millis) if request http failed
|
-| connect_timeout_ms | Int | No | 12000 | Connection
timeout setting, default 12s.
|
-| socket_timeout_ms | Int | No | 60000 | Socket timeout
setting, default 60s.
|
-| key_timestamp | Int | NO | - | prometheus
timestamp key .
|
-| key_label | String | yes | - | prometheus label
key
|
-| key_value | Double | yes | - | prometheus value
|
-| batch_size | Int | false | 1024 | prometheus
batch size write
|
-| flush_interval | Long | false | 300000L | prometheus
flush commit interval |
-| common-options | | No | - | Sink plugin
common parameters, please refer to [Sink Common
Options](../common-options/sink-common-options.md) for details |
+| name | type | required | default value |
description |
+|-----------------------------|--------|----------|---------------|-------------|
+| url | String | Yes | - |
Prometheus-compatible remote write URL. |
+| headers | Map | No | - | HTTP
request headers. |
+| retry | Int | No | - | Maximum
retry times when the HTTP request fails with `IOException`. |
+| retry_backoff_multiplier_ms | Int | No | 100 | Retry
backoff multiplier, in milliseconds. |
+| retry_backoff_max_ms | Int | No | 10000 | Maximum
retry backoff, in milliseconds. |
+| key_label | String | Yes | - | Field name
whose value is used as Prometheus labels. The field value must be a map. |
+| key_value | String | Yes | - | Field name
whose value is used as the Prometheus sample value. |
+| key_timestamp | String | No | - | Field name
whose value is used as the sample timestamp. If omitted, the current system
time is used. |
+| batch_size | Int | No | 1024 | Maximum
number of samples written in one request. Must be greater than 0. |
+| flush_interval | Long | No | 300000 | Scheduled
flush interval in milliseconds. |
+| common-options | config | No | - | Sink
common options. |
+
+### key_label [String]
+
+The named field must be `map<string, string>`. It is converted into Prometheus
labels. Include `__name__` in the map to set the metric name.
+
+### key_value [String]
+
+The named field is converted into the Prometheus sample value. A `double`
field is recommended.
+
+### key_timestamp [String]
+
+Optional timestamp field.
+
+Supported field types:
+
+- `timestamp`: converted to epoch milliseconds with the local time zone
+- `bigint`: treated as epoch milliseconds
+- `double`: treated as Unix seconds and converted to milliseconds
+- `string`: parsed as epoch milliseconds
+
+When this option is not configured, the sink uses the current system time.
+
+### common options
+
+Sink plugin common parameters, please refer to [Sink Common
Options](../common-options/sink-common-options.md) for details.
## Example
-simple:
+### Write to Prometheus remote write
```hocon
env {
@@ -73,21 +96,21 @@ source {
}
plugin_output = "fake"
rows = [
- {
- kind = INSERT
- fields = [{"__name__": "test1"}, 1.23, "2024-08-15T17:00:00"]
- },
- {
- kind = INSERT
- fields = [{"__name__": "test2"}, 1.23, "2024-08-15T17:00:00"]
- }
+ {
+ kind = INSERT
+ fields = [{"__name__" : "metric_1"}, 1.23, CURRENT_TIMESTAMP]
+ },
+ {
+ kind = INSERT
+ fields = [{"__name__" : "metric_2"}, 1.23, CURRENT_TIMESTAMP]
+ }
]
}
}
-
sink {
Prometheus {
+ plugin_input = "fake"
url = "http://prometheus:9090/api/v1/write";
key_label = "c_map"
key_value = "c_double"
@@ -95,9 +118,23 @@ sink {
batch_size = 1
}
}
+```
+
+### Write to a Prometheus-compatible remote write API
+```hocon
+sink {
+ Prometheus {
+ plugin_input = "fake"
+ url = "http://victoria-metrics:8428/api/v1/write";
+ key_label = "c_map"
+ key_value = "c_double"
+ key_timestamp = "c_timestamp"
+ batch_size = 5
+ }
+}
```
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/en/connectors/source/Prometheus.md
b/docs/en/connectors/source/Prometheus.md
index 7b6afaea76..e7c59f55d4 100644
--- a/docs/en/connectors/source/Prometheus.md
+++ b/docs/en/connectors/source/Prometheus.md
@@ -6,7 +6,9 @@ import ChangeLog from '../changelog/connector-prometheus.md';
## Description
-Used to read data from Prometheus.
+Reads metric samples from Prometheus-compatible HTTP APIs.
+
+The connector uses the Prometheus query API. Configure `url` as the base
address, such as `http://prometheus:9090` or `http://victoria-metrics:8428`.
SeaTunnel appends `/api/v1/query` for `Instant` queries and
`/api/v1/query_range` for `Range` queries.
## Key features
@@ -16,133 +18,144 @@ Used to read data from Prometheus.
## Options
-| name | type | required | default value |
-|-----------------------------|---------|----------|-----------------|
-| url | String | Yes | - |
-| query | String | Yes | - |
-| query_type | String | Yes | Instant |
-| content_field | String | Yes | $.data.result.* |
-| schema.fields | Config | Yes | - |
-| format | String | No | json |
-| params | Map | Yes | - |
-| poll_interval_millis | int | No | - |
-| retry | int | No | - |
-| retry_backoff_multiplier_ms | int | No | 100 |
-| retry_backoff_max_ms | int | No | 10000 |
-| enable_multi_lines | boolean | No | false |
-| common-options | config | No | - |
-
-### url [String]
-
-http request url
-
-### query [String]
-
-Prometheus expression query string
+| name | type | required | default value |
description |
+|-----------------------------|---------|----------|---------------|-------------|
+| url | String | Yes | - |
Prometheus-compatible server base URL. |
+| query | String | Yes | - | PromQL
expression. |
+| query_type | String | No | Instant | Query
type. Valid values are `Instant` and `Range`. |
+| start | String | Required when `query_type = Range` |
- | Range query start time. |
+| end | String | Required when `query_type = Range` |
- | Range query end time. |
+| step | String | Required when `query_type = Range` |
- | Range query resolution step, for example `15s`. |
+| time | Long | No | - | Instant
query evaluation time, as a Unix timestamp. |
+| timeout | Long | No | - | Query
timeout passed to Prometheus. |
+| headers | Map | No | - | HTTP
request headers. |
+| params | Map | No | - | Extra
HTTP request parameters. SeaTunnel adds `query` and query time parameters
automatically. |
+| content_field | String | No | - | JSONPath
used to extract the sample list. For Prometheus responses, use
`$.data.result.*`. |
+| schema.fields | Config | Required when `format = json` | - |
Output schema. |
+| format | String | No | text | Response
format. Use `json` for Prometheus metric samples. |
+| poll_interval_millis | int | No | - | Request
interval in stream mode, in milliseconds. |
+| retry | int | No | - | Maximum
retry times when the HTTP request fails with `IOException`. |
+| retry_backoff_multiplier_ms | int | No | 100 | Retry
backoff multiplier, in milliseconds. |
+| retry_backoff_max_ms | int | No | 10000 | Maximum
retry backoff, in milliseconds. |
+| common-options | config | No | - | Source
common options. |
### query_type [String]
-Instant/Range
-
-1. Instant : The following endpoint evaluates an instant query at a single
point in time
-2. Range : The following endpoint evaluates an expression query over a range
of time
-
-https://prometheus.io/docs/prometheus/latest/querying/api/
-
-### params [Map]
-
-http request params
-
-### poll_interval_millis [int]
+`Instant` evaluates the query at a single time. `Range` evaluates the query
over a time range.
-request http api interval(millis) in stream mode
+### start / end [String]
-### retry [int]
+Used only when `query_type = Range`.
-The max retry times if request http return to `IOException`
+Supported values:
-### retry_backoff_multiplier_ms [int]
+- `CURRENT_TIMESTAMP`
+- ISO-8601 timestamp, for example `2025-05-13T02:25:23Z`
+- Unix timestamp in seconds, for example `1747103123.083`
-The retry-backoff times(millis) multiplier if request http failed
+### step [String]
-### retry_backoff_max_ms [int]
-
-The maximum retry-backoff times(millis) if request http failed
-
-### format [String]
-
-the format of upstream data, default `json`.
+Used only when `query_type = Range`. It is the query resolution step accepted
by Prometheus, such as `15s`, `1m`, or a number of seconds.
### schema [Config]
-Fill in a fixed value
+Prometheus source returns three fields in this order:
```hocon
- schema = {
- fields {
- metric = "map<string, string>"
- value = double
- time = long
- }
- }
-
+schema = {
+ fields {
+ metric = "map<string, string>"
+ value = double
+ time = long
+ }
+}
```
-#### fields [Config]
-
-the schema fields of upstream data
+The `metric` field contains labels, including `__name__`. The `value` field is
the metric value. The `time` field is the sample timestamp in milliseconds.
### common options
-Source plugin common parameters, please refer to [Source Common
Options](../common-options/source-common-options.md) for details
+Source plugin common parameters, please refer to [Source Common
Options](../common-options/source-common-options.md) for details.
## Example
-### Instant
+### Instant query
```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
source {
Prometheus {
- plugin_output = "http"
- url = "http://mockserver:1080";
- query = "up"
+ plugin_output = "prometheus_metrics"
+ url = "http://prometheus:9090";
+ query = "metric_1"
query_type = "Instant"
content_field = "$.data.result.*"
format = "json"
schema = {
- fields {
- metric = "map<string, string>"
- value = double
- time = long
- }
- }
+ fields {
+ metric = "map<string, string>"
+ value = double
+ time = long
+ }
}
+ }
}
```
-### Range
+### Range query
```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
source {
Prometheus {
- plugin_output = "http"
- url = "http://mockserver:1080";
- query = "up"
+ plugin_output = "prometheus_metrics"
+ url = "http://prometheus:9090";
+ query = "metric_1"
query_type = "Range"
+ start = "CURRENT_TIMESTAMP"
+ end = "CURRENT_TIMESTAMP"
+ step = "15s"
content_field = "$.data.result.*"
format = "json"
- start = "2024-07-22T20:10:30.781Z"
- end = "2024-07-22T20:11:00.781Z"
- step = "15s"
schema = {
- fields {
- metric = "map<string, string>"
- value = double
- time = long
- }
- }
+ fields {
+ metric = "map<string, string>"
+ value = double
+ time = long
+ }
}
}
+}
+```
+
+### Read from a Prometheus-compatible API
+
+```hocon
+source {
+ Prometheus {
+ plugin_output = "metrics"
+ url = "http://victoria-metrics:8428";
+ query = "metric_1"
+ query_type = "Instant"
+ content_field = "$.data.result.*"
+ format = "json"
+ schema = {
+ fields {
+ metric = "map<string, string>"
+ value = double
+ time = long
+ }
+ }
+ }
+}
```
## Changelog
diff --git a/docs/zh/connectors/sink/Prometheus.md
b/docs/zh/connectors/sink/Prometheus.md
index 0f90018970..fcecca5be1 100644
--- a/docs/zh/connectors/sink/Prometheus.md
+++ b/docs/zh/connectors/sink/Prometheus.md
@@ -14,46 +14,69 @@ import ChangeLog from
'../changelog/connector-prometheus.md';
- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md)
- [ ] [cdc](../../introduction/concepts/connector-v2-features.md)
-- [x] [support multiple table
write](../../introduction/concepts/connector-v2-features.md)
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+- [x] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
+- [x] [定时刷新](../../introduction/concepts/connector-v2-features.md)
## 描述
-接收Source端传入的数据，利用数据触发 web hooks。
+向 Prometheus 兼容的 remote write 接口写入指标样本。
-> 例如，来自上游的数据为 [`label: {"__name__": "test1"}, value:
1.2.3,time:2024-08-15T17:00:00`], 则body内容如下: `{"label":{"__name__": "test1"},
"value":"1.23","time":"2024-08-15T17:00:00"}`
+这个 sink 会把每一行 SeaTunnel 数据转换成一个 Prometheus 样本，再按 remote write 协议序列化，用 Snappy
压缩后通过 HTTP POST 发出。常见地址类似 `http://prometheus:9090/api/v1/write` 或
`http://victoria-metrics:8428/api/v1/write`。
-**Tips: Prometheus 数据接收器 仅支持 `post json` 类型的 web hook，source 数据将被视为 webhook 中的
body 内容。并且不支持传递过去太久的数据**
+如果样本时间太早，目标服务可能会按自己的保留策略或 remote write 规则拒绝写入。
## 支持的数据源信息
-想使用 Prometheus 连接器，需要安装以下必要的依赖。可以通过运行 install-plugin.sh 脚本或者从 Maven 中央仓库下载这些依赖
+想使用 Prometheus 连接器，需要安装以下依赖。可以通过 `install-plugin.sh` 脚本安装，也可以从 Maven 中央仓库下载。
-| 数据源 | 支持版本 | 依赖
|
-|------|-----------|------------------------------------------------------------------------------------------------------------------|
-| Http | universal |
[Download](https://mvnrepository.com/artifact/org.apache.seatunnel/seatunnel-connectors-v2/connector-prometheus)
|
+| 数据源 | 支持版本 | 依赖 |
+|--------|----------|------|
+| Prometheus | universal |
[Download](https://mvnrepository.com/artifact/org.apache.seatunnel/seatunnel-connectors-v2/connector-prometheus)
|
## 接收器选项
-| Name | Type | Required | Default | Description
|
-|-----------------------------|--------|----------|---------|-------------------------------------------------------------------|
-| url | String | Yes | - | Http 请求链接
|
-| headers | Map | No | - | Http 标头
|
-| retry | Int | No | - |
如果请求http返回`IOException`的最大重试次数 |
-| retry_backoff_multiplier_ms | Int | No | 100 |
http请求失败，重试回退次数（毫秒）乘数 |
-| retry_backoff_max_ms | Int | No | 10000 |
http请求失败，最大重试回退时间(毫秒) |
-| connect_timeout_ms | Int | No | 12000 | 连接超时设置，默认12s
|
-| socket_timeout_ms | Int | No | 60000 | 套接字超时设置，默认为60s
|
-| key_timestamp | Int | NO | - |
prometheus时间戳的key. |
-| key_label | String | yes | - | prometheus标签的key
|
-| key_value | Double | yes | - | prometheus值的key
|
-| batch_size | Int | false | 1024 |
prometheus批量写入大小 |
-| flush_interval | Long | false | 300000L |
prometheus定时写入 |
-| common-options | | No | - | Sink插件常用参数，请参考
[Sink常用选项 ](../common-options/sink-common-options.md) 了解详情 |
+| 名称 | 类型 | 是否必填 | 默认值 | 描述 |
+|-----------------------------|--------|----------|--------|------|
+| url | String | 是 | - | Prometheus 兼容
remote write 地址。 |
+| headers | Map | 否 | - | HTTP 请求头。 |
+| retry | Int | 否 | - | HTTP 请求出现
`IOException` 时的最大重试次数。 |
+| retry_backoff_multiplier_ms | Int | 否 | 100 | 重试退避时间乘数，单位毫秒。 |
+| retry_backoff_max_ms | Int | 否 | 10000 | 最大重试退避时间，单位毫秒。 |
+| key_label | String | 是 | - | 作为 Prometheus
标签的字段名。字段值必须是 map。 |
+| key_value | String | 是 | - | 作为 Prometheus
样本值的字段名。 |
+| key_timestamp | String | 否 | - |
作为样本时间戳的字段名。不配置时使用当前系统时间。 |
+| batch_size | Int | 否 | 1024 | 单次请求最多写入的样本数，必须大于
0。 |
+| flush_interval | Long | 否 | 300000 | 定时刷新间隔，单位毫秒。 |
+| common-options | config | 否 | - | Sink 通用选项。 |
+
+### key_label [String]
+
+对应字段必须是 `map<string, string>`，会被转换为 Prometheus 标签。建议在 map 中包含
`__name__`，用来表示指标名。
+
+### key_value [String]
+
+对应字段会被转换为 Prometheus 样本值。推荐使用 `double` 类型字段。
+
+### key_timestamp [String]
+
+可选的时间戳字段。
+
+支持以下字段类型：
+
+- `timestamp`：按本地时区转换为毫秒级时间戳
+- `bigint`：按毫秒级时间戳处理
+- `double`：按 Unix 秒级时间戳处理，并转换为毫秒
+- `string`：按毫秒级时间戳解析
+
+不配置这个选项时，sink 会使用当前系统时间。
+
+### common options
+
+Sink 插件通用参数，请参考 [Sink Common
Options](../common-options/sink-common-options.md)。
## 示例
-简单示例:
+### 写入 Prometheus remote write
```hocon
env {
@@ -72,21 +95,21 @@ source {
}
plugin_output = "fake"
rows = [
- {
- kind = INSERT
- fields = [{"__name__": "test1"}, 1.23, "2024-08-15T17:00:00"]
- },
- {
- kind = INSERT
- fields = [{"__name__": "test2"}, 1.23, "2024-08-15T17:00:00"]
- }
+ {
+ kind = INSERT
+ fields = [{"__name__" : "metric_1"}, 1.23, CURRENT_TIMESTAMP]
+ },
+ {
+ kind = INSERT
+ fields = [{"__name__" : "metric_2"}, 1.23, CURRENT_TIMESTAMP]
+ }
]
}
}
-
sink {
Prometheus {
+ plugin_input = "fake"
url = "http://prometheus:9090/api/v1/write";
key_label = "c_map"
key_value = "c_double"
@@ -96,6 +119,21 @@ sink {
}
```
+### 写入 Prometheus 兼容 remote write 接口
+
+```hocon
+sink {
+ Prometheus {
+ plugin_input = "fake"
+ url = "http://victoria-metrics:8428/api/v1/write";
+ key_label = "c_map"
+ key_value = "c_double"
+ key_timestamp = "c_timestamp"
+ batch_size = 5
+ }
+}
+```
+
## 变更日志
<ChangeLog />
diff --git a/docs/zh/connectors/source/Prometheus.md
b/docs/zh/connectors/source/Prometheus.md
index 98ed165e0f..14c11552ce 100644
--- a/docs/zh/connectors/source/Prometheus.md
+++ b/docs/zh/connectors/source/Prometheus.md
@@ -6,7 +6,9 @@ import ChangeLog from '../changelog/connector-prometheus.md';
## 描述
-用于读取prometheus数据。
+从 Prometheus 兼容的 HTTP API 读取指标样本。
+
+连接器使用 Prometheus 查询 API。`url` 只需要填写基础地址，例如 `http://prometheus:9090` 或
`http://victoria-metrics:8428`。SeaTunnel 会按查询类型自动追加 `/api/v1/query` 或
`/api/v1/query_range`。
## 主要特性
@@ -16,135 +18,146 @@ import ChangeLog from
'../changelog/connector-prometheus.md';
## 源选项
-| 名称 | 类型 | 是否必填 | 默认值 |
-|-----------------------------|---------|------|-----------------|
-| url | String | Yes | - |
-| query | String | Yes | - |
-| query_type | String | Yes | Instant |
-| content_field | String | Yes | $.data.result.* |
-| schema.fields | Config | Yes | - |
-| format | String | No | json |
-| params | Map | Yes | - |
-| poll_interval_millis | int | No | - |
-| retry | int | No | - |
-| retry_backoff_multiplier_ms | int | No | 100 |
-| retry_backoff_max_ms | int | No | 10000 |
-| enable_multi_lines | boolean | No | false |
-| common-options | config | No | |
-
-### url [String]
-
-http 请求路径。
-
-### query [String]
-
-Prometheus 表达式查询字符串
+| 名称 | 类型 | 是否必填 | 默认值 | 描述 |
+|-----------------------------|---------|----------|--------|------|
+| url | String | 是 | - | Prometheus
兼容服务的基础地址。 |
+| query | String | 是 | - | PromQL 查询表达式。 |
+| query_type | String | 否 | Instant | 查询类型，可选值为
`Instant` 和 `Range`。 |
+| start | String | `query_type = Range` 时必填 | - |
范围查询开始时间。 |
+| end | String | `query_type = Range` 时必填 | - |
范围查询结束时间。 |
+| step | String | `query_type = Range` 时必填 | - |
范围查询步长，例如 `15s`。 |
+| time | Long | 否 | - | 即时查询的评估时间，使用 Unix
时间戳。 |
+| timeout | Long | 否 | - | 传给 Prometheus
的查询超时时间。 |
+| headers | Map | 否 | - | HTTP 请求头。 |
+| params | Map | 否 | - | 额外的 HTTP
请求参数。SeaTunnel 会自动加入 `query` 和查询时间参数。 |
+| content_field | String | 否 | - | 用来提取样本列表的
JSONPath。Prometheus 响应通常填写 `$.data.result.*`。 |
+| schema.fields | Config | `format = json` 时必填 | - | 输出字段结构。 |
+| format | String | 否 | text | 响应格式。读取
Prometheus 指标样本时请设置为 `json`。 |
+| poll_interval_millis | int | 否 | - | 流模式下请求间隔，单位毫秒。 |
+| retry | int | 否 | - | HTTP 请求出现
`IOException` 时的最大重试次数。 |
+| retry_backoff_multiplier_ms | int | 否 | 100 | 重试退避时间乘数，单位毫秒。 |
+| retry_backoff_max_ms | int | 否 | 10000 | 最大重试退避时间，单位毫秒。 |
+| common-options | config | 否 | - | Source 通用选项。 |
### query_type [String]
-Instant/Range
-
-1. Instant : 简单指标的即时查询。
-2. Range : 一段时间内指标数据。
-
-https://prometheus.io/docs/prometheus/latest/querying/api/
-
-### params [Map]
-
-http 请求参数
-
-### poll_interval_millis [int]
+`Instant` 表示查询某一个时间点的指标值。`Range` 表示查询一段时间范围内的指标值。
-流模式下请求HTTP API间隔(毫秒)
+### start / end [String]
-### retry [int]
+仅在 `query_type = Range` 时使用。
-The max retry times if request http return to `IOException`
+支持以下写法：
-### retry_backoff_multiplier_ms [int]
+- `CURRENT_TIMESTAMP`
+- ISO-8601 时间，例如 `2025-05-13T02:25:23Z`
+- Unix 秒级时间戳，例如 `1747103123.083`
-请求http返回到' IOException '的最大重试次数
+### step [String]
-### retry_backoff_max_ms [int]
-
-http请求失败，最大重试回退时间(毫秒)
-
-### format [String]
-
-上游数据的格式，默认为json。
+仅在 `query_type = Range` 时使用。它表示 Prometheus 接受的查询步长，例如 `15s`、`1m`，也可以是秒数。
### schema [Config]
-按照如下填写一个固定值
+Prometheus source 固定输出下面三个字段，顺序也固定：
```hocon
- schema = {
- fields {
- metric = "map<string, string>"
- value = double
- time = long
- }
- }
-
+schema = {
+ fields {
+ metric = "map<string, string>"
+ value = double
+ time = long
+ }
+}
```
-#### fields [Config]
-
-上游数据的模式字段
+`metric` 字段保存指标标签，包括 `__name__`。`value` 字段是指标值。`time` 字段是毫秒级样本时间戳。
### common options
-源插件常用参数，请参考[Source Common Options](../common-options/source-common-options.md)
了解详细信息
+源插件通用参数，请参考 [Source Common
Options](../common-options/source-common-options.md)。
## 示例
-### Instant
+### 即时查询
```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
source {
Prometheus {
- plugin_output = "http"
- url = "http://mockserver:1080";
- query = "up"
+ plugin_output = "prometheus_metrics"
+ url = "http://prometheus:9090";
+ query = "metric_1"
query_type = "Instant"
content_field = "$.data.result.*"
format = "json"
schema = {
- fields {
- metric = "map<string, string>"
- value = double
- time = long
- }
- }
+ fields {
+ metric = "map<string, string>"
+ value = double
+ time = long
+ }
}
+ }
}
```
-### Range
+### 范围查询
```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
source {
Prometheus {
- plugin_output = "http"
- url = "http://mockserver:1080";
- query = "up"
+ plugin_output = "prometheus_metrics"
+ url = "http://prometheus:9090";
+ query = "metric_1"
query_type = "Range"
+ start = "CURRENT_TIMESTAMP"
+ end = "CURRENT_TIMESTAMP"
+ step = "15s"
content_field = "$.data.result.*"
format = "json"
- start = "2024-07-22T20:10:30.781Z"
- end = "2024-07-22T20:11:00.781Z"
- step = "15s"
schema = {
- fields {
- metric = "map<string, string>"
- value = double
- time = long
- }
- }
+ fields {
+ metric = "map<string, string>"
+ value = double
+ time = long
+ }
}
}
+}
+```
+
+### 读取 Prometheus 兼容接口
+
+```hocon
+source {
+ Prometheus {
+ plugin_output = "metrics"
+ url = "http://victoria-metrics:8428";
+ query = "metric_1"
+ query_type = "Instant"
+ content_field = "$.data.result.*"
+ format = "json"
+ schema = {
+ fields {
+ metric = "map<string, string>"
+ value = double
+ time = long
+ }
+ }
+ }
+}
```
## 变更日志
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
