zentol commented on a change in pull request #17067:
URL: https://github.com/apache/flink/pull/17067#discussion_r699159076
##########
File path: docs/content/docs/connectors/datastream/kafka.md
##########
@@ -213,26 +213,115 @@ Note that Kafka source does **NOT** rely on committed
offsets for fault toleranc
is only for exposing the progress of consumer and consuming group for
monitoring.
### Monitoring
-Kafka source exposes metrics in Flink's metric group for monitoring and
diagnosing.
-#### Scope of Metric
-All metrics of Kafka source reader are registered under group
```KafkaSourceReader```, which is a
-child group of operator metric group. Metrics related to a specific topic
partition will be registered
-in the group
```KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
-For example, current consuming offset of topic "my-topic" and partition 1 will
be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```
,
+Kafka source exposes the following metrics in the respective [scope]({{< ref
"docs/ops/metrics" >}}/#scope).
-and number of successful commits will be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.commitsSucceeded``` .
+#### Scope of Metric
-#### List of Metrics
+All metrics of Kafka source reader are registered under group ```<job metric
group>.<operator name>.<subtask>.KafkaSourceReader```, which is a
+child group of operator metric group. Metrics related to a specific topic
partition will be registered
+in the group ```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
+
+For example, current consuming offset of topic "my-topic" and partition 1 will
be reported in metric
+```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```,
and number of successful commits will be reported in metric
+```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader.commitsSucceeded```.
+
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 15%">Scope</th>
+ <th class="text-left" style="width: 18%">Metrics</th>
+ <th class="text-left" style="width: 18%">User Variables</th>
+ <th class="text-left" style="width: 39%">Description</th>
+ <th class="text-left" style="width: 10%">Type</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <th rowspan="2">Task</th>
+ <td>numBytesIn</td>
+ <td>n/a</td>
+ <td>The total number of input bytes since the source started. Count
towards the numBytesIn in TaskIOMetricsGroup.</td>
+ <td>Counter</td>
+ </tr>
+ <tr>
+ <td>numBytesInPerSecond</td>
+ <td>n/a</td>
+ <td>The input bytes per second.</td>
+ <td>Meter</td>
+ </tr>
+ <tr>
+ <th rowspan="10">Operator</th>
+ <td>numRecordsIn</td>
Review comment:
hmm these are all already documented as general operator task/operator
metrics; I don't think we need to document them again.
##########
File path: docs/content/docs/connectors/datastream/kafka.md
##########
@@ -213,26 +213,115 @@ Note that Kafka source does **NOT** rely on committed
offsets for fault toleranc
is only for exposing the progress of consumer and consuming group for
monitoring.
### Monitoring
-Kafka source exposes metrics in Flink's metric group for monitoring and
diagnosing.
-#### Scope of Metric
-All metrics of Kafka source reader are registered under group
```KafkaSourceReader```, which is a
-child group of operator metric group. Metrics related to a specific topic
partition will be registered
-in the group
```KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
-For example, current consuming offset of topic "my-topic" and partition 1 will
be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```
,
+Kafka source exposes the following metrics in the respective [scope]({{< ref
"docs/ops/metrics" >}}/#scope).
-and number of successful commits will be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.commitsSucceeded``` .
+#### Scope of Metric
-#### List of Metrics
+All metrics of Kafka source reader are registered under group ```<job metric
group>.<operator name>.<subtask>.KafkaSourceReader```, which is a
+child group of operator metric group. Metrics related to a specific topic
partition will be registered
+in the group ```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
+
+For example, current consuming offset of topic "my-topic" and partition 1 will
be reported in metric
+```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```,
and number of successful commits will be reported in metric
+```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader.commitsSucceeded```.
+
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 15%">Scope</th>
+ <th class="text-left" style="width: 18%">Metrics</th>
+ <th class="text-left" style="width: 18%">User Variables</th>
+ <th class="text-left" style="width: 39%">Description</th>
+ <th class="text-left" style="width: 10%">Type</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <th rowspan="2">Task</th>
+ <td>numBytesIn</td>
+ <td>n/a</td>
+ <td>The total number of input bytes since the source started. Count
towards the numBytesIn in TaskIOMetricsGroup.</td>
Review comment:
```suggestion
<td>The total number of input bytes since the source started.</td>
```
##########
File path: docs/content/docs/connectors/datastream/kafka.md
##########
@@ -213,26 +213,115 @@ Note that Kafka source does **NOT** rely on committed
offsets for fault toleranc
is only for exposing the progress of consumer and consuming group for
monitoring.
### Monitoring
-Kafka source exposes metrics in Flink's metric group for monitoring and
diagnosing.
-#### Scope of Metric
-All metrics of Kafka source reader are registered under group
```KafkaSourceReader```, which is a
-child group of operator metric group. Metrics related to a specific topic
partition will be registered
-in the group
```KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
-For example, current consuming offset of topic "my-topic" and partition 1 will
be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```
,
+Kafka source exposes the following metrics in the respective [scope]({{< ref
"docs/ops/metrics" >}}/#scope).
-and number of successful commits will be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.commitsSucceeded``` .
+#### Scope of Metric
-#### List of Metrics
+All metrics of Kafka source reader are registered under group ```<job metric
group>.<operator name>.<subtask>.KafkaSourceReader```, which is a
Review comment:
And which metrics belong to the Kafka source reader?
##########
File path: docs/content/docs/connectors/datastream/kafka.md
##########
@@ -213,26 +213,115 @@ Note that Kafka source does **NOT** rely on committed
offsets for fault toleranc
is only for exposing the progress of consumer and consuming group for
monitoring.
### Monitoring
-Kafka source exposes metrics in Flink's metric group for monitoring and
diagnosing.
-#### Scope of Metric
-All metrics of Kafka source reader are registered under group
```KafkaSourceReader```, which is a
-child group of operator metric group. Metrics related to a specific topic
partition will be registered
-in the group
```KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
-For example, current consuming offset of topic "my-topic" and partition 1 will
be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```
,
+Kafka source exposes the following metrics in the respective [scope]({{< ref
"docs/ops/metrics" >}}/#scope).
-and number of successful commits will be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.commitsSucceeded``` .
+#### Scope of Metric
-#### List of Metrics
+All metrics of Kafka source reader are registered under group ```<job metric
group>.<operator name>.<subtask>.KafkaSourceReader```, which is a
Review comment:
organizing kafka-specific metrics into their own group is fine imo, but
KafkaSourceReader is quite specific isn't it.
If these metrics existed before 1.14 we shouldn't change things to to not
break existing setups.
##########
File path: docs/content/docs/connectors/datastream/kafka.md
##########
@@ -213,26 +213,115 @@ Note that Kafka source does **NOT** rely on committed
offsets for fault toleranc
is only for exposing the progress of consumer and consuming group for
monitoring.
### Monitoring
-Kafka source exposes metrics in Flink's metric group for monitoring and
diagnosing.
-#### Scope of Metric
-All metrics of Kafka source reader are registered under group
```KafkaSourceReader```, which is a
-child group of operator metric group. Metrics related to a specific topic
partition will be registered
-in the group
```KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
-For example, current consuming offset of topic "my-topic" and partition 1 will
be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```
,
+Kafka source exposes the following metrics in the respective [scope]({{< ref
"docs/ops/metrics" >}}/#scope).
-and number of successful commits will be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.commitsSucceeded``` .
+#### Scope of Metric
-#### List of Metrics
+All metrics of Kafka source reader are registered under group ```<job metric
group>.<operator name>.<subtask>.KafkaSourceReader```, which is a
+child group of operator metric group. Metrics related to a specific topic
partition will be registered
+in the group ```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
+
+For example, current consuming offset of topic "my-topic" and partition 1 will
be reported in metric
+```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```,
and number of successful commits will be reported in metric
+```<job metric group>.<operator
name>.<subtask>.KafkaSourceReader.commitsSucceeded```.
+
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 15%">Scope</th>
+ <th class="text-left" style="width: 18%">Metrics</th>
+ <th class="text-left" style="width: 18%">User Variables</th>
+ <th class="text-left" style="width: 39%">Description</th>
+ <th class="text-left" style="width: 10%">Type</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <th rowspan="2">Task</th>
+ <td>numBytesIn</td>
+ <td>n/a</td>
+ <td>The total number of input bytes since the source started. Count
towards the numBytesIn in TaskIOMetricsGroup.</td>
+ <td>Counter</td>
+ </tr>
+ <tr>
+ <td>numBytesInPerSecond</td>
+ <td>n/a</td>
+ <td>The input bytes per second.</td>
+ <td>Meter</td>
+ </tr>
+ <tr>
+ <th rowspan="10">Operator</th>
+ <td>numRecordsIn</td>
Review comment:
correct
##########
File path: docs/content/docs/connectors/datastream/kafka.md
##########
@@ -213,26 +213,115 @@ Note that Kafka source does **NOT** rely on committed
offsets for fault toleranc
is only for exposing the progress of consumer and consuming group for
monitoring.
### Monitoring
-Kafka source exposes metrics in Flink's metric group for monitoring and
diagnosing.
-#### Scope of Metric
-All metrics of Kafka source reader are registered under group
```KafkaSourceReader```, which is a
-child group of operator metric group. Metrics related to a specific topic
partition will be registered
-in the group
```KafkaSourceReader.topic.<topic_name>.partition.<partition_id>```.
-For example, current consuming offset of topic "my-topic" and partition 1 will
be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.topic.my-topic.partition.1.currentOffset```
,
+Kafka source exposes the following metrics in the respective [scope]({{< ref
"docs/ops/metrics" >}}/#scope).
-and number of successful commits will be reported in metric:
-```<some_parent_groups>.operator.KafkaSourceReader.commitsSucceeded``` .
+#### Scope of Metric
-#### List of Metrics
+All metrics of Kafka source reader are registered under group ```<job metric
group>.<operator name>.<subtask>.KafkaSourceReader```, which is a
Review comment:
Considering that they were not present in 1.13.0/1.13.1 it _may_ be fine
to change things?
Why do things like this get casually backported in a bugfix release -.-
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
