Sxnan commented on code in PR #249:
URL: https://github.com/apache/flink-agents/pull/249#discussion_r2406193209
##########
docs/content/docs/operations/monitoring.md:
##########
@@ -24,24 +24,115 @@ under the License.
## Metric
-{{< hint warning >}}
-**TODO**: How to use add custom metrics.
+### Built-in Metrics
-**TODO**: List of all built-in Metrics.
+We offer data monitoring for built-in metrics, which includes events and
actions. Events provide general-level metrics, while actions deliver both
general-level and type-specific data metrics.
-**TODO**: How to check the metrics with Flink executor.
-{{< /hint >}}
+| Component Type | Count | Meter
|
+|-------------------------------|------------------------------|--------------------------------|
+| Agent (Operator Builtin) | NumOfInput<br>NumOfOutput |
NumOfInputPerSec<br>NumOfOutputPerSec |
Review Comment:
We should describe what the metric measures and its implications. We can
take Flink docs as an example
https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/ops/metrics/#cpu
##########
docs/content/docs/operations/monitoring.md:
##########
@@ -24,24 +24,115 @@ under the License.
## Metric
-{{< hint warning >}}
-**TODO**: How to use add custom metrics.
+### Built-in Metrics
-**TODO**: List of all built-in Metrics.
+We offer data monitoring for built-in metrics, which includes events and
actions. Events provide general-level metrics, while actions deliver both
general-level and type-specific data metrics.
-**TODO**: How to check the metrics with Flink executor.
-{{< /hint >}}
+| Component Type | Count | Meter
|
+|-------------------------------|------------------------------|--------------------------------|
+| Agent (Operator Builtin) | NumOfInput<br>NumOfOutput |
NumOfInputPerSec<br>NumOfOutputPerSec |
+| Event | numOfEventProcessed |
numOfEventProcessedPerSec |
+| Action | numOfActionsExecuted |
numOfActionsExecutedPerSec |
+| Pre-Action | numOfActionsExecuted |
numOfActionsExecutedPerSec |
+
+####
+
+### How to add custom metrics
+
+In Flink Agents, users implement their logic by defining custom Actions that
respond to various Events throughout the Agent lifecycle. To support
user-defined metrics, we introduce two new properties: `agent_metric_group` and
`action_metric_group` in the RunnerContext. These properties allow users to
create or update global metrics and independent metrics for actions. For an
introduction to metric types, please refer to the [Metric types
documentation](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/ops/metrics/#metric-types).
+
+Here is the user case example:
+
+``````python
+class MyAgent(Agent):
+ @action(InputEvent)
+ @staticmethod
+ def first_action(event: Event, ctx: RunnerContext): # noqa D102
+ start_time = time.time_ns()
Review Comment:
How about using ... to omit the code that is unrelated to the custom metric
usage.
##########
docs/content/docs/operations/monitoring.md:
##########
@@ -24,24 +24,115 @@ under the License.
## Metric
-{{< hint warning >}}
-**TODO**: How to use add custom metrics.
+### Built-in Metrics
-**TODO**: List of all built-in Metrics.
+We offer data monitoring for built-in metrics, which includes events and
actions. Events provide general-level metrics, while actions deliver both
general-level and type-specific data metrics.
-**TODO**: How to check the metrics with Flink executor.
-{{< /hint >}}
+| Component Type | Count | Meter
|
+|-------------------------------|------------------------------|--------------------------------|
+| Agent (Operator Builtin) | NumOfInput<br>NumOfOutput |
NumOfInputPerSec<br>NumOfOutputPerSec |
+| Event | numOfEventProcessed |
numOfEventProcessedPerSec |
+| Action | numOfActionsExecuted |
numOfActionsExecutedPerSec |
+| Pre-Action | numOfActionsExecuted |
numOfActionsExecutedPerSec |
+
+####
+
+### How to add custom metrics
+
+In Flink Agents, users implement their logic by defining custom Actions that
respond to various Events throughout the Agent lifecycle. To support
user-defined metrics, we introduce two new properties: `agent_metric_group` and
`action_metric_group` in the RunnerContext. These properties allow users to
create or update global metrics and independent metrics for actions. For an
introduction to metric types, please refer to the [Metric types
documentation](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/ops/metrics/#metric-types).
+
+Here is the user case example:
+
+``````python
+class MyAgent(Agent):
+ @action(InputEvent)
+ @staticmethod
+ def first_action(event: Event, ctx: RunnerContext): # noqa D102
+ start_time = time.time_ns()
+ input = event.input
+ content = input.get_review() + " first action."
+ ctx.send_event(MyEvent(value=content))
+
+ # Access the main agent metric group
+ metrics = ctx.agent_metric_group
+
+ # Update global metrics
+ metrics.get_counter("numInputEvent").inc()
+ metrics.get_meter("numInputEventPerSec").mark()
+
+ # Access the per-action metric group
+ action_metrics = ctx.action_metric_group
+ action_metrics.get_histogram("actionLatencyMs") \
+ .update(int(time.time_ns() - start_time) // 1000000)
+
+ @action(MyEvent)
+ @staticmethod
+ def second_action(event: Event, ctx: RunnerContext): # noqa D102
Review Comment:
Is there any extra usage that you want to showcase? If so, we should show
that in just one action. Otherwise, we should remove this action.
In this example, we want to just show the essential code to demonstrate the
usage of the component. So we want as few distractions as possible.
##########
docs/content/docs/operations/monitoring.md:
##########
@@ -24,24 +24,115 @@ under the License.
## Metric
-{{< hint warning >}}
-**TODO**: How to use add custom metrics.
+### Built-in Metrics
-**TODO**: List of all built-in Metrics.
+We offer data monitoring for built-in metrics, which includes events and
actions. Events provide general-level metrics, while actions deliver both
general-level and type-specific data metrics.
-**TODO**: How to check the metrics with Flink executor.
-{{< /hint >}}
+| Component Type | Count | Meter
|
+|-------------------------------|------------------------------|--------------------------------|
+| Agent (Operator Builtin) | NumOfInput<br>NumOfOutput |
NumOfInputPerSec<br>NumOfOutputPerSec |
+| Event | numOfEventProcessed |
numOfEventProcessedPerSec |
+| Action | numOfActionsExecuted |
numOfActionsExecutedPerSec |
+| Pre-Action | numOfActionsExecuted |
numOfActionsExecutedPerSec |
+
+####
+
+### How to add custom metrics
+
+In Flink Agents, users implement their logic by defining custom Actions that
respond to various Events throughout the Agent lifecycle. To support
user-defined metrics, we introduce two new properties: `agent_metric_group` and
`action_metric_group` in the RunnerContext. These properties allow users to
create or update global metrics and independent metrics for actions. For an
introduction to metric types, please refer to the [Metric types
documentation](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/ops/metrics/#metric-types).
+
+Here is the user case example:
+
+``````python
+class MyAgent(Agent):
+ @action(InputEvent)
+ @staticmethod
+ def first_action(event: Event, ctx: RunnerContext): # noqa D102
+ start_time = time.time_ns()
+ input = event.input
+ content = input.get_review() + " first action."
+ ctx.send_event(MyEvent(value=content))
+
+ # Access the main agent metric group
+ metrics = ctx.agent_metric_group
+
+ # Update global metrics
+ metrics.get_counter("numInputEvent").inc()
+ metrics.get_meter("numInputEventPerSec").mark()
+
+ # Access the per-action metric group
+ action_metrics = ctx.action_metric_group
+ action_metrics.get_histogram("actionLatencyMs") \
+ .update(int(time.time_ns() - start_time) // 1000000)
+
+ @action(MyEvent)
+ @staticmethod
+ def second_action(event: Event, ctx: RunnerContext): # noqa D102
+ input = event.value
+ content = input + " second action."
+ ctx.send_event(OutputEvent(output=content))
+
+ # Access the main agent metric group
+ metrics = ctx.agent_metric_group
+
+ # Update global metrics
+ metrics.get_counter("numMyEvent").inc()
+ metrics.get_meter("numMyEventPerSecond").mark()
+
+ # Creating and tracking metrics for MyEvent using submetric group
+ if isinstance(event, MyEvent):
+ sub_metrics = metrics.action_metric_group
+ sub_metrics.get_counter("numEvent").inc()
+ sub_metrics.get_meter("numEventPerSecond").mark()
+``````
+
+
+
+### How to check the metrics with Flink executor
+
+We can check the metric result in the WebUI of Flink Job:
Review Comment:
Or we can check the metric at the location where the Flink metric is
reporting to.
We can link to this Flink docs:
https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/deployment/metric_reporters/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
