PeterZh6 opened a new pull request, #11130:
URL: https://github.com/apache/inlong/pull/11130
Fixes #11129
### Motivation
The primary goal of this PR is to enhance metric instrumentation for the
InLong Sort Flink Connector, specifically for the Postgres-CDC connector. This
change is aimed at improving observability by introducing additional metrics
that track serialization/deserialization, snapshot states, and checkpoint
completion.
### Modifications
**Deserialization Metrics:**
Added counters to track successful and failed deserialization attempts
(`numDeserializeSuccess`, `numDeserializeError`).
Added latency gauge to measure time taken for deserialization
(`deserializeTimeLag`).
**SnapshotState Metrics:**
Added counters for the number of snapshots created (`numSnapshotCreate`) and
errors encountered during snapshot operations (`numSnapshotError`).
**NotifyComplete Metrics:**
Added a counter to track completed snapshots (`numCompletedSnapshots`).
Added latency gauge for the time between snapshot creation and checkpoint
completion (`snapshotToCheckpointTimeLag`).
### Verifying this change
*(Please pick either of the following options)*
- [ ] This change is a trivial rework/code cleanup without any test coverage.
- [ ] This change is already covered by existing tests, such as:
*(please describe tests)*
- [x] This change added tests and can be verified as follows:
Make Flink `takemanager `report the metrics, with Slf4jReporter for example.
Add the following configurations to `conf/flink-conf.yaml` of flink
`taskmanager` and the above mentioned metrics will be printed to the logging
file once `inlong-sort` starts to process data.
``` properties
metrics.reporter.slf4j.class: org.apache.flink.metrics.slf4j.Slf4jReporter
metrics.reporter.slf4j.interval: 15 SECONDS
```
### Documentation
- Does this pull request introduce a new feature? **Yes**
- If yes, how is the feature documented? **JavaDocs**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
