tjiuming opened a new issue, #18560:
URL: https://github.com/apache/pulsar/issues/18560
### Motivation
Currently, we have WebExecutor metrics and Jetty metrics in the pulsar
broker.
The granularity of these metrics is relatively large, we can only know the
metrics of the Server dimension, but we cannot know the metrics of each
specific endpoint.
This can not provide more help when finding and troubleshooting problems.
For the purpose of solve the above questions, it is necessary to introduce
per-endpoint metrics.
### Goal
Provide per-endpoint metrics.
### API Changes
_No response_
### Implementation
### 1. Implementation
Pulsar using Jersey framework to provide RestAPI service, and Jersey
provided `ContainerRequestFilter` and `ContainerResponseFilter`.
We can use these two filters to monitoring each endpoint's metrics:
```
public class RestEndpointMetricsFilter implements ContainerResponseFilter,
ContainerRequestFilter {
private static final String REQUEST_START_TIME = "requestStartTime";
@Override
public void filter(ContainerRequestContext req, ContainerResponseContext
resp) throws IOException {
String path = req.getUriInfo().getPath();
String method = req.getMethod();
long requestStartTime = req.getProperty(REQUEST_START_TIME);
// ignore....
}
@Override
public void filter(ContainerRequestContext req) throws IOException {
// Set the request start time into properties.
req.setProperty(REQUEST_START_TIME, System.currentTimeMillis());
}
```
But there is a problem, for an Rest endpoint
```
@Path("/user")
public class Users {
@Path("/{id}/query")
public User queryById(@PathParam("id") long userId) {
return xxx;
}
}
```
the request paths can be
`/user/1111/query`,`/user/2222/query`,`/user/3333/query` and etc. It will lead
to tooooo many time series and may even lead to downtime of Prometheus.
So we have to group these paths to `/user/:id/query`, and this will be a
very good optimization.
Fortunately, we can get the method of providing services through
`req.getUriInfo()`, and get the content of `@Path` through reflection:
```
@Override
public void filter(ContainerRequestContext req, ContainerResponseContext
resp) throws IOException {
UriRoutingContext info = (UriRoutingContext) req.getUriInfo();
ResourceMethod rm = info.getMatchedResourceMethod();
Invocable inv = method.getInvocable();
// The endpoint handler class
Class<?> handlingClass = inv.getHandler().getHandlerClass();
// The endpoint handler method
Method handlingMethod = inv.getHandlingMethod();
Path parent = handlingClass.getDeclaredAnnotation(Path.class);
Path child = handlingMethod.getDeclaredAnnotation(Path.class);
// The base path
String parent0 = parent == null ? "" : parent.value();
// The method path
String child0 = child == null ? "" : child.value();
String path = parent0.endsWith("/") || child0.startsWith("/")
? parent0 + child0 : parent0 + "/" + child0;
}
```
### 2. Metrics
| Name | Type
| Description
| Tags |
| ----------------------------------------- | ----------- |
---------------------------------------------------- | ----- |
| pulsar_broker_rest_endpoint_latency_ms | Histogram | The latency of the
request execution. | path, method |
| pulsar_broker_rest_endpoint_failed | Counter | The number of
times this Endpoint failed to execute. | path, method, httpcode |
### Alternatives
_No response_
### Anything else?
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
