xBis7 opened a new pull request, #4044: URL: https://github.com/apache/ozone/pull/4044
## What changes were proposed in this pull request? In this patch we are exposing performance metrics in the `/jmx` and `/prom` endpoints, for using gRPC between OM and S3G clients. The metrics are similar to the ones gathered by hadoop RPC. Hadoop RPC uses a proxy server that monitors the socket and gathers the metrics based on it's traffic. We can achieve similar results using gRPC interceptors to get all the messages sent and received by the server and monitor the number of active client connections using gRPC transport filter. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-7517 ## How was this patch tested? We can't have unit tests under OzoneManager without adding a pom dependency to S3Gateway and the same goes for S3Gateway. We can't have unit tests under it without adding a dependency to OzoneManager. Furthermore, we can't have an integration test with Grpc enabled because Mini Ozone Cluster tries to configure Grpc server ports on multiple OMs with the same port causing binding errors. In order to test this patch, a new acceptance test was added. This acceptance test gets executed from the s3g container from a new compose directory that is configured with grpc enabled. This patch can also be tested manually using the new directory `compose/ozone-grpc-om-s3` that has the configuration for grpc enabled. In addition, we can configure and the two new flags for percentiles and quantile. add in `docker-config` ``` OZONE-SITE.XML_ozone.om.s3.grpc.metrics.percentiles.intervals=60,300,900 OZONE-SITE.XML_ozone.om.s3.grpc.metrics.quantile.enabled=true ``` ``` docker-compose up --scale datanode=3 -d docker exec -it ozone-grpc-om-s3_s3g_1 bash bash-4.2$ aws configure AWS Access Key ID [****************test]: test AWS Secret Access Key [****************pass]: pass Default region name [None]: Default output format [None]: bash-4.2$ aws s3api --endpoint http://localhost:9878 create-bucket --bucket bucket1 { "Location": "http://localhost:9878/bucket1"; } bash-4.2$ aws s3api --endpoint http://localhost:9878 put-object --bucket bucket1 --key dir1/key1 --body README.md ``` And then check `/jmx` and `/prom` endpoints for the om. For example in `/jmx` we have something like ``` { "name" : "Hadoop:service=OzoneManager,name=GrpcOzoneManagerMetrics", "modelerType" : "GrpcOzoneManagerMetrics", "tag.port" : "8981", "tag.Context" : "ozone", "tag.Hostname" : "84865778768c", "GrpcOmProcessingTimeNumOps" : 7, "GrpcOmProcessingTimeAvgTime" : 2.6270819E7, "GrpcOmQueueTimeNumOps" : 7, "GrpcOmQueueTimeAvgTime" : 590112.5, "NumOpenClientConnections" : 1, "ReceivedBytes" : 3765, "SentBytes" : 2715 }, ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
