pg-yang commented on issue #9883: URL: https://github.com/apache/skywalking/issues/9883#issuecomment-1384131290
From what I know 1. [Amazon CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html) could collect metrics from most AWS services(e.g. S3, API Gateway). 2. [CloudWatch metrics stream ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Metric-Streams.html) is a feature of CloudWatch, it could stream CloudWatch metrics to a destination of your choice. Actually by my test, the only option of destination is [Amazon Kinesis Data Firehose](https://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.html) 3. [Amazon Kinesis Data Firehose](https://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.html) could send data to HTTP endpoint(refer to [Choose HTTP Endpoint](https://docs.aws.amazon.com/firehose/latest/dev/create-destination.html#create-destination-http)), and the HTTP endpoint should follow[ the specification](https://docs.aws.amazon.com/firehose/latest/dev/httpdeliveryrequestresponse.html) So we could develop a module(let's name it ` aws-firehose-receiver ` temporarily). ` aws-firehose-receiver ` will follow [ the specification](https://docs.aws.amazon.com/firehose/latest/dev/httpdeliveryrequestresponse.html) to receive HTTP request from [Amazon Kinesis Data Firehose](https://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.html), and convert the HTTP request to OTLP request, and then invoke `org.apache.skywalking.oap.server.receiver.otel.otlp.OpenTelemetryMetricHandler#export` for reusing code in `otel-receiver-plugin`. Here is the data follow of S3(others is similar) ```mermaid graph LR; S3 --> CloudWatch --> CloudWatch-stream --> Amazon-Kinesis-Data-Firehose --> aws-firehose-receiver --> OpenTelemetryMetricHandler ``` Here is some doc maybe you want to read 1. [CloudWatch Metric Streams – Send AWS Metrics to Partners and to Your Apps in Real Time](https://aws.amazon.com/cn/blogs/aws/cloudwatch-metric-streams-send-aws-metrics-to-partners-and-to-your-apps-in-real-time/) 2. [S3 metrics with Cloud Watch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) 3. [API Gateway metrics with Cloud Watch](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-metrics.html) Here is S3 metrics sample data in json format ```json {"metric_stream_name":"QuickFull-QwkHX6","account_id":"008835616606","region":"ap-northeast-1","namespace":"AWS/S3","metric_name":"HeadRequests","dimensions":{"BucketName":"metricstreams-quickfull-qwkhx6-cyp2xuov","FilterId":"test-"},"timestamp":1673092920000,"value":{"max":1.0,"min":1.0,"sum":1.0,"count":1.0},"unit":"Count"} {"metric_stream_name":"QuickFull-QwkHX6","account_id":"008835616606","region":"ap-northeast-1","namespace":"AWS/S3","metric_name":"4xxErrors","dimensions":{"BucketName":"metricstreams-quickfull-qwkhx6-cyp2xuov","FilterId":"test-"},"timestamp":1673092920000,"value":{"max":0.0,"min":0.0,"sum":0.0,"count":1.0},"unit":"Count"} ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
