This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking-website.git
The following commit(s) were added to refs/heads/master by this push:
new 55be2c32f62 blog: Monitoring DynamoDB with SkyWalking (#583)
55be2c32f62 is described below
commit 55be2c32f62135efa47aff2d3444fd06c8f7eced
Author: yswdqz <[email protected]>
AuthorDate: Wed Mar 15 15:46:36 2023 +0800
blog: Monitoring DynamoDB with SkyWalking (#583)
---
.../aws-service.png | Bin 0 -> 41520 bytes
.../cloudwatch.png | Bin 0 -> 76862 bytes
.../endpoint.png | Bin 0 -> 150219 bytes
.../gateway_to_ui.png | Bin 0 -> 18977 bytes
.../2023-03-13-skywalking-aws-dynamodb/icon.png | Bin 0 -> 43282 bytes
.../2023-03-13-skywalking-aws-dynamodb/index.md | 353 +++++++++++++++++++++
.../2023-03-13-skywalking-aws-dynamodb/kinesis.png | Bin 0 -> 76369 bytes
.../2023-03-13-skywalking-aws-dynamodb/service.png | Bin 0 -> 181788 bytes
8 files changed, 353 insertions(+)
diff --git a/content/blog/2023-03-13-skywalking-aws-dynamodb/aws-service.png
b/content/blog/2023-03-13-skywalking-aws-dynamodb/aws-service.png
new file mode 100644
index 00000000000..adc130c9894
Binary files /dev/null and
b/content/blog/2023-03-13-skywalking-aws-dynamodb/aws-service.png differ
diff --git a/content/blog/2023-03-13-skywalking-aws-dynamodb/cloudwatch.png
b/content/blog/2023-03-13-skywalking-aws-dynamodb/cloudwatch.png
new file mode 100644
index 00000000000..7dfbac44c88
Binary files /dev/null and
b/content/blog/2023-03-13-skywalking-aws-dynamodb/cloudwatch.png differ
diff --git a/content/blog/2023-03-13-skywalking-aws-dynamodb/endpoint.png
b/content/blog/2023-03-13-skywalking-aws-dynamodb/endpoint.png
new file mode 100644
index 00000000000..595f9a107fe
Binary files /dev/null and
b/content/blog/2023-03-13-skywalking-aws-dynamodb/endpoint.png differ
diff --git a/content/blog/2023-03-13-skywalking-aws-dynamodb/gateway_to_ui.png
b/content/blog/2023-03-13-skywalking-aws-dynamodb/gateway_to_ui.png
new file mode 100644
index 00000000000..2aad0400ac4
Binary files /dev/null and
b/content/blog/2023-03-13-skywalking-aws-dynamodb/gateway_to_ui.png differ
diff --git a/content/blog/2023-03-13-skywalking-aws-dynamodb/icon.png
b/content/blog/2023-03-13-skywalking-aws-dynamodb/icon.png
new file mode 100644
index 00000000000..de2fd3dcef0
Binary files /dev/null and
b/content/blog/2023-03-13-skywalking-aws-dynamodb/icon.png differ
diff --git a/content/blog/2023-03-13-skywalking-aws-dynamodb/index.md
b/content/blog/2023-03-13-skywalking-aws-dynamodb/index.md
new file mode 100644
index 00000000000..788b55f8640
--- /dev/null
+++ b/content/blog/2023-03-13-skywalking-aws-dynamodb/index.md
@@ -0,0 +1,353 @@
+---
+title: "Monitoring DynamoDB with SkyWalking"
+date: 2023-03-13
+author: Yueqin Zhang
+description: This article shows how to use SkyWalking to monitor DynamoDB.
+---
+
+
+
+
+
+## Background
+[Apache SkyWalking](https://skywalking.apache.org/) is an open-source
application performance management system that helps users collect and
aggregate logs, traces, metrics, and events, and display them on the UI.
Starting from OAP 9.4.0, SkyWalking has added [AWS Firehose
receiver](https://skywalking.apache.org/docs/main/next/en/setup/backend/aws-firehose-receiver/),
which is used to receive and calculate the data of CloudWatch metrics. In this
article, we will take DynamoDB as an examp [...]
+
+## What are Amazon CloudWatch and Amazon Kinesis Data Firehose？
+
+[Amazon CloudWatch](https://aws.amazon.com/cloudwatch/) is a metrics
repository, this tool can collect raw data from AWS (e.g. DynamoDB) and process
it into readable metrics in near real-time. Also, we can use **Metric Stream**
to continuously stream CloudWatch metrics to a selected target location for
near real-time delivery and low latency. SkyWalking takes advantage of this
feature to create metric streams and direct them to Amazon Kinesis Data
Firehose transport streams for further t [...]
+
+[Amazon Kinesis Data
Firehose](https://aws.amazon.com/kinesis/data-firehose/)is an extract,
transform, and load (ETL) service that reliably captures, transforms, and
delivers streaming data to data lakes, data stores, and analytics services.
SkyWalking takes advantage of this feature to eventually direct the metrics
stream to the aws-firehose-receiver for OAP to calculate and ultimately display
the metrics.
+
+The flow chart is as follows.
+
+
+
+###### Notice
+
+- Due to Kinesis Data Firehose specifications, the URL of the HTTP endpoint
must use the HTTPS protocol and must use port 443. Also, this URL must be
proxied by Gateway and forwarded to the real aws-firehose-receiver.
+- The TLS certificate must be signed by a CA and the self-signed certificate
will not be trusted by Kinesis Data Firehose.
+
+## Setting up DynamoDB monitoring
+Next, let's take DynamoDB as an example to illustrate the necessary settings
in aws before using OAP to collect CloudWatch metrics:
+
+1. Go to [Kinesis Console](https://console.aws.amazon.com/kinesis/home),
create a data stream, and select `Direct PUT` for `Source` and `HTTP Endpoint`
for `Destination`. And set `HTTP Endpoint URL` to `Gateway URL`. The rest of
the configuration options can be configured as needed.
+
+! [image.png](. /kinesis.png)
+
+2. Go to the [CloudWatch
Console](https://console.aws.amazon.com/cloudwatch/home), select
`Metrics-Stream` in the left control panel, and click Create metric stream.
Select `AWS/DynamoDB` for `namespace`. Also, you can add other namespaces as
needed. `Kinesis Data Firehose` selects the data stream created in the first
step. Finally, set the output format to opentelemetry0.7. The rest of the
configuration options can be configured as needed.
+
+! [cloudwatch.png](. /cloudwatch.png)
+
+At this point, the AWS side of DynamoDB monitoring configuration is set up.
+## SkyWalking OAP metrics processing analysis
+SkyWalking uses aws-firehose-receiver to receive and decode AWS metrics
streams forwarded by Gateway, and send it to
[Opentelemetry-receiver](https://github.com/apache/skywalking/tree/master/oap-server/server-receiver-plugin/otel-receiver-plugin)
for processing and transforming into SkyWalking metrics. Then, the metrics are
analyzed and aggregated by [Meter Analysis Language
(MAL)](https://skywalking.apache.org/docs/main/next/en/concepts-and-designs/mal/)
and finally presented on the UI.
+
+The MAL part and the UI part of SkyWalking support users' customization, to
display the metrics data in a more diversified way. For details, please refer
to [MAL
doc](https://skywalking.apache.org/docs/main/next/en/concepts-and-designs/mal/)
and [UI doc](https://skywalking.apache.org/docs/main/next/en/ui/readme/).
+
+
+
+## Typical metrics analysis
+### Scope
+In SkyWalking, there is the concept of scope. By using scopes, we can classify
and aggregate metrics more rationally. In the monitoring of DynamoDB, two of
these scopes are used - Service and Endpoint.
+
+Service represents a set of workloads that provide the same behavior for
incoming requests. Commonly used as cluster-level scopes for services, user
accounts are closer to the concept of clusters in AWS. So SkyWalking uses AWS
account id as a key to map AWS accounts to Service types.
+
+Similarly, Endpoint represents a logical concept, often used in services for
the path of incoming requests, such as HTTP URI path or gRPC service class +
method signature, and can also represent the table structure in the database.
So SkyWalking maps DynamoDB tables to Endpoint type.
+### Metrics
+
+| Metric Name | Meaning |
+| --- | --- |
+| AccountMaxReads / AccountMaxWrites | The maximum number of read/write
capacity units that can be used by an account. |
+| AccountMaxTableLevelReads / AccountMaxTableLevelWrites | The maximum number
of read/write capacity units that can be used by a table or global secondary
index of an account. |
+| AccountProvisionedReadCapacityUtilization /
AccountProvisionedWriteCapacityUtilization | The percentage of provisioned
read/write capacity units utilized by an account. |
+| MaxProvisionedTableReadCapacityUtilization /
MaxProvisionedTableWriteCapacityUtilization | The percentage of provisioned
read/write capacity utilized by the highest provisioned read table or global
secondary index of an account. |
+
+Above are some common account metrics (Serivce scope). They are various
configuration information in DynamoDB, and SkyWalking can show a complete
picture of the database configuration changes by monitoring these metrics.
+
+| Metric Name | Meaning |
+| --- | --- |
+| ConsumedReadCapacityUnits / ConsumedWriteCapacityUnits | The number of
read/write capacity units consumed over the specified time period. |
+| ReturnedItemCount | The number of items returned by Query, Scan or
ExecuteStatement (select) operations during the specified time period. |
+| SuccessfulRequestLatency | The latency of successful requests to DynamoDB or
Amazon DynamoDB Streams during the specified time period. |
+| TimeToLiveDeletedItemCount | The number of items deleted by Time to Live
(TTL) during the specified time period. |
+
+The above are some common table metrics (Endpoint scope), which will also be
aggregated into account metrics. These metrics are generally used to analyze
the performance of the database, and users can use them to determine the
reasonable level of database configuration. For example, users can track how
much of their provisioned throughput is used through ConsumedReadCapicityUnits
/ ConsumedReadCapicityUnits to determine the reasonableness of the
preconfigured throughput of a table or acc [...]
+
+| Metric Name | Meaning |
+| --- | --- |
+| UserErrors |Requests to DynamoDB or Amazon DynamoDB Streams that generate an
HTTP 400 status code during the specified time period. |
+| SystemErrors |The requests to DynamoDB or Amazon DynamoDB Streams that
generate an HTTP 500 status code during the specified time period. |
+| ThrottledRequests | Requests to DynamoDB that exceed the provisioned
throughput limits on a resource.|
+| TransactionConflict | Rejected item-level requests due to transactional
conflicts between concurrent requests on the same items. |
+
+The above are some common error metrics, among which UserErrors are
account-level metrics and the rest are table-level metrics. Users can set
alarms on these metrics, and if warnings appear, then it may indicate that
there are some problems with the use of the database, and users need to check
and verify by themselves.
+
+### Notice
+SkyWalking's metrics selection for DynamoDB comes directly from CloudWatch
metrics, which can also be found at [CloudWatch metrics
doc](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/metrics-dimensions.html)
to get metrics details.
+
+## Demo
+In this section, we will demonstrate how to use terraform to create a DynamoDB
table and other AWS services that can generate metrics streams, and deploy
Skywalking to complete the metrics collection.
+
+First, you need a running gateway instance, such as
[NGINX](https://www.nginx.com/), which is responsible for receiving metrics
streams from AWS and forwarding them to the aws-firehose-receiver. Note that
the gateway needs to be configured with certificates to accept HTTPS protocol
requests.
+
+Below is an example configuration for NGINX. The configuration does not need
to be identical, as long as it can send incoming HTTPS requests to `oap
host:12801/aws/firehose/metrics`.
+
+```
+server {
+ listen 443 ssl;
+
+ ssl_certificate /crt/test.pem;
+ ssl_certificate_key /crt/test.key;
+
+ ssl_session_timeout 5m;
+ ssl_ciphers
ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
+ ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
+ ssl_prefer_server_ciphers on;
+
+ location /aws/firehose/metrics {
+ proxy_pass http://test.xyz:12801/aws/firehose/metrics;
+ }
+ }
+```
+
+### Deploying SkyWalking
+There are various ways to deploy SkyWalking, and you can get them directly
from the [release
page](https://github.com/apache/skywalking/releases/tag/v9.4.0).
+
+Of course, if you are more comfortable with Kubernetes, you can also find the
appropriate deployment method from
[SkyWalking-kubernetes](https://github.com/apache/skywalking-kubernetes).
+
+Please note that no matter which deployment method you use, please make sure
that the OAP and UI version is 9.4.0 or higher and that port 12801 needs to be
open.
+
+The following is an example of a deployment using the helm command.
+
+```
+export SKYWALKING_RELEASE_VERSION=4.3.0
+export SKYWALKING_RELEASE_NAME=skywalking
+export SKYWALKING_RELEASE_NAMESPACE=default
+
+helm install "${SKYWALKING_RELEASE_NAME}" \
+ oci://registry-1.docker.io/apache/skywalking-helm \
+ --version "${SKYWALKING_RELEASE_VERSION}" \
+ -n "${SKYWALKING_RELEASE_NAMESPACE}" \
+ --set oap.image.tag=9.4.0 \
+ --set oap.storageType=elasticsearch \
+ --set ui.image.tag=9.4.0 \
+ --set oap.ports.firehose=12801
+```
+
+### Start the corresponding AWS service
+
+The terraform configuration file is as follows (example modified in[Terraform
Registry -
kinesis_firehose_delivery_stream](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_firehose_delivery_stream)）：
+<details>
+<summary>terraform configuration file </summary>
+
+```
+provider "aws" {
+ region = "ap-northeast-1"
+ access_key = "[need change]your access_key"
+ secret_key = "[need change]your secret_key"
+}
+resource "aws_dynamodb_table" "basic-dynamodb-table" {
+ name = "GameScores"
+ billing_mode = "PROVISIONED"
+ read_capacity = 20
+ write_capacity = 20
+ hash_key = "UserId"
+ range_key = "GameTitle"
+
+ attribute {
+ name = "UserId"
+ type = "S"
+ }
+
+ attribute {
+ name = "GameTitle"
+ type = "S"
+ }
+
+ attribute {
+ name = "TopScore"
+ type = "N"
+ }
+
+ ttl {
+ attribute_name = "TimeToExist"
+ enabled = true
+ }
+
+ global_secondary_index {
+ name = "GameTitleIndex"
+ hash_key = "GameTitle"
+ range_key = "TopScore"
+ write_capacity = 10
+ read_capacity = 10
+ projection_type = "INCLUDE"
+ non_key_attributes = ["UserId"]
+ }
+
+ tags = {
+ Name = "dynamodb-table-1"
+ Environment = "production"
+ }
+}
+
+resource "aws_cloudwatch_metric_stream" "main" {
+ name = "my-metric-stream"
+ role_arn = aws_iam_role.metric_stream_to_firehose.arn
+ firehose_arn = aws_kinesis_firehose_delivery_stream.http_stream.arn
+ output_format = "opentelemetry0.7"
+
+ include_filter {
+ namespace = "AWS/DynamoDB"
+ }
+}
+
+#
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-metric-streams-trustpolicy.html
+data "aws_iam_policy_document" "streams_assume_role" {
+ statement {
+ effect = "Allow"
+
+ principals {
+ type = "Service"
+ identifiers = ["streams.metrics.cloudwatch.amazonaws.com"]
+ }
+
+ actions = ["sts:AssumeRole"]
+ }
+}
+
+resource "aws_iam_role" "metric_stream_to_firehose" {
+ name = "metric_stream_to_firehose_role"
+ assume_role_policy = data.aws_iam_policy_document.streams_assume_role.json
+}
+
+#
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-metric-streams-trustpolicy.html
+data "aws_iam_policy_document" "metric_stream_to_firehose" {
+ statement {
+ effect = "Allow"
+
+ actions = [
+ "firehose:PutRecord",
+ "firehose:PutRecordBatch",
+ ]
+
+ resources = [aws_kinesis_firehose_delivery_stream.http_stream.arn]
+ }
+}
+resource "aws_iam_role_policy" "metric_stream_to_firehose" {
+ name = "default"
+ role = aws_iam_role.metric_stream_to_firehose.id
+ policy = data.aws_iam_policy_document.metric_stream_to_firehose.json
+}
+
+resource "aws_s3_bucket" "bucket" {
+ bucket = "metric-stream-test-bucket"
+}
+
+resource "aws_s3_bucket_acl" "bucket_acl" {
+ bucket = aws_s3_bucket.bucket.id
+ acl = "private"
+}
+
+data "aws_iam_policy_document" "firehose_assume_role" {
+ statement {
+ effect = "Allow"
+
+ principals {
+ type = "Service"
+ identifiers = ["firehose.amazonaws.com"]
+ }
+
+ actions = ["sts:AssumeRole"]
+ }
+}
+
+resource "aws_iam_role" "firehose_to_s3" {
+ assume_role_policy = data.aws_iam_policy_document.firehose_assume_role.json
+}
+
+data "aws_iam_policy_document" "firehose_to_s3" {
+ statement {
+ effect = "Allow"
+
+ actions = [
+ "s3:AbortMultipartUpload",
+ "s3:GetBucketLocation",
+ "s3:GetObject",
+ "s3:ListBucket",
+ "s3:ListBucketMultipartUploads",
+ "s3:PutObject",
+ ]
+
+ resources = [
+ aws_s3_bucket.bucket.arn,
+ "${aws_s3_bucket.bucket.arn}/*",
+ ]
+ }
+}
+
+resource "aws_iam_role_policy" "firehose_to_s3" {
+ name = "default"
+ role = aws_iam_role.firehose_to_s3.id
+ policy = data.aws_iam_policy_document.firehose_to_s3.json
+}
+
+resource "aws_kinesis_firehose_delivery_stream" "http_stream" {
+ name = "metric-stream-test-stream"
+ destination = "http_endpoint"
+
+ http_endpoint_configuration {
+ name = "test_http_endpoint"
+ url = "[need change]Gateway url"
+ role_arn = aws_iam_role.firehose_to_s3.arn
+ }
+ s3_configuration {
+ role_arn = aws_iam_role.firehose_to_s3.arn
+ bucket_arn = aws_s3_bucket.bucket.arn
+ }
+}
+```
+</details>
+
+Steps to use.
+
+1. Get the access_key and secret_key of the AWS account.( For how to get them,
please refer to
[create-access-key](https://aws.amazon.com/premiumsupport/knowledge-center/create-access-key/)
)
+
+2. Fill in the access_key and secret_key you got in the previous step, and
fill in the corresponding URL of your gateway in the corresponding location of
`aws_kinesis_firehose_delivery_stream` configuration.
+
+3. Copy the above content and save it to the main.tf file.
+
+4. Execute the following code in the corresponding path.
+
+```
+terraform init
+terraform apply
+```
+
+At this point, all the required AWS services have been successfully created,
and you can check your console to see if the services were successfully created.
+### Done!
+
+If all the above steps were successful, please wait for about five minutes.
After that, you can visit the SkyWalking UI to see the metrics.
+
+
+
+
+Currently, the metrics collected by SkyWalking by default are displayed as
follows.
+
+**account metrics:**
+
+
+
+**table metrics：**
+
+
+
+### Other services
+
+Currently, SkyWalking officially supports EKS, S3, DynamoDB monitoring. Users
also refer to [the OpenTelemetry
receiver](https://skywalking.apache.org/docs/main/next/en/setup/backend/opentelemetry-receiver/)
to configure OTel rules to collect and analyze CloudWatch metrics of other AWS
services and display them through [a custom
dashboard](https://skywalking.apache.org/docs/main/next/en/ui/readme/).
+
+### Material
+
+* [Monitoring S3 metrics with Amazon
CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html)
+* [Monitoring DynamoDB metrics with Amazon
CloudWatch](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/monitoring-cloudwatch.html)
+* [Supported metrics in AWS Firehose receiver of
OAP](https://skywalking.apache.org/docs/main/next/en/setup/backend/aws-firehose-receiver/)
+* [Configuration Vocabulary | Apache
SkyWalking](https://skywalking.apache.org/docs/main/next/en/setup/backend/configuration-vocabulary/)
diff --git a/content/blog/2023-03-13-skywalking-aws-dynamodb/kinesis.png
b/content/blog/2023-03-13-skywalking-aws-dynamodb/kinesis.png
new file mode 100644
index 00000000000..dcd0d6c31de
Binary files /dev/null and
b/content/blog/2023-03-13-skywalking-aws-dynamodb/kinesis.png differ
diff --git a/content/blog/2023-03-13-skywalking-aws-dynamodb/service.png
b/content/blog/2023-03-13-skywalking-aws-dynamodb/service.png
new file mode 100644
index 00000000000..d88bfe30f25
Binary files /dev/null and
b/content/blog/2023-03-13-skywalking-aws-dynamodb/service.png differ
