This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new 3794b3dc91 Add a FAQ about other storage options. (#12059)
3794b3dc91 is described below
commit 3794b3dc91022c8847672b88f758cf80c2782d11
Author: 吴晟 Wu Sheng <wu.sh...@foxmail.com>
AuthorDate: Fri Mar 29 11:55:18 2024 +0800
Add a FAQ about other storage options. (#12059)
---
docs/en/FAQ/README.md | 3 ++-
docs/en/FAQ/why-clickhouse-not-supported.md | 38 +++++++++++++++++++++++++++++
docs/en/FAQ/why_mq_not_involved.md | 4 +--
docs/en/changes/changes.md | 1 +
4 files changed, 43 insertions(+), 3 deletions(-)
diff --git a/docs/en/FAQ/README.md b/docs/en/FAQ/README.md
index ed43bbf49a..09329589f2 100644
--- a/docs/en/FAQ/README.md
+++ b/docs/en/FAQ/README.md
@@ -2,7 +2,8 @@
These are known and frequently asked questions about SkyWalking. We welcome
you to contribute here.
## Design
-* [Why doesn't SkyWalking involve MQ in its
architecture?](why_mq_not_involved.md)
+* [Why does SkyWalking use RPC(gRPC and RESTful) rather than MQ as transport
layer by default?](why_mq_not_involved.md)
+* [Why is Clickhouse or Loki or xxx not supported as a storage
option?](why-clickhouse-not-supported.md)
## Compiling
* [Protoc plugin fails in maven build](Protoc-Plugin-Fails-When-Build.md)
diff --git a/docs/en/FAQ/why-clickhouse-not-supported.md
b/docs/en/FAQ/why-clickhouse-not-supported.md
new file mode 100644
index 0000000000..dc85e296c3
--- /dev/null
+++ b/docs/en/FAQ/why-clickhouse-not-supported.md
@@ -0,0 +1,38 @@
+# Why is Clickhouse or Loki or xxx not supported as a storage option?
+
+## Background
+
+In the past several years, community users have asked why Clickhouse, Loki, or
some other storage is not supported in the upstream. We have repeated the
answer many times, but it is still happening, at here, I would like to write
down the summary to help people understand more
+
+## Previous Discussions
+All the following issues were about discussing new storage extension topics.
+- Loki as storage
+ - https://github.com/apache/skywalking/discussions/9836
+- ClickHouse
+ - https://github.com/apache/skywalking/issues/11924
+ - https://github.com/apache/skywalking/discussions/9011
+- Vertica
+ - https://github.com/apache/skywalking/discussions/8817
+
+Generally, all those asking are about adding a new kind of storage.
+
+## Why they don't exist ?
+First of all, `WHY` is not a suitable question. SkyWalking is a
volunteer-driven community, the volunteers build this project including bug
fixes, maintenance work, and new features from their personal and employer
interests. What you saw about the current status is the combination of all
those interests rather than responsibilities.
+So, in SkyWalking, anything you saw existing is/was someone's interest and
contributed to upstream.
+
+This logic is the same as this question, SkyWalking active maintainers are
focusing on JDBC(MySQL and PostgreSQL ecosystem) Database and Elasticsearch for
existing users, and moving forward on BanyanDB as the native one. We for now
don't have people interested in ClickHouse or any other database. That is why
they are not there.
+
+## How could add one?
+To add a new feature, including a new storage plugin, you should go through
[SWIP - SkyWalking Improvement
Proposal](https://skywalking.apache.org/docs/main/next/en/swip/readme/)
workflow, and have a full discussion with the maintenance team.
+SkyWalking has a pluggable storage system, so, ideally new storage option is
possible to implement a new provider for the storage module. Meanwhile, in
practice, as storage implementation should be in high performance and well
optimized, considering our experiences with JDBC and Elasticsearch
implementations, some flags and annotations may need to be added in the kernel
level and data model declarations.
+
+Furthermore, as current maintainers are not a fun of Clickhouse or
others(otherwise, you should have seen those implementations), they are not
going to be involved in the code implementations and they don't know much more
from a general perspective about which kind of implementation in that specific
database will have a better behavior and performance. So, if you want to
propose this to upstream, you should be very experienced in that database, and
have enough scale and environments to p [...]
+
+## What happens next if the new implementation gets accepted/merged/released?
+Who proposed this new implementation(such as clickhouse storage), has to take
the responsibilities of the maintenance. The maintenance means they need to
+1. Join storage relative discussion to make sure SkyWalking can move forward
on a kernel-level optimization without being blocked by these specific storage
options.
+2. Respond to this storage relative questions, bugs, CVEs, and performance
issues.
+3. Make the implementation performance match the expectation of the original
proposal. Such as, about clickhouse, people are talking about how they are
faster and have higher efficiency than Elasticsearch for large-scale
deployments. Then we should always be able to see it has better benchmark and
product side practice.
+
+Even if the storage gets accepted/merged/released, but **no one can't take the
above responsibilities** or **the community doesn't receive the feedback and
questions about those storages**, SkyWalking PMC(Project Management Committee)
will start the process to remove the implementations. This happened before for
Apache IoTDB and InfluxDB storage options. Here is the last vote about this,
+- https://github.com/apache/skywalking/discussions/9059
\ No newline at end of file
diff --git a/docs/en/FAQ/why_mq_not_involved.md
b/docs/en/FAQ/why_mq_not_involved.md
index 21b1bb39e6..3b02cda418 100644
--- a/docs/en/FAQ/why_mq_not_involved.md
+++ b/docs/en/FAQ/why_mq_not_involved.md
@@ -1,4 +1,4 @@
-# Why doesn't SkyWalking involve MQ in its architecture?
+# Why does SkyWalking use RPC(gRPC and RESTful) rather than MQ as transport
layer by default?
This is often asked by those who are first introduced to SkyWalking. Many
believe that MQ should have better performance and should be able to support
higher throughput, like the following:
<img src="MQ-involved-architecture.png"/>
@@ -23,4 +23,4 @@ Even though MQ transport is not recommended from the
production perspective, Sky
`kafka-reporter` and `kafka-fetcher` for this feature since 8.1.0.
### How about MQ metrics data exporter?
-The answer is that the MQ metrics data exporter is already readily available.
The exporter module with gRPC default mechanism is there, and you can easily
provide a new implementor of this module.
+Log and trace exporters are using MQ as transport channel. And metrics
exporter uses gRPC, as considering the scale.
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index f45aaf4933..5b8e99c470 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -129,5 +129,6 @@
* Add `SWIP-5 Support ClickHouse Monitoring`.
* Remove `OpenTelemetry Exporter` support from meter doc, as this has been
flagged as unmaintained on OTEL upstream.
* Add doc of one-line quick start script for different storage types.
+* Add FAQ for `Why is Clickhouse or Loki or xxx not supported as a storage
option?`.
All issues and pull requests are
[here](https://github.com/apache/skywalking/milestone/202?closed=1)
