This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 6683a349e0c [release] Fix EN 3.0.0 release pics (#1209)
6683a349e0c is described below
commit 6683a349e0ceb713ed1ec812c442ed5029512b5a
Author: KassieZ <[email protected]>
AuthorDate: Mon Oct 21 12:02:51 2024 +0800
[release] Fix EN 3.0.0 release pics (#1209)
# Versions
- [ ] dev
- [ ] 3.0
- [ ] 2.1
- [ ] 2.0
# Languages
- [ ] Chinese
- [ ] English
---
blog/release-note-3.0.0.md | 15 ++++++-------
common_docs_zh/releasenotes/v2.0/release-2.0.0.md | 25 +++++++++++----------
common_docs_zh/releasenotes/v2.1/release-2.1.0.md | 22 +++++++++---------
releasenotes/v2.1/release-2.1.0.md | 6 ++---
releasenotes/v3.0/release-3.0.0.md | 14 ++++++------
.../images/blind-test-performance-improvement.png | Bin 0 -> 56237 bytes
.../comparison-with-alternative-solutions.png | Bin 0 -> 55444 bytes
static/images/design-hightlight.PNG | Bin 0 -> 78392 bytes
static/images/query-performance-comparison.png | Bin 0 -> 71871 bytes
static/images/real-time-write-performance..png | Bin 0 -> 82037 bytes
static/images/write-speed-comparison.png | Bin 0 -> 65659 bytes
11 files changed, 41 insertions(+), 41 deletions(-)
diff --git a/blog/release-note-3.0.0.md b/blog/release-note-3.0.0.md
index 0ca688d4426..b6e9854d69e 100644
--- a/blog/release-note-3.0.0.md
+++ b/blog/release-note-3.0.0.md
@@ -30,10 +30,9 @@ under the License.
-->
-
We are excited to announce the release of Apache Doris 3.0!
-**Starting with version 3.X, Apache Doris supports a compute-storage decoupled
mode inaddition to the compute-storage mode for cluster deployment. With the
cloud-native architecture that decouples the computation and storage layers,
users can achieve physical isolation between query loads across multiple
compute clusters, as well as isolation between read and write loads.
Additionally, users can take advantage of low-cost shared storage systems such
as object storage or HDFS to significa [...]
+**Starting from version 3.X, Apache Doris supports a compute-storage decoupled
mode in addition to the compute-storage coupled mode for cluster deployment.
With the cloud-native architecture that decouples the computation and storage
layers, users can achieve physical isolation between query loads across
multiple compute clusters, as well as isolation between read and write loads.
Additionally, users can take advantage of low-cost shared storage systems such
as object storage or HDFS to [...]
Version 3.0 marks a milestone in the evolution of Apache Doris towards a
unified data lake and data warehouse architecture. This version introduces the
ability to write data back to data lakes, allowing users to perform data
analysis, sharing, processing, and storage operations across multiple data
sources within Apache Doris. With capabilities such as asynchronous
materialized views, Apache Doris can serve as a unified data processing engine
for enterprises, helping users better manage [...]
@@ -90,7 +89,7 @@ In the compute-storage decoupled mode, the Apache Doris
architecture consists of
The design of the compute-storage decoupled mode of Apache Doris highlights
the transformation of the FE's in-memory metadata model into a shared metadata
service. This approach offers a globally consistent state view, allowing any
node to directly submit writes without needing to go through the FE for
publishing. During write operations, data is stored in shared storage, while
metadata is managed by the metadata service. **This effectively controls the
number of small files in shared st [...]
-
+
Based on the globally consistent state view, for data garbage collection, we
have adopted a design approach for data deletion that is easier to prove
correct and more efficient.
@@ -104,7 +103,7 @@ Another design of decoupling compute and storage in the
industry is to store the
- **Inability to support real-time writes**: During data writes, the data is
mapped to tablets based on the partitioning and bucketing rules, generating
segment files and rowset meta data. During the write process, a two-phase
commit (Publish) is performed through the FE. When a BE node receives the
Publish request, it then sets the rowset as visible. The Publish operation must
not fail. If the rowset meta data is stored in the shared storage, the total
small file data during the real-ti [...]
- 
+ 
We compared the real-time data write performance of Apache Doris 3.0 with
the above-described solution. We simulated 500 concurrent tasks writing 10,000
data files with 500 rows each, and 50 concurrent tasks writing 250 data files
with 20,000 rows each, using the same computational resources.
@@ -112,7 +111,7 @@ Another design of decoupling compute and storage in the
industry is to store the
At 500 concurrent tasks, the performance of Apache Doris in the
compute-storage decoupled mode showed slight degradation, but it still
maintained an 11X advantage over the industry solution. To ensure a fair test,
Apache Doris did not enable the Group Commit feature (which the industry
solution lacks). Enabling Group Commit would further enhance real-time write
performance.
- 
+ 
Additionally, the industry solution also faces stability and cost issues in
terms of real-time data ingestion:
@@ -144,7 +143,7 @@ In this context, we have conducted performance tests with
different caching stra
- When the cache is completely missed (i.e., the cache is cleared before every
SQL execution, simulating an extreme case), the performance loss is around 35%.
**Even so, Apache Doris in the compute-storage decoupled mode delivers much
higher performance than its alternative solutions.**
-
+
### 1-5 Write speed comparison
@@ -154,7 +153,7 @@ In terms of write performance, we have simulated two test
cases under the same c
- **High-concurrency real-time import**: as described in the "Comparison with
alternative solutions" section.
-
+
### 1-6 Tips for production environment
@@ -383,7 +382,7 @@ In V3.0, Doris implements a self-adaptive Runtime Filter
calculation approach. I
Our blind tests on V3.0 and V2.1 show that the new version is 7.3% and 6.2%
faster in TPC-DS and TPC-H benchmark tests, respectively.
-
+
## 7. New features
diff --git a/common_docs_zh/releasenotes/v2.0/release-2.0.0.md
b/common_docs_zh/releasenotes/v2.0/release-2.0.0.md
index b520d88c142..2fa384bac27 100644
--- a/common_docs_zh/releasenotes/v2.0/release-2.0.0.md
+++ b/common_docs_zh/releasenotes/v2.0/release-2.0.0.md
@@ -24,6 +24,7 @@ specific language governing permissions and limitations
under the License.
-->
+
亲爱的社区小伙伴们，我们很高兴地向大家宣布，Apache Doris 2.0.0 Release 版本已于 2023 年 8 月 11 日正式发布，有超过
275 位贡献者为 Apache Doris 提交了超过 4100 个优化与修复。
在 2.0.0 版本中，Apache Doris 在标准 Benchmark 数据集上盲测查询性能得到超过 10
倍的提升、在日志分析和湖仓一体场景能力得到全面加强、数据更新效率和写入效率都更加高效稳定、支持了更加完善的多租户和资源隔离机制、在资源弹性与存算分离方向踏上了新的台阶、增加了一系列面向企业用户的易用性特性。在经过近半年的开发、测试与稳定性调优后，这一版本已经正式稳定可用，欢迎大家下载使用！
@@ -33,7 +34,7 @@ under the License.
> GitHub
> 源码：[https://github.com/apache/doris/tree/2.0.0-rc04](https://github.com/apache/doris/tree/2.0.0-rc04)
-# 盲测性能 10 倍以上提升！
+## 盲测性能 10 倍以上提升！
在 Apache Doris 2.0.0
版本中，我们引入了全新查询优化器和自适应的并行执行模型，结合存储层、执行层以及执行算子上的一系列性能优化手段，实现了盲测性能 10 倍以上的提升。以
SSB-Flat 和 TPC-H 标准测试数据集为例，在相同的集群和机器配置下，新版本宽表场景盲测较之前版本性能提升 10 倍、多表关联场景盲测提升了 13
倍，实现了巨大的性能飞跃。
@@ -43,7 +44,7 @@ under the License.
以 TPC-H 为例，全新优化器在未进行任何手工调优和 SQL 改写的情况下，绝大多数 SQL 仍领先于旧优化器手工调优后的性能表现！而在超过百家 2.0
版本提前体验用户的真实业务场景中，绝大多数原始 SQL 执行效率得以极大提升！
-参考文档：[https://doris.apache.org/zh-CN/docs/dev/query-acceleration/nereids](https://doris.apache.org/zh-CN/docs/dev/query-acceleration/nereids)
+参考文档：[更智能的全新查询优化器](../../query/nereids/nereids-new)
如何开启：`SET enable_nereids_planner=true` 在 Apache Doris 2.0-beta 版本中全新查询优化器已经默认开启
@@ -69,14 +70,14 @@ under the License.
在实现极速分析体验的同时，为了保证多个混合分析负载的执行效率以及查询的稳定性，在 2.0.0 版本中我们引入了 Pipeline
执行模型作为查询执行引擎。在 Pipeline 执行引擎中，查询的执行是由数据来驱动控制流变化的，各个查询执行过程之中的阻塞算子被拆分成不同
Pipeline，各个 Pipeline
能否获取执行线程调度执行取决于前置数据是否就绪，实现了阻塞操作的异步化、可以更加灵活地管理系统资源，同时减少了线程频繁创建和销毁带来的开销，并提升了
Apache Doris 对于 CPU 的利用效率。因此 Apache Doris 在混合负载场景中的查询性能和稳定性都得到了全面提升。
-参考文档：[https://doris.apache.org/zh-CN/docs/dev/query-acceleration/pipeline-execution-engine](https://doris.apache.org/zh-CN/docs/dev/query-acceleration/pipeline-execution-engine)
+参考文档：[查询执行引擎](../../query/pipeline-execution-engine)
如何开启：` Set enable_pipeline_engine = true `
- 该功能在 Apache Doris 2.0 版本中将默认开启，BE 在进行查询执行时默认将 SQL 的执行模型转变 Pipeline 的执行方式。
- `parallel_pipeline_task_num`代表了 SQL 查询进行查询并发的 Pipeline Task 数目。Apache Doris
默认配置为`0`，此时 Apache Doris 会自动感知每个 BE 的 CPU 核数并把并发度设置为 CPU
核数的一半，用户也可以根据自己的实际情况进行调整。
- 对于从老版本升级的用户，系统自动将该参数设置成老版本中`parallel_fragment_exec_instance_num`的值。
-# 更统一多样的分析场景
+## 更统一多样的分析场景
作为最初诞生于报表分析场景的 OLAP 系统，Apache Doris
在这一擅长领域中做到了极致，凭借自身优异的分析性能和极简的使用体验收获到了众多用户的认可，在诸如实时看板（Dashboard）、实时大屏、业务报表、管理驾驶舱等实时报表场景以及自助
BI 平台、用户行为分析等即席查询场景获得了极为广泛的运用。
@@ -102,7 +103,7 @@ under the License.
- 统一数据分析网关：利用 Apache Doris 构建完善可扩展的数据源连接框架，支持用户将这些外部数据源统一到 Doris
的元数据映射结构上，当用户通过 Doris 查询这些外部数据源时，能够提供一致的查询体验。
-# 高效的数据更新
+## 高效的数据更新
在实时分析场景中，数据更新是非常普遍的需求。用户不仅希望能够实时查询最新数据，也希望能够对数据进行灵活的实时更新。典型场景如电商订单分析、物流运单分析、用户画像等，需要支持数据更新类型包括整行更新、部分列更新、按条件进行批量更新或删除以及整表或者整个分区的重写（inser
overwrite）。
@@ -116,7 +117,7 @@ under the License.
- 支持复杂条件更新和条件删除。在 2.0.0 版本之前 Unique Key 主键模型仅支持简单 Update 和 Delete 操作，在 2.0.0
版本中我们基于 Merge-on-Write 实现了复杂条件的数据更新和删除，并且执行效率更加高效。基于以上优化，Apache Doris
对于各类数据更新需求都有完备的能力支持！
-# 更加高效稳定的数据写入
+## 更加高效稳定的数据写入
### 导入性能进一步提升
@@ -137,7 +138,7 @@ under the License.
在过去版本中我们引入了毫秒级别的 Schema Change，而在最新版本 Flink-Doris-Connector 中，我们实现了从 MySQL
等关系型数据库到 Apache Doris
的一键整库同步。在实际测试中单个同步任务可以承载数千张表的实时并行写入，从此彻底告别过去繁琐复杂的同步流程，通过简单命令即可实现上游业务数据库的表结构及数据同步。同时当上游数据结构发生变更时，也可以自动捕获
Schema 变更并将 DDL 动态同步到 Doris 中，保证业务的无缝运行。
-# 更加完善的多租户资源隔离
+## 更加完善的多租户资源隔离
多租户与资源隔离的主要目的是为了保证高负载时避免相互发生资源抢占，Apache Doris 在过去版本中推出了资源组（Resource
Group）的硬隔离方案，通过对同一个集群内部的 BE 打上标签，标签相同的 BE
会组成一个资源组。数据入库时会按照资源组配置将数据副本写入到不同的资源组中，查询时按照资源组的划分使用对应资源组上的计算资源进行计算，例如将读、写流量放在不同的副本上从而实现读写分离，或者将在线与离线业务划分在不同的资源组、避免在离线分析任务之间的资源抢占。
@@ -147,7 +148,7 @@ under the License.
与此同时，在 Workload Group 中我们还引入了查询排队的功能，在创建 Workload Group
时可以设置最大查询数，超出最大并发的查询将会进行队列中等待执行，以此来缓解高负载下系统的压力。
-# 极致弹性与存算分离
+## 极致弹性与存算分离
过去 Apache Doris 凭借在易用性方面的诸多设计帮助用户大幅节约了计算与存储资源成本，而面向未来的云原生架构，我们已经走出了坚实的一步。
@@ -178,7 +179,7 @@ under the License.
面对更加彻底的存储计算分离需求，飞轮科技（SelectDB）技术团队设计并实现了全新的云原生存算分离架构（SelectDB
Cloud），近一年来经历了大量企业客户的大规模使用，在性能、功能成熟度、系统稳定性等方面经受了真实生产环境的考验。在 Apache Doris 2.0.0
版本发布之际，飞轮科技宣布将这一经过大规模打磨后的成熟架构贡献至 Apache Doris 社区。这一工作预计将于 2023 年 10
月前后完成，届时全部存算分离的代码都将会提交到 Apache Doris 社区主干分支中，预计在 9 月广大社区用户就可以提前体验到基于存算分离架构的预览版本。
-# 易用性进一步提升
+## 易用性进一步提升
除了以上功能需求外，在 Apache Doris 还增加了许多面向企业级特性的体验改进：
@@ -190,7 +191,7 @@ under the License.
在 Apache Doris 2.0.0 版本中，我们可以通过 CCR
的功能在库/表级别将源集群的数据变更同步到目标集群，可根据场景精细控制同步范围；用户也可以根据需求灵活选择全量或者增量同步，有效提升了数据同步的灵活性和效率；此外
Dors CCR 还支持 DDL 同步，源集群执行的 DDL 语句可以自动同步到目标集群，从而保证了数据的一致性。Doris CCR
配置和使用也非常简单，简单操作即可快速完成跨集群数据复制。基于 Doris CCR
优异的能力，可以更好实现读写负载分离以及多机房备份，并可以更好支持不同场景的跨集群复制需求。
-# 其他升级注意事项
+## 其他升级注意事项
- 1.2-lts 需要停机升级到 2.0.0，2.0-alpha 需要停机升级到 2.0.0
- 查询优化器开关默认开启 `enable_nereids_planner=true`；
@@ -209,13 +210,13 @@ under the License.
- 对 Doris 的进程名重命名为 DorisFE 和 DorisBE；
- AES 和 SM4
加解密函数的两参数版本行为变化，详见[对应函数文档](../../sql-manual/sql-functions/encrypt-digest-functions/sm4-encrypt.md)
-# 正式踏上 2.0 之旅
+## 正式踏上 2.0 之旅
在 Apache Doris 2.0.0
版本发布过程中，我们邀请了数百家企业参与新版本的打磨，力求为所有用户提供性能更佳、稳定性更高、易用性更好的数据分析体验。后续我们将会持续敏捷发版来响应所有用户对功能和稳定性的更高追求，预计
2.0 系列的第一个迭代版本 2.0.1 将于 8 月下旬发布，9 月会进一步发布 2.0.2 版本。在快速 Bugfix
的同时，也会不断将一些最新特性加入到新版本中。9 月份我们还将发布 2.1 版本的尝鲜版本，会增加一系列呼声已久的新能力，包括 Variant
可变数据类型以更好满足半结构化数据 Schema Free
的分析需求，多表物化视图，在导入性能方面持续优化、增加新的更加简洁的数据导入方式，通过自动攒批实现更加实时的数据写入，复合数据类型的嵌套能力等。
期待 Apache Doris 2.0 版本的正式发布为更多社区用户提供实时统一的分析体验，我们也相信 Apache Doris 2.0
版本会成为您在实时分析场景中的最理想选择。
-# 致谢
+## 致谢
再次向所有参与 Apache Doris 2.0.0 版本开发和测试的贡献者们表示最衷心的感谢，他们分别是：
diff --git a/common_docs_zh/releasenotes/v2.1/release-2.1.0.md
b/common_docs_zh/releasenotes/v2.1/release-2.1.0.md
index e6749f9eda8..8882cbb2644 100644
--- a/common_docs_zh/releasenotes/v2.1/release-2.1.0.md
+++ b/common_docs_zh/releasenotes/v2.1/release-2.1.0.md
@@ -104,7 +104,7 @@ under the License.
:::note
-参考文档：[Pipeline X 执行引擎](../query/pipeline/pipeline-x-execution-engine.md)
+参考文档：[Pipeline X 执行引擎](../../query-acceleration/pipeline-execution-engine)
:::
## ARM 架构深度适配，性能提升 230%
@@ -143,7 +143,7 @@ under the License.
:::note
- 演示 Demo: https://www.bilibili.com/video/BV1cS421A7kA/?spm_id_from=333.999.0.0
-- 参考文档：[SQL 方言兼容](../lakehouse/sql-dialect.md)
+- 参考文档：[SQL 方言兼容](../../lakehouse/sql-dialect.md)
:::
### 高速数据读取，数据传输效率提升 100 倍
@@ -302,7 +302,7 @@ CREATE MATERIALIZED VIEW mv1
:::note
- 演示 Demo: https://www.bilibili.com/video/BV1s2421T71z/?spm_id_from=333.999.0.0
-- 参考文档：[异步物化视图](../query/view-materialized-view/async-materialized-view.md)
+-
参考文档：[异步物化视图](../../query-acceleration/materialized-view/async-materialized-view/overview)
:::
## 存储能力增强
@@ -407,7 +407,7 @@ PROPERTIES (
:::
:::note
-参考文档：https://doris.apache.org/zh-CN/docs/table-design/data-partition#%E8%87%AA%E5%8A%A8%E5%88%86%E5%8C%BA
+参考文档：[分区分桶](../../table-design/data-partition)
:::
### INSERT INTO SELECT 导入性能提升 100%
@@ -469,7 +469,7 @@ MemTable 前移在 2.1 版本中默认开启，用户无需修改原有的导入
:::note
- 演示 Demo：https://www.bilibili.com/video/BV1um411o7Ha/?spm_id_from=333.999.0.0
-- 参考文档和完整测试报告：[Group Commit](../data-operate/import/group-commit-manual.md)
+- 参考文档和完整测试报告：[Group
Commit](../../data-operate/import/import-way/group-commit-manual)
:::
## 半结构化数据分析
@@ -540,7 +540,7 @@ SELECT v["properties"]["title"] from ${table_name}
:::note
- 演示 Demo: https://www.bilibili.com/video/BV13u4m1g7ra/?spm_id_from=333.999.0.0
-- 参考文档：[VARIANT](../sql-manual/sql-types/Data-Types/VARIANT.md)
+- 参考文档：[VARIANT](../../sql-manual/sql-data-types/semi-structured/VARIANT.md)
:::
### IP 数据类型
@@ -554,7 +554,7 @@ SELECT v["properties"]["title"] from ${table_name}
- INET_ATON：获取包含 IPv4 地址的字符串，格式为 A.B.C.D（点分隔的十进制数字）
:::note
-参考文档：[IPV6](../sql-manual/sql-types/Data-Types/IPV6.md)
+参考文档：[IPV6](../../sql-manual/sql-data-types/ip/IPV6)
:::
### 复杂数据类型分析函数完善
@@ -696,7 +696,7 @@ mysql> select struct(1,"2") not in (struct(1,3),
struct(1,"2"), struct(1,1), nul
:::note
- 演示 Demo：https://www.bilibili.com/video/BV1Fz421X7XE/?spm_id_from=333.999.0.0
-- 参考文档：[Workload Group](../admin-manual/resource-admin/workload-group.md)
+- 参考文档：[Workload Group](../../admin-manual/resource-admin/workload-group.md)
:::
### TopSQL
@@ -753,7 +753,7 @@ select QueryId,max(BePeakMemoryBytes) as be_peak_mem from
active_queries() group
目前主要展示的负载类型包括 Select 和`Insert Into……Select`，预计在 2.1 版本之上的三位迭代版本中会支持 Stream
Load 和 Broker Load 的资源用量展示。
:::note
-参考文档：[ACTIVE_QUERIES](../sql-manual/sql-functions/table-functions/active_queries.md)
+参考文档：[ACTIVE_QUERIES](../../sql-manual/sql-functions/table-functions/active_queries.md)
:::
@@ -855,7 +855,7 @@ JOB e_daily
:::caution
注意事项
-当前 Job Scheduler 仅支持 Insert
内表，参考文档：[CREATE-JOB](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-JOB.md)
+当前 Job Scheduler 仅支持 Insert
内表，参考文档：[CREATE-JOB](../../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-JOB.md)
:::
## Behavior Changed
@@ -874,7 +874,7 @@ JOB e_daily
- 对于之前已经安装过审计日志插件的用户，升级后可以继续使用原有插件，也可以通过 uninstall
命令卸载原有插件后，使用新的插件。但注意，切换插件后，审计日志表也将切换到新的表中。
- - 具体可参阅：[审计日志插件](../admin-manual/audit-plugin.md)
+ - 具体可参阅：[审计日志插件](../../admin-manual/audit-plugin.md)
diff --git a/releasenotes/v2.1/release-2.1.0.md
b/releasenotes/v2.1/release-2.1.0.md
index a86f5143396..b0b88f715ee 100644
--- a/releasenotes/v2.1/release-2.1.0.md
+++ b/releasenotes/v2.1/release-2.1.0.md
@@ -68,7 +68,7 @@ According to TPC-DS benchmark tests (1TB) of Doris V2.1
against Trino,
### SQL dialects compatibility
-To facilitate migration to Doris and increase its compatibility with other
DBMS, we have enabled SQL dialect conversion in V2.1. ([read
more](https://doris.apache.org/docs/lakehouse/sql-dialect/)) For example, by
set sql_dialect = "trino" in Doris, you can use the Trino SQL dialect as you're
used to, without modifying your current business logic, and Doris will execute
the corresponding queries for you. Tests in user production environment show
that Doris V2.1 is compatible with 99% of T [...]
+To facilitate migration to Doris and increase its compatibility with other
DBMS, we have enabled SQL dialect conversion in V2.1. ([read
more](../../lakehouse/sql-dialect)) For example, by set sql_dialect = "trino"
in Doris, you can use the Trino SQL dialect as you're used to, without
modifying your current business logic, and Doris will execute the corresponding
queries for you. Tests in user production environment show that Doris V2.1 is
compatible with 99% of Trino SQL.
### Arrow Flight SQL protocol
@@ -95,7 +95,7 @@ To further release burden on operation and maintenance, V2.1
allows auto data pa
### High-concurrency real-time data ingestion
-For data writing, a back pressure mechanism is in place to avoid execessive
data versions, so as to reduce resource consumption by data version merging. In
addition, V2.1 supports group commit ([read
more](https://doris.apache.org/docs/data-operate/import/group-commit-manual)),
which means to accumulate multiple writing and commit them as one. Benchmark
tests on group commit with JDBC ingestion and the Stream Load method present
great results.
+For data writing, a back pressure mechanism is in place to avoid execessive
data versions, so as to reduce resource consumption by data version merging. In
addition, V2.1 supports group commit ([read
more](../../data-operate/import/import-way/group-commit-manual)), which means
to accumulate multiple writing and commit them as one. Benchmark tests on group
commit with JDBC ingestion and the Stream Load method present great results.
## Semi-structured data analysis
@@ -150,7 +150,7 @@ Please note that this version is only suitable for quick
experience and function
- Audit log plug-in
- Since V2.1.0, Doris has a built-in audit log plug-in. Users can simply
enable or disable it by setting the enable_audit_plugin parameter.
- If you have already installed your own audit log plug-in, you can either
continue using it after upgrading to Doris V2.1, or uninstall it and use the
one in Doris. Please note that the audit log table will be relocated after
switching plug-in.
- - For more details, please see the
[docs](https://doris.apache.org/docs/admin-manual/audit-plugin).
+ - For more details, please see the [docs](../../admin-manual/audit-plugin).
## Credits
diff --git a/releasenotes/v3.0/release-3.0.0.md
b/releasenotes/v3.0/release-3.0.0.md
index 71dd150a023..baa62b37e1e 100644
--- a/releasenotes/v3.0/release-3.0.0.md
+++ b/releasenotes/v3.0/release-3.0.0.md
@@ -27,7 +27,7 @@ under the License.
We are excited to announce the release of Apache Doris 3.0!
-**Starting with version 3.X, Apache Doris supports a compute-storage decoupled
mode inaddition to the compute-storage mode for cluster deployment. With the
cloud-native architecture that decouples the computation and storage layers,
users can achieve physical isolation between query loads across multiple
compute clusters, as well as isolation between read and write loads.
Additionally, users can take advantage of low-cost shared storage systems such
as object storage or HDFS to significa [...]
+**Starting from version 3.X, Apache Doris supports a compute-storage decoupled
mode in addition to the compute-storage coupled mode for cluster deployment.
With the cloud-native architecture that decouples the computation and storage
layers, users can achieve physical isolation between query loads across
multiple compute clusters, as well as isolation between read and write loads.
Additionally, users can take advantage of low-cost shared storage systems such
as object storage or HDFS to [...]
Version 3.0 marks a milestone in the evolution of Apache Doris towards a
unified data lake and data warehouse architecture. This version introduces the
ability to write data back to data lakes, allowing users to perform data
analysis, sharing, processing, and storage operations across multiple data
sources within Apache Doris. With capabilities such as asynchronous
materialized views, Apache Doris can serve as a unified data processing engine
for enterprises, helping users better manage [...]
@@ -84,7 +84,7 @@ In the compute-storage decoupled mode, the Apache Doris
architecture consists of
The design of the compute-storage decoupled mode of Apache Doris highlights
the transformation of the FE's in-memory metadata model into a shared metadata
service. This approach offers a globally consistent state view, allowing any
node to directly submit writes without needing to go through the FE for
publishing. During write operations, data is stored in shared storage, while
metadata is managed by the metadata service. **This effectively controls the
number of small files in shared st [...]
-
+
Based on the globally consistent state view, for data garbage collection, we
have adopted a design approach for data deletion that is easier to prove
correct and more efficient.
@@ -98,7 +98,7 @@ Another design of decoupling compute and storage in the
industry is to store the
- **Inability to support real-time writes**: During data writes, the data is
mapped to tablets based on the partitioning and bucketing rules, generating
segment files and rowset meta data. During the write process, a two-phase
commit (Publish) is performed through the FE. When a BE node receives the
Publish request, it then sets the rowset as visible. The Publish operation must
not fail. If the rowset meta data is stored in the shared storage, the total
small file data during the real-ti [...]
- 
+ 
We compared the real-time data write performance of Apache Doris 3.0 with
the above-described solution. We simulated 500 concurrent tasks writing 10,000
data files with 500 rows each, and 50 concurrent tasks writing 250 data files
with 20,000 rows each, using the same computational resources.
@@ -106,7 +106,7 @@ Another design of decoupling compute and storage in the
industry is to store the
At 500 concurrent tasks, the performance of Apache Doris in the
compute-storage decoupled mode showed slight degradation, but it still
maintained an 11X advantage over the industry solution. To ensure a fair test,
Apache Doris did not enable the Group Commit feature (which the industry
solution lacks). Enabling Group Commit would further enhance real-time write
performance.
- 
+ 
Additionally, the industry solution also faces stability and cost issues in
terms of real-time data ingestion:
@@ -138,7 +138,7 @@ In this context, we have conducted performance tests with
different caching stra
- When the cache is completely missed (i.e., the cache is cleared before every
SQL execution, simulating an extreme case), the performance loss is around 35%.
**Even so, Apache Doris in the compute-storage decoupled mode delivers much
higher performance than its alternative solutions.**
-
+
### 1-5 Write speed comparison
@@ -148,7 +148,7 @@ In terms of write performance, we have simulated two test
cases under the same c
- **High-concurrency real-time import**: as described in the "Comparison with
alternative solutions" section.
-
+
### 1-6 Tips for production environment
@@ -377,7 +377,7 @@ In V3.0, Doris implements a self-adaptive Runtime Filter
calculation approach. I
Our blind tests on V3.0 and V2.1 show that the new version is 7.3% and 6.2%
faster in TPC-DS and TPC-H benchmark tests, respectively.
-
+
## 7. New features
diff --git a/static/images/blind-test-performance-improvement.png
b/static/images/blind-test-performance-improvement.png
new file mode 100644
index 00000000000..e77276d3a58
Binary files /dev/null and
b/static/images/blind-test-performance-improvement.png differ
diff --git a/static/images/comparison-with-alternative-solutions.png
b/static/images/comparison-with-alternative-solutions.png
new file mode 100644
index 00000000000..79417036fa1
Binary files /dev/null and
b/static/images/comparison-with-alternative-solutions.png differ
diff --git a/static/images/design-hightlight.PNG
b/static/images/design-hightlight.PNG
new file mode 100644
index 00000000000..34b46931b48
Binary files /dev/null and b/static/images/design-hightlight.PNG differ
diff --git a/static/images/query-performance-comparison.png
b/static/images/query-performance-comparison.png
new file mode 100644
index 00000000000..e142e775996
Binary files /dev/null and b/static/images/query-performance-comparison.png
differ
diff --git a/static/images/real-time-write-performance..png
b/static/images/real-time-write-performance..png
new file mode 100644
index 00000000000..0880959f409
Binary files /dev/null and b/static/images/real-time-write-performance..png
differ
diff --git a/static/images/write-speed-comparison.png
b/static/images/write-speed-comparison.png
new file mode 100644
index 00000000000..95075b0cbaf
Binary files /dev/null and b/static/images/write-speed-comparison.png differ
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
