This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7ccaf3e908a [doc](sql-dialect)(audit) add doc for sql dialect and
audit plugin (#29775)
7ccaf3e908a is described below
commit 7ccaf3e908afe6a459d8b927a0655c9ce788278b
Author: Mingyu Chen <morning...@163.com>
AuthorDate: Fri Jan 12 21:12:19 2024 +0800
[doc](sql-dialect)(audit) add doc for sql dialect and audit plugin (#29775)
---
docs/en/docs/ecosystem/audit-plugin.md | 40 ++++++++++++++++-
docs/en/docs/lakehouse/sql-dialect.md | 71 +++++++++++++++++++++++++++++++
docs/sidebars.json | 1 +
docs/zh-CN/docs/ecosystem/audit-plugin.md | 41 +++++++++++++++++-
docs/zh-CN/docs/lakehouse/sql-dialect.md | 70 ++++++++++++++++++++++++++++++
5 files changed, 220 insertions(+), 3 deletions(-)
diff --git a/docs/en/docs/ecosystem/audit-plugin.md
b/docs/en/docs/ecosystem/audit-plugin.md
index 401891c8aaf..3698caeb18e 100644
--- a/docs/en/docs/ecosystem/audit-plugin.md
+++ b/docs/en/docs/ecosystem/audit-plugin.md
@@ -28,9 +28,45 @@ under the License.
Doris's audit log plugin was developed based on FE's plugin framework. Is an
optional plugin. Users can install or uninstall this plugin at runtime.
-This plugin can periodically import the FE audit log into the specified Doris
cluster, so that users can easily view and analyze the audit log through SQL.
+This plugin can periodically import the FE audit log into the specified system
table, so that users can easily view and analyze the audit log through SQL.
-## Compile, Configure and Deploy
+## Use the audit log plug-in
+
+Starting from Doris version 2.1, the audit log plug-in is directly integrated
into the Doris as a built-in plug-in. Users do not need to install additional
plug-ins.
+
+After the cluster is started, a system table named `audit_log` will be created
under the `__internal_schema` database to store audit logs.
+
+> 1. If you upgrade from an old version, you can continue to use the previous
plug-in. You can also uninstall the previous plug-in and use the new built-in
plug-in. But note that the built-in plug-in will write the new audit log to a
new table instead of the original audit log table.
+>
+> 2. If it is a version before Doris 2.1, please refer to the following
**Compilation, Configuration and Deployment** chapters.
+
+### Enable plug-in
+
+The audit log plug-in can be turned on or off at any time through the global
variable `enable_audit_plugin` (the default is off), such as:
+
+`set global enable_audit_plugin = true;`
+
+After it is enabled, Doris will write the audit log after it is enabled to the
`audit_log` table.
+
+The audit log plugin can be turned off at any time:
+
+`set global enable_audit_plugin = false;`
+
+After disable, Doris will stop writing to the `audit_log` table. Audit logs
that have been written will not change.
+
+### Related configuration
+
+The audit log table is a dynamic partitioned table, partitioned by day, and
retains the data of the last 30 days by default.
+
+The following 3 global variables can control some writing behaviors of the
audit log table:
+
+- `audit_plugin_max_batch_interval_sec`: The maximum write interval for the
audit log table. Default 60 seconds.
+- `audit_plugin_max_batch_bytes`: The maximum amount of data written in each
batch of the audit log table. Default 50MB.
+- `audit_plugin_max_sql_length`: The maximum length of statements recorded in
the audit log table. Default 4096.
+
+Can be set via `set global xxx=yyy`.
+
+## Compilation, Configuration and Deployment
### FE Configuration
diff --git a/docs/en/docs/lakehouse/sql-dialect.md
b/docs/en/docs/lakehouse/sql-dialect.md
new file mode 100644
index 00000000000..fe89d8e485f
--- /dev/null
+++ b/docs/en/docs/lakehouse/sql-dialect.md
@@ -0,0 +1,71 @@
+---
+{
+ "title": "SQL Dialect",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# SQL Dialect
+
+Starting from version 2.1, Doris can support multiple SQL dialects, such as
Presto, Trino, Hive, PostgreSQL, Spark, Oracle, Clickhouse, and more. Through
this feature, users can directly use the corresponding SQL dialect to query
data in Doris, which facilitates users to smoothly migrate their original
business to Doris.
+
+> 1. This function is currently an experimental function. If you encounter any
problems during use, you are welcome to provide feedback through the mail
group, [Github issue](https://github.com/apache/doris/issues), etc. .
+>
+> 2. This function only supports query statements and does not support other
DDL and DML statements including Explain.
+
+## Deploy service
+
+1. Download latest [SQL Transform
Tool](https://doris-build-1308700295.cos.ap-beijing.myqcloud.com/transform-doris-tool/transform-doris-tool-1.0.0-bin-x86).
+2. On any FE node, start the service through the following command:
+
+ `nohup ./transform-doris-tool-1.0.0-bin-x86 run --host=0.0.0.0
--port=5001 &`
+
+ > 1. This service is a stateless service and can be started and stopped
at any time.
+ >
+ > 2. `5001` is the service port and can be arbitrarily specified as an
available port.
+ >
+ > 3. It is recommended to start a separate service on each FE node.
+
+3. Start the Doris cluster (version 2.1 or higher)
+4. Set the URL of the SQL Dialect Conversion Service with the following
command in Doris:
+
+ `MySQL> set global sql_converter_service_url =
"http://127.0.0.1:5001/api/v1/convert"`
+
+ > 1. `127.0.0.1:5001` is the deployment node IP and port of the SQL
dialect conversion service.
+
+## Use SQL dialect
+
+Here we take the Presto/Trino SQL dialect as an example:
+
+1. `set sql_dialect = "presto";`
+2. Execute any Presto/Trino SQL syntax for data query.
+
+Currently supported dialect types include:
+
+- `presto`
+- `trino`
+- `hive`
+- `spark`
+- `postgres`
+- `clickhouse`
+- `oracle`
+
diff --git a/docs/sidebars.json b/docs/sidebars.json
index 16fb3af32ed..5d6a34e2a0a 100644
--- a/docs/sidebars.json
+++ b/docs/sidebars.json
@@ -238,6 +238,7 @@
"lakehouse/file",
"lakehouse/filecache",
"lakehouse/external-statistics",
+ "lakehouse/sql-dialect",
"lakehouse/fs-benchmark-tool",
"lakehouse/faq"
]
diff --git a/docs/zh-CN/docs/ecosystem/audit-plugin.md
b/docs/zh-CN/docs/ecosystem/audit-plugin.md
index 9bc608765d4..a6fd56c3468 100644
--- a/docs/zh-CN/docs/ecosystem/audit-plugin.md
+++ b/docs/zh-CN/docs/ecosystem/audit-plugin.md
@@ -28,10 +28,49 @@ under the License.
Doris 的审计日志插件是在 FE 的插件框架基础上开发的。是一个可选插件。用户可以在运行时安装或卸载这个插件。
-该插件可以将 FE 的审计日志定期的导入到指定 Doris 集群中，以方便用户通过 SQL 对审计日志进行查看和分析。
+该插件可以将 FE 的审计日志定期的导入到指定的系统表中，以方便用户通过 SQL 对审计日志进行查看和分析。
+
+## 使用审计日志插件
+
+从 Doris 2.1 版本开始，审计日志插件作为内置插件，直接集成到了 Doris 内核中。用户无需在额外安装插件。
+
+集群启动后，会在 `__internal_schema` 库下创建名为 `audit_log` 的系统表，用于存储审计日志。
+
+> 1.
如果是从老版本升级上来的用户，可以继续使用之前的方式。也可以卸载之前的插件，使用内置插件。但注意内置插件会将新的审计日志写入到新的表中，而不是原有的审计日志表中。
+>
+> 2. 如果是 Doris 2.1 之前的版本，请参阅之后的 **编译、配置和部署** 章节。
+
+### 开启插件
+
+通过全局变量 `enable_audit_plugin` 可以随时开启或关闭审计日志插件（默认为关闭状态），如：
+
+`set global enable_audit_plugin = true;`
+
+开启后，Doris 会将开启后的审计日志写入 `audit_log` 表。
+
+可以随时关闭审计日志插件：
+
+`set global enable_audit_plugin = false;`
+
+关闭后，Doris 将会停止 `audit_log` 表的写入。已写入的审计日志不会变化。
+
+### 相关配置
+
+审计日志表是一张动态分区表，按天分区，默认保留最近30天的数据。
+
+以下3个全局变量可以控制审计日志表的一些写入行为：
+
+- `audit_plugin_max_batch_interval_sec`：审计日志表的最大写入间隔。默认 60 秒。
+- `audit_plugin_max_batch_bytes`：审计日志表每批次最大写入数据量。默认 50MB。
+- `audit_plugin_max_sql_length`：审计日志表里记录的语句的最大长度。默认 4096。
+
+可以通过 `set global xxx=yyy` 进行设置。
+
## 编译、配置和部署
+Doris 2.1 版本之前的用户，请参阅如下方式使用审计日志插件。
+
### FE 配置
审计日志插件框架在 Doris 中是默认开启的的，由 FE 的配置 `plugin_enable` 控制
diff --git a/docs/zh-CN/docs/lakehouse/sql-dialect.md
b/docs/zh-CN/docs/lakehouse/sql-dialect.md
new file mode 100644
index 00000000000..f54e5af74c0
--- /dev/null
+++ b/docs/zh-CN/docs/lakehouse/sql-dialect.md
@@ -0,0 +1,70 @@
+---
+{
+ "title": "SQL方言兼容",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# SQL方言兼容
+
+从 2.1 版本开始，Doris 可以支持多种 SQL 方言，如 Presto、Trino、Hive、PostgreSQL、Spark、Clickhouse
等等。通过这个功能，用户可以直接使用对应的 SQL 方言查询 Doris 中的数据，方便用户将原先的业务平滑的迁移到 Doris 中。
+
+> 1. 该功能目前是实验性功能，您在使用过程中如遇到任何问题，欢迎通过邮件组、[Github
issue](https://github.com/apache/doris/issues) 等方式进行反馈。
+>
+> 2. 该功能只支持查询语句，不支持包括 Explain 在内的其他 DDL、DML 语句。
+
+## 部署服务
+
+1. 下载最新版本的 [SQL
方言转换工具](https://doris-build-1308700295.cos.ap-beijing.myqcloud.com/transform-doris-tool/transform-doris-tool-1.0.0-bin-x86)。
+2. 在任意FE节点，通过以下命令启动服务：
+
+ `nohup ./transform-doris-tool-1.0.0-bin-x86 run --host=0.0.0.0
--port=5001 &`
+
+ > 1. 该服务是一个无状态的服务，可随时启停。
+ >
+ > 2. `5001` 是服务端口，可以任意指定为一个可用端口。
+ >
+ > 3. 建议在每个 FE 节点都单独启动一个服务。
+
+3. 启动 Doris 集群（2.1 或更高版本）
+4. 通过以下命令，在Doris中设置 SQL 方言转换服务的 URL：
+
+ `MySQL> set global sql_converter_service_url =
"http://127.0.0.1:5001/api/v1/convert"`
+
+ > 1. `127.0.0.1:5001` 是 SQL 方言转换服务的部署节点 ip 和端口。
+
+## 使用 SQL 方言
+
+这里我们以 Presto/Trino SQL 方言为例。
+
+1. `set sql_dialect = "presto";`
+2. 执行任意 Presto/Trino SQL 语法进行数据查询。
+
+目前支持的方言类型包括：
+
+- `presto`
+- `trino`
+- `hive`
+- `spark`
+- `postgres`
+- `clickhouse`
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
