This is an automated email from the ASF dual-hosted git repository.
kassiez pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new f124eb18d5 Audit (#1786)
f124eb18d5 is described below
commit f124eb18d554cdfa27e5f4b132ddff70b9a1e8e2
Author: wangtianyi2004 <[email protected]>
AuthorDate: Tue Jan 14 20:59:13 2025 +0800
Audit (#1786)
## Versions
- [x] dev
- [x] 3.0
- [x] 2.1
- [ ] 2.0
## Languages
- [x] Chinese
- [x] English
## Docs Checklist
- [ ] Checked by AI
- [ ] Test Cases Built
---
docs/admin-manual/audit-plugin.md | 180 +++++----------------
.../current/admin-manual/audit-plugin.md | 151 +++--------------
.../version-2.1/admin-manual/audit-plugin.md | 151 +++--------------
.../version-3.0/admin-manual/audit-plugin.md | 151 +++--------------
.../version-2.1/admin-manual/audit-plugin.md | 178 ++++----------------
.../version-3.0/admin-manual/audit-plugin.md | 180 +++++----------------
6 files changed, 184 insertions(+), 807 deletions(-)
diff --git a/docs/admin-manual/audit-plugin.md
b/docs/admin-manual/audit-plugin.md
index 90b9846403..80798071ed 100644
--- a/docs/admin-manual/audit-plugin.md
+++ b/docs/admin-manual/audit-plugin.md
@@ -1,6 +1,6 @@
---
{
- "title": "Audit Log Plugin",
+ "title": "Audit Log",
"language": "en"
}
---
@@ -24,156 +24,56 @@ specific language governing permissions and limitations
under the License.
-->
-This plugin can regularly import the audit logs of FE into a specified system
table, making it convenient for users to view and analyze audit logs through
SQL.
+Doris provides auditing capabilities for database operations, allowing the
recording of user logins, queries, and modification operations on the database.
In Doris, audit logs can be queried directly through built-in system tables or
by viewing Doris's audit log files.
-## Using the Audit Log Plugin
+## Enabling Audit Logs
-Starting from Doris 2.1, the audit log plugin is integrated directly into the
Doris kernel as a built-in plugin. Users do not need to install the plugin
separately.
+The audit log plugin can be enabled or disabled at any time using the global
variable `enable_audit_plugin` (disabled by default), for example:
-After the cluster starts, a system table named `audit_log` will be created
under the `__internal_schema` database to store audit logs.
+`set global enable_audit_plugin = true;`
-:::note
-For versions prior to Doris 2.1, please refer to the 2.0 version documentation.
-:::
+Once enabled, Doris will write the audit logs to the `audit_log` table.
+
+You can disable the audit log plugin at any time:
+
+`set global enable_audit_plugin = false;`
+
+After disabling, Doris will stop writing to the `audit_log` table. The already
written audit logs will remain unchanged.
+
+## Viewing the Audit Log Table
+
+:::caution Note
+
+Before version 2.1.8, as the system version was upgraded, the audit log table
fields may have increased. After upgrading, you need to add fields to the
`audit_log` table using the `ALTER TABLE` command based on the fields in the
audit log table.
-:::warning
-After upgrading to version 2.1, the original audit log plugin will be
unavailable. Refer to the **Audit Log Migration** section to see how to migrate
audit log table data.
:::
-### Enabling the Plugin
+Starting from Doris version 2.1, Doris can write user behavior operations to
the [`audit_log`](../admin-manual/system-tables/internal_schema/audit_log)
table in the `__internal_schema` database by enabling the audit log feature.
-The audit log plugin can be enabled or disabled at any time using the global
variable `enable_audit_plugin` (default is disabled), for example:
+The audit log table is a dynamically partitioned table, partitioned daily by
default, retaining the most recent 30 days of data. You can adjust the
retention period of dynamic partitions by modifying the
`dynamic_partition.start` property using the `ALTER TABLE` statement.
-`set global enable_audit_plugin = true;`
+## Audit Log Files
-Once enabled, Doris will write the enabled audit logs to the `audit_log` table.
+In `fe.conf`, `LOG_DIR` defines the storage path for FE logs. All database
operations executed by this FE node are recorded in `${LOG_DIR}/fe.audit.log`.
To view all operations in the cluster, you need to traverse the audit logs of
each FE node.
-The audit log plugin can be disabled at any time:
+## Audit Log Configuration
-`set global enable_audit_plugin = false;`
+**Global Variables:**
+
+Audit log variables can be modified using `set [global] <var_name> =
<var_value>`.
+
+| Variable | Default Value | Description
|
+| -------------------------------------- | ------------- |
----------------------------------------------- |
+| `audit_plugin_max_batch_interval_sec` | 60 seconds | Maximum write
interval for the audit log table. |
+| `audit_plugin_max_batch_bytes` | 50MB | Maximum data volume
per batch for the audit log table. |
+| `audit_plugin_max_sql_length` | 4096 | Maximum length of
SQL statements recorded in the audit log table. |
+| `audit_plugin_load_timeout` | 600 seconds | Default timeout for
audit log import jobs. |
+
+**FE Configuration Items:**
+
+FE configuration items can be modified by editing the `fe.conf` directory.
-After disabling, Doris will stop writing to the `audit_log` table. The
existing audit logs will not be affected.
-
-### Audit log table
-
-With the upgrade of Doris version, the fields of the audit log table will also
increase. For details, please refer to
[audit_log](./system-tables/internal_schema/audit_log.md)
-
-Starting from version 2.1.8 and 3.0.3, the `audit_log` system table will
automatically add new fields of `audit_log` table as the Doris version is
upgraded.
-
-In previous versions, users need to manually add fields to the `audit_log`
system table through the `ALTER TABLE` command.
-
-### Related Configurations
-
-The audit log table is a dynamic partitioned table, partitioned by day, and
retains data for the last 30 days by default.
+| Configuration Item | Description
|
+| -------------------------- |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
+| `skip_audit_user_list` | If you do not want operations of certain users
to be recorded in the audit logs, you can modify this configuration (supported
since version 3.0.01). For example, use the following command to exclude
`user1` and `user2` from audit log recording: |
-The following global variables can control some writing behaviors of the audit
log table:
-
-- `audit_plugin_max_batch_interval_sec`: The maximum write interval for the
audit log table. Default is 60 seconds.
-- `audit_plugin_max_batch_bytes`: The maximum amount of data written per batch
to the audit log table. Default is 50MB.
-- `audit_plugin_max_sql_length`: The maximum length of statements recorded in
the audit log table. Default is 4096.
-- `audit_plugin_load_timeout`: Default timeout for the audit log import job.
Default is 600 seconds.
-
-These can be set using `set global xxx=yyy`.
-
-FE Configuration:
-
-- `skip_audit_user_list` (supported since 3.0.1)
-
- If you do not want the operations of certain users to be recorded in the
audit log, you can modify this configuration.
-
- ```
- skip_audit_user_list=root
- -- or
- skip_audit_user_list=user1,user2
- ```
-
-## Audit Log Migration
-
-After upgrading to version 2.1, the original audit log plugin will be
unavailable. This section explains how to migrate data from the original audit
log table to the new audit log table.
-
-1. Confirm the field information of the old and new audit log tables.
-
- The default audit log table should be
`doris_audit_db__`.`doris_audit_log_tbl__`.
-
- The new audit log table is `__internal_schema`.`audit_log`.
-
- You can check if the field information of the two tables matches by using
the `DESC table_name` command. Typically, the fields of the old table should be
a subset of the new table.
-
-2. Migrate Audit Log Table Data
-
- You can use the following statement to migrate data from the original
table to the new table:
-
- ```sql
- INSERT INTO __internal_schema.audit_log (
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- )
- SELECT
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- FROM doris_audit_db__.doris_audit_log_tbl__;
- ```
-
-3. Remove Original Plugin
-
- After migration, you can remove the original plugin by using the
`UNINSTALL PLUGIN AuditLoader;` command.
-
-## FAQ
-
-1. No data in the audit log table, or no new data is being ingested after
running for a period of time
-
- You can troubleshoot by following these steps:
-
- - Check if partitions are created correctly
-
- The audit log table is a dynamic partition table partitioned by day.
By default, it creates partitions for the next 3 days and retains historical
partitions for 30 days. Data can only be written to the audit log if partitions
are created correctly.
-
- You can check the scheduling status of dynamic partitions by using
`show dynamic partition tables from __internal_schema` and troubleshoot based
on error reasons. Possible error reasons may include:
-
- - Number of nodes is less than the required replication number: The
audit log table defaults to 3 replicas, so at least 3 BE nodes are required.
You can modify the replication number using the `alter table` statement, for
example:
-
- `alter table __internal_schema.audit_log set
("dynamic_partition.replication_num" = "2")`
-
- - No suitable storage medium: You can check the `storage_medium`
property by using `show create table __internal_schema.audit_log`. If there is
no corresponding storage medium on the BE, partition creation may fail.
-
- - No suitable resource group: The audit log table defaults to the
default resource group. You can check if the resource group has enough node
resources by using the `show backends` command.
-
- - Search for `AuditLoad` in the `fe.log` on the Master FE to see if there
are any related error logs
-
- The audit log is imported into the table through internal stream load
operations. If there are issues with the import process, error logs will be
printed in the `fe.log`.
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/audit-plugin.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/audit-plugin.md
index 1700b35eeb..e89a8aba99 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/audit-plugin.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/audit-plugin.md
@@ -1,6 +1,6 @@
---
{
- "title": "审计日志插件",
+ "title": "审计日志",
"language": "zh-CN"
}
---
@@ -24,23 +24,9 @@ specific language governing permissions and limitations
under the License.
-->
-该插件可以将 FE 的审计日志定期的导入到指定的系统表中，以方便用户通过 SQL 对审计日志进行查看和分析。
+Doris 提供了对于数据库操作的审计能力，可以记录用户对数据库的登陆、查询、修改操作。在 Doris
中，可以直接通过内置系统表查询审计日志，也可以直接查看 Doris 的审计日志文件。
-## 使用审计日志插件
-
-从 Doris 2.1 版本开始，审计日志插件作为内置插件，直接集成到了 Doris 内核中。用户无需在额外安装插件。
-
-集群启动后，会在 `__internal_schema` 库下创建名为 `audit_log` 的系统表，用于存储审计日志。
-
-:::note
-如果是 Doris 2.1 之前的版本，请参阅 2.0 版本文档。
-:::
-
-:::warning
-升级到 2.1 版本后，原有的审计日志插件将不可用。请参阅 **审计日志迁移** 章节查看如何迁移审计日志表数据。
-:::
-
-### 开启插件
+## 开启审计日志
通过全局变量 `enable_audit_plugin` 可以随时开启或关闭审计日志插件（默认为关闭状态），如：
@@ -54,127 +40,40 @@ under the License.
关闭后，Doris 将会停止 `audit_log` 表的写入。已写入的审计日志不会变化。
-### 审计日志表
-
-随着 Doris 的版本升级，审计日志表的字段也会增加，具体请参阅
[audit_log](./system-tables/internal_schema/audit_log.md)
-
-自 2.1.8 和 3.0.3 版本开始，`audit_log` 系统表会随着 Doris 版本升级，自动添加新增字段。
-
-之前的版本，用户需手动通过 `ALTER TABLE` 命令为 `audit_log` 系统表增加字段。
-
-### 相关配置
-
-审计日志表是一张动态分区表，按天分区，默认保留最近 30 天的数据。
-
-以下全局变量可以控制审计日志表的一些写入行为：
-
-- `audit_plugin_max_batch_interval_sec`：审计日志表的最大写入间隔。默认 60 秒。
-- `audit_plugin_max_batch_bytes`：审计日志表每批次最大写入数据量。默认 50MB。
-- `audit_plugin_max_sql_length`：审计日志表里记录的语句的最大长度。默认 4096。
-- `audit_plugin_load_timeout`: 审计日志导入作业的默认超时时间。默认 600 秒。
-
-可以通过 `set global xxx=yyy` 进行设置。
+## 查看审计日志表
-FE 配置项：
+:::caution 注意
-- `skip_audit_user_list` (自 3.0.1 支持)
+在 2.1.8 版本之前，随着系统版本的升级，审计日志字段会有增加，在升级后需要根据审计日志表中的字段，通过 `ALTER TALBE` 命令为
`audit_log` 表增加字段。
- 如果不希望某些用户的操作被审计日志记录，可以通过这个配置修改。
-
- ```
- skip_audit_user_list=root
- -- or
- skip_audit_user_list=user1,user2
- ```
-
-## 审计日志迁移
-
-升级到 2.1 版本后，原有的审计日志插件将不可用。本小节介绍如何将原有审计日志表中的数据迁移到新的审计日志表中。
-
-1. 确认新旧审计日志表的字段信息
-
- 原有审计日志表默认情况下应为：`doris_audit_db__`.`doris_audit_log_tbl__`。
-
- 新的审计日志表为：`__internal_schema`.`audit_log`
-
- 可以通过 `DESC table_name` 命令查看两种表的字段信息是否匹配。通常情况下，旧表的字段应为新表的子集。
-
-2. 迁移审计日志表数据。
-
- 可以使用如下语句将原表中数据迁移到新表中：
-
- ```sql
- INSERT INTO __internal_schema.audit_log (
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- )
- SELECT
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- FROM doris_audit_db__.doris_audit_log_tbl__;
- ```
-
-3. 删除原有插件
-
- 迁移后，可以通过 `UNINSTALL PLUGIN AuditLoader;` 命令删除原有插件即可。
-
-## FAQ
+:::
-1. 审计日志表中没有数据，或运行一段时间后，不再进入新的数据
+从 Doris 2.1 版本开始，Doirs 可以通过开启审计日志功能，将用户行为操作写入到 `__internal_schema` 库的
[`audit_log`](../admin-manual/system-tables/internal_schema/audit_log) 表中。
- 可以通过以下步骤排查：
+审计日志表是一张动态分区表，按天进行分区，默认保留最近 30 天的数据。可以通过 ALTER TABLE 语句修改动态分区的
`dynamic_partition.start` 属性调整动态分区的保留天数。
- - 检查分区是否被正常创建
+## 审计日志文件
- 审计日志表是一张按天分区的动态分区表，默认会创建未来 3 天的分区，并保留历史 30 天的分区。只有分区被正确创建后，才能正常写入审计日志。
+在 fe.conf 中，LOG\_DIR 定义了 FE 日志的存储路径。在 ${LOG\_DIR}/fe.audit.log 中记录了这台 FE
节点执行的所有数据库操作。如果需要查看集群所有的操作，需要便利每一台 FE 的审计日志。
- 可以通过 `show dynamic partition tables from __internal_schema`
查看动态分区的调度情况，并根据错误原因排查。可能得错误原因包括：
+## 审计日志相关配置
- - 节点数小于所需副本数：审计日志表默认 3 副本，所以至少需要 3 台 BE 节点。或者通过 `alter table`
语句修改副本数，如：
+**全局变量：**
- `alter table __internal_schema.audit_log set
("dynamic_partition.replication_num" = "2")`
+可以通过 `set [global] <var_name> = <var_value>` 修改审计日志变量。
- - 没有合适的存储介质：可以通过 `show create table __internal_schema.audit_log` 查看
`storage_medium` 属性，如果 BE 没有对应的存储介质，则分区可能创建失败。
+| 变量 | 默认值 | 说明 |
+| ------------------------------------- | ----- | ---------------- |
+| `audit_plugin_max_batch_interval_sec` | 60 秒 | 审计日志表的最大写入间隔。 |
+| `audit_plugin_max_batch_bytes` | 50MB | 审计日志表每批次最大写入数据量 |
+| `audit_plugin_max_sql_length` | 4096 | 审计日志表里记录的语句的最大长度 |
+| `audit_plugin_load_timeout` | 600 秒 | 审计日志导入作业的默认超时时间 |
- - 没有合适的资源组：审计日志表默认在 default 资源组。可以通过 `show backends`
命令查看该资源自是否有足够的节点资源。
+**FE 配置项：**
- - 在 Master FE 的 `fe.log` 中搜索 `AuditLoad` 字样，查看是否有相关错误日志
+通过修改 fe.conf 目录可以修改 FE 配置项。
- 审计日志是通过内部的 stream load 操作导入到表中的，有可能是导入流程出现了问题，这些问题会在 `fe.log` 中打印错误日志。
+| 配置项 | 说明
|
+| ---------------------- |
---------------------------------------------------------------------------- |
+| `skip_audit_user_list` | 如果不希望某些用户的操作被审计日志记录，可以通过这个配置修改（自 3.0.01
支持）。如通过以下命令屏蔽 user1 与 user2 的审计日志记录： |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/audit-plugin.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/audit-plugin.md
index 9e8e2d8c90..e89a8aba99 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/audit-plugin.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/audit-plugin.md
@@ -1,6 +1,6 @@
---
{
- "title": "审计日志插件",
+ "title": "审计日志",
"language": "zh-CN"
}
---
@@ -24,23 +24,9 @@ specific language governing permissions and limitations
under the License.
-->
-该插件可以将 FE 的审计日志定期的导入到指定的系统表中，以方便用户通过 SQL 对审计日志进行查看和分析。
+Doris 提供了对于数据库操作的审计能力，可以记录用户对数据库的登陆、查询、修改操作。在 Doris
中，可以直接通过内置系统表查询审计日志，也可以直接查看 Doris 的审计日志文件。
-## 使用审计日志插件
-
-从 Doris 2.1 版本开始，审计日志插件作为内置插件，直接集成到了 Doris 内核中。用户无需在额外安装插件。
-
-集群启动后，会在 `__internal_schema` 库下创建名为 `audit_log` 的系统表，用于存储审计日志。
-
-:::note
-如果是 Doris 2.1 之前的版本，请 2.0 版本文档。
-:::
-
-:::warning
-升级到 2.1 版本后，原有的审计日志插件将不可用。请参阅 **审计日志迁移** 章节查看如何迁移审计日志表数据。
-:::
-
-### 开启插件
+## 开启审计日志
通过全局变量 `enable_audit_plugin` 可以随时开启或关闭审计日志插件（默认为关闭状态），如：
@@ -54,127 +40,40 @@ under the License.
关闭后，Doris 将会停止 `audit_log` 表的写入。已写入的审计日志不会变化。
-### 审计日志表
-
-随着 Doris 的版本升级，审计日志表的字段也会增加，具体请参阅
[audit_log](./system-tables/internal_schema/audit_log.md)
-
-自 2.1.8 和 3.0.3 版本开始，`audit_log` 系统表会随着 Doris 版本升级，自动添加新增字段。
-
-之前的版本，用户需手动通过 `ALTER TABLE` 命令为 `audit_log` 系统表增加字段。
-
-### 相关配置
-
-审计日志表是一张动态分区表，按天分区，默认保留最近 30 天的数据。
-
-以下全局变量可以控制审计日志表的一些写入行为：
-
-- `audit_plugin_max_batch_interval_sec`：审计日志表的最大写入间隔。默认 60 秒。
-- `audit_plugin_max_batch_bytes`：审计日志表每批次最大写入数据量。默认 50MB。
-- `audit_plugin_max_sql_length`：审计日志表里记录的语句的最大长度。默认 4096。
-- `audit_plugin_load_timeout`: 审计日志导入作业的默认超时时间。默认 600 秒。
-
-可以通过 `set global xxx=yyy` 进行设置。
+## 查看审计日志表
-FE 配置项：
+:::caution 注意
-- `skip_audit_user_list` (自 3.0.1 支持)
+在 2.1.8 版本之前，随着系统版本的升级，审计日志字段会有增加，在升级后需要根据审计日志表中的字段，通过 `ALTER TALBE` 命令为
`audit_log` 表增加字段。
- 如果不希望某些用户的操作被审计日志记录，可以通过这个配置修改。
-
- ```
- skip_audit_user_list=root
- -- or
- skip_audit_user_list=user1,user2
- ```
-
-## 审计日志迁移
-
-升级到 2.1 版本后，原有的审计日志插件将不可用。本小节介绍如何将原有审计日志表中的数据迁移到新的审计日志表中。
-
-1. 确认新旧审计日志表的字段信息
-
- 原有审计日志表默认情况下应为：`doris_audit_db__`.`doris_audit_log_tbl__`。
-
- 新的审计日志表为：`__internal_schema`.`audit_log`
-
- 可以通过 `DESC table_name` 命令查看两种表的字段信息是否匹配。通常情况下，旧表的字段应为新表的子集。
-
-2. 迁移审计日志表数据。
-
- 可以使用如下语句将原表中数据迁移到新表中：
-
- ```sql
- INSERT INTO __internal_schema.audit_log (
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- )
- SELECT
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- FROM doris_audit_db__.doris_audit_log_tbl__;
- ```
-
-3. 删除原有插件
-
- 迁移后，可以通过 `UNINSTALL PLUGIN AuditLoader;` 命令删除原有插件即可。
-
-## FAQ
+:::
-1. 审计日志表中没有数据，或运行一段时间后，不再进入新的数据
+从 Doris 2.1 版本开始，Doirs 可以通过开启审计日志功能，将用户行为操作写入到 `__internal_schema` 库的
[`audit_log`](../admin-manual/system-tables/internal_schema/audit_log) 表中。
- 可以通过以下步骤排查：
+审计日志表是一张动态分区表，按天进行分区，默认保留最近 30 天的数据。可以通过 ALTER TABLE 语句修改动态分区的
`dynamic_partition.start` 属性调整动态分区的保留天数。
- - 检查分区是否被正常创建
+## 审计日志文件
- 审计日志表是一张按天分区的动态分区表，默认会创建未来 3 天的分区，并保留历史 30 天的分区。只有分区被正确创建后，才能正常写入审计日志。
+在 fe.conf 中，LOG\_DIR 定义了 FE 日志的存储路径。在 ${LOG\_DIR}/fe.audit.log 中记录了这台 FE
节点执行的所有数据库操作。如果需要查看集群所有的操作，需要便利每一台 FE 的审计日志。
- 可以通过 `show dynamic partition tables from __internal_schema`
查看动态分区的调度情况，并根据错误原因排查。可能得错误原因包括：
+## 审计日志相关配置
- - 节点数小于所需副本数：审计日志表默认 3 副本，所以至少需要 3 台 BE 节点。或者通过 `alter table`
语句修改副本数，如：
+**全局变量：**
- `alter table __internal_schema.audit_log set
("dynamic_partition.replication_num" = "2")`
+可以通过 `set [global] <var_name> = <var_value>` 修改审计日志变量。
- - 没有合适的存储介质：可以通过 `show create table __internal_schema.audit_log` 查看
`storage_medium` 属性，如果 BE 没有对应的存储介质，则分区可能创建失败。
+| 变量 | 默认值 | 说明 |
+| ------------------------------------- | ----- | ---------------- |
+| `audit_plugin_max_batch_interval_sec` | 60 秒 | 审计日志表的最大写入间隔。 |
+| `audit_plugin_max_batch_bytes` | 50MB | 审计日志表每批次最大写入数据量 |
+| `audit_plugin_max_sql_length` | 4096 | 审计日志表里记录的语句的最大长度 |
+| `audit_plugin_load_timeout` | 600 秒 | 审计日志导入作业的默认超时时间 |
- - 没有合适的资源组：审计日志表默认在 default 资源组。可以通过 `show backends`
命令查看该资源自是否有足够的节点资源。
+**FE 配置项：**
- - 在 Master FE 的 `fe.log` 中搜索 `AuditLoad` 字样，查看是否有相关错误日志
+通过修改 fe.conf 目录可以修改 FE 配置项。
- 审计日志是通过内部的 stream load 操作导入到表中的，有可能是导入流程出现了问题，这些问题会在 `fe.log` 中打印错误日志。
+| 配置项 | 说明
|
+| ---------------------- |
---------------------------------------------------------------------------- |
+| `skip_audit_user_list` | 如果不希望某些用户的操作被审计日志记录，可以通过这个配置修改（自 3.0.01
支持）。如通过以下命令屏蔽 user1 与 user2 的审计日志记录： |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/admin-manual/audit-plugin.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/admin-manual/audit-plugin.md
index 9e8e2d8c90..e89a8aba99 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/admin-manual/audit-plugin.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/admin-manual/audit-plugin.md
@@ -1,6 +1,6 @@
---
{
- "title": "审计日志插件",
+ "title": "审计日志",
"language": "zh-CN"
}
---
@@ -24,23 +24,9 @@ specific language governing permissions and limitations
under the License.
-->
-该插件可以将 FE 的审计日志定期的导入到指定的系统表中，以方便用户通过 SQL 对审计日志进行查看和分析。
+Doris 提供了对于数据库操作的审计能力，可以记录用户对数据库的登陆、查询、修改操作。在 Doris
中，可以直接通过内置系统表查询审计日志，也可以直接查看 Doris 的审计日志文件。
-## 使用审计日志插件
-
-从 Doris 2.1 版本开始，审计日志插件作为内置插件，直接集成到了 Doris 内核中。用户无需在额外安装插件。
-
-集群启动后，会在 `__internal_schema` 库下创建名为 `audit_log` 的系统表，用于存储审计日志。
-
-:::note
-如果是 Doris 2.1 之前的版本，请 2.0 版本文档。
-:::
-
-:::warning
-升级到 2.1 版本后，原有的审计日志插件将不可用。请参阅 **审计日志迁移** 章节查看如何迁移审计日志表数据。
-:::
-
-### 开启插件
+## 开启审计日志
通过全局变量 `enable_audit_plugin` 可以随时开启或关闭审计日志插件（默认为关闭状态），如：
@@ -54,127 +40,40 @@ under the License.
关闭后，Doris 将会停止 `audit_log` 表的写入。已写入的审计日志不会变化。
-### 审计日志表
-
-随着 Doris 的版本升级，审计日志表的字段也会增加，具体请参阅
[audit_log](./system-tables/internal_schema/audit_log.md)
-
-自 2.1.8 和 3.0.3 版本开始，`audit_log` 系统表会随着 Doris 版本升级，自动添加新增字段。
-
-之前的版本，用户需手动通过 `ALTER TABLE` 命令为 `audit_log` 系统表增加字段。
-
-### 相关配置
-
-审计日志表是一张动态分区表，按天分区，默认保留最近 30 天的数据。
-
-以下全局变量可以控制审计日志表的一些写入行为：
-
-- `audit_plugin_max_batch_interval_sec`：审计日志表的最大写入间隔。默认 60 秒。
-- `audit_plugin_max_batch_bytes`：审计日志表每批次最大写入数据量。默认 50MB。
-- `audit_plugin_max_sql_length`：审计日志表里记录的语句的最大长度。默认 4096。
-- `audit_plugin_load_timeout`: 审计日志导入作业的默认超时时间。默认 600 秒。
-
-可以通过 `set global xxx=yyy` 进行设置。
+## 查看审计日志表
-FE 配置项：
+:::caution 注意
-- `skip_audit_user_list` (自 3.0.1 支持)
+在 2.1.8 版本之前，随着系统版本的升级，审计日志字段会有增加，在升级后需要根据审计日志表中的字段，通过 `ALTER TALBE` 命令为
`audit_log` 表增加字段。
- 如果不希望某些用户的操作被审计日志记录，可以通过这个配置修改。
-
- ```
- skip_audit_user_list=root
- -- or
- skip_audit_user_list=user1,user2
- ```
-
-## 审计日志迁移
-
-升级到 2.1 版本后，原有的审计日志插件将不可用。本小节介绍如何将原有审计日志表中的数据迁移到新的审计日志表中。
-
-1. 确认新旧审计日志表的字段信息
-
- 原有审计日志表默认情况下应为：`doris_audit_db__`.`doris_audit_log_tbl__`。
-
- 新的审计日志表为：`__internal_schema`.`audit_log`
-
- 可以通过 `DESC table_name` 命令查看两种表的字段信息是否匹配。通常情况下，旧表的字段应为新表的子集。
-
-2. 迁移审计日志表数据。
-
- 可以使用如下语句将原表中数据迁移到新表中：
-
- ```sql
- INSERT INTO __internal_schema.audit_log (
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- )
- SELECT
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- FROM doris_audit_db__.doris_audit_log_tbl__;
- ```
-
-3. 删除原有插件
-
- 迁移后，可以通过 `UNINSTALL PLUGIN AuditLoader;` 命令删除原有插件即可。
-
-## FAQ
+:::
-1. 审计日志表中没有数据，或运行一段时间后，不再进入新的数据
+从 Doris 2.1 版本开始，Doirs 可以通过开启审计日志功能，将用户行为操作写入到 `__internal_schema` 库的
[`audit_log`](../admin-manual/system-tables/internal_schema/audit_log) 表中。
- 可以通过以下步骤排查：
+审计日志表是一张动态分区表，按天进行分区，默认保留最近 30 天的数据。可以通过 ALTER TABLE 语句修改动态分区的
`dynamic_partition.start` 属性调整动态分区的保留天数。
- - 检查分区是否被正常创建
+## 审计日志文件
- 审计日志表是一张按天分区的动态分区表，默认会创建未来 3 天的分区，并保留历史 30 天的分区。只有分区被正确创建后，才能正常写入审计日志。
+在 fe.conf 中，LOG\_DIR 定义了 FE 日志的存储路径。在 ${LOG\_DIR}/fe.audit.log 中记录了这台 FE
节点执行的所有数据库操作。如果需要查看集群所有的操作，需要便利每一台 FE 的审计日志。
- 可以通过 `show dynamic partition tables from __internal_schema`
查看动态分区的调度情况，并根据错误原因排查。可能得错误原因包括：
+## 审计日志相关配置
- - 节点数小于所需副本数：审计日志表默认 3 副本，所以至少需要 3 台 BE 节点。或者通过 `alter table`
语句修改副本数，如：
+**全局变量：**
- `alter table __internal_schema.audit_log set
("dynamic_partition.replication_num" = "2")`
+可以通过 `set [global] <var_name> = <var_value>` 修改审计日志变量。
- - 没有合适的存储介质：可以通过 `show create table __internal_schema.audit_log` 查看
`storage_medium` 属性，如果 BE 没有对应的存储介质，则分区可能创建失败。
+| 变量 | 默认值 | 说明 |
+| ------------------------------------- | ----- | ---------------- |
+| `audit_plugin_max_batch_interval_sec` | 60 秒 | 审计日志表的最大写入间隔。 |
+| `audit_plugin_max_batch_bytes` | 50MB | 审计日志表每批次最大写入数据量 |
+| `audit_plugin_max_sql_length` | 4096 | 审计日志表里记录的语句的最大长度 |
+| `audit_plugin_load_timeout` | 600 秒 | 审计日志导入作业的默认超时时间 |
- - 没有合适的资源组：审计日志表默认在 default 资源组。可以通过 `show backends`
命令查看该资源自是否有足够的节点资源。
+**FE 配置项：**
- - 在 Master FE 的 `fe.log` 中搜索 `AuditLoad` 字样，查看是否有相关错误日志
+通过修改 fe.conf 目录可以修改 FE 配置项。
- 审计日志是通过内部的 stream load 操作导入到表中的，有可能是导入流程出现了问题，这些问题会在 `fe.log` 中打印错误日志。
+| 配置项 | 说明
|
+| ---------------------- |
---------------------------------------------------------------------------- |
+| `skip_audit_user_list` | 如果不希望某些用户的操作被审计日志记录，可以通过这个配置修改（自 3.0.01
支持）。如通过以下命令屏蔽 user1 与 user2 的审计日志记录： |
diff --git a/versioned_docs/version-2.1/admin-manual/audit-plugin.md
b/versioned_docs/version-2.1/admin-manual/audit-plugin.md
index b1399800c7..80798071ed 100644
--- a/versioned_docs/version-2.1/admin-manual/audit-plugin.md
+++ b/versioned_docs/version-2.1/admin-manual/audit-plugin.md
@@ -1,6 +1,6 @@
---
{
- "title": "Audit Log Plugin",
+ "title": "Audit Log",
"language": "en"
}
---
@@ -24,176 +24,56 @@ specific language governing permissions and limitations
under the License.
-->
-This plugin can regularly import the audit logs of FE into a specified system
table, making it convenient for users to view and analyze audit logs through
SQL.
+Doris provides auditing capabilities for database operations, allowing the
recording of user logins, queries, and modification operations on the database.
In Doris, audit logs can be queried directly through built-in system tables or
by viewing Doris's audit log files.
-## Using the Audit Log Plugin
+## Enabling Audit Logs
-Starting from Doris 2.1, the audit log plugin is integrated directly into the
Doris kernel as a built-in plugin. Users do not need to install the plugin
separately.
-
-After the cluster starts, a system table named `audit_log` will be created
under the `__internal_schema` database to store audit logs.
-
-:::note
-For versions prior to Doris 2.1, please refer to the 2.0 version documentation.
-:::
-
-:::warning
-After upgrading to version 2.1, the original audit log plugin will be
unavailable. Refer to the **Audit Log Migration** section to see how to migrate
audit log table data.
-:::
-
-### Enabling the Plugin
-
-The audit log plugin can be enabled or disabled at any time using the global
variable `enable_audit_plugin` (default is disabled), for example:
+The audit log plugin can be enabled or disabled at any time using the global
variable `enable_audit_plugin` (disabled by default), for example:
`set global enable_audit_plugin = true;`
-Once enabled, Doris will write the enabled audit logs to the `audit_log` table.
+Once enabled, Doris will write the audit logs to the `audit_log` table.
-The audit log plugin can be disabled at any time:
+You can disable the audit log plugin at any time:
`set global enable_audit_plugin = false;`
-After disabling, Doris will stop writing to the `audit_log` table. The
existing audit logs will not be affected.
-
-### Audit log table
-
-With the upgrade of Doris version, the fields of the audit log table will also
increase. For details, please refer to
[audit_log](./system-tables/internal_schema/audit_log.md)
+After disabling, Doris will stop writing to the `audit_log` table. The already
written audit logs will remain unchanged.
-Starting from version 2.1.8 and 3.0.3, the `audit_log` system table will
automatically add new fields of `audit_log` table as the Doris version is
upgraded.
+## Viewing the Audit Log Table
-In previous versions, users need to manually add fields to the `audit_log`
system table through the `ALTER TABLE` command.
+:::caution Note
-### Related configurations
+Before version 2.1.8, as the system version was upgraded, the audit log table
fields may have increased. After upgrading, you need to add fields to the
`audit_log` table using the `ALTER TABLE` command based on the fields in the
audit log table.
-The audit log table is a dynamic partitioned table, partitioned by day, and
retains data for the last 30 days by default.
-
-The following global variables can control some writing behaviors of the audit
log table:
-
-- `audit_plugin_max_batch_interval_sec`: The maximum write interval for the
audit log table. Default 60 seconds.
-- `audit_plugin_max_batch_bytes`: The maximum amount of data written in each
batch of the audit log table. Default 50MB.
-- `audit_plugin_max_sql_length`: The maximum length of statements recorded in
the audit log table. Default 4096.
-- `audit_plugin_load_timeout`: The default timeout of audit log load job.
Default 600 seconds.
-
-Can be set via `set global xxx=yyy`.
-
-FE configurations:
-
-- `skip_audit_user_list` (Since 3.0.1)
-
- If you do not want certain users' operations to be recorded in the audit
log, you can modify this configuration.
-
- ```
- skip_audit_user_list=root
- -- or
- skip_audit_user_list=user1,user2
- ```
-
-## Compilation, Configuration and Deployment
-
-### FE Configuration
+:::
-The audit log plug-in framework is enabled by default in Doris and is
controlled by the FE configuration `plugin_enable`
+Starting from Doris version 2.1, Doris can write user behavior operations to
the [`audit_log`](../admin-manual/system-tables/internal_schema/audit_log)
table in the `__internal_schema` database by enabling the audit log feature.
-These can be set using `set global xxx=yyy`.
+The audit log table is a dynamically partitioned table, partitioned daily by
default, retaining the most recent 30 days of data. You can adjust the
retention period of dynamic partitions by modifying the
`dynamic_partition.start` property using the `ALTER TABLE` statement.
-FE Configuration:
+## Audit Log Files
-- `skip_audit_user_list` (supported since 3.0.1)
+In `fe.conf`, `LOG_DIR` defines the storage path for FE logs. All database
operations executed by this FE node are recorded in `${LOG_DIR}/fe.audit.log`.
To view all operations in the cluster, you need to traverse the audit logs of
each FE node.
- If you do not want the operations of certain users to be recorded in the
audit log, you can modify this configuration.
+## Audit Log Configuration
- ```
- skip_audit_user_list=root
- -- or
- skip_audit_user_list=user1,user2
- ```
+**Global Variables:**
-## Audit Log Migration
+Audit log variables can be modified using `set [global] <var_name> =
<var_value>`.
-After upgrading to version 2.1, the original audit log plugin will be
unavailable. This section explains how to migrate data from the original audit
log table to the new audit log table.
+| Variable | Default Value | Description
|
+| -------------------------------------- | ------------- |
----------------------------------------------- |
+| `audit_plugin_max_batch_interval_sec` | 60 seconds | Maximum write
interval for the audit log table. |
+| `audit_plugin_max_batch_bytes` | 50MB | Maximum data volume
per batch for the audit log table. |
+| `audit_plugin_max_sql_length` | 4096 | Maximum length of
SQL statements recorded in the audit log table. |
+| `audit_plugin_load_timeout` | 600 seconds | Default timeout for
audit log import jobs. |
-1. Confirm the field information of the old and new audit log tables.
+**FE Configuration Items:**
- The default audit log table should be
`doris_audit_db__`.`doris_audit_log_tbl__`.
-
- The new audit log table is `__internal_schema`.`audit_log`.
-
- You can check if the field information of the two tables matches by using
the `DESC table_name` command. Typically, the fields of the old table should be
a subset of the new table.
+FE configuration items can be modified by editing the `fe.conf` directory.
-2. Migrate Audit Log Table Data
+| Configuration Item | Description
|
+| -------------------------- |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
+| `skip_audit_user_list` | If you do not want operations of certain users
to be recorded in the audit logs, you can modify this configuration (supported
since version 3.0.01). For example, use the following command to exclude
`user1` and `user2` from audit log recording: |
- You can use the following statement to migrate data from the original
table to the new table:
-
- ```sql
- INSERT INTO __internal_schema.audit_log (
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- )
- SELECT
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- FROM doris_audit_db__.doris_audit_log_tbl__;
- ```
-
-3. Remove Original Plugin
-
- After migration, you can remove the original plugin by using the
`UNINSTALL PLUGIN AuditLoader;` command.
-
-## FAQ
-
-1. No data in the audit log table, or no new data is being ingested after
running for a period of time
-
- You can troubleshoot by following these steps:
-
- - Check if partitions are created correctly
-
- The audit log table is a dynamic partition table partitioned by day.
By default, it creates partitions for the next 3 days and retains historical
partitions for 30 days. Data can only be written to the audit log if partitions
are created correctly.
-
- You can check the scheduling status of dynamic partitions by using
`show dynamic partition tables from __internal_schema` and troubleshoot based
on error reasons. Possible error reasons may include:
-
- - Number of nodes is less than the required replication number: The
audit log table defaults to 3 replicas, so at least 3 BE nodes are required.
You can modify the replication number using the `alter table` statement, for
example:
-
- `alter table __internal_schema.audit_log set
("dynamic_partition.replication_num" = "2")`
-
- - No suitable storage medium: You can check the `storage_medium`
property by using `show create table __internal_schema.audit_log`. If there is
no corresponding storage medium on the BE, partition creation may fail.
-
- - No suitable resource group: The audit log table defaults to the
default resource group. You can check if the resource group has enough node
resources by using the `show backends` command.
-
- - Search for `AuditLoad` in the `fe.log` on the Master FE to see if there
are any related error logs
-
- The audit log is imported into the table through internal stream load
operations. If there are issues with the import process, error logs will be
printed in the `fe.log`.
diff --git a/versioned_docs/version-3.0/admin-manual/audit-plugin.md
b/versioned_docs/version-3.0/admin-manual/audit-plugin.md
index 9f60c93ef2..80798071ed 100644
--- a/versioned_docs/version-3.0/admin-manual/audit-plugin.md
+++ b/versioned_docs/version-3.0/admin-manual/audit-plugin.md
@@ -1,6 +1,6 @@
---
{
- "title": "Audit Log Plugin",
+ "title": "Audit Log",
"language": "en"
}
---
@@ -24,156 +24,56 @@ specific language governing permissions and limitations
under the License.
-->
-This plugin can regularly import the audit logs of FE into a specified system
table, making it convenient for users to view and analyze audit logs through
SQL.
+Doris provides auditing capabilities for database operations, allowing the
recording of user logins, queries, and modification operations on the database.
In Doris, audit logs can be queried directly through built-in system tables or
by viewing Doris's audit log files.
-## Using the Audit Log Plugin
+## Enabling Audit Logs
-Starting from Doris 2.1, the audit log plugin is integrated directly into the
Doris kernel as a built-in plugin. Users do not need to install the plugin
separately.
+The audit log plugin can be enabled or disabled at any time using the global
variable `enable_audit_plugin` (disabled by default), for example:
-After the cluster starts, a system table named `audit_log` will be created
under the `__internal_schema` database to store audit logs.
+`set global enable_audit_plugin = true;`
-:::note
-For versions prior to Doris 2.1, please refer to the 2.0 version documentation.
-:::
+Once enabled, Doris will write the audit logs to the `audit_log` table.
+
+You can disable the audit log plugin at any time:
+
+`set global enable_audit_plugin = false;`
+
+After disabling, Doris will stop writing to the `audit_log` table. The already
written audit logs will remain unchanged.
+
+## Viewing the Audit Log Table
+
+:::caution Note
+
+Before version 2.1.8, as the system version was upgraded, the audit log table
fields may have increased. After upgrading, you need to add fields to the
`audit_log` table using the `ALTER TABLE` command based on the fields in the
audit log table.
-:::warning
-After upgrading to version 2.1, the original audit log plugin will be
unavailable. Refer to the **Audit Log Migration** section to see how to migrate
audit log table data.
:::
-### Enabling the Plugin
+Starting from Doris version 2.1, Doris can write user behavior operations to
the [`audit_log`](../admin-manual/system-tables/internal_schema/audit_log)
table in the `__internal_schema` database by enabling the audit log feature.
-The audit log plugin can be enabled or disabled at any time using the global
variable `enable_audit_plugin` (default is disabled), for example:
+The audit log table is a dynamically partitioned table, partitioned daily by
default, retaining the most recent 30 days of data. You can adjust the
retention period of dynamic partitions by modifying the
`dynamic_partition.start` property using the `ALTER TABLE` statement.
-`set global enable_audit_plugin = true;`
+## Audit Log Files
-Once enabled, Doris will write the enabled audit logs to the `audit_log` table.
+In `fe.conf`, `LOG_DIR` defines the storage path for FE logs. All database
operations executed by this FE node are recorded in `${LOG_DIR}/fe.audit.log`.
To view all operations in the cluster, you need to traverse the audit logs of
each FE node.
-The audit log plugin can be disabled at any time:
+## Audit Log Configuration
-`set global enable_audit_plugin = false;`
+**Global Variables:**
+
+Audit log variables can be modified using `set [global] <var_name> =
<var_value>`.
+
+| Variable | Default Value | Description
|
+| -------------------------------------- | ------------- |
----------------------------------------------- |
+| `audit_plugin_max_batch_interval_sec` | 60 seconds | Maximum write
interval for the audit log table. |
+| `audit_plugin_max_batch_bytes` | 50MB | Maximum data volume
per batch for the audit log table. |
+| `audit_plugin_max_sql_length` | 4096 | Maximum length of
SQL statements recorded in the audit log table. |
+| `audit_plugin_load_timeout` | 600 seconds | Default timeout for
audit log import jobs. |
+
+**FE Configuration Items:**
+
+FE configuration items can be modified by editing the `fe.conf` directory.
-After disabling, Doris will stop writing to the `audit_log` table. The
existing audit logs will not be affected.
-
-### Audit log table
-
-With the upgrade of Doris version, the fields of the audit log table will also
increase. For details, please refer to
[audit_log](./system-tables/internal_schema/audit_log.md)
-
-Starting from version 2.1.8 and 3.0.3, the `audit_log` system table will
automatically add new fields of `audit_log` table as the Doris version is
upgraded.
-
-In previous versions, users need to manually add fields to the `audit_log`
system table through the `ALTER TABLE` command.
-
-### Related configurations
-
-The audit log table is a dynamic partitioned table, partitioned by day, and
retains data for the last 30 days by default.
+| Configuration Item | Description
|
+| -------------------------- |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
+| `skip_audit_user_list` | If you do not want operations of certain users
to be recorded in the audit logs, you can modify this configuration (supported
since version 3.0.01). For example, use the following command to exclude
`user1` and `user2` from audit log recording: |
-The following global variables can control some writing behaviors of the audit
log table:
-
-- `audit_plugin_max_batch_interval_sec`: The maximum write interval for the
audit log table. Default is 60 seconds.
-- `audit_plugin_max_batch_bytes`: The maximum amount of data written per batch
to the audit log table. Default is 50MB.
-- `audit_plugin_max_sql_length`: The maximum length of statements recorded in
the audit log table. Default is 4096.
-- `audit_plugin_load_timeout`: Default timeout for the audit log import job.
Default is 600 seconds.
-
-These can be set using `set global xxx=yyy`.
-
-FE Configuration:
-
-- `skip_audit_user_list` (supported since 3.0.1)
-
- If you do not want the operations of certain users to be recorded in the
audit log, you can modify this configuration.
-
- ```
- skip_audit_user_list=root
- -- or
- skip_audit_user_list=user1,user2
- ```
-
-## Audit Log Migration
-
-After upgrading to version 2.1, the original audit log plugin will be
unavailable. This section explains how to migrate data from the original audit
log table to the new audit log table.
-
-1. Confirm the field information of the old and new audit log tables.
-
- The default audit log table should be
`doris_audit_db__`.`doris_audit_log_tbl__`.
-
- The new audit log table is `__internal_schema`.`audit_log`.
-
- You can check if the field information of the two tables matches by using
the `DESC table_name` command. Typically, the fields of the old table should be
a subset of the new table.
-
-2. Migrate Audit Log Table Data
-
- You can use the following statement to migrate data from the original
table to the new table:
-
- ```sql
- INSERT INTO __internal_schema.audit_log (
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- )
- SELECT
- query_id ,
- time ,
- client_ip ,
- user ,
- db ,
- state ,
- error_code ,
- error_message ,
- query_time ,
- scan_bytes ,
- scan_rows ,
- return_rows ,
- stmt_id ,
- is_query ,
- frontend_ip ,
- cpu_time_ms ,
- sql_hash ,
- sql_digest ,
- peak_memory_bytes,
- stmt
- FROM doris_audit_db__.doris_audit_log_tbl__;
- ```
-
-3. Remove Original Plugin
-
- After migration, you can remove the original plugin by using the
`UNINSTALL PLUGIN AuditLoader;` command.
-
-## FAQ
-
-1. No data in the audit log table, or no new data is being ingested after
running for a period of time
-
- You can troubleshoot by following these steps:
-
- - Check if partitions are created correctly
-
- The audit log table is a dynamic partition table partitioned by day.
By default, it creates partitions for the next 3 days and retains historical
partitions for 30 days. Data can only be written to the audit log if partitions
are created correctly.
-
- You can check the scheduling status of dynamic partitions by using
`show dynamic partition tables from __internal_schema` and troubleshoot based
on error reasons. Possible error reasons may include:
-
- - Number of nodes is less than the required replication number: The
audit log table defaults to 3 replicas, so at least 3 BE nodes are required.
You can modify the replication number using the `alter table` statement, for
example:
-
- `alter table __internal_schema.audit_log set
("dynamic_partition.replication_num" = "2")`
-
- - No suitable storage medium: You can check the `storage_medium`
property by using `show create table __internal_schema.audit_log`. If there is
no corresponding storage medium on the BE, partition creation may fail.
-
- - No suitable resource group: The audit log table defaults to the
default resource group. You can check if the resource group has enough node
resources by using the `show backends` command.
-
- - Search for `AuditLoad` in the `fe.log` on the Master FE to see if there
are any related error logs
-
- The audit log is imported into the table through internal stream load
operations. If there are issues with the import process, error logs will be
printed in the `fe.log`.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
