This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 9d690ee37a [INLONG-636][Doc] Postgresql cdc suggest setting replica
identity to FULL (#637)
9d690ee37a is described below
commit 9d690ee37a23f9a71907092af3b07b7f908ef211
Author: Liao Rui <[email protected]>
AuthorDate: Mon Dec 12 14:23:57 2022 +0800
[INLONG-636][Doc] Postgresql cdc suggest setting replica identity to FULL
(#637)
Co-authored-by: ryanrliao <[email protected]>
---
docs/data_node/extract_node/postgresql-cdc.md | 31 +++++++++++++++++++--
.../data_node/extract_node/postgresql-cdc.md | 32 ++++++++++++++++++++--
2 files changed, 57 insertions(+), 6 deletions(-)
diff --git a/docs/data_node/extract_node/postgresql-cdc.md
b/docs/data_node/extract_node/postgresql-cdc.md
index 9252e03906..e41285f9c7 100644
--- a/docs/data_node/extract_node/postgresql-cdc.md
+++ b/docs/data_node/extract_node/postgresql-cdc.md
@@ -30,6 +30,7 @@ build automation tool (such as Maven or SBT) and SQL Client
with Sort Connectors
</dependency>
`}
</code></pre>
+
## Setup PostgreSQL server
Change Data Capture (CDC) allows you to track and propagate changes in a
PostgreSQL database to downstream consumers based on its Write-Ahead Log (WAL).
@@ -69,6 +70,27 @@ shared_preload_libraries = 'decoderbufs'
wal_level = logical
```
+### replica identity
+
+`REPLICA IDENTITY` is a PostgreSQL-specific table-level setting that
determines the amount of information that is available to the logical decoding
plug-in for UPDATE and DELETE events. See
[more](https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-replica-identity).
+
+Please keep the `replica indentity` level of source tables to `FULL`. You can
check and change this value by SQL:
+
+```sql
+-- show replica identity
+SELECT CASE relreplident
+ WHEN 'd' THEN 'default'
+ WHEN 'n' THEN 'nothing'
+ WHEN 'f' THEN 'full'
+ WHEN 'i' THEN 'index'
+ END AS replica_identity
+FROM pg_class
+WHERE oid = 'mytablename'::regclass;
+
+-- change replica identity
+ALTER TABLE mytablename REPLICA IDENTITY FULL;
+```
+
## How to create a PostgreSQL Extract Node
### Usage for SQL API
@@ -116,10 +138,13 @@ TODO: It will be supported in the future.
| debezium.* | optional | (none) | String | Pass-through Debezium's properties
to Debezium Embedded Engine which is used to capture data changes from Postgres
server. For example: 'debezium.snapshot.mode' = 'never'. See more about the
[Debezium's Postgres Connector
properties](https://debezium.io/documentation/reference/1.5/connectors/postgresql.html#postgresql-connector-properties).
|
| inlong.metric.labels | optional | (none) | String | Inlong metric label,
format of value is groupId=xxgroup&streamId=xxstream&nodeId=xxnode. |
-**Note**: `slot.name` is recommended to set for different tables to avoid the
potential PSQLException: ERROR: replication slot "flink" is active for PID 974
error.
-**Note**: PSQLException: ERROR: all replication slots are in use Hint: Free
one or increase max_replication_slots. We can delete slot by the following
statement.
+:::caution
+- `slot.name` is recommended to set for different tables to avoid the
potential PSQLException: ERROR: replication slot "flink" is active for PID 974
error.
+- PSQLException: ERROR: all replication slots are in use Hint: Free one or
increase max_replication_slots. We can delete slot by the following statement.
+:::
```sql
-SELECT*FROM pg_replication_slots;
+SELECT * FROM pg_replication_slots;
+
-- get slot name is flink. delete it
SELECT pg_drop_replication_slot('flink');
```
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/postgresql-cdc.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/postgresql-cdc.md
index 7ad1f01bcb..ff15c847fd 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/postgresql-cdc.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/postgresql-cdc.md
@@ -29,6 +29,7 @@ import {siteVariables} from '../../version';
</dependency>
`}
</code></pre>
+
## 设置 PostgreSQL 服务
更改数据捕获 (CDC) 允许您跟踪 PostgreSQL 数据库中的更改并将其传播到基于其预写日志 (WAL) 的下游消费者。
@@ -69,6 +70,27 @@ shared_preload_libraries = 'decoderbufs'
wal_level = logical
```
+### replica identity
+
+`REPLICA
IDENTITY`是PostgreSQL表级别的一个设置参数，它决定了逻辑解码插件在捕获更新和删除事件时获取足够的信息。可以从[这里](https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-replica-identity)查看更多信息。
+
+请确保需要同步的表的`replica identity`的级别为`FULL`。您可以通过以下SQL代码查看并修改相应参数。
+
+```sql
+-- 查看 replica identity
+SELECT CASE relreplident
+ WHEN 'd' THEN 'default'
+ WHEN 'n' THEN 'nothing'
+ WHEN 'f' THEN 'full'
+ WHEN 'i' THEN 'index'
+ END AS replica_identity
+FROM pg_class
+WHERE oid = 'mytablename'::regclass;
+
+-- 修改 replica identity
+ALTER TABLE mytablename REPLICA IDENTITY FULL;
+```
+
## 如何创建 PostgreSQL Extract 节点
### SQL API 用法
@@ -116,10 +138,14 @@ TODO: 将在未来支持此功能。
| debezium.* | 可选 | (none) | String | 将 Debezium 的属性传递给用于从 Postgres 服务器捕获数据更改的
Debezium Embedded Engine。 例如：“debezium.snapshot.mode”=“never”。 查看更多关于 [Debezium
的 Postgres
连接器属性](https://debezium.io/documentation/reference/1.5/connectors/postgresql.html#postgresql-connector-properties)。
|
| inlong.metric | 可选 | (none) | String | inlong metric
的标签值，该值的构成为groupId&streamId&nodeId。|
-**Note**: `slot.name` 建议为不同的表设置以避免潜在的 PSQLException: ERROR: replication slot
"flink" is active for PID 974 error。
-**Note**: PSQLException: ERROR: all replication slots are in use Hint: Free
one or increase max_replication_slots. 我们可以通过以下语句删除槽。
+:::caution
+- `slot.name` 建议为不同的表设置以避免潜在的 PSQLException: ERROR: replication slot "flink"
is active for PID 974 error。
+- PSQLException: ERROR: all replication slots are in use Hint: Free one or
increase max_replication_slots. 我们可以通过以下语句删除槽。
+:::
+
```sql
-SELECT*FROM pg_replication_slots;
+SELECT * FROM pg_replication_slots;
+
-- 获取插槽名称为 flink。 删除它
SELECT pg_drop_replication_slot('flink');
```
