This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/paimon.git
The following commit(s) were added to refs/heads/master by this push:
new 7c7bcd9b75 [doc] Add manage partition introduction page (#5357)
7c7bcd9b75 is described below
commit 7c7bcd9b75c7ae86f845c8eb063ace5947fa1bd7
Author: xiangyu0xf <[email protected]>
AuthorDate: Fri Mar 28 15:22:07 2025 +0800
[doc] Add manage partition introduction page (#5357)
---
docs/content/flink/sql-ddl.md | 2 +-
.../manage-partitions.md} | 36 ++++++++++++++++++++--
2 files changed, 34 insertions(+), 4 deletions(-)
diff --git a/docs/content/flink/sql-ddl.md b/docs/content/flink/sql-ddl.md
index a373348861..4626704270 100644
--- a/docs/content/flink/sql-ddl.md
+++ b/docs/content/flink/sql-ddl.md
@@ -187,7 +187,7 @@ If you need cross partition upsert (primary keys not
contain all partition field
{{< /hint >}}
{{< hint info >}}
-By configuring [partition.expiration-time]({{< ref "flink/expire-partition"
>}}), expired partitions can be automatically deleted.
+By configuring [partition.expiration-time]({{< ref
"maintenance/manage-partitions" >}}), expired partitions can be automatically
deleted.
{{< /hint >}}
### Specify Statistics Mode
diff --git a/docs/content/flink/expire-partition.md
b/docs/content/maintenance/manage-partitions.md
similarity index 79%
rename from docs/content/flink/expire-partition.md
rename to docs/content/maintenance/manage-partitions.md
index 226017513f..7b851fe055 100644
--- a/docs/content/flink/expire-partition.md
+++ b/docs/content/maintenance/manage-partitions.md
@@ -1,9 +1,9 @@
---
-title: "Expire Partition"
-weight: 96
+title: "Manage Partitions"
+weight: 12
type: docs
aliases:
-- /flink/expire-partition.html
+- /maintenance/manage-partitions.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
@@ -24,6 +24,10 @@ specific language governing permissions and limitations
under the License.
-->
+# Manage Partitions
+Paimon provides multiple ways to manage partitions, including expire
historical partitions by different strategies or
+mark a partition done to notify the downstream application that the partition
has finished writing.
+
## Expiring Partitions
You can set `partition.expiration-time` when creating a partitioned table.
Paimon streaming sink will periodically check
@@ -138,3 +142,29 @@ More options:
</tr>
</tbody>
</table>
+
+## Partition Mark Done
+
+You can use the option `'partition.mark-done-action'` to configure the action
when a partition needs to be mark done.
+- `success-file`: add '_success' file to directory.
+- `done-partition`: add 'xxx.done' partition to metastore.
+- `mark-event`: mark partition event to metastore.
+- `http-report`: report partition mark done to remote http server.
+- `custom`: use policy class to create a mark-partition policy.
+These actions can be configured at the same time:
'done-partition,success-file,mark-event,custom'.
+
+Paimon partition mark done can be triggered both by streaming write and batch
write.
+
+### Streaming Mark Done
+
+You can use the options `'partition.idle-time-to-done'` to set a partition
idle time to done duration. When a partition
+has no new data after this time duration, the mark done action will be
triggered to indicate that the data is ready.
+
+By default, Flink will use process time as idle time to trigger partition mark
done. You can also use watermark to
+trigger partition mark done. This will make the partition mark done time more
accurate when data is delayed. You can
+enable this by setting `'partition.mark-done-action.mode' = 'watermark'`.
+
+### Batch Mark Done
+
+For batch mode, you can trigger partition mark done when end input by setting
`'partition.end-input-to-done'='true'`.
+
