This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-paimon.git
The following commit(s) were added to refs/heads/master by this push:
new e2517ffa1 [doc] Add doc for tag (#1449)
e2517ffa1 is described below
commit e2517ffa183b249722fc658c8206836807e47627
Author: yuzelin <[email protected]>
AuthorDate: Thu Jun 29 10:31:51 2023 +0800
[doc] Add doc for tag (#1449)
---
docs/content/how-to/querying-tables.md | 8 +-
docs/content/how-to/system-tables.md | 19 ++++
docs/content/maintenance/manage-tags.md | 168 ++++++++++++++++++++++++++++++++
3 files changed, 194 insertions(+), 1 deletion(-)
diff --git a/docs/content/how-to/querying-tables.md
b/docs/content/how-to/querying-tables.md
index 7be3add0a..26cab1dde 100644
--- a/docs/content/how-to/querying-tables.md
+++ b/docs/content/how-to/querying-tables.md
@@ -34,7 +34,7 @@ Paimon's batch read returns all the data in a snapshot of the
table. By default,
### Batch Time Travel
-Paimon batch reads with time travel can specify a snapshot and read the
corresponding data.
+Paimon batch reads with time travel can specify a snapshot or a tag and read
the corresponding data.
{{< tabs "time-travel-example" >}}
@@ -45,6 +45,9 @@ SELECT * FROM t /*+ OPTIONS('scan.snapshot-id' = '1') */;
-- read the snapshot from specified timestamp in unix milliseconds
SELECT * FROM t /*+ OPTIONS('scan.timestamp-millis' = '1678883047356') */;
+
+-- read tag 'my-tag'
+SELECT * FROM t /*+ OPTIONS('scan.tag-name' = 'my-tag') */;
```
{{< /tab >}}
@@ -63,6 +66,9 @@ SELECT * FROM t TIMESTAMP AS OF '2023-06-01 00:00:00.123';
-- read the snapshot from specified timestamp in unix seconds
SELECT * FROM t TIMESTAMP AS OF 1678883047;
+
+-- read tag 'my-tag'
+SELECT * FROM t VERSION AS OF 'my-tag';
```
{{< /tab >}}
diff --git a/docs/content/how-to/system-tables.md
b/docs/content/how-to/system-tables.md
index c5d9e565b..da06a995d 100644
--- a/docs/content/how-to/system-tables.md
+++ b/docs/content/how-to/system-tables.md
@@ -157,3 +157,22 @@ SELECT * FROM MyTable$files /*+
OPTIONS('scan.snapshot-id'='1') */;
+-----------+--------+--------------------------------+-------------+-----------+-------+--------------+--------------------+---------+---------+------------------------+-------------------------+-------------------------+-----------------------+
3 rows in set
```
+
+## Tags Table
+
+You can query the tag history information of the table through tags table,
including which snapshots are the tags based on
+and some historical information of the snapshots. You can also get all tag
names and time travel to a specific tag data by name.
+
+```sql
+SELECT * FROM MyTable$snapshots;
+
+/*
++----------+-------------+-----------+-------------------------+--------------+
+| tag_name | snapshot_id | schema_id | commit_time | record_count |
++----------+-------------+-----------+-------------------------+--------------+
+| tag1 | 1 | 0 | 2023-06-28 14:55:29.344 | 3 |
+| tag3 | 3 | 0 | 2023-06-28 14:58:24.691 | 7 |
++----------+-------------+-----------+-------------------------+--------------+
+2 rows in set
+*/
+```
diff --git a/docs/content/maintenance/manage-tags.md
b/docs/content/maintenance/manage-tags.md
new file mode 100644
index 000000000..25e5abb70
--- /dev/null
+++ b/docs/content/maintenance/manage-tags.md
@@ -0,0 +1,168 @@
+---
+title: "Manage Tags"
+weight: 7
+type: docs
+aliases:
+- /maintenance/manage-tags.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Manage Tags
+
+Paimon's snapshots can provide a easy way to query historical data. But in
most scenarios, a job will generate too many
+snapshots and table will expire old snapshots according to table
configuration. Snapshot expiration will also delete old
+data files, and the historical data of expired snapshots cannot be queried
anymore.
+
+To solve this problem, you can create a tag based on a snapshot. The tag will
maintain the manifests and data files of the
+snapshot. A typical usage is creating tags daily, then you can maintain the
historical data of each day for batch reading.
+
+## Create Tags
+
+You can create a tag with given name (cannot be number) and snapshot ID.
+
+{{< tabs "create-tag" >}}
+
+{{< tab "Flink" >}}
+
+ Run the following command:
+
+```bash
+<FLINK_HOME>/bin/flink run \
+ /path/to/paimon-flink-action-{{< version >}}.jar \
+ create-tag \
+ --warehouse <warehouse-path> \
+ --database <database-name> \
+ --table <table-name> \
+ --tag-name <tag-name> \
+ --snapshot <snapshot-id> \
+ [--catalog-conf <paimon-catalog-conf> [--catalog-conf
<paimon-catalog-conf> ...]]
+```
+
+{{< /tab >}}
+
+{{< tab "Java API" >}}
+
+```java
+import org.apache.paimon.table.Table;
+
+public class CreateTag {
+
+ public static void main(String[] args) {
+ Table table = ...;
+ table.createTag("my-tag", 1);
+ }
+}
+```
+
+{{< /tab >}}
+
+{{< /tabs >}}
+
+## Delete Tags
+
+You can delete a tag by its name.
+
+{{< tabs "delete-tag" >}}
+
+{{< tab "Flink" >}}
+
+Run the following command:
+
+```bash
+<FLINK_HOME>/bin/flink run \
+ /path/to/paimon-flink-action-{{< version >}}.jar \
+ delete-tag \
+ --warehouse <warehouse-path> \
+ --database <database-name> \
+ --table <table-name> \
+ --tag-name <tag-name> \
+ [--catalog-conf <paimon-catalog-conf> [--catalog-conf
<paimon-catalog-conf> ...]]
+```
+
+{{< /tab >}}
+
+{{< tab "Java API" >}}
+
+```java
+import org.apache.paimon.table.Table;
+
+public class DeleteTag {
+
+ public static void main(String[] args) {
+ Table table = ...;
+ table.deleteTag("my-tag");
+ }
+}
+```
+
+{{< /tab >}}
+
+{{< /tabs >}}
+
+## Rollback to Tag
+
+Rollback table to a specific tag. All snapshots and tags whose snapshot id is
larger than the tag will be deleted (and
+the data will be deleted too).
+
+{{< tabs "rollback-to" >}}
+
+{{< tab "Flink" >}}
+
+Run the following command:
+
+```bash
+<FLINK_HOME>/bin/flink run \
+ /path/to/paimon-flink-action-{{< version >}}.jar \
+ rollback-to \
+ --warehouse <warehouse-path> \
+ --database <database-name> \
+ --table <table-name> \
+ --vesion <tag-name> \
+ [--catalog-conf <paimon-catalog-conf> [--catalog-conf
<paimon-catalog-conf> ...]]
+```
+
+{{< /tab >}}
+
+{{< tab "Java API" >}}
+
+```java
+import org.apache.paimon.table.Table;
+
+public class RollbackTo {
+
+ public static void main(String[] args) {
+ // before rollback:
+ // snapshot-3 [expired] -> tag3
+ // snapshot-4 [expired]
+ // snapshot-5 -> tag5
+ // snapshot-6
+ // snapshot-7
+
+ table.rollbackTo("tag3");
+
+ // after rollback:
+ // snapshot-3 -> tag3
+ }
+}
+```
+
+{{< /tab >}}
+
+{{< /tabs >}}
\ No newline at end of file
