massdosage commented on a change in pull request #2067:
URL: https://github.com/apache/iceberg/pull/2067#discussion_r558118063
##########
File path: site/docs/spark.md
##########
@@ -814,3 +815,320 @@ This type conversion table describes how Iceberg types
are converted to the Spar
| struct | struct | |
| list | array | |
| map | map | |
+
+## Procedures
+
+In Spark 3 Iceberg provides an SQL API for accomplishing the [maintenance
actions](maintenance.md). Support for
+stored procedures is tied to the DataSourceV2 catalog and requires that the
Iceberg Extensions are enabled for the
+Spark Session.
+
+### General Usage
+
+To call an Iceberg stored procedure, execute a `CALL` command against the
iceberg catalog. All procedures are added to
+the `system` keyspace. Procedures can take positional or named arguments.
+
+#### Generic Call with Positional Arguments
+```sql
+ CALL catalog_name.system.procedure_name(arg_1, arg_2, ... arg_n)
+```
+
+#### Generic Call with Named Arguments
+```sql
+ CALL catalog_name.system.procedure_name(arg_name_2 => arg_2, arg_name_1 =>
arg_1)
+```
+
+### Cherrypick Snapshot Procedure
+
+A procedure that applies changes in a given snapshot and creates a new
snapshot which will
+be set as the current snapshot in a table.
+
+**Note** this procedure invalidates all cached Spark plans that reference the
affected table
+
+#### Usage
+
+| Argument Name | Required? | Type | Description |
+|---------------|-----------|------|-------------|
+| table | ✔️ | String | Name of table to perform cherrypick on |
+| snapshot_id | ✔️ | Long | The snapshot ID to cherrypick |
+
+#### Output
+
+| Output Name | Type | Description |
+| ------------|------|-------------|
+| source_snapshot_id | Long | The snapshot before applying the cherrypick |
+| current_snapshot_id | Long | The current snapshot now that the cherrypick
has been applied|
+
+#### Examples
+
+Cherrypick Snapshot 1
+```sql
+ CALL catalog_name hive_prod.system.cherrypick_snapshot('my_table', 1)
+```
+
+Cherrypick Snapshot 1 with named args
+```sql
+ CALL catalog_name hive_prod.system.cherrypick_snapshot(snapshot_id => 1,
table => 'my_table' )
+```
+
+### Expire Snapshot Procedure
+
+Each write/update/delete/upsert/compaction in Iceberg produces a new snapshot
while keeping the old data and metadata
+around for snapshot isolation and time travel. The `expire_snapshots`
procedure can be used to remove older snapshots
+and their files which are no longer needed.
+
+This procedure will remove old snapshots and data files which are uniquely
required by those old snapshots. This means
+the ``expire_snapshots`` procedure will never remove files which are still
required by a non-expired snapshot.
+
+#### Usage
+
+| Argument Name | Required? | Type | Description |
+|---------------|-----------|------|-------------|
+| table | ✔️ | String | Name of table to expire snapshots from |
+| older_than | ️ | Timestamp | Remove snapshots older than this date
(Defaults to 5 days ago) |
+| retain_last | | Int | Length of history to preserve regardless of
older_than target (Defaults to 1)|
+
+#### Output
+
+| Output Name | Type | Description |
+| ------------|------|-------------|
+| deleted_data_files_count | Long | Data files deleted by this operation |
+| deleted_manifest_files_count | Long | Manifest files deleted by this
operation |
+| deleted_manifest_lists_count | Long | Manifest List files deleted by this
operation |
+
+#### Examples
+
+Remove snapshots older than 10 days ago, but retain the last 100 Snapshots.
+```sql
+ CALL hive_prod.system.expire_snapshots('db.sample',
date_sub(current_date(), 10), 100)
+```
+
+Erase all snapshots older than the current timestamp but retains the last 5
snapshots made.
+```sql
+ CALL hive_prod.system.expire_snapshots(table => 'db.sample', retain_last
=> 5)
+```
+
+### Migrate Table Procedure
+
+Converts a table known to Spark in this catalog into a Iceberg table. This is
manily for taking
+hive tables and fully converting them into Iceberg tables. The
[snapshot](#snapshot-table-procedure) procedure
+can be used for making Iceberg tables without effecting the underlying table.
The migrated table will preserve all
+properties set on the original table.
+
+#### Usage
+
+| Argument Name | Required? | Type | Description |
+|---------------|-----------|------|-------------|
+| table | ✔️ | String | Table to be migrated to Iceberg |
+| properties | ️ | Map<String, String> | Table properties to add to the
table |
+
+#### Output
+
+| Output Name | Type | Description |
+| ------------|------|-------------|
+| migrated_files_count | Long | Number of files migrated to Iceberg|
+
+#### Examples
+
+Migrate the table db.sample into an Iceberg table and add a property 'foo' set
to 'bar'
+```sql
+ CALL catalog_name hive_prod.system.migrate('db.sample', map('foo', 'bar'))
+```
+
+Migrate the table db.sample
+```sql
+ CALL catalog_name hive_prod.system.migrate('db.sample')
+```
+
+### Remove Orphan Files Procedure
+
+Remove files in a directory which a given Iceberg table does not own. Used for
removing files which are no longer
+required by an Iceberg table and Iceberg has no references to.
+
+#### Usage
+
+| Argument Name | Required? | Type | Description |
+|---------------|-----------|------|-------------|
+| table | ✔️ | String | Name of table to remove files from |
+| older_than | ️ | Timestamp | Remove orphan files older than this time
(Defaults to 3 days ago) |
Review comment:
OK, yes, makes sense to reword to keep "ago" (or some other indication
that it's older than this point in time).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
