szehon-ho commented on code in PR #13532:
URL: https://github.com/apache/iceberg/pull/13532#discussion_r2205557077
##########
docs/docs/spark-procedures.md:
##########
@@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of
table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id
=> 'snap1', columns => array('col1', 'col2'));
```
+## Partition Statistics
+
+### `compute_partition_stats`
+
+This procedure computes the stats incrementally from the last snapshot that
has `PartitionStatisticsFile` until the given
Review Comment:
nit: a 'PartitionStatisticsFile' (need article for complete sentence)
##########
docs/docs/spark-procedures.md:
##########
@@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of
table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id
=> 'snap1', columns => array('col1', 'col2'));
```
+## Partition Statistics
+
+### `compute_partition_stats`
+
+This procedure computes the stats incrementally from the last snapshot that
has `PartitionStatisticsFile` until the given
+snapshot (uses current snapshot if not specified) and writes the combined
result into a `PartitionStatisticsFile`
+after merging the partition stats. It performs a full compute if previous
statistics file does not exist. It also registers the
+`PartitionStatisticsFile` to table metadata.
Review Comment:
nit: `the table metadata` (need article)
##########
docs/docs/spark-procedures.md:
##########
@@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of
table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id
=> 'snap1', columns => array('col1', 'col2'));
```
+## Partition Statistics
+
+### `compute_partition_stats`
+
+This procedure computes the stats incrementally from the last snapshot that
has `PartitionStatisticsFile` until the given
+snapshot (uses current snapshot if not specified) and writes the combined
result into a `PartitionStatisticsFile`
+after merging the partition stats. It performs a full compute if previous
statistics file does not exist. It also registers the
Review Comment:
lets get rid of 'merging with the previous statistics file'? I feel its
implied from 'incrementally' and 'combined'
##########
docs/docs/spark-procedures.md:
##########
@@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of
table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id
=> 'snap1', columns => array('col1', 'col2'));
```
+## Partition Statistics
+
+### `compute_partition_stats`
+
+This procedure computes the stats incrementally from the last snapshot that
has `PartitionStatisticsFile` until the given
+snapshot (uses current snapshot if not specified) and writes the combined
result into a `PartitionStatisticsFile`
+after merging the partition stats. It performs a full compute if previous
statistics file does not exist. It also registers the
Review Comment:
nit (needs article and specify stats file): 'the previous partition
statistics file'
##########
docs/docs/spark-procedures.md:
##########
@@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of
table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id
=> 'snap1', columns => array('col1', 'col2'));
```
+## Partition Statistics
+
+### `compute_partition_stats`
+
+This procedure computes the stats incrementally from the last snapshot that
has `PartitionStatisticsFile` until the given
+snapshot (uses current snapshot if not specified) and writes the combined
result into a `PartitionStatisticsFile`
+after merging the partition stats. It performs a full compute if previous
statistics file does not exist. It also registers the
+`PartitionStatisticsFile` to table metadata.
+
+| Argument Name | Required? | Type | Description
|
+|---------------|-----------|---------------|---------------------------------------------------------------------------------------|
+| `table` | ✔️ | string | Name of the table
|
+| `snapshot_id` | | string | Id of the snapshot to collect
partition stats. By default current snapshot id is used |
+
+#### Output
+
+| Output Name | Type | Description
|
+|-------------------|--------|-----------------------------------------------------------|
+| `partition_statistics_file` | string | Path to partition stats file created
from by this command |
Review Comment:
'to the partition stats file'. (add article)
Also , 'from by', only need one. Use `by`?
##########
docs/docs/spark-procedures.md:
##########
@@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of
table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id
=> 'snap1', columns => array('col1', 'col2'));
```
+## Partition Statistics
+
+### `compute_partition_stats`
+
+This procedure computes the stats incrementally from the last snapshot that
has `PartitionStatisticsFile` until the given
+snapshot (uses current snapshot if not specified) and writes the combined
result into a `PartitionStatisticsFile`
+after merging the partition stats. It performs a full compute if previous
statistics file does not exist. It also registers the
Review Comment:
should specify type of stats file, and also add article:
`if the previous partition statistics file`
##########
docs/docs/spark-procedures.md:
##########
@@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of
table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id
=> 'snap1', columns => array('col1', 'col2'));
```
+## Partition Statistics
+
+### `compute_partition_stats`
+
+This procedure computes the stats incrementally from the last snapshot that
has `PartitionStatisticsFile` until the given
+snapshot (uses current snapshot if not specified) and writes the combined
result into a `PartitionStatisticsFile`
+after merging the partition stats. It performs a full compute if previous
statistics file does not exist. It also registers the
+`PartitionStatisticsFile` to table metadata.
+
+| Argument Name | Required? | Type | Description
|
+|---------------|-----------|---------------|---------------------------------------------------------------------------------------|
+| `table` | ✔️ | string | Name of the table
|
+| `snapshot_id` | | string | Id of the snapshot to collect
partition stats. By default current snapshot id is used |
Review Comment:
collect => compute (follows the command name)
##########
docs/docs/spark-procedures.md:
##########
@@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of
table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id
=> 'snap1', columns => array('col1', 'col2'));
```
+## Partition Statistics
+
+### `compute_partition_stats`
+
+This procedure computes the stats incrementally from the last snapshot that
has `PartitionStatisticsFile` until the given
+snapshot (uses current snapshot if not specified) and writes the combined
result into a `PartitionStatisticsFile`
+after merging the partition stats. It performs a full compute if previous
statistics file does not exist. It also registers the
+`PartitionStatisticsFile` to table metadata.
+
+| Argument Name | Required? | Type | Description
|
+|---------------|-----------|---------------|---------------------------------------------------------------------------------------|
+| `table` | ✔️ | string | Name of the table
|
+| `snapshot_id` | | string | Id of the snapshot to collect
partition stats. By default current snapshot id is used |
Review Comment:
'is used' sounds unnecessary. Hm how about just: `defaults to current
snapshot id`?
##########
docs/docs/spark-procedures.md:
##########
@@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of
table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id
=> 'snap1', columns => array('col1', 'col2'));
```
+## Partition Statistics
+
+### `compute_partition_stats`
+
+This procedure computes the stats incrementally from the last snapshot that
has `PartitionStatisticsFile` until the given
Review Comment:
stats => partitions statistics.
By the way, can we link to the spec about partition stats?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
