rdblue commented on a change in pull request #3482:
URL: https://github.com/apache/iceberg/pull/3482#discussion_r744297010
##########
File path: site/docs/spark-queries.md
##########
@@ -243,6 +269,101 @@ Note:
2. `contains_nan` could return null, which indicates that this information is
not available from files' metadata.
This usually occurs when reading from V1 table, where `contains_nan` is not
populated.
+### All Manifests
+
+To show a table's valid file manifests and each file's metadata, run:
+
+!!! Note
+ a valid manifest file is one that is referenced from any snapshot
currently tracked by the table.
+
+!!! WARNING
+ this table may return **duplicate** rows
+
+```sql
+SELECT * FROM prod.db.table.all_manifests
+```
+
+```text
++------------------------------------------+------+-----------------+-------------------+----------------------+-------------------------+------------------------+------------------------------------------------------------+
+| path|length|partition_spec_id|
added_snapshot_id|added_data_files_count|existing_data_files_count|deleted_data_files_count|
partition_summaries|
++------------------------------------------+------+-----------------+-------------------+----------------------+-------------------------+------------------------+------------------------------------------------------------+
+|s3:/.../table/data/dt=20210102/xxx.parquet| 6376|
0|6272782676904868561| 2| 0|
0|[{false, false, 10, 11}, {false, false, 20210101, 20210101}]|
++------------------------------------------+------+-----------------+-------------------+----------------------+-------------------------+------------------------+------------------------------------------------------------+
+```
+
+!!! Note
+1. Fields within `partition_summaries` column of the manifests table
correspond to `field_summary` structs within [manifest
list](./spec.md#manifest-lists), with the following order:
+ - `contains_null`
+ - `contains_nan`
+ - `lower_bound`
+ - `upper_bound`
+2. `contains_nan` could return null, which indicates that this information is
not available from files' metadata.
+ This usually occurs when reading from V1 table, where `contains_nan` is not
populated.
+
+### Partitions
+
+To show a table's current partitions
+
+```sql
+SELECT * FROM prod.db.table.partitions
+```
+
+```text
++----------------+------------+----------+
+| partition|record_count|file_count|
++----------------+------------+----------+
+| {20211001, 11}| 1| 1|
+| {20211002, 11}| 1| 1|
+| {20211001, 10}| 1| 1|
+| {20211002, 10}| 1| 1|
++----------------+------------+----------+
+```
+
+### Entries
Review comment:
While we support this, I'm not sure that we want this in the standard
documentation for Spark users. The entries table will show all entries in
manifests, for both deleted and live files in the table. I think interpreting
this table is more of an advanced use.
What do you think @samredai?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
