samredai commented on a change in pull request #3482:
URL: https://github.com/apache/iceberg/pull/3482#discussion_r746089346
##########
File path: site/docs/spark-queries.md
##########
@@ -243,6 +269,101 @@ Note:
2. `contains_nan` could return null, which indicates that this information is
not available from files' metadata.
This usually occurs when reading from V1 table, where `contains_nan` is not
populated.
+### All Manifests
+
+To show a table's valid file manifests and each file's metadata, run:
+
+!!! Note
+ a valid manifest file is one that is referenced from any snapshot
currently tracked by the table.
+
+!!! WARNING
+ this table may return **duplicate** rows
+
+```sql
+SELECT * FROM prod.db.table.all_manifests
+```
+
+```text
++------------------------------------------+------+-----------------+-------------------+----------------------+-------------------------+------------------------+------------------------------------------------------------+
+| path|length|partition_spec_id|
added_snapshot_id|added_data_files_count|existing_data_files_count|deleted_data_files_count|
partition_summaries|
++------------------------------------------+------+-----------------+-------------------+----------------------+-------------------------+------------------------+------------------------------------------------------------+
+|s3:/.../table/data/dt=20210102/xxx.parquet| 6376|
0|6272782676904868561| 2| 0|
0|[{false, false, 10, 11}, {false, false, 20210101, 20210101}]|
++------------------------------------------+------+-----------------+-------------------+----------------------+-------------------------+------------------------+------------------------------------------------------------+
+```
+
+!!! Note
+1. Fields within `partition_summaries` column of the manifests table
correspond to `field_summary` structs within [manifest
list](./spec.md#manifest-lists), with the following order:
+ - `contains_null`
+ - `contains_nan`
+ - `lower_bound`
+ - `upper_bound`
+2. `contains_nan` could return null, which indicates that this information is
not available from files' metadata.
+ This usually occurs when reading from V1 table, where `contains_nan` is not
populated.
+
+### Partitions
+
+To show a table's current partitions
+
+```sql
+SELECT * FROM prod.db.table.partitions
+```
+
+```text
++----------------+------------+----------+
+| partition|record_count|file_count|
++----------------+------------+----------+
+| {20211001, 11}| 1| 1|
+| {20211002, 11}| 1| 1|
+| {20211001, 10}| 1| 1|
+| {20211002, 10}| 1| 1|
++----------------+------------+----------+
+```
+
+### Entries
Review comment:
+1 I totally agree. I can't see a reason for a typical user to use this
table and having it in this section of the docs could be a signal that they
should be using it. There's also an instinct of data engineers coming from Hive
tables to want to find the location of the data files and start messing around
there so it's beneficial if we strengthen that abstraction by example in the
docs (maybe even explicitly call it out)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
