szehon-ho commented on code in PR #8251:
URL: https://github.com/apache/iceberg/pull/8251#discussion_r1291624886
##########
docs/spark-procedures.md:
##########
@@ -379,11 +403,23 @@ Iceberg can rewrite position delete files, which serves
two purposes:
| `table` | ✔️ | string | Name of the table to update |
| `options` | ️ | map<string, string> | Options to be used for procedure
|
-See the [`SizeBasedFileRewriter` Javadoc](../../../javadoc/{{% icebergVersion
%}}/org/apache/iceberg/actions/SizeBasedFileRewriter.html#field.summary),
-for list of all the supported options for this procedure.
-
Dangling deletes are always filtered out during rewriting.
+#### Options
+
+| Name | Default Value | Description |
+|------|---------------|-------------|
+| `max-concurrent-file-group-rewrites` | 5 | Maximum number of file groups to
be simultaneously rewritten |
+| `partial-progress.enabled` | false | Enable committing groups of files prior
to the entire rewrite completing |
+| `partial-progress.max-commits` | 10 | Maximum amount of commits that this
rewrite is allowed to produce if partial progress is enabled |
+| `rewrite-job-order` | none | Force the rewrite job order based on the value.
<ul><li>If rewrite-job-order=bytes-asc, then rewrite the smallest job groups
first.</li><li>If rewrite-job-order=bytes-desc, then rewrite the largest job
groups first.</li><li>If rewrite-job-order=files-asc, then rewrite the job
groups with the least files first.</li><li>If rewrite-job-order=files-desc,
then rewrite the job groups with the most files first.</li><li>If
rewrite-job-order=none, then rewrite job groups in the order they were planned
(no specific ordering).</li></ul> |
+| `target-file-size-bytes` | 67108864 (default value of
`write.delete.target-file-size-bytes` from [table
properties](../configuration/#write-properties)) | Target output file size |
Review Comment:
Sorry just noticed a small discrepancy, we mention the number here but not
in the same config of rewrite_data_files where we just mention link, can we
make make it consistent ? (no strong preference either way)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
