andygrove opened a new pull request, #6828: URL: https://github.com/apache/datafusion-comet/pull/6828
## Which issue does this PR close? No issue. This follows up on #6808. ## Rationale for this change #6808 makes the native Iceberg write gate ignore five more S3A settings: `fs.s3a.committer.name`, `fs.s3a.committer.magic.enabled`, `fs.s3a.committer.threads`, `fs.s3a.bulk.delete.page.size` and `fs.s3a.experimental.input.fadvise`. The user guide and the contributor guide both name the ignored settings, and both list only the vectored-read and `downgrade.syncable.exceptions` ones, so they would be out of date once #6808 merges. The S3A row of the user guide's eligibility table also says every other `fs.s3a.*` setting falls back, without pointing at the exceptions. This describes the behavior #6808 adds, so it should merge after it. ## What changes are included in this PR? User guide (`iceberg-writes.md`): - The S3A row of the eligibility table points at the ignored settings listed below it. - The paragraph under the table lists all of the ignored settings and says Spark 4.1 sets two of them itself. It also notes that they are matched by their plain `fs.s3a.<name>` keys, so a per-bucket form still falls back. Contributor guide (`iceberg-writes.md`): - The gate description names `IgnoredHadoopS3Keys` and lists what is on it. It says when a key belongs there: only if S3A would not apply it to a data-file write request. Settings S3A does apply must keep falling back. ## How are these changes tested? Docs only, so there are no tests. `npx prettier@latest --check` passes on both files. The matching rule (exact key, no per-bucket normalization) was read from `unsupportedHadoopS3Settings` and `effectiveHadoopConf` in `CometIcebergNativeWrite`, and the Spark 4.1 seeding from `SparkContext.enableMagicCommitterIfNeeded` in Spark 4.1.1. Neither was run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
