comphead commented on code in PR #6961: URL: https://github.com/apache/datafusion-comet/pull/6961#discussion_r4488371858
########## docs/source/user-guide/latest/iceberg.md: ########## @@ -574,6 +574,82 @@ or GCS each read is a GET request, so the task waits two request round trips per small next to uploading a large file, but a write that produces many small files, such as a fanout write over many partitions, pays it for every one of them. +#### Plans for each fallback + +Each eligibility rule above is planned to be lifted, waiting on work in another project, or not +planned. A rule is lifted only once the native writer matches iceberg-java's output for the +setting it guards. + +Planned to be lifted: + +- `write.object-storage.enabled=true` + ([#6956](https://github.com/apache/datafusion-comet/issues/6956)) +- S3 settings that cannot change the written object or where it is written, such as `fs.s3a.impl`, + connection pool sizes, timeouts, retries and multipart sizes; S3A and `FileIO` credential + providers and S3A server-side encryption settings that have a native equivalent; and ACL, + storage class, requester-pays and `dsse-kms` settings + ([#6957](https://github.com/apache/datafusion-comet/issues/6957)) +- Merge-on-read `DELETE`, `UPDATE` and `MERGE` on a format version 2 table + ([#6240](https://github.com/apache/datafusion-comet/issues/6240)) +- Bloom filters (`write.parquet.bloom-filter-*`) + ([PR #5724](https://github.com/apache/datafusion-comet/pull/5724)) +- Copy-on-write `DELETE`, `UPDATE` and `MERGE`, and `rewrite_data_files`, on a format version 3 + table with Iceberg 1.10 or newer + ([#6669](https://github.com/apache/datafusion-comet/issues/6669)) +- Azure (`abfs`, `abfss`, `wasb`, `wasbs`) and `oss` data locations, S3-compatible scheme + aliases, and a `gs` data location opened through `HadoopFileIO` + ([#6958](https://github.com/apache/datafusion-comet/issues/6958)) +- `parquet.*` table properties and session Hadoop settings that iceberg-java's writer never reads, + `write.parquet.shred-variants=true`, and an `io-impl` table property + ([#6955](https://github.com/apache/datafusion-comet/issues/6955)) +- `parquet.enable.dictionary`, `parquet.columnindex.truncate.length`, + `write.parquet.page-version=v2`, `write.parquet.stats-enabled.column.*`, non-default + `row-group-check-*` counts, and a zstd level of `0` + ([#6954](https://github.com/apache/datafusion-comet/issues/6954)) +- `uuid` and `unknown` columns + ([#6959](https://github.com/apache/datafusion-comet/issues/6959)) +- An identity partition on a `float` or `double` column + ([PR #6851](https://github.com/apache/datafusion-comet/pull/6851)) Review Comment: #6851 merged on 2026-10-17, so float and double identity partitions are written natively on `main` and the eligibility table there no longer lists the rule. Could this entry be dropped when you rebase? The other linked issues and #5724 are still open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
