andygrove opened a new pull request, #6150: URL: https://github.com/apache/datafusion-comet/pull/6150
## Which issue does this PR close? Closes #6147. ## Rationale for this change The user guide pages for Iceberg had several statements about native writes that no longer match the code, and were missing two things a user needs to know before enabling the native writer. Found in an audit of the native Iceberg write path (#5649). ## What changes are included in this PR? `docs/source/user-guide/latest/iceberg-writes.md`: - The accepted-divergences list said float and double partition directories are rendered with Rust's shortest representation (`f=1` where iceberg-java writes `f=1.0`). Since #5840 they use Java's `Float.toString` / `Double.toString` rules and match iceberg-java, so that divergence is removed. The claim that Iceberg deprecated float and double partitioning in 1.3 is also removed; identity partitioning on float and double is accepted on every Iceberg version Comet tests. - The manifest section said the parity suite compares committed manifests byte for byte. `CometIcebergWriteActionSuite` compares the committed `readable_metrics` of tables written by both writers, so the sentence now says that. - New paragraph under "Native Parquet write eligibility": the native writer needs a Comet input, so writes fed by a local relation (`INSERT ... VALUES`, `df.writeTo(...).append()` on local data) only use it when `spark.comet.exec.localTableScan.enabled=true`. The configuration example now includes that setting, and its comment for `spark.comet.iceberg.write.enabled` describes the flag as the native writer rather than "eligibility detection". - New accepted divergence: high-cardinality columns keep a dictionary page that iceberg-java would not write (#6114). `docs/source/user-guide/latest/iceberg.md`: - The native scan's fallback list included "Iceberg writes (reads are accelerated, writes use Spark)". Writes are not a scan fallback, and there is now a native writer, so the bullet is replaced by a sentence after the list that points to the writes page. ## How are these changes tested? Documentation only. `prettier --check` passes on both files. Each corrected statement was checked against the code on `main`: `java_float_string` in `iceberg_partition_path.rs`, the `readable_metrics` comparisons in `CometIcebergWriteActionSuite`, and `CometIcebergNativeWrite.requiresNativeChildren`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
