andygrove opened a new pull request, #7052: URL: https://github.com/apache/datafusion-comet/pull/7052
## Which issue does this PR close? Closes #7050. ## Rationale for this change The nested-type audits of the native Parquet and Iceberg scans (#7040) and writers (#7051) filed or confirmed a set of open issues in which Comet gives a different result from Spark without falling back. The user guide named only one of them, datetime rebasing (#5010), so a user who sees an unexpected value has no page that lists the rest. The audits' results, including what was checked and matched and which candidates did not reproduce, are recorded only in those two epics, which is not where the next audit of the same area would look. The expression audits have a log in the contributor guide, and the data sources have none. The Iceberg writes page also says the float/double null-count difference can't change `IS NULL` pruning. It can: iceberg-java's counts are the wrong ones (apache/iceberg#17561), and the native counts are correct (#7050). ## What changes are included in this PR? - A new compatibility page, "Known Correctness Issues in Parquet and Iceberg Reads and Writes". It lists each open issue in which a Parquet or Iceberg read or write gives a different result from Spark without falling back, with a link to the issue, in sections for Parquet reads, Iceberg reads, Parquet writes and Iceberg writes. A table at the top says which native paths are on by default, and a last section lists the differences that are bugs in Spark or iceberg-java. Queries that fail, and the cases that fall back, stay on the existing pages. - Pointers to the new page from the compatibility overview (its topic list and "Known result-value divergences"), from `scans.md`, whose datetime rebasing item moves to the new page, from the Iceberg guide's current limitations, and from the Iceberg writes page's accepted divergences. - The corrected float/double null-count bullet in `iceberg-writes.md`. - A new "Data Source Audits" section in the contributor guide, next to Expression Audits, with a Parquet page and an Iceberg page. Each holds dated entries for the 2026-10-15 scan audit and the 2026-10-16 writer audit: scope, method and versions, the issues filed, what matched, what did not reproduce, the differences that are not Comet bugs, and what was not covered. ## How are these changes tested? Documentation only. `prettier --check` (3.9.10) passes on the changed files. A local Sphinx build gives the same 80 warnings as `main`, so every new link and heading anchor resolves. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
