andygrove commented on PR #6469: URL: https://github.com/apache/datafusion-comet/pull/6469#issuecomment-5982057318
Closing this. Every regression it listed is now fixed on `branch-1.1` except #6254, and #6544, its fix, is approved and labelled for backport, so the 1.1.0 page would be empty or close to it. That isn't worth a new top-level docs section. The part users still need, the upgrade-guide note that native Iceberg reads on EKS no longer fall back from IRSA (#6025), is now #6603, and it will be backported to `branch-1.1`. #6402 keeps the full list of regressions and their fixes. Two things from here belong in the 1.1.0 release blog post instead: - The headline from the audit (#6399): 1.1.0 fixes about 90 bugs that shipped in 1.0.0 (91 of rc1's 131 `fix:` pull requests), and about 40 of those returned results that differed from Spark. - The #5421 change: when the final aggregate runs in Spark, which happens for every aggregate if the Comet shuffle manager isn't installed, the partial `avg`, decimal `sum`, `stddev`, `variance`, `corr`, `first`, `last` and a few others now run in Spark too. In 1.0.0 they ran natively, and `avg` could return NULL in that plan (#5419). Installing the Comet shuffle manager keeps them native. If #6544 doesn't make rc2, the #6254 workaround can go in the post too: more off-heap memory, or `spark.comet.exec.aggregate.enabled=false` for the affected job. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
