andygrove opened a new pull request, #4933: URL: https://github.com/apache/datafusion-comet/pull/4933
## Which issue does this PR close? N/A ## Rationale for this change Comet has accumulated a campaign of scalar expression performance optimizations (for example `spark_cast_int_to_int`, `to_json`, `parse_url`, `spark_size`, `spark_unhex`, and many more in flight). The recurring methodology and pitfalls have lived only in individual PR descriptions. This PR captures that knowledge in one place so both human contributors and automated agents follow the same approach, and so we can track which expressions have already been tuned. ## What changes are included in this PR? - New contributor guide page `Optimizing Scalar Expressions` (added to the Debugging and Performance section) covering the benchmark-first workflow, criterion benchmark shapes that expose regressions, a catalog of proven techniques mapped to the PRs that landed them, correctness rules (bit-identical output, null-slot error handling, eval-mode semantics), and the no-regression gate. - New `optimize-comet-expression` skill that points at the guide as the shared source of truth and adds the agent execution loop and submission gate. - Extended the per-expression audit pages to record performance audits alongside correctness audits, with a dated `Performance (tuned ...)` line naming the technique, speedup, PR, and benchmark file. - Backfilled performance audit entries for the recently merged `cast` (int-to-int), `to_json`, `parse_url`, `size`, and `unhex` optimizations. ## How are these changes tested? Documentation and skill content only; no code changes. Verified the new page is wired into the contributor guide toctree and that the audit entries land under the correct expression headings. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
