andygrove opened a new issue, #5052: URL: https://github.com/apache/datafusion-comet/issues/5052
Triage pass over the open `requires-triage` queue, per the project [Bug Triage Guide](https://github.com/apache/datafusion-comet/blob/main/docs/source/contributor-guide/bug_triage.md). - Date: 2026-07-27 - Total issues processed: 20 (16 triaged, 4 skipped, 0 failed) - Type counts: 7 bugs, 9 enhancements - Priority counts applied: `priority:critical` 2, `priority:high` 2, `priority:medium` 2, `priority:low` 1 - Guide: [docs/source/contributor-guide/bug_triage.md](https://github.com/apache/datafusion-comet/blob/main/docs/source/contributor-guide/bug_triage.md) Labels have already been applied. A reviewer should spot-check the calls below and close this issue when satisfied; corrections should be made directly on the affected issue. ## Bugs ### priority:critical - RANGE window frame boundary overflow diverges from Spark for DATE and DECIMAL ORDER BY ([#5022](https://github.com/apache/datafusion-comet/issues/5022)) - Area labels: `area:expressions` - Rationale: Native and Spark produce different window frames on boundary overflow with no error — silent wrong results, which the guide defines as `priority:critical`. - Object store cache key drops the ABFS container, so two containers in one storage account share a store instance ([#4993](https://github.com/apache/datafusion-comet/issues/4993)) - Area labels: `area:scan` - Rationale: Reads from one ABFS container can silently return another container's data — a data-corruption / silent wrong-results case that the guide places at `priority:critical`. ### priority:high - Sporadic SIGSEGV in macOS [scans] workflow in CI ([#5023](https://github.com/apache/datafusion-comet/issues/5023)) - Area labels: `area:ci`, `area:scan` - Rationale: JVM SIGSEGV in the native scan path — a crash on a supported code path, matching the guide's `priority:high` definition; kept off critical because it appears only sporadically in CI and has not been shown to produce wrong results. - Native panic casting to negative-scale decimal when spark.sql.legacy.allowNegativeScaleOfDecimal=true ([#5013](https://github.com/apache/datafusion-comet/issues/5013)) - Area labels: `area:expressions` - Rationale: Reproducible native panic on a supported cast path; `priority:high` per the crash rule. See Escalations below — the release build silently wraps instead of panicking, which is a critical-tier hazard once confirmed. ### priority:medium - Datetime rebase: track the documented scan limitation, and spark.comet.exceptionOnDatetimeRebase is dead code ([#5010](https://github.com/apache/datafusion-comet/issues/5010)) - Area labels: `area:scan`, `documentation`, `correctness` - Rationale: Silent wrong results on legacy-calendar Parquet, but the reader has a documented workaround (`spark.comet.scan.enabled=false`) and the surface is narrow — pre-3.0 files or writes with `datetimeRebaseModeInWrite=LEGACY`. See Escalations below. - Native Parquet scan ignores session-level datafusion.execution.parquet.* options ([#4990](https://github.com/apache/datafusion-comet/issues/4990)) - Area labels: `area:scan` - Rationale: A documented config path is silently inert but the scan still returns correct data; broken feature with a workaround → `priority:medium`. ### priority:low - CI: non-gating label (e.g. auto-added `performance`) spawns a phantom all-skipped CI run that shadows real checks ([#5007](https://github.com/apache/datafusion-comet/issues/5007)) - Area labels: `area:ci` - Rationale: CI/tooling issue — the real CI run still executes; the phantom run only cosmetically misleads the merge box → `priority:low`. ## Enhancements - Follow-up: docs and test-coverage hardening for native collect_list / array_agg (#4720) ([#5024](https://github.com/apache/datafusion-comet/issues/5024)) - Area labels: `area:aggregation`, `area:expressions` - Rationale: Deferred docs additions and test-coverage improvements from PR #4720 review; nothing is broken, so this is enhancement work. - Centralize data-type support predicates so new types don't have to be added in multiple places ([#5021](https://github.com/apache/datafusion-comet/issues/5021)) - Area labels: none - Rationale: The reporter explicitly notes "This is a maintainability cleanup, not a correctness bug" — a refactor to consolidate duplicated type predicates. - Allow individual cast conversions to be enabled/disabled ([#5015](https://github.com/apache/datafusion-comet/issues/5015)) - Area labels: `area:expressions` - Rationale: New config to selectively disable individual `(fromType, toType)` cast pairs — feature addition to the Cast serde. - perf: reuse Arrow IPC CompressionContext across shuffle blocks instead of per-batch allocation ([#5008](https://github.com/apache/datafusion-comet/issues/5008)) - Area labels: `area:shuffle`, `performance` - Rationale: Shuffle-writer performance optimization — reuse an existing allocation across blocks; behavior is unchanged. - Share Parquet footer and page index metadata across tasks in an executor ([#4994](https://github.com/apache/datafusion-comet/issues/4994)) - Area labels: `area:scan`, `performance` - Rationale: New feature to share Parquet metadata caches across tasks in an executor; a performance improvement over parity with Spark. - Support native Iceberg scans in Spark Structured Streaming ([#4992](https://github.com/apache/datafusion-comet/issues/4992)) - Area labels: `area:scan` - Rationale: New functionality — allow the existing native Iceberg scan to be used in Structured Streaming queries that are currently skipped by `CometExecRule`. - Row-level Parquet filter pushdown regresses uncorrelated-predicate scans (TPC-H Q6 +80%) ([#4988](https://github.com/apache/datafusion-comet/issues/4988)) - Area labels: `area:scan`, `performance` - Rationale: Investigation into whether `spark.comet.parquet.rowFilterPushdown.enabled` can be turned on by default; the flag is off by default so no user is affected today — enhancement, not a user-visible regression. - Upstream Spark-compatible approx_count_distinct (HyperLogLogPlusPlus) to the datafusion-spark crate ([#4984](https://github.com/apache/datafusion-comet/issues/4984)) - Area labels: `area:aggregation`, `area:expressions` - Rationale: Refactor/upstreaming of an existing native aggregate to the upstream `datafusion-spark` crate; no behavior change. - Implement TimeType support - Time extraction (from TimeType to Int/Decimal) ([#4983](https://github.com/apache/datafusion-comet/issues/4983)) - Area labels: `area:expressions` - Rationale: New expression support for TimeType extraction — a subtask of the TimeType umbrella (#4288). ## Escalations to consider - Native panic casting to negative-scale decimal when spark.sql.legacy.allowNegativeScaleOfDecimal=true ([#5013](https://github.com/apache/datafusion-comet/issues/5013)) - Guide trigger: "A `priority:high` crash is discovered to also produce wrong results silently in some cases → escalate to `priority:critical`." The reporter observed that debug builds panic but the release profile has `overflow-checks = false`, so a release build would silently wrap and produce wrong values. If confirmed on a release build this should move to `priority:critical`. - Datetime rebase: track the documented scan limitation, and spark.comet.exceptionOnDatetimeRebase is dead code ([#5010](https://github.com/apache/datafusion-comet/issues/5010)) - Guide trigger: correctness principle — "A bug that silently returns wrong results is worse than one that crashes." Left at `priority:medium` because the surface is narrow (legacy-calendar Parquet only) and a working workaround exists (`spark.comet.scan.enabled=false`), but a reviewer familiar with the deployed workloads may want to escalate to `priority:critical`. ## Skipped — needs more info - Bug triage results: 2026-07-20 ([#4980](https://github.com/apache/datafusion-comet/issues/4980)) - Prior triage summary issue that was left carrying the auto-applied `requires-triage` label; not itself a bug or enhancement — reviewer should sanity-check the referenced results and close. - Bug triage results: 2026-07-13 ([#4905](https://github.com/apache/datafusion-comet/issues/4905)) - Prior triage summary issue that was left carrying the auto-applied `requires-triage` label; not itself a bug or enhancement — reviewer should sanity-check the referenced results and close. - Bug triage results: 2026-07-06 ([#4838](https://github.com/apache/datafusion-comet/issues/4838)) - Prior triage summary issue that was left carrying the auto-applied `requires-triage` label; not itself a bug or enhancement — reviewer should sanity-check the referenced results and close. - Bug triage results: 2026-06-29 ([#4751](https://github.com/apache/datafusion-comet/issues/4751)) - Prior triage summary issue that was left carrying the auto-applied `requires-triage` label; not itself a bug or enhancement — reviewer should sanity-check the referenced results and close. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
