andygrove opened a new issue, #7230: URL: https://github.com/apache/datafusion-comet/issues/7230
## Summary - Date: 2026-10-26 - `requires-triage` issues found: 48. Labeled: 40 (31 bugs, 9 enhancements). Skipped: 8. Failed to label: 0. - Bugs by priority: `priority:critical` 11, `priority:high` 3, `priority:medium` 10, `priority:low` 7 - Guide: [docs/source/contributor-guide/bug_triage.md](https://github.com/apache/datafusion-comet/blob/main/docs/source/contributor-guide/bug_triage.md) Labels have already been applied and `requires-triage` removed from every labeled issue. Please spot-check the calls and close this issue when satisfied. Several issues already carried a type label, a priority, or both from their author; those were kept when they matched the guide. ## Bugs ### priority:critical - Native string to timestamp cast returns NULL for zone offsets with a non-zero seconds part ([#7224](https://github.com/apache/datafusion-comet/issues/7224)) - Area labels: `area:expressions` - Rationale: A valid input silently becomes NULL where Spark returns a value, which is a silent wrong result. - Native Iceberg scan returns NULL for an INT96 column read through a name mapping when other columns are selected ([#7212](https://github.com/apache/datafusion-comet/issues/7212)) - Area labels: `area:scan`, `area:Iceberg` - Rationale: Nothing fails and the column comes back NULL, which is a silent wrong result. - String-to-timestamp cast at the far-past range limit uses -08:00 instead of local mean time in America/Los_Angeles ([#7210](https://github.com/apache/datafusion-comet/issues/7210)) - Area labels: `area:expressions` - Rationale: The cast returns a value 422 seconds off from Spark's, a silent wrong result, although only at the extreme end of the range. - Native string-to-timestamp cast rejects zone suffixes Spark accepts (PST, CET, Zulu, Etc/GMT+8, +08:00:00, no space) ([#7209](https://github.com/apache/datafusion-comet/issues/7209)) - Area labels: `area:expressions` - Rationale: Inputs Spark accepts return NULL with ANSI off, which is a silent wrong result. - REBALANCE followed by GROUP BY or a window returns duplicate groups on skewed data ([#7208](https://github.com/apache/datafusion-comet/issues/7208)) - Area labels: `area:aggregation` - Rationale: Skewed keys come back as several partial rows, a silent wrong aggregation result. - collect_set over floats beside count(DISTINCT) diverges from Spark on signed zeros and NaN ([#7203](https://github.com/apache/datafusion-comet/issues/7203)) - Area labels: `area:aggregation` - Rationale: The set contents differ from Spark with no error; this is a 1.2.0 regression for the distinct-aggregate shape. - Native Iceberg scan drops rows from data files that store a narrower type than the column ([#7138](https://github.com/apache/datafusion-comet/issues/7138)) - Area labels: `area:scan`, `area:Iceberg` - Rationale: Rows are silently dropped by the scan and the post-scan filter cannot restore them. - Native Iceberg scan drops NULL rows for NOT (col <=> value) ([#7137](https://github.com/apache/datafusion-comet/issues/7137)) - Area labels: `area:scan`, `area:Iceberg` - Rationale: NULL rows are silently dropped, which is a silent wrong result. - Native Iceberg scan drops -0.0 rows for floating-point predicates ([#7136](https://github.com/apache/datafusion-comet/issues/7136)) - Area labels: `area:scan`, `area:Iceberg` - Rationale: Rows are silently dropped because -0.0 and 0.0 compare as different values, which is a silent wrong result. - Native Iceberg scan applies equality deletes to the wrong column when column names differ only in case ([#7129](https://github.com/apache/datafusion-comet/issues/7129)) - Area labels: `area:scan`, `area:Iceberg` - Rationale: Deleted rows come back or the wrong rows are dropped with no error, a silent wrong result. - Native object-store errors and debug logs print whole URLs outside the Azure store ([#7128](https://github.com/apache/datafusion-comet/issues/7128)) - Area labels: `area:scan` - Rationale: A password in the URL userinfo or a token in the query can reach task errors and logs; the guide places security vulnerabilities at `priority:critical`. The reviewer may want to downgrade it, since it needs credentials embedded in the URL. ### priority:high - Native ABFS scans fail every task instead of falling back when Hadoop declines the auth type ([#7202](https://github.com/apache/datafusion-comet/issues/7202)) - Area labels: `area:scan` - Rationale: Every task of affected ABFS jobs fails where 1.1.0 worked, which is major functional breakage on a production path. - Bloom aggregate merge panics on batched states and ignores sliced offsets ([#7172](https://github.com/apache/datafusion-comet/issues/7172)) - Area labels: `area:aggregation` - Rationale: The merge asserts and panics when spill replay supplies several states for one group. - Executor is OOM-killed while Comet's pool and allocation counter stay under the limit: glibc keeps several GiB of freed native memory ([#7108](https://github.com/apache/datafusion-comet/issues/7108)) - Area labels: none - Rationale: The cgroup OOM killer terminates the executor (exit 137) on a TPC-H SF10 run, which is a crash on a real workload. ### priority:medium - Native window over a sorted cache fails with "Expects PARTITION BY expression to be ordered" ([#7223](https://github.com/apache/datafusion-comet/issues/7223)) - Area labels: none - Rationale: The query fails but only with `spark.comet.convert.inMemoryCache.enabled=true` and `spark.comet.exec.inMemoryCache.enabled=false`, so the user can turn the config off. - CAST(timestamp_ntz AS TIMESTAMP) fails the task for values beyond year 262143 where Spark returns a value ([#7211](https://github.com/apache/datafusion-comet/issues/7211)) - Area labels: `area:expressions` - Rationale: Loud failure (not a panic) on extreme values only, and the same bug is in 1.1.0. - fair_unified: a sort fails on its first request when consumers that cannot spill hold the task's share ([#7197](https://github.com/apache/datafusion-comet/issues/7197)) - Area labels: none - Rationale: A query fails with a memory error under a non-default pool, and switching the pool is a workaround. - Broadcast exchange fails with `OversizedAllocationException` when a string or binary column holds more than 1 GiB ([#7192](https://github.com/apache/datafusion-comet/issues/7192)) - Area labels: none - Rationale: A broken feature for very large build sides, with the workaround of avoiding the broadcast. - Native string to negative-scale decimal cast doesn't match Spark 4.0.5, 4.1.4 and 4.2.1 (SPARK-60119) ([#7187](https://github.com/apache/datafusion-comet/issues/7187)) - Area labels: `area:expressions` - Rationale: The divergence only applies to Spark releases that are still in RC and to `spark.sql.legacy.allowNegativeScaleOfDecimal=true`; see the escalation note. - sequence and string to decimal cast don't match Spark 4.0.5, 4.1.4 and 4.2.1 (SPARK-58821, SPARK-60118) ([#7170](https://github.com/apache/datafusion-comet/issues/7170)) - Area labels: `area:expressions` - Rationale: The divergence only applies to Spark releases that are still in RC; see the escalation note. - Native Parquet scan: casts the planner inserts into pushed-down filters are rejected as Parquet type promotions (Spark 3.x, row-level filter pushdown) ([#7158](https://github.com/apache/datafusion-comet/issues/7158)) - Area labels: `area:scan` - Rationale: Queries fail at runtime, but only with `spark.comet.parquet.rowFilterPushdown.enabled=true` on Spark 3.x. - Native filters and non-collect limits can evaluate ANSI casts on rows Spark skips ([#7147](https://github.com/apache/datafusion-comet/issues/7147)) - Area labels: `area:expressions` - Rationale: A query Spark completes raises `CAST_INVALID_INPUT`; it is a loud failure limited to ANSI mode and LIMIT or filter shapes. - NativeUtil.exportBatch rejects large_string and large_binary vectors ([#7109](https://github.com/apache/datafusion-comet/issues/7109)) - Area labels: `area:ffi` - Rationale: The export fails with `Unsupported Arrow Vector for export` for a native operator above `CometMapInBatchExec`. - Arrow struct writer should respect projected schema width ([#6576](https://github.com/apache/datafusion-comet/issues/6576)) - Area labels: `area:Iceberg` - Rationale: The writer can walk past the Arrow children, but only on the in-progress merge-on-read path. ### priority:low - Native `NUMERIC_OUT_OF_SUPPORTED_RANGE` error surfaces as `SparkException` instead of `SparkArithmeticException` ([#7216](https://github.com/apache/datafusion-comet/issues/7216)) - Area labels: `area:expressions` - Rationale: The error class and message match Spark; only the exception type differs. - Native Parquet scan reads timestamp columns as string, int, double or decimal, and INT96 as bigint, where Spark rejects them ([#7214](https://github.com/apache/datafusion-comet/issues/7214)) - Area labels: `area:scan` - Rationale: Comet is more lenient than Spark on reads Spark rejects; valid queries are unaffected. - Native Parquet scan read errors name no file, or every file in the task, instead of the file that failed ([#7213](https://github.com/apache/datafusion-comet/issues/7213)) - Area labels: `area:scan` - Rationale: Cosmetic; the read fails as it should and only the error message is wrong. - Native string to decimal cast raises the wrong error under ANSI for values with more than 38 integral digits ([#7188](https://github.com/apache/datafusion-comet/issues/7188)) - Area labels: `area:expressions` - Rationale: Both engines raise an error and return NULL outside ANSI; only the error class differs. - CometTaskMemoryManager.releaseMemory lowers used before Spark accepts the release ([#7176](https://github.com/apache/datafusion-comet/issues/7176)) - Area labels: none - Rationale: Accounting is wrong only on the path where Spark throws during a release. - A short grant stays with Spark when handing it back fails, but the pool records nothing ([#7175](https://github.com/apache/datafusion-comet/issues/7175)) - Area labels: none - Rationale: Accounting is wrong only when the JNI hand-back call fails. - Concurrent unified pool try_grow calls can both ask Spark for the same overcommit ([#7174](https://github.com/apache/datafusion-comet/issues/7174)) - Area labels: none - Rationale: A race that makes a request larger than needed; no wrong results or crash. ## Enhancements - Reduce duplicate right-side keys before sorting a native left-semi join ([#7189](https://github.com/apache/datafusion-comet/issues/7189)) - Area labels: none - Rationale: An opt-in performance optimization request. - Run the MinIO-backed S3 suites in CI, starting with ParquetWriteWithMagicCommitterSuite ([#7150](https://github.com/apache/datafusion-comet/issues/7150)) - Area labels: `area:ci` - Rationale: A request for new CI coverage. - Run SortAggregateExec as a native hash aggregate without its input sort ([#7146](https://github.com/apache/datafusion-comet/issues/7146)) - Area labels: `area:aggregation` - Rationale: A performance and robustness improvement request, not a defect in existing behavior. - Tune nested take compaction threshold with a query-level benchmark ([#7145](https://github.com/apache/datafusion-comet/issues/7145)) - Area labels: none - Rationale: A benchmarking and tuning request. - test: move make_date SQL regressions into SQL fixtures ([#7144](https://github.com/apache/datafusion-comet/issues/7144)) - Area labels: none - Rationale: Test reorganization; the issue says current coverage already exercises the behavior. - perf: retain native make_date filters behind redundant NULL checks ([#7143](https://github.com/apache/datafusion-comet/issues/7143)) - Area labels: `area:expressions` - Rationale: Results are correct; the request is to keep a native filter that currently falls back, and the issue says the impact is unmeasured. - refactor: consolidate make_date and shared evaluation guards ([#7142](https://github.com/apache/datafusion-comet/issues/7142)) - Area labels: `area:expressions` - Rationale: A refactor request. - Keep wide-decimal scalar AVG native with a wider running sum ([#7116](https://github.com/apache/datafusion-comet/issues/7116)) - Area labels: `area:aggregation` - Rationale: Asks to keep a currently conservative fallback native. - Support join runtime reader filters through column-only projections ([#7113](https://github.com/apache/datafusion-comet/issues/7113)) - Area labels: `area:scan` - Rationale: A request to extend runtime-filter attachment. ## Escalations to consider - Native string to negative-scale decimal cast doesn't match Spark 4.0.5, 4.1.4 and 4.2.1 (SPARK-60119) ([#7187](https://github.com/apache/datafusion-comet/issues/7187)) - The native cast returns NULL where the new Spark releases return 0, a silent wrong result. It is `priority:medium` only because those releases are still in RC; consider `priority:critical` when they ship. - sequence and string to decimal cast don't match Spark 4.0.5, 4.1.4 and 4.2.1 (SPARK-58821, SPARK-60118) ([#7170](https://github.com/apache/datafusion-comet/issues/7170)) - `'0e79'` casts to NULL where the new releases return 0, a silent wrong result. It is `priority:medium` only because those releases are still in RC; consider `priority:critical` when they ship. - Native window over a sorted cache fails with "Expects PARTITION BY expression to be ordered" ([#7223](https://github.com/apache/datafusion-comet/issues/7223)) - Currently a loud failure behind two non-default configs. If the cache path becomes default, move to `priority:high`. ## Skipped — needs more info - [EPIC] Regressions in 1.2.0 since 1.1.0 ([#7204](https://github.com/apache/datafusion-comet/issues/7204)) - This is a tracking epic, not a single defect, so no priority fits. It already carries `bug` and `EPIC`. `requires-triage` was left in place for a human to decide. - Audit the PRs in 1.1.0 for regressions since 1.0.0 ([#6399](https://github.com/apache/datafusion-comet/issues/6399)) - A multi-phase audit tracker; it is neither a bug nor an enhancement request. - TPC-DS q54/q68: local SF100 results and distributed reproduction limits ([#6946](https://github.com/apache/datafusion-comet/issues/6946)) - A reproduction report that explicitly claims no defect; it does not say whether there is a bug. - Bug triage results: 2026-10-18 ([#7105](https://github.com/apache/datafusion-comet/issues/7105)) - A previous triage summary that picked up the auto-applied `requires-triage`. Not a bug or enhancement; remove the label when closing. - Bug triage results: 2026-10-12 ([#6892](https://github.com/apache/datafusion-comet/issues/6892)) - Same as above. - Bug triage results: 2026-10-05 ([#6665](https://github.com/apache/datafusion-comet/issues/6665)) - Same as above. - Bug triage results: 2026-09-28 ([#6321](https://github.com/apache/datafusion-comet/issues/6321)) - Same as above. - Bug triage results: 2026-08-24 ([#5454](https://github.com/apache/datafusion-comet/issues/5454)) - Same as above. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
