andygrove opened a new issue, #5999: URL: https://github.com/apache/datafusion-comet/issues/5999
The nightly Miri workflow (`.github/workflows/miri.yml`, 04:00 UTC) has failed every night since 2026-07-01 — 79 consecutive runs. The last green run was 2026-06-30. Nothing surfaced it because `miri.yml` has no failure-reporting step and a scheduled run has no pull request to turn red, so the job has effectively not been providing any signal for eleven weeks. ## What is failing The job runs the whole native workspace under Miri: ``` cd native MIRIFLAGS="-Zmiri-disable-isolation" cargo miri test --lib --bins --tests --examples ``` Miri aborts on the first error, so the run only ever reports one. The reported error has changed at least three times over the streak, and none of the three is Comet's own `unsafe` code: | Window | Error | | --- | --- | | 2026-07-01 onwards | `unsupported operation: can't call foreign function \`aws_lc_0_41_0_CRYPTO_library_init\` on OS \`linux\`` | | around 2026-09-08 | `unsupported operation: can't call foreign function \`llistxattr\` on OS \`linux\`` | | 2026-09-12 to now | `Undefined Behavior: incorrect c-variadic argument type for \`syscall(SYS_futex, ...)\`: expected argument #2 to have type \`*mut u32\` but got incompatible type \`&std::sync::atomic::Atomic<i32>\`` | The current one is inside `parking_lot_core` 0.9.12's `ThreadParker::futex_wait` (`thread_parker/linux.rs:112`), reached through a plain `parking_lot::Mutex::lock`. It surfaces via `execution::memory_pools::task_shared::tests::concurrent_acquire_and_drop_leaves_a_consistent_registry` in `native/core/src/execution/memory_pools/task_shared.rs`, but that test is only the first one to take a contended `parking_lot` lock — any of them would trip it. Miri nightly recently started type-checking c-variadic arguments, which is what made this visible. Latest failing run: https://github.com/apache/datafusion-comet/actions/runs/35181494817 ## Why this keeps happening `cargo miri test` over the entire workspace pulls in every test in every crate, including tests that reach real FFI (AWS-LC via the object store stack), unusual syscalls (`llistxattr`), and third-party synchronization primitives. Miri cannot execute foreign functions at all, and it models platform synchronization only approximately, so any such test aborts the whole run regardless of whether Comet's own code is sound. That is a much wider net than the job was set up to cast. The original motivation (position 1 in the history: issue #634, then #1161) was to check Comet's own `unsafe` code — the unaligned access in `SparkUnsafeRow` and `spark_compatible_murmur3_hash`. Under the current invocation that code is never reached, because the run aborts earlier on something else. ## Suggested fix Two options, not mutually exclusive: 1. Scope the run to the crates and tests whose `unsafe` code Miri is meant to cover, rather than `--lib --bins --tests --examples` across the workspace. 2. Mark the tests that reach FFI or platform synchronization with `#[cfg_attr(miri, ignore)]` so Miri skips them and reports on the rest. Whichever way it goes, the job should be able to go green, otherwise the next regression in it is invisible for another eleven weeks. Separately, `miri.yml` and `publish_snapshot.yml` are both scheduled workflows with no failure-reporting step. `ci.yml`'s nightly tier opens a `ci-nightly-failure` issue when it goes red; these two do not. Worth considering the same treatment, or folding them into the nightly tier so they inherit it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
