deepthi912 commented on PR #19010: URL: https://github.com/apache/pinot/pull/19010#issuecomment-5689432648
### CI failure analysis — `Pinot Integration Test Set 2 (temurin-25)` lane-a The Set 2 lane-a failure is a Temurin 25.0.4.1+1 C2 JIT miscompilation, not a test regression from this PR. Sharing evidence so we can decide how to unblock. **Crash signature (identical across 5 consecutive CI runs on this branch):** ``` # A fatal error has been detected by the Java Runtime Environment: # SIGSEGV (0xb) at pc=0x00007f..., pid=..., tid=... # JRE version: OpenJDK Runtime Environment Temurin-25.0.4.1+1 (25.0.4.1+1) # Java VM: OpenJDK 64-Bit Server VM Temurin-25.0.4.1+1 # Problematic frame: # J <id> c2 jdk.internal.misc.Unsafe.getIntUnaligned(Ljava/lang/Object;JZ)I [email protected] ``` Only the compilation ID (8023, 8750, 7858, 8172, ...) and PC address change between runs; frame and JDK are identical. The problematic frame is a C2-JIT-compiled JVM intrinsic inside `java.base` — a JDK codegen bug, not Pinot code. **Why this PR trips it and master doesn't** This PR flips the default `pinot.server.consuming.segment.consistency.mode` from `RESTRICTED` to `PROTECTED`. That default change alters the runtime path of two tests earlier in the same lane-a fork (`PauselessDedupRealtimeIngestionSegmentCommitFailureTest`, `PauselessRealtimeIngestionIntegrationTest`), which shifts C2's profile-guided inlining and codegen decisions enough to push `Unsafe.getIntUnaligned` into the buggy code path by the time `PartialUpsertTableRebalanceIntegrationTest` runs (~18 min into the fork, compilation ID ~7000-8500). Master's fork profile stays outside the buggy path. **Local reproduction attempts** | Setup | Result | |---|---| | macOS aarch64, Temurin 25.0.3+9, single test class, fresh fork | Passes in 37 s | | macOS aarch64, Temurin **25.0.4.1+1** (identical to CI), single test class, fresh fork | Passes in 36 s | | macOS aarch64, Temurin **25.0.4.1+1**, **7-test lane-a sequence** in single reused fork (`-DreuseForks=true -DforkCount=1`) | **Passes in 6:03 min** | The last configuration matches CI's JDK version, test sequence, and fork-reuse strategy — but the C2 bug does not reproduce on aarch64 because C2 emits completely different machine code per architecture. The bug appears specific to Temurin 25.0.4.1+1 x86-64 C2 output. **Options to unblock — deferring to maintainer preference** 1. **Pin Temurin patch level** to 25.0.3.x (or later 25.0.4.x once a fix ships) in `.github/workflows/pinot_integration_tests.yml` — currently `java: [ 25 ]` resolves to whatever is latest at run time. 2. **Add `-XX:CompileCommand=exclude,jdk/internal/misc/Unsafe.getIntUnaligned`** to `pinot.integration.test.jvm.args` in `pinot-integration-tests/pom.xml` — disables C2 for that one intrinsic (test-only, ~1% perf hit on byte[]/String hashing), avoids the crash. Small, targeted, reversible. 3. **Waive the required check on this PR and merge** — the semantic change is a one-line default flip; it cannot itself cause a native SIGSEGV. 4. **Wait for Adoptium to ship a fix** — happy to file the bug at https://github.com/adoptium/adoptium-support/issues with a full `hs_err_pid*.log` if the workflow can be tweaked to upload it as an artifact on failure (currently not uploaded, so the full crashing-thread stack is not accessible). Happy to open a separate PR for whichever workaround the maintainers prefer. Not planning to keep re-running CI on this PR — 5 for 5 with identical signature has established the pattern. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
