weiqingy commented on issue #893: URL: https://github.com/apache/flink-agents/issues/893#issuecomment-4938513983
Update — the flake has a **dated onset**, which points at an external change (most likely the unpinned Ollama version) rather than anything in the repo, and revises the mitigation priority. `main`'s CI history: - 07-06: 2 runs, all pass - 07-07: 2 runs, all pass - 07-08: 4 runs, all pass - 07-09: no runs (nothing merged) - 07-10: 2 runs, **both fail** — and one of them is a docs-only change that cannot touch the inference path So `main` was green for eight straight runs through 07-08 and both runs on 07-10 failed with these same two signatures (one `llama-server` segfault, one py4j-1.20 gateway death). A regression introduced by a merge would have reddened the *first* main run after that merge; instead the failures appear on a day (07-09) when nothing merged. That rules out a repo change as the cause. The most likely trigger is the **unpinned Ollama version**: `tools/start_ollama_server.sh` installs the latest Ollama with no version lock, so an upstream release around 07-09 would land in CI automatically with no repo change — which matches the green-through-07-08 / red-from-07-10 boundary exactly. (I could not extract the exact installed version from the job logs to prove a version bump specifically — a runner-image update is a secondary possibility — so this is a strong hypothesis rather than a proven cause. The "external, dated onset, not a PR" conclusion is solid regardless.) **This changes the recommended first step.** An earlier read here was that pinning a version would only add determinism, based on same-binary mixed pass/fail within a single run. But a dated onset means something *changed*, so pinning Ollama back to the last-known-good version (whatever was latest around 07-06/07-08) is now the **cheapest thing to try first** and could restore green outright — before investing in the more involved restart-Ollama-on-crash approach. Suggested order: (1) identify the last-good Ollama version and pin `OLLAMA_VERSION`; (2) if segfaults persist at that pin, fall back to restart-on-crash; (3) the py4j-1.20 gateway flake is separate and may only warrant a tracking note. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
