To compare behavior, we intentionally kept one resource (an OCF agent
whose start consistently fails) in a permanent start-failure state.
Can you show me the OCF agent and its configuration in the CIB so I can
try to reproduce this? Once I can, I'll dig into your bug report more
thoroughly. Thanks for the detail.
* A single resource that repeatedly fails 'start' drives Pacemaker
into a continuous stop -> start -> stop recovery loop (roughly every
30s in our setup).
* After *~2 days / ~2300 failed start attempts*, pacemaker-controld
crashed with signal 11 (SIGSEGV), and because the controld respawn
budget was exhausted, pacemakerd declared a fatal failure and shut
the whole stack down, and it stays down.
Hopefully we can figure this out so no one ever runs into a segfault
after two days, but I hate the idea of segfaults lurking in the code
anywhere. Can you show me a backtrace of that as well so we can take a
look?
1. Is this a known issue / already fixed after 3.0.3 in the controller
or IPC (libcrmcommon mainloop_gio / event-notification) code?
No, I don't think this has been encountered before. It definitely
hasn't been fixed.
2. Is there a recommended configuration workaround to avoid the fatal
path while a fix is pending (e.g. tuning around resource failure
handling)?
You could try changing the on-fail setting for the start action of the
resource to be "stop" instead of "restart". There's likely some
timeouts and thresholds you could configure as well, but I am not
familiar with all the details. You'd need to check "Pacemaker
Explained" or "Pacemaker Adimistration" for more details.
Is there any chance you've tried on other releases as well? There's a
*ton* of differences between 2.1.6 and 3.0.3:
$ git diff --stat Pacemaker-2.1.6..Pacemaker-3.0.3
...
3636 files changed, 222158 insertions(+), 149509 deletions(-)
In particular, if you tried any other 3.0.x releases, that would be
especially helpful to know.
- Chris
_______________________________________________
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users
ClusterLabs home: https://www.clusterlabs.org/
