How do I get rid of that restriction?
once you have fixed whatever the problem was on node 3, you can use:
crm_resource -C -H node3 -r name_of_the_resource_that_failed_there
BTW, in a long running cluster, even with sporadic failures, this can
accumulate over time. Say that due to whatever reason we have a start
failure (or, in the future, a monitor failure causing a failover) once
every two months or so. Still, after a year, suddenly all nodes will
have been "exhausted".
What I'm trying to say is that this "resource foo failed somewhere"
probably should have a timeout - say, a day or something by default.
Is that a sane suggestion? If so, I'll go file a bugzilla.
its not the craziest thing i've heard all week... by all means file an enhancement :-)