Re: [Linux-ha-dev] Restart service on hb_takeover.
Everything is possible. Heartbeat is end of life, haresources mode even more so. No need to argue, I'm just stating facts here ;-) Probably the most easy way for you to achieve what you want: use yet an other wrapper script. If you elaborate on the actual problem, with some specific example, someone may come up with a better idea altogether. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [ClusterLabs Developers] moving cluster-glue to github
On Mon, Oct 10, 2016 at 12:07:48PM +0200, Kristoffer Grönlund wrote: > Adam Spiers <aspi...@suse.com> writes: > > > Kristoffer Gronlund <kgronl...@suse.com> wrote: > >> We've discussed moving cluster-glue to github.com and the ClusterLabs > >> organization, but no one has actually done it yet. ;) > > > > Out of curiosity what needs to be done for this, other than the > > obvious "git push" to github, and maybe updating a README / wiki page > > or two? > > > > The main thing would be to ensure that everyone who maintains it agrees > to the move. AFAIK at least Lars Ellenberg and Dejan are both in favor, > but I am not sure who else might be considered an owner of > cluster-glue. Appart from LINBIT, SuSE is the only relevant entity here, I'd say. We are apparently the only ones that (still) ship cluster-glue. I'd argue to "let cluster-glue die in peace" [*] Contributions, if any, will probably only be new stonith/external scripts. Bring them over to "fence_agents". Cannot be that difficult to generically bring "stonith/*" or even only "stonith/external" agents over to the "fence_agents" compatible API. Not even necessarily convert to python, probably one python wrapper would be good enough for all stonith/external style agents. [*] the same as you all let "heartbeat" (the membership and comm layer, not the concept) die in (mostly) peace. Yes, still working and maintained and all. Still alive. But not really kicking. It's not that there is a steady stream of changes and contributions for cluster glue. The only relevant commits in the last six years have been from Linbit (almost exclusively me), or from SuSE (almost exclusively by Dejan), in part inspired or on behalf of work or contributions by NTT. Apart from a few cosmetic patches, the only changes have been to "hb_report" (which pacemaker feels is obsoleted by crm_report, which was a fork of hb_report; probably any feature in hb_report that is not (yet/anymore) in crm_report could and should be merged back, and hb_report be dropped; maybe that has already happened). Now, with Dejan no longer being SuSE, that probably leaves lge, lmb, and krig to "agree" on anything there. I have no problem using git-remote-hg or mercurial-git to push it over, or even keep them in sync, having one reflect the other. Not that it makes any difference, really. But as I said, I'd prefer to leave it alone if possible. Some historical background for those interested: the original idea of "stonith plugins" was to have stonith agents available as a collection of binary shared objects that could be loaded and mlock'ed early on, so fencing would work even in "bad situations", without requiring additional IO or (much) allocations. Then that was weakened for the "stonith/external" agents, which would usually be (shell) scripts. The result was that usually only those scripts would be used. Now, the "fence agents" are a collection of python scripts, with few agents still being in C, but the idea of having the stonith mechanism be "preloaded" has been abandoned completely. Arguably, if you have not enough resources left to timely execute some fencing script, you should not be in control of the cluster, and certainly not try to take over more resources of the to-be-fenced node. -- : Lars Ellenberg : LINBIT | Keeping the Digital World Running : DRBD -- Heartbeat -- Corosync -- Pacemaker : R, Integration, Ops, Consulting, Support DRBD® and LINBIT® are registered trademarks of LINBIT ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Some minor patches for cluster-glue
On Tue, Aug 30, 2016 at 06:23:15PM +0200, Kristoffer Grönlund wrote: > Dejan Muhamedagic <deja...@fastmail.fm> writes: > > > > > Apparently, the patches as they are cannot be imported with "hg > > import", i.e. the metadata gets lost. Did you do "hg export"? Can > > you supply them with "hg export"? > > > > I thought I did.. let me try again. I think what Dejan was expecting is the result of "hg export", which should look more like # HG changeset patch # User Lars Ellenberg <l...@linbit.com> # Date 1413480257 -7200 # Thu Oct 16 19:24:17 2014 +0200 # Node ID 0a7add1d9996b6d869d441da6c82fb7b8abcef4f # Parent f2227d4971baed13958306b2c7cabec0eda93e82 fix syslogmsgfmt logging inconsistency for stderr/stdout ... not the output of "hg log -v -p", which looks like what you sent. Though the formats are very similar, and possibly could be massaged by hand, even, hg import is best used with the output created by hg export. Or sent dejan a hg bundle which he then can unbundle. Cheers, Lars > changeset: 2820:13875518ed6b > parent: 2815:643ac28499bd > user:Kristoffer Grönlund <kgronl...@suse.com> > date:Wed Aug 10 12:13:13 2016 +0200 > files: doc/stonith.xml.in > description: > Low: stonith: Update man page with -E, -m parameters (bsc#970307) ... ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
[Linux-ha-dev] Call for review of undocumented parameters in resource agent meta data
On Fri, Jan 30, 2015 at 09:52:49PM +0100, Dejan Muhamedagic wrote:
Hello,
We've tagged today (Jan 30) a new stable resource-agents release
(3.9.6) in the upstream repository.
Big thanks go to all contributors! Needless to say, without you
this release would not be possible.
Big thanks to Dejan.
Who once again finally did,
what I meant to do in late 2013 already, but simply pushed off
for over a year (and no-one else stepped up, either...)
So: Thank You.
I just today noticed that apparently some resource agents
accept and use parameters that are not documented in their meta data.
I now came up with a bash two-liner,
which likely still produces a lot of noise,
because it does not take into account that some agents
source additional helper files.
But here is the list:
--- used, but not described
+++ described, but apparently not used.
EvmsSCC +OCF_RESKEY_ignore_deprecation
Evmsd +OCF_RESKEY_ignore_deprecation
?? intentionally undocumented ??
IPaddr+OCF_RESKEY_iflabel
IPaddr-OCF_RESKEY_netmask
Not sure.
IPaddr2 -OCF_RESKEY_netmask
intentional, backward compat, quoting the agent:
# Note: We had a version out there for a while which used
# netmask instead of cidr_netmask. Don't remove this aliasing code!
Please help review these:
IPsrcaddr -OCF_RESKEY_ip
IPsrcaddr +OCF_RESKEY_cidr_netmask
IPv6addr.c-OCF_RESKEY_cidr_netmask
IPv6addr.c-OCF_RESKEY_ipv6addr
IPv6addr.c-OCF_RESKEY_nic
LinuxSCSI +OCF_RESKEY_ignore_deprecation
Squid -OCF_RESKEY_squid_confirm_trialcount
Squid -OCF_RESKEY_squid_opts
Squid -OCF_RESKEY_squid_suspend_trialcount
SysInfo -OCF_RESKEY_clone
WAS6 -OCF_RESKEY_profileName
apache+OCF_RESKEY_use_ipv6
conntrackd-OCF_RESKEY_conntrackd
dnsupdate -OCF_RESKEY_opts
dnsupdate +OCF_RESKEY_nsupdate_opts
docker-OCF_RESKEY_container
ethmonitor-OCF_RESKEY_check_level
ethmonitor-OCF_RESKEY_multiplicator
galera+OCF_RESKEY_additional_parameters
galera+OCF_RESKEY_binary
galera+OCF_RESKEY_client_binary
galera+OCF_RESKEY_config
galera+OCF_RESKEY_datadir
galera+OCF_RESKEY_enable_creation
galera+OCF_RESKEY_group
galera+OCF_RESKEY_log
galera+OCF_RESKEY_pid
galera+OCF_RESKEY_socket
galera+OCF_RESKEY_user
Probably all bogus, it source mysql-common.sh.
Someone please have a more detailed look.
iSCSILogicalUnit +OCF_RESKEY_product_id
iSCSILogicalUnit +OCF_RESKEY_vendor_id
false positive
surprise: florian learned some wizardry back then ;-)
for var in scsi_id scsi_sn vendor_id product_id; do
envar=OCF_RESKEY_${var}
if [ -n ${!envar} ]; then
params=${params} ${var}=${!envar}
fi
done
If such magic is used elsewhere,
that could mask Used but not documented cases.
iface-bridge -OCF_RESKEY_multicast_querier
!! Yep, that needs to be documented!
mysql-proxy -OCF_RESKEY_group
mysql-proxy -OCF_RESKEY_user
Oops, apparently my magic scriptlet below needs to learn to
ignore script comments...
named -OCF_RESKEY_rootdir
!! Probably a bug:
named_rootdir is documented.
nfsserver -OCF_RESKEY_nfs_notify_cmd
!! Yep, that needs to be documented!
nginx -OCF_RESKEY_client
nginx +OCF_RESKEY_testclient
!! client is used, but not documented,
!! testclient is documented, but unused...
Bug?
nginx -OCF_RESKEY_nginx
Bogus. Needs to be dropped from leading comment block.
oracle-OCF_RESKEY_tns_admin
!! Yep, that needs to be documented!
pingd +OCF_RESKEY_ignore_deprecation
?? intentionally undocumented ??
pingd -OCF_RESKEY_update
!! Yep, is undocumented.
sg_persist+OCF_RESKEY_binary
sg_persist-OCF_RESKEY_sg_persist_binary
!! BUG? binary vs sg_persist_binary
varnish -OCF_RESKEY_binary
!! Yep, is undocumented.
Please someone find the time to prepare pull requests
to fix these...
Thanks,
Lars
-
List was generated by below scriptlet,
which can be improved. The improved version should probably be part of
a unit test check, when building resource-agents.
# In the git checkout of the resource agents,
# get a list of files that look like actual agent scripts.
cd heartbeat
A=$(git ls-files | xargs grep -s -l 'resource-agent ')
# and for each of these files,
# diff the list of OCF_RESKEY_* occurrences
# with the list of parameter name=* ones.
for a in $A; do
diff -U0 \
( grep -h -o
[Linux-ha-dev] Announcing the Heartbeat 3.0.6 Release
TL;DR: If you intend to set up a new High Availability cluster using the Pacemaker cluster manager, you typically should not care for Heartbeat, but use recent releases (2.3.x) of Corosync. If you don't care for Heartbeat, don't read further. Unless you are beekhof... there's a question below ;-) After 3½ years since the last officially tagged release of Heartbeat, I have seen the need to do a new maintenance release. The Heartbeat 3.0.6 release tag: 3d59540cf28d and the change set it points to: cceeb47a7d8f The main reason for this was that pacemaker more recent than somewhere between 1.1.6 and 1.1.7 would no longer work properly on the Heartbeat cluster stack. Because some of the daemons have moved from glue to pacemaker proper, and changed their paths. This has been fixed in Heartbeat. And because during that time, stonith-ng was refactored, and would still reliably fence, but not understand its own confirmation message, so it was effectively broken. This I fixed in pacemaker. If you chose to run new Pacemaker with the Heartbeat communication stack, it should be at least 1.1.12 with a few patches, see my December 2014 commits at the top of https://github.com/lge/pacemaker/commits/linbit-cluster-stack-pcmk-1.1.12 I'm not sure if they got into pacemaker upstream yet. beekhof? Do I need to rebase? Or did I miss you merging these? --- If you have those patches, consider setting this new ha.cf configuration parameter: # If pacemaker crmd spawns the pengine itself, # it sometimes forgets to kill the pengine on shutdown, # which later may confuse the system after cluster restart. # Tell the system that Heartbeat is supposed to # control the pengine directly. crmd_spawns_pengine off Here is the shortened Heartbeat changelog, the longer version is available in mercurial: http://hg.linux-ha.org/heartbeat-STABLE_3_0/shortlog - fix emergency shutdown due to broken update_ackseq - fix node dead detection problems - fix converging of membership (ccm) - fix init script startup glitch (caused by changes in glue/resource-agents) - heartbeat.service file for systemd platforms - new ucast6 UDP IPv6 communication plugin - package ha_api.py in standard package - update some man pages, specifically the example ha.cf - also report ccm membership status for cl_status hbstatus -v - updated some log messages, or their log levels - reduce max_delay in broadcast client_status query to one second - apply various (mostly cosmetic) patches from Debian - drop HBcompress compression plugins: they are part of cluster glue - drop openais HBcomm plugin - better support for current pacemaker versions - try to not miss a SIGTERM (fix problem with very fast respawn/stop cycle) - dopd: ignore dead ping nodes - cl_status improvements - api internals: reduce IPC round-trips to get at status information - uid=root is sufficient to use heartbeat api (gid=haclient remains sufficient) - fix /dev/null as log- or debugfile setting - move daemon binaries into libexecdir - document movement of compression plugins into cluster-glue - fix usage of SO_REUSEPORT in ucast sockets - fix compile issues with recent gcc and -Werror Note that a number of the mentioned fixes have been created two years ago already, and may have been released in packages for a long time, where vendors have chosen to package them. As to future plans for Heartbeat: Heartbeat is still useful for non-pacemaker, haresources-mode clusters. We (Linbit) will maintain Heartbeat for the foreseeable future. That should not be too much of a burden, as it is stable, and due to long years of field exposure, all bugs are known ;-) The most notable shortcoming when using Heartbeat with Pacemaker clusters would be the limited message size. There are currently no plans to remove that limitation. With its wide choice of communications paths, even exotic communication plugins, and the ability to run arbitrarily many paths, some deployments may even favor it over Corosync still. But typically, for new deployments involving Pacemaker, in most cases you should chose Corosync 2.3.x as your membership and communication layer. For existing deployments using Heartbeat, upgrading to this Heartbeat version is strongly recommended. Thanks, Lars Ellenberg signature.asc Description: Digital signature ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] RFC: pidfile handling; current worst case: stop failure and node level fencing
On Tue, Oct 21, 2014 at 02:06:24PM +0100, Tim Small wrote: On 20/10/14 20:17, Lars Ellenberg wrote: In other OSes, ps may be able to give a good enough equivalent? Debian's start-stop-daemon executable might be worth considering here - it's used extensively in the init script infrastructure of Debian (and derivatives, over several different OS kernels), and so is well debugged, and in my experience beats re-implementing it's functionality. http://anonscm.debian.org/cgit/dpkg/dpkg.git/tree/utils/start-stop-daemon.c I've used it in pacemaker resource control scripts before successfully - it's kill expression support is very useful in particular on HA. Tim. NAME start-stop-daemon - start and stop system daemon programs Really? pasting a man page to a mailing list? But yes... If we want to require presence of start-stop-daemon, we could make all this somebody elses problem. I need find some time to browse through the code to see if it can be improved further. But in any case, using (a tool like) start-stop-daemon consistently throughout all RAs would improve the situation already. Do we want to do that? Dejan? David? Anyone? Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] RFC: pidfile handling; current worst case: stop failure and node level fencing
On Wed, Oct 22, 2014 at 03:09:12PM +0200, Dejan Muhamedagic wrote: On Wed, Oct 22, 2014 at 06:50:37AM -0600, Alan Robertson wrote: On 10/22/2014 03:33 AM, Dejan Muhamedagic wrote: Hi Alan, On Mon, Oct 20, 2014 at 02:52:13PM -0600, Alan Robertson wrote: For the Assimilation code I use the full pathname of the binary from /proc to tell if it's one of mine. That's not perfect if you're using an interpreted language. It works quite well for compiled languages. Yes, though not perfect, that may be good enough. I supposed that the probability that the very same program gets the same recycled pid is rather low. (Or is it?) From my 'C' code I could touch the lock file to match the timestamp of the /proc/pid/stat (or /proc/pid/exe) symlink -- and verify that they match. If there is no /proc/pid/stat, then you won't get that extra safeguard. But as you suggest, it decreases the probability by orders of magnitude even without the The /proc/pid/exe symlink appears to have the same timestamp as /proc/pid/stat Hmm, not here: $ sudo ls -lt /proc/1 ... lrwxrwxrwx 1 root root 0 Aug 27 13:51 exe - /sbin/init dr-x-- 2 root root 0 Aug 27 13:51 fd -r--r--r-- 1 root root 0 Aug 27 13:20 cmdline -r--r--r-- 1 root root 0 Aug 27 13:18 stat We can not rely on properties of the inodes in /proc/. These inodes get dropped and recreated as the system sees fit. and their properties re-initialized to something. Ok, the uid/gid is consistent, obviously. But neither inode numbers or a,m,ctime is stable. I demo'ed that in my first email, I demo it again here: sleep 120 k=$! ; stat /proc/$k ; echo 3 /proc/sys/vm/drop_caches ; sleep 2; find /proc/ -ls /dev/null; stat /proc/$k File: `/proc/8862' Size: 0 Blocks: 0 IO Block: 1024 directory Device: 3h/3d Inode: 4295899 Links: 8 Access: (0555/dr-xr-xr-x) Uid: (0/root) Gid: (0/root) Access: 2014-10-23 18:43:25.53506 +0200 Modify: 2014-10-23 18:43:25.53506 +0200 Change: 2014-10-23 18:43:25.53506 +0200 File: `/proc/8862' Size: 0 Blocks: 0 IO Block: 1024 directory Device: 3h/3d Inode: 4296016 Links: 8 Access: (0555/dr-xr-xr-x) Uid: (0/root) Gid: (0/root) Access: 2014-10-23 18:43:27.561002753 +0200 Modify: 2014-10-23 18:43:27.561002753 +0200 Change: 2014-10-23 18:43:27.561002753 +0200 Note how the inode number and a,m,ctime changes. the starttime I was talking about is the 22nd field of /proc/$pid/stat see proc(5): starttime %llu (was %lu before Linux 2.6) (22) The time the process started after system boot. In kernels before Linux 2.6, this value was expressed in jiffies. Since Linux 2.6, the value is expressed in clock ticks (divide by sysconf(_SC_CLK_TCK)). Thats a monotonic time counting from system boot. Which makes it so attractive. Even if someone fiddles with date --set (or ntp or ...), even if that would be done on purpose, this field would not care. Anyways: making this somebody elses problem, using (a tool like) start-stop-daemon, require that to be present, and help make that do the best thing possible, and as portable as possible, could be a good way to go. Still the cd /proc/$pid, then work from there would avoid various race conditions nicely. Where available, open() then openat() will do nicely, as well, no need to chdir. So the quick fix to solve the issue that triggered the discussion (not noticing that a pid has died): is my first suggestion: # wait for pid to die: - while kill -0 $pid; do sleep 1; done + ( if cd /proc/$pid ; then while test -d . ; do sleep 1; done ; fi ) /dev/null Should we do an ocf-shellfuncs helper for this? Suggested names? What should go in there -- only the waiting, the kill TERM? A timeout? Escalation to kill KILL? Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] RFC: pidfile handling; current worst case: stop failure and node level fencing
On Wed, Oct 22, 2014 at 02:11:21PM +0100, Tim Small wrote: On 22/10/14 13:50, Alan Robertson wrote: Does anyone know which OSes have either or both of those /proc names? Once again, can I recommend taking a look at the start-stop-daemon source (see earlier posting), which does this stuff, and includes checks for Linux/Hurd/Sun/OpenBSD/FreeBSD/NetBSD/DragonFly, and whilst I've only ever used it on Linux, at the very least the BSD side seems to be maintained: http://anonscm.debian.org/cgit/dpkg/dpkg.git/tree/utils/start-stop-daemon.c Does not solve the problem I was talking about at all. If you only have a pid, it has the exact same problem, it may miss the pid dead event because of pid recycling. If you are more specific (user, parent pid, exe name...) it becomes less and less likely -- but it would still be possible. So you may want to add a similar trick to pid_fd = open(/proc/pid); fstat(pid_fd) ... Yet an other crazy idea, at least for linux: do not poll, monitor! -- CONFIG_PROC_EVENTS, subscribe to CN_IDX_PROC, wait for PROC_EVENT_EXIT ;-) No, seriously, that's too much trouble to replace a shell oneliner... If however that would be added to start-stop-daemon, it could drop the funky algorithm for the kill -0 polling. (which still would need to be supported, because of old kernels) Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] RFC: pidfile handling; current worst case: stop failure and node level fencing
On Mon, Oct 20, 2014 at 11:21:36PM +0200, Lars Ellenberg wrote: On Mon, Oct 20, 2014 at 03:04:31PM -0600, Alan Robertson wrote: On 10/20/2014 02:52 PM, Alan Robertson wrote: For the Assimilation code I use the full pathname of the binary from /proc to tell if it's one of mine. That's not perfect if you're using an interpreted language. It works quite well for compiled languages. It works just as well (or as bad) from interpreted languages: readlink /proc/$pid/exe (very old linux has a fsid:inode encoding there, but I digress) But that does solve a different subset of problems, has race conditions in itself, and breaks if you have updated the binary since start of that service (which does happen). Sorry, I lost the original. Alan then wrote: It only breaks if you change the *name* of the binary. Updating the binary contents has no effect. Changing the name of the binary is pretty unusual - or so it seems to me. Did I miss something? And if you do, you should stop with the binary with the old version and start it with the new one. Very few methods are going to deal well with radical changes in the service without stopping it with the old script, updating, and starting with the new script. Well, the pid starttime method does... I don't believe I see the race condition. Does not matter. It won't loop, and it's not fooled by pid wraparound. What else are you looking for? [Guess I missed something else here] pid + exe is certainly is better than the pid alone. It may even be good enough. But it still has shortcomings. /proc/pid/exe is not stable, (changes to deleted if the binary is deleted) could be accounted for. /proc/pid/exe links to the interpreter (python, bash, java, whatever) Even if it is a real binary, (pid, /proc/pid/exe) is still NOT unique for pid re-use after wrap around: think different instances of mysql or whatever. (yes, it gets increasingly unlikely...) However, (pid, starttime) *is* unique (for the lifetime of the pidfile, as long as that is stored on tmpfs resp. cleared after reboot). (unless you tell me you can eat through pid_max, or at least the currently unused pids, within the granularity of starttime...) So that's why I propose to use (pid, starttime) tuple. If you see problems with (pid, starttime), please speak up. If you have something *better*, please speak up. If you just have something different, feel free to tell us anyways :-) Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
[Linux-ha-dev] RFC: pidfile handling; current worst case: stop failure and node level fencing
Recent discussions with Dejan made me again more prominently aware of a
few issues we probably all know about, but usually dismis as having not
much relevance in the real-world.
The facts:
* a pidfile typically only stores a pid
* a pidfile may stale, not properly cleaned up
when the pid it references died.
* pids are recycled
This is more an issue if kernel.pid_max is small
wrt the number of processes created per unit time,
for example on some embeded systems,
or on some very busy systems.
But it may be an issue on any system,
even a mostly idle one, given bad luck^W timing,
see below.
A common idiom in resource agents is to
kill_that_pid_and_wait_until_dead()
{
local pid=$1
is_alive $pid || return 0
kill -TERM $pid
while is_alive $pid ; sleep 1; done
return 0
}
The naïve implementation of is_alive() is
is_alive() { kill -0 $1 ; }
This is the main issue:
---
If the last-used-pid is just a bit smaller then $pid,
during the sleep 1, $pid may die,
and the OS may already have created a new process with that exact pid.
Using above is_alive, kill_that_pid() will not notice that the
to-be-killed pid has actually terminated while that new process runs.
Which may be a very long time if that is some other long running daemon.
This may result in stop failure and resulting node level fencing.
The question is, which better way do we have to detect if some pid died
after we killed it. Or, related, and even better: how to detect if the
process currently running with some pid is in fact still the process
referenced by the pidfile.
I have two suggestions.
(I am trying to avoid bashisms in here.
But maybe I overlook some.
Also, the code is typed, not sourced from some working script,
so there may be logic bugs and typos.
My intent should be obvious enough, though.)
using cd /proc/$pid; stat .
-
# this is most likely linux specific
kill_that_pid_and_wait_until_dead()
{
local pid=$1
(
cd /proc/$pid || return 0
kill -TERM $pid
while stat . ; sleep 1; done
)
return 0
}
Once pid dies, /proc/$pid will become stale (but not completely go away,
because it is our cwd), and stat . will return No such process.
Variants:
using test -ef
--
exec 7/proc/$pid || return 0
kill -TERM $pid
while :; do
exec 8/proc/$pid || break
test /proc/self/fd/7 -ef /proc/self/fd/8 || break
sleep 1
done
exec 7- 8-
using stat -c %Y /proc/$pid
---
ctime0=$(stat -c %Y /proc/$pid)
kill -TERM $pid
while ctime=$(stat -c %Y /proc/$pid) [ $ctime = $ctime0 ] ; do sleep
1; done
Why not use the inode number I hear you say.
Because it is not stable. Sorry.
Don't believe me? Don't want to read kernel source?
Try it yourself:
sleep 120 k=$!
stat /proc/$k
echo 3 /proc/sys/vm/drop_caches
stat /proc/$k
But that leads me to an other proposal:
store the starttime together with the pid in a pidfile.
For linux that would be:
(see proc(5) for /proc/pid/stat field meanings.
note that (comm) may contain both whitespace and ),
which is the reason for my sed | cut below)
spawn_create_exclusive_pid_starttime()
{
local pidfile=$1
shift
local reset
case $- in *C*) reset=:;; *) set -C; reset=set +C;; esac
if ! exec 3$pidfile ; then
$reset
return 1
fi
$reset
setsid sh -c '
read pid _ /proc/self/stat
starttime=$(sed -e 's/^.*) //' /proc/$pid/stat | cut -d' ' -f
20)
3 echo $pid $starttime
3- exec $@
' -- $@
return 0
}
It does not seem possible to cycle through all available pids
within fractions of time smaller than the granularity of starttime,
so pid starttime should be a unique tuple (until the next reboot --
at least on linux, starttime is measured as strictly monotonic uptime).
If we have pid starttime in the pidfile,
we can:
get_proc_pid_starttime()
{
proc_pid_starttime=$(sed -e 's/^.*) //' /proc/$pid/stat) || return 1
proc_pid_starttime=$(echo $proc_pid_starttime | cut -d' ' -f 20)
}
kill_using_pidfile()
{
local pidfile=$1
local pid starttime proc_pid_starttime
test -e $pidfile|| return # already dead
read pid starttime $pidfile|| return # unreadable
# check pid and starttime are both present, numeric only, ...
# I have a version that distinguishes 16 distinct error
# conditions; this is the short version only...
local i=0
while
get_proc_pid_starttime
[ $starttime = $proc_pid_starttime ]
do
: $(( i+=1 ))
Re: [Linux-ha-dev] RFC: pidfile handling; current worst case: stop failure and node level fencing
On Mon, Oct 20, 2014 at 03:04:31PM -0600, Alan Robertson wrote: On 10/20/2014 02:52 PM, Alan Robertson wrote: For the Assimilation code I use the full pathname of the binary from /proc to tell if it's one of mine. That's not perfect if you're using an interpreted language. It works quite well for compiled languages. It works just as well (or as bad) from interpreted languages: readlink /proc/$pid/exe (very old linux has a fsid:inode encoding there, but I digress) But that does solve a different subset of problems, has race conditions in itself, and breaks if you have updated the binary since start of that service (which does happen). It does not fully address what I am talking about. I somehow missed that you were talking about resource agents. Not exclusively. But any solution should easily work for any caller and an easily unserstood set of conventions, regarless of implementation language. But shouldn't the CRM guarantee that no more than one is running anyway - making some conditions a lot less likely? You miss the point. Point being: kill -TERM $pid while kill -0 $pid ; do sleep 1; done may never terminate. If you care, the source for the code I mentioned is here: http://hg.linux-ha.org/assimilation/file/tip/clientlib/misc.c Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] support LSBs /run directory
On Fri, Dec 20, 2013 at 12:20:26PM +0100, Timur I. Bakeyev wrote:
On Wed, Dec 18, 2013 at 2:08 PM, Serge Dubrouski serge...@gmail.com wrote:
Lars -
I'm a bit lost on path=$varrun/$path. Why do we need to add $varrun if
it doesn't present? The goal is to cover the case when /run , or any other
directory, is used instead of /var/run
Ignore for a moment that I used the variable name varrun.
/run is used instead of /var/run, and /var/run is a symlink to /run. Within
a distribution RUNDIR is, kind of, hardcoded, no matter what is it.
That's why I used the @@__autofoo_notation__@@ and I'm too lazy to look
up what the correct name between @@ @@ would be; last time I checked,
people where still discussing --rundir, --varrundir, --runstatedir and
others.
Anyways, that's supposed to be the distribution wide hardcoded default,
and that would be replaced by autofoo/configure at build time of this
package.
I think we have it already in ocf-directories.in as
: ${HA_VARRUN:=@localstatedir@/run/}
and we probably should change that to
: ${HA_RUNSTATEDIR:=@runstatedir@}
(or HA_RUNDIR:=@rundir@; whatever will be the proper autofoo name for it)
and keep the alias
HA_VARRUN=$HA_RUNSTATEDIR
And now, assume that this variable, despite the name,
has a value of /run on those distributions that use /run,
and /var/run on those that use that,
and $home/var/run or $prefix/var/run when people install locally
(not that that'd make much sense for resoruce-agents...)
Ok?
So real question is to distinct usage of the plain RUNDIR and sub-directory
under it, as later one requires some special handling on memory based file
systems.
With regards,
Timur Bakeyev.
On Tue, Dec 17, 2013 at 1:48 PM, Timur I. Bakeyev ti...@com.bat.ruwrote:
Hi, Lars!
On Tue, Dec 17, 2013 at 1:43 PM, Lars Ellenberg
lars.ellenb...@linbit.com wrote:
On Tue, Dec 17, 2013 at 02:39:52AM +0100, Timur I. Bakeyev wrote:
Hi, guys!
Any reaction, please?
Probably best to add a helper to ocf-functions, say,
# require_run_dir mode user:group path
require_run_dir()
{
local mode=$1 owner=$2 path=$3
local $varrun=@@__varrun_or_whatever_autofoo_calls_it__@@
case $path in
$varrun/*) : nothing ;;
*)
path=$varrun/$path ;;
esac
test -d $path return 0
[ $(id -u) = 0 ] || return 1
# (or some helper function mkdir_p, in case we doubt -p is
available...)
mkdir -p $path chown $owner $path chmod $mode $path
}
Then use that early in the various resource agents,
maybe where the defaults are defined.
Yes?
That would be even better! There are few more RAs that would benefit from
that.
I'd only invert the parameters, as path is mandatory, permissions are
semi-optional and owner, as in 99% we run as root - optional. And I'd put
some meaningful defaults for the later two parameters:
local path=$1 mode=$2 owner=$3
: ${mode:=0755}
: ${owner:=root}
Also, as 'path' is usually is smth. like '/var/run/named' or
'/var/run/zabbix' I'm afraid that switch will do nothing in any case.
need a bit more magic, smth. like:
if [ -n ${path##$varrun} ]
or alike.
With best regards,
Timur.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
--
Serge Dubrouski.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Fwd: ldirectord fails to test HTTPS real servers.
On Fri, Dec 20, 2013 at 12:31:05PM +0100, Timur I. Bakeyev wrote: So maybe exporting PERL_LWP_SSL_VERIFY_HOSTNAME=0 somewhere is the more elegant solution? That was my original idea, but playing with ENV variables within ldirectord sounds a bit dangerous. We can play with $ua-ssl_opts( $key = $value ); instead. Not sure, if it exists in the previous versions of LWP, so, possibly, have to be verified with 'can' call. Also, it seems, that similar code was committed to the repository 2 months ago: https://github.com/ClusterLabs/resource-agents/commit/b4bb755ec4fcfb2c5900282d81f9fa88bf86 In that case, your issue 361 does no longer apply, because already fixed, right? And we could wait for the next issue complaining about too noisy warnings on older LWP versions. Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] support LSBs /run directory
On Wed, Dec 18, 2013 at 06:08:44AM -0700, Serge Dubrouski wrote:
Lars -
I'm a bit lost on path=$varrun/$path. Why do we need to add $varrun if it
doesn't present? The goal is to cover the case when /run , or any other
directory, is used instead of /var/run
Intention was to not hardcode paths in the resource agents,
when those paths may depend on build-time configuration of the software
they are supposed to control.
You'd only do require_run_dir httpd,
and that would create /var/run/httpd, /run/httpd/,
/usr/local/whatever-I-told-autofoo-at-build-time/httpd
appropriately.
But in case the prefix is already there, do not double the prefix.
This is to avoid distribution (version) specific resource agents,
or the transition of all resource agents to *.in.
If that makes sense.
Lars
On Tue, Dec 17, 2013 at 1:48 PM, Timur I. Bakeyev ti...@com.bat.ru wrote:
Hi, Lars!
On Tue, Dec 17, 2013 at 1:43 PM, Lars Ellenberg lars.ellenb...@linbit.com
wrote:
On Tue, Dec 17, 2013 at 02:39:52AM +0100, Timur I. Bakeyev wrote:
Hi, guys!
Any reaction, please?
Probably best to add a helper to ocf-functions, say,
# require_run_dir mode user:group path
require_run_dir()
{
local mode=$1 owner=$2 path=$3
local $varrun=@@__varrun_or_whatever_autofoo_calls_it__@@
case $path in
$varrun/*) : nothing ;;
*)
path=$varrun/$path ;;
esac
test -d $path return 0
[ $(id -u) = 0 ] || return 1
# (or some helper function mkdir_p, in case we doubt -p is
available...)
mkdir -p $path chown $owner $path chmod $mode $path
}
Then use that early in the various resource agents,
maybe where the defaults are defined.
Yes?
That would be even better! There are few more RAs that would benefit from
that.
I'd only invert the parameters, as path is mandatory, permissions are
semi-optional and owner, as in 99% we run as root - optional. And I'd put
some meaningful defaults for the later two parameters:
local path=$1 mode=$2 owner=$3
: ${mode:=0755}
: ${owner:=root}
Also, as 'path' is usually is smth. like '/var/run/named' or
'/var/run/zabbix' I'm afraid that switch will do nothing in any case.
need a bit more magic, smth. like:
if [ -n ${path##$varrun} ]
or alike.
With best regards,
Timur.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
--
Serge Dubrouski.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Fwd: ldirectord fails to test HTTPS real servers.
On Tue, Dec 03, 2013 at 12:57:24PM +0100, Timur I. Bakeyev wrote:
Hi guys!
I've posted bug report regarding ldirectord, can you please review it and
commit, if possible?
https://github.com/ClusterLabs/resource-agents/issues/361
Ldirectord is using LWP for it's negotiate checks for the HTTP/HTTPS sites.
Since LWP 6.0 by default it verifies the correspondence of the SSL
certificate and the server hostname. In 99.9% of the cases this is the VIP
hostname and RIP are identified by their internal hostnames or, most common
- by their IP addresses.
That breaks hostname verification and hence - marks HTTPS backends as
invalid and kicks them off the pool. This problem did hit me in the
production when we've upgraded from Debian squeeze to Debian wheezy, which
brought newer version of LWP.
http://search.cpan.org/~gaas/LWP-Protocol-https-6.04/lib/LWP/Protocol/https.pm
Luckily, the fix to the problem is easy:
--- ldirectord.orig 2013-12-03 11:59:11.114983525 +0100
+++ ldirectord 2013-12-03 11:59:34.703026282 +0100
@@ -2834,7 +2834,7 @@
ld_debug(2, check_http: url=\$$r{url}\
. virtualhost=\$virtualhost\);
- my $ua = new LWP::UserAgent();
+ my $ua = new LWP::UserAgent(ssl_opts = { verify_hostname = 0 });
my $h = undef;
if ($$v{service} eq http_proxy) {
I haven't verified that with older version of LWP, but I believe it should
just ignore unknown parameters to the constructor.
It does ignore, but warn, which will cause all sorts of complaints, I guess.
The more recent LWP::UserAgent states:
verify_hostname = $bool
When TRUE LWP will for secure protocol schemes ensure it connects
to servers that have a valid certificate matching the expected
hostname. If FALSE no checks are made and you can't be sure that
you communicate with the expected peer. The no checks behaviour
was the default for libwww-perl-5.837 and earlier releases.
This option is initialized from the PERL_LWP_SSL_VERIFY_HOSTNAME
environment variable. If this environment variable isn't set;
then verify_hostname defaults to 1.
So maybe exporting PERL_LWP_SSL_VERIFY_HOSTNAME=0 somewhere
is the more elegant solution?
Lars
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Linux-HA] announcement: planning resource-agents release 3.9.6
On Mon, Sep 30, 2013 at 03:33:35PM +0200, Dejan Muhamedagic wrote: Hello, We released resource-agents v3.9.5 back in February. In the meantime there have been quite a few fixes and new features pushed to the repository and it is high time for another release. Lars Ellenberg will run the release this time and do whatever is necessary that we have a good set of resource agents. Thanks Lars! Is that so ;-) I thought I'd only try to poke all contributers, authors and maintainers, as well as the community, to either raise issues now, or don't get to complain about it later ;) Two milestones were created at github.com today and this is the tentative schedule: 3.9.6-rc1: October 9. 3.9.6: October 16. If there's anything you think should be part of the release please open an issue, a pull request, or a bugzilla, as you see fit. If there's anything that hasn't received due attention, please let us know. Finally, if you can help with resolving issues consider yourself invited to do so. Thanks, -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] High: ccm: fix a memory leak when a client exits
On Wed, Sep 04, 2013 at 08:16:44PM +0900, Keisuke MORI wrote: Hi, The attached patch will fix a memory leak in ccm that occurs whenever a ccm client disconnect. Thank you. This may introduce double free for client_delete_all() now? All this aparently useless indirection seems to be from a time when client_destroy explicitly called into a -ops-destroy virtual function. Which it no longer does. So I think dropping the explicit calls to client_destroy, as well as the other then useless indirection functions, but instead do a g_hash_table_new_full with g_free in client_init would be the way to go. Could you have a look? It would not affect to most of the installations because only crmd and cib are the client, but if you run any ccm client such as crm_node command periodically, ccm will increase its memory consumption. The valgrind outputs are also attached as the evidence of the leakage and the fix by the patch; The results are taken after crm_node command is executed 100 times. There still exists some definitely / indirectly / possibly lost , but as long as I've investigated they are all allocated only at the invocation time and not considered as a leak. Double checks are welcome. Thanks, -- Keisuke MORI Cheers, Lars -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] Patch to make Route RA IPv6 ready
On Thu, May 30, 2013 at 09:36:34AM +0200, Robert Sander wrote: This little patch makes the Route RA IPv6 ready. Sorry for the huge delay. Will make sure this is merged right away. Thanks, Lars -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Linux-HA] LVM Resource agent, exclusive activation - implementation bugs
On Tue, May 14, 2013 at 01:22:08PM +0200, Lars Ellenberg wrote:
[ on linux-ha,
a lot of stuff about why I think we don't even want that feature,
or at least not how it is implemented ]
This post is related, but about the implementation bugs in the current
implementation as I spotted them. So this time to linux-ha-dev.
If you want me to trimm the Cc, just say so ;-)
So assuming we even want this feature, here are some bugs I found:
--
in lvm_vg.sh,
vg_stop_tagged()
probably only strip, if you are the owner,
NOT if you could claim it.
vg_tag_owner
-if [ $? -ne 0 ] ; then
+if [ $? = 1 ] ; then
strip_tags
--
I see quite a number of
lvs --options | while read line; do
some_variable=whatever
possibly return $SOMETHING
done
refer to $some_variable
e.g in vg_start_exclusive().
THIS DOES NOT WORK.
the whole while loop, because it reads from a pipe,
executes in a subshell.
variable assignments do not propagate to the main script,
return values do not return from the current function,
but only from the subshell, changing the exit code of that pipeline only.
Note that I think most (all?) of these slipped in
with the attempt to
Low: LVM: Restore LVM agent portability
so I guess the original is probably ok, doing
result=($(subcommand));
while [ ! -z ${result[iterator]} ] ...
See the example further down for how to do that
same thing in portable shell.
# BROKEN
t() {
var=initial-value
echo bla | while read line; do
var=other-value
return 42
done
echo still here
}
t
echo return code: $?
echo var=$var
output:
still here
return code: 0
var=initial-value
D'oh. All that fine error handling for nothing :-(
Assuming they are otherwise correct,
just convert all those into
for some_var in $(lvs --options); do
done
That should do what I think you mean.
# OK
t() {
var=initial-value
for line in $(echo bla); do
var=other-value
return 42
done
echo still here?
}
t
echo return code: $?
echo var=$var
output:
return code: 42
var=other-value
For the two (or more) field case,
where the original used bash array variables,
you can use the positional argument array.
Do something like this:
t()
{
local result name attr
# doing it this way, first declare local,
# then do the backtick assign
# preserves the exit code
result=$(lvs --noheadings -o name,attr $whatever)
# so you can now check the exit status, if you want to.
# then assign to positional argument array
set -- $result
# if $# % 2 != 0 then panic ;-)
while [ $# -ge 2 ]; do
name=$1 attr=$2
shift 2
do-what-you-have-to
case $attr in
[rR]???p)
it-is-a-partial ;;
[mM]???p)
a-partial-we-may-need-to-repair ;;
*a?)
: it-is-active-great ;;
*)
not-activated-now-what ;;
esac
done
}
shell can split lines to words and do pattern matching just fine...
no need to call out to echo | awk | grep, unless word splitting and
patterns are not sufficient.
Or, of course, if we decide we so deperatly want this feature,
maybe just ignore the /bin/bash != /bin/sh is a regression argument,
use the hopefully proven bash code from the rgmanager branch,
drop that compat commit,
and ignore people whining about bash being bloated.
This is Linux specific code, and I very much doubt there is a linux
shipping LVM + pacemaker but no bash.
-
I have to stop this now.
But if I come back later,
I'll followup with whatever else I find.
Cheers,
Lars
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] cluster-glue memory leak
: NewPILInterfaceUniv (pils.c:1723) ==3484==by 0x4E672DC: NewPILPluginUniv (pils.c:487) ==3484==by 0x9EB8FE3: init_pluginsys (stonith.c:75) ==3484==by 0x9EB90EC: stonith_new (stonith.c:105) ==3484==by 0x3F51008137: get_stonith_provider (st_client.c:1434) ==3484==by 0x3F51006E28: stonith_api_device_metadata (st_client.c:1059) ==3484==by 0x3F52407E22: stonith_get_metadata (lrmd_client.c:1478) ==3484==by 0x3F52408DB6: lrmd_api_get_metadata (lrmd_client.c:1736) ==3484==by 0x427FB2: lrm_state_get_metadata (lrm_state.c:555) ==3484==by 0x41F991: get_rsc_metadata (lrm.c:436) ==3484==by 0x41FCD4: get_rsc_restart_list (lrm.c:521) ==3484==by 0x4201B0: append_restart_list (lrm.c:607) ==3484==by 0x420670: build_operation_update (lrm.c:672) ==3484==by 0x425AE1: do_update_resource (lrm.c:1906) ==3484==by 0x42622E: process_lrm_event (lrm.c:2016) ==3484==by 0x41EE10: lrm_op_callback (lrm.c:242) ==3484==by 0x3F52404339: lrmd_dispatch_internal (lrmd_client.c:289) ==3484==by 0x3F524043DF: lrmd_ipc_dispatch (lrmd_client.c:311) ==3484==by 0x3F504308A9: mainloop_gio_callback (mainloop.c:587) ==3484==by 0x373FA38F0D: g_main_context_dispatch (gmain.c:1960) ==3484==by 0x373FA3C937: g_main_context_iterate (gmain.c:2591) -- Yuichi SEINO METROSYSTEMS CORPORATION E-mail:seino.clust...@gmail.com -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Linux-HA] Announcing release 0.1.0 of the Assimilation Monitoring Project!
On Fri, Apr 19, 2013 at 03:01:51PM -0600, Alan Robertson wrote: Hi all, Hi Alan! Good to see progress on this Project. Did you know about NeDi www.nedi.ch ? I put Remo Rickli on Cc; aparently NeDi prefers forum over mailing list. Nedi has a few years head start, and a different focus maybe. But as both projects seem to have a lot in common at least for the discovery part, you still should be able to find some potential synergies, or at least productively cooperate. Cheers, Lars This announcement is likely to be of interest to people like you who are concerned about availability. I founded the Linux-HA project in 1998 and led it for nearly 10 years. Back in about November 2010, I announced the beginnings of what would become the Assimilation Monitoring Project on this mailing list. The Assimilation Monitoring project [http://assimmon.org http://assimmon.org/] is a new open source monitoring project with a revolutionary architecture. It provides highly scalable [~*/O/*(1)] monitoring driven by integrated continuous Stealth Discovery(TM). This first release is intended as a proof of concept, to demonstrate the architecture, get feedback, add early adopters, and grow the community. The project has basically two thrusts: * It provides /extremely/ scalable exception monitoring (100K servers -- no problem) * It discovers all the details of your infrastructure (servers, services, dependencies, switches, switch port connections, etc.), builds a Neo4j graph database of all the gory details and updates it as things change - without setting off network security alarms. * The two functions are integrated in a way that will permit much easier configuration than traditional systems, and support the creation of simple audits to see if everything is being monitored. Release description: http://linux-ha.org/source-doc/assimilation/html/_release_descriptions.html Technology video:http://bit.ly/OD6bY6 TechTarget Interview: http://bit.ly/17M6DK2 Join the mailing list: http://lists.community.tummy.com/cgi-bin/mailman/listinfo/assimilation Join the mailing list, download the code, try it out, and send your comments and questions to the list! Thanks and have a great weekend! -- Alan Robertson al...@unix.sh - @OSSAlanR Openness is the foundation and preservative of friendship... Let me claim from you at all times your undisguised opinions. - William Wilberforce ___ Linux-HA mailing list linux...@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha See also: http://linux-ha.org/ReportingProblems -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH][crmsh] deal with the case-insentive hostname
On Wed, Apr 10, 2013 at 06:13:45PM +0900, Junko IKEDA wrote: Hi, I set upper-case hostname (GUEST03/GUEST4) and run Pacemaker 1.1.9 + Corosync 2.3.0. [root@GUEST04 ~]# crm_mon -1 Last updated: Wed Apr 10 15:12:48 2013 Last change: Wed Apr 10 14:02:36 2013 via crmd on GUEST04 Stack: corosync Current DC: GUEST04 (3232242817) - partition with quorum Version: 1.1.9-e8caee8 2 Nodes configured, unknown expected votes 1 Resources configured. Online: [ GUEST03 GUEST04 ] dummy (ocf::pacemaker:Dummy): Started GUEST03 for example, call crm shell with lower-case hostname. [root@GUEST04 ~]# crm node standby guest03 ERROR: bad lifetime: guest03 crm node standby GUEST03 surely works well, so crm shell just doesn't take into account the hostname conversion. It's better to accept the both of the upper/lower-case. node standby, node delete, resource migrate(move) get hit with this issue. Please see the attached. Thanks, Junko Sorry for the late reaction. diff -r da93d3523e6a modules/ui.py.in --- a/modules/ui.py.inTue Mar 26 11:44:17 2013 +0100 +++ b/modules/ui.py.inMon Apr 08 17:49:00 2013 +0900 @@ -924,10 +924,14 @@ lifetime = None opt_l = fetch_opts(argl, [force]) if len(argl) == 1: -if not argl[0] in listnodes(): -lifetime = argl[0] -else: -node = argl[0] + for i in listnodes(): + pattern = re.compile(i, re.IGNORECASE) + if pattern.match(argl[1]) and len(i) == len(argl[1]): + node = argl[1] This is not exactly equivalent. Before, we had a string comparison. Now we have a regexp match. This may be considered as a new feature. But it should then be done intentionally. Otherwise, i would need to be quote-metaed first. In Perl I'd write \Q$i\E, in python we probably have to insert some '\' into it first. I admit in most setups it would not make any difference, as there should at most be dots in there ., and they should be at places where they won't be ambiguous, especially with the additional len() check. Maybe rather compare argl[0].lower() with listnodes(), which should also return all elements as .lower(). Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] ManageVE prints bogus errors to the syslog
On Fri, Apr 05, 2013 at 12:39:46PM +0200, Dejan Muhamedagic wrote:
Hi Lars,
On Thu, Apr 04, 2013 at 09:28:00PM +0200, Lars Ellenberg wrote:
On Wed, Apr 03, 2013 at 06:25:58PM +0200, Dejan Muhamedagic wrote:
Hi,
On Fri, Mar 22, 2013 at 08:41:30AM +0100, Roman Haefeli wrote:
Hi,
When stopping a node of our cluster managing a bunch of OpenVZ CTs, I
get a lot of such messages in the syslog:
Mar 20 17:20:44 localhost ManageVE[2586]: ERROR: vzctl status 10002
returned: 10002 does not exist.
Mar 20 17:20:44 localhost lrmd: [2547]: info: operation monitor[6] on
opensim for client 2550: pid 2586 exited with return code 7
It looks to me as if lrmd is making sure the CT is not running anymore.
However, this triggers ManageVE to print an error.
Could be. Looking at the RA, there's a bunch of places where the
status is invoked and where this message could get logged. It
could be improved. The following patch should help:
https://github.com/ClusterLabs/resource-agents/commit/ca987afd35226145f48fb31bef911aa3ed3b6015
BTW, why call `vzctl | awk` *twice*,
just to get two items out of the vzctl output?
how about lose the awk, and the second invokation?
something like this:
(should veexists and vestatus be local as well?)
diff --git a/heartbeat/ManageVE b/heartbeat/ManageVE
index 56a3d03..53f9bab 100755
--- a/heartbeat/ManageVE
+++ b/heartbeat/ManageVE
@@ -182,10 +182,12 @@ migrate_from_ve()
status_ve()
{
declare -i retcode
-
- veexists=`$VZCTL status $VEID 2/dev/null | $AWK '{print $3}'`
- vestatus=`$VZCTL status $VEID 2/dev/null | $AWK '{print $5}'`
+ local vzstatus
+ vzstatus=`$VZCTL status $VEID 2/dev/null`
retcode=$?
+ set -- $vzstatus
+ veexists=$3
+ vestatus=$5
if [[ $retcode != 0 ]]; then
ocf_log err vzctl status $VEID returned: $retcode
Well, you do have commit rights, don't you? :)
Sure, but I don't have a vz handy to test even obviously correct
patches with, before I commit...
Lars
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] ManageVE prints bogus errors to the syslog
On Wed, Apr 03, 2013 at 06:25:58PM +0200, Dejan Muhamedagic wrote:
Hi,
On Fri, Mar 22, 2013 at 08:41:30AM +0100, Roman Haefeli wrote:
Hi,
When stopping a node of our cluster managing a bunch of OpenVZ CTs, I
get a lot of such messages in the syslog:
Mar 20 17:20:44 localhost ManageVE[2586]: ERROR: vzctl status 10002
returned: 10002 does not exist.
Mar 20 17:20:44 localhost lrmd: [2547]: info: operation monitor[6] on
opensim for client 2550: pid 2586 exited with return code 7
It looks to me as if lrmd is making sure the CT is not running anymore.
However, this triggers ManageVE to print an error.
Could be. Looking at the RA, there's a bunch of places where the
status is invoked and where this message could get logged. It
could be improved. The following patch should help:
https://github.com/ClusterLabs/resource-agents/commit/ca987afd35226145f48fb31bef911aa3ed3b6015
BTW, why call `vzctl | awk` *twice*,
just to get two items out of the vzctl output?
how about lose the awk, and the second invokation?
something like this:
(should veexists and vestatus be local as well?)
diff --git a/heartbeat/ManageVE b/heartbeat/ManageVE
index 56a3d03..53f9bab 100755
--- a/heartbeat/ManageVE
+++ b/heartbeat/ManageVE
@@ -182,10 +182,12 @@ migrate_from_ve()
status_ve()
{
declare -i retcode
-
- veexists=`$VZCTL status $VEID 2/dev/null | $AWK '{print $3}'`
- vestatus=`$VZCTL status $VEID 2/dev/null | $AWK '{print $5}'`
+ local vzstatus
+ vzstatus=`$VZCTL status $VEID 2/dev/null`
retcode=$?
+ set -- $vzstatus
+ veexists=$3
+ vestatus=$5
if [[ $retcode != 0 ]]; then
ocf_log err vzctl status $VEID returned: $retcode
[ BTW, what's all the declare -i doing in there?
local would have done nicely. ]
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] ocft updated (test tool of resource-agnets)
On Sun, Jan 20, 2013 at 09:23:05AM -0700, John Shi wrote: Medium: tools: ocft: update to version 0.44 Added incremental mode (ocft test -i) , cache results and not repeatedly rerun those cases which succeeded. Improved *ocft make*, the ocft will only make the updated test-case file. The ocft test prints the actual case names instead of just case numbers. From efaeb8c675c8dfbbeb2e6c6068a281f2ce53bf22 Mon Sep 17 00:00:00 2001 From: John Shi j...@suse.com Date: Sun, 20 Jan 2013 23:35:00 +0800 Subject: [PATCH] Medium: tools: ocft: update to version 0.44 Added incremental mode (ocft test -i) , cache results and not repeatedly rerun those cases which succeeded. Improved *ocft make*, the ocft will only make the updated test-case file. The ocft test prints the actual case names instead of just case numbers. There is nothing in the commit message about changed kill behaviour. And I don't think I understand nor like that change. You do: - setsid $aroot/$agent $cmd /tmp/.ocft_runlog 21 + $aroot/$agent $cmd /tmp/.ocft_runlog 21 pid=$! i=0 @@ -111,9 +111,9 @@ agent_run() done if [ $i -ge $timeout ]; then -kill -SIGTERM -$pid /dev/null 21 +kill -SIGTERM $pid /dev/null 21 sleep 3 -kill -SIGKILL -$pid /dev/null 21 +kill -SIGKILL $pid /dev/null 21 The setsid and kill -SIG -$pid was intentional, to be sure to kill the process group. Agents may spawn childs, killing the agent should cleanup those childs as well. Why did you change that? Did it not work? Break something? Thanks, -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [resource-agents] Low: pgsql: check existence of instance number in replication mode (#159)
On Thu, Oct 25, 2012 at 01:24:40AM -0700, Takatoshi MATSUO wrote: check existence of instance number in replication mode because Pacemaker 1.1.8 or higher do not append instance numbers. I think this is wrong. It seems this became necessary because of commit 427c7fe6ea94a566aaa714daf8d214290632f837 Author: Andrew Beekhof and...@beekhof.net Date: Fri Jul 13 13:37:42 2012 +1000 High: PE: Do not append instance numbers to anonymous clones Benefits: - they shouldnt have been exposed in the first place, but I didnt know how not to back then - if admins don't know what they are, they can't be misunderstood or misused - more reliable failcount and promotion scores (since you dont have to check for all possible permutations) - smaller status section since there cant be entries for each possible :N suffix - the name in the config corresponds to the resource in the logs So if pgsql thinks it needs these instance numbers, maybe it is not so anonymous a clone, after all? Would the existing resource agent work with globally-unique=true ? Lars You can merge this Pull Request by running: git pull https://github.com/t-matsuo/resource-agents check-instance-number Or you can view, comment on it, or merge it online at: https://github.com/ClusterLabs/resource-agents/pull/159 -- Commit Summary -- * Low: pgsql: check existence of instance number in replication mode -- File Changes -- M heartbeat/pgsql (44) -- Patch Links -- https://github.com/ClusterLabs/resource-agents/pull/159.patch https://github.com/ClusterLabs/resource-agents/pull/159.diff --- Reply to this email directly or view it on GitHub: https://github.com/ClusterLabs/resource-agents/pull/159 -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [resource-agents] [RFC] IPaddr2: Proposal patch to support the dual stack of IPv4 and IPv6. (#97)
support. IPv6addr.c is reused to make this command.
Note that IPv6addr RA is still there and you can continue to use it
for the backward compatibility.
## Acknowledgement
Thanks to Tomo Nozawa-san for his hard work for writing and testing
this patch.
Thanks to Lars Ellenberg for the first findif.sh implementation.
You can merge this Pull Request by running:
git pull https://github.com/kskmori/resource-agents
IPaddr2-dualstack-devel
Or you can view, comment on it, or merge it online at:
https://github.com/ClusterLabs/resource-agents/pull/97
-- Commit Summary --
* [RFC] IPaddr2: Proposal patch to support the dual stack of IPv4 and
IPv6.
-- File Changes --
M heartbeat/IPaddr2 (59) M heartbeat/IPv6addr.c (89) M
heartbeat/Makefile.am (9) A heartbeat/findif.sh (184) M
tools/ocft/IPaddr2 (24) A tools/ocft/IPaddr2v4 (315) A
tools/ocft/IPaddr2v6 (257) M tools/ocft/Makefile.am (2)
-- Patch Links --
https://github.com/ClusterLabs/resource-agents/pull/97.patch
diff --git a/heartbeat/findif.sh b/heartbeat/findif.sh
new file mode 100644
index 000..6a2807c
--- /dev/null
+++ b/heartbeat/findif.sh
@@ -0,0 +1,184 @@
+#!/bin/sh
+ipcheck_ipv4() {
local r1_to_255=([1-9][0-9]?|1[0-9][0-9]|2[0-4][0-9]|25[0-5])
local r0_to_255=([0-9][0-9]?|1[0-9][0-9]|2[0-4][0-9]|25[0-5])
local r_ipv4=^$r1_to_255\.$r0_to_255\.$r0_to_255\.$r0_to_255$
echo $1 | grep -q -Ee $r_ipv4
}
+ipcheck_ipv6() {
! echo $1 | grep -qs [^0-9:a-fA-F]
}
+ifcheck_ipv4() {
+ local ifcheck=$1
+ local ifstr
+ local counter=0
+ local procfile=/proc/net/dev
+ while read LINE
+ do
+if [ $counter -ge 2 ] ; then
+ ifstr=`echo $LINE | cut -d ':' -f 1`
+ if [ $ifstr = $ifcheck ] ; then
+return 0
+ fi
+fi
+counter=`expr $counter + 1`
+ done $procfile
local ifstr rest
while read ifstr rest ; do
[ $ifstr = $ifcheck: ] return 0
done
return 1
+ return 1
+}
+ifcheck_ipv6() {
+ local ifcheck=$1
+ local ifstr
+ local procfile=/proc/net/if_inet6
+ while read LINE
+ do
+ifstr=`echo $LINE | awk -F ' ' '{print $6}'`
+if [ $ifstr = $ifcheck ] ; then
+ return 0
+fi
+ done $procfile
# btw, in bash, I tend to use _ as dummy
# much more readable
local tmp ifstr
while read tmp tmp tmp tmp tmp ifstr ; do
[ $ifstr = $ifcheck ] return 0
done
+ return 1
+}
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Using /etc/ha.d/nodeinfo--supported?
On Thu, Sep 20, 2012 at 08:59:36AM -0400, Paul Smith wrote: Hi; anyone have any thoughts about the nodeinfo file in modern heartbeat implementations? Subject: /etc/ha.d/nodeinfo--supported? Short answer: probably not. Longer answer: It is still in there, and it was not changed. I doubt has ever been much used or tested. Guess you have to just try it, to find out if it works for you. It will pretend that whatever is read from that nodeinfo file (up to the first newline, if any) is the local node name, and it will export the HA_CURHOST environment variable for any child processes. AFAIK, Pacemaker is not aware of this at all. Awareness is probably missing from some client libs as well, so it may or may not work for some of them. Most external scripts will not be aware of it either. DRBD certainly is not. From a quick glance at the heartbeat source, at least heartbeat -s is not aware of it either, so would print the true uname in its message, but still the correct status. Why fake the uname for the cluster? Why not fake it for that other application, which thinks it needs to depend on it? Or maybe even just add some entry into /etc/hosts, so the reverse lookup for that other application returns whatever is expected? You could also look into ldpreload'ing the uname() call... Oh, and you certainly do NOT want to use 2.1.2. Well, if you want haresources mode, maybe you could. But still you should use latest heartbeat. If you want to use crm mode, use pacemaker. Hth, Lars Thanks! On Sun, 2012-09-16 at 17:52 -0400, Paul Smith wrote: Hi all. I'm investigating an HA environment with a simple active/standby configuration, just two nodes in a cluster with DRBD to provide a shared partition (using standard Red Hat EL 6.2 packages, such as heartbeat 2.1.2 and DRBD 8.4). Uhm, those are standard RHEL 6.2 packages? Really? These servers have multiple network interfaces: an internal private network, over which DRBD, heartbeat, etc. are run, and also a separate set of interfaces to provide customer access. The internal interfaces have static IP addresses and unchanging hostnames. The external interfaces do NOT: those interfaces are owned by the customer, not by the cluster. They might use DHCP to get IP addresses, and it's important that the hostnames of the systems, when the customer runs uname etc., be _their_ hostname and not the internal hostname of the cluster. Users of the system must be free to change these values. Why. I'm frustrated trying to get this to work robustly with heartbeat. Explaining why the entire system must be brought down and restarted merely to change the hostname is somewhat embarrassing as well. If I could get heartbeat to use my internal, forever-constant names rather BTW, heartbeat *does* use UUIDs internally, so would even be able to detect uname changes. But still many parts of the whole cluster stack rely on uname to be constant accros at least their process lifetime, and pacemaker requires special treatment when renaming nodes as well. than the results of uname -n my system would work so much more smoothly and reliably, provide more uptime, and require a lot less effort from me. Because this is a working environment, moving to completely different technology like corosync is not really feasible. I don't think moving to corosync would change *this* particular problem. I found a thread from 2004 discussing the (then?) undocumented support for the /etc/ha.d/nodeinfo file with heartbeat. This seems like the obviously correct solution. I can't find any information on this subject more current than that thread, though. Is this feature still available/supported? Does it work with DRBD as well? Is it something I can rely on going forward, insofar as heartbeat is still supported? I must confess myself somewhat taken aback to read in that 2004 thread a robust defense of the idea that uname -n would be the sole true infallible identifier for a node. Hostnames may be relied upon to be unique _at any given moment_, yes, but they are a very far ways from being _constants_. They do change. While it's useful for status output, logs, etc. to utilize hostnames as user-readable identifiers, a design using an internal (constant) identifier for nodes in the cluster seems to me to be far more reliable and straightforward to manage. I'm no HA guru however; is there a technical reason why this is difficult or sub-optimal? -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Probable sub-optimal behavior in ucast
On Mon, Jul 30, 2012 at 02:02:38PM -0600, Alan Robertson wrote: Hi, I have a 10-node system with each system having 2 interfaces, and therefore each ha.cf file has 18 ucast lines in it. If I read the code correctly, I think each heartbeat packet is then being received 18 times and sent to the master control process - where each is then uncompressed and 17 of them are thrown away... Could someone else offer your thoughts on this? It looks to be a 2 or 3 line fix in ucast.c to thrown away ucast packets that aren't from the address we expect - which would cut us down to only one of them being sent from each of the interfaces - a 9 to 1 reduction in work on the master control process. And I don't have to uncompress them to throw them away - I can just look at the source IP address... What do you think? Besides that a ten node cluster is likely to break the 64k message size limit, even after compression... You probably should re-organize the code so that you only have one receiving ucast socket per nic/ip/port. But I think that a single UDP packet will be delivered to a single socket, even though you have 18 receiving sockets bound to the same port (possible because of SO_REUSEPORT, only). If we, as I think we do, receive on just one of them, where which one is determined by the kernel, not us, your suggested ingress filter on expected source IP would break communications. Do you have evidence for the assumption that you receive incoming packets on all sockets, and not on just one of them? -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] Fix ha.cf node parsing in ha_propagate
On Tue, Jun 26, 2012 at 05:46:44PM +0700, Philippe Alcoy wrote: On 22 June 2012 17:20, Lars Ellenberg lars.ellenb...@linbit.com wrote: On Wed, Jun 20, 2012 at 06:29:43PM +0700, Philippe Alcoy wrote: # HG changeset patch # User Philippe Alcoy phili...@alcoy.co.uk # Date 1340188944 -25200 # Node ID f6ce2daddba6234643f715c46b28d5562940eb1f # Parent add12b838ef461b20eec223e551bf508038dc640 Fix ha.cf node parsing in ha_propagate Someone is actually *using* this? and even with multiple nodes per node line in ha.cf? Probably not. I was just curious and tested it in the lab. diff -r add12b838ef4 -r f6ce2daddba6 heartbeat/lib/ha_propagate.in --- a/heartbeat/lib/ha_propagate.in Mon Apr 09 17:50:27 2012 +0200 +++ b/heartbeat/lib/ha_propagate.in Wed Jun 20 17:42:24 2012 +0700 @@ -27,9 +27,8 @@ for line in f: if line.startswith(node): toks = line.split() - if (len(toks) == 2): - nodeName = toks[1] - nodes.append(nodeName) + if (len(toks) 0): I'd suggest to use 1. If you decide to keep ha_propagate, I would recommend 0. The script would work even if for some unknown reasons there is only one node in the list. python -c 'print len(node.split())' 1 The first token in toks is node. A node line without node names is invalid. Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] Fix ha.cf node parsing in ha_propagate
On Wed, Jun 20, 2012 at 06:29:43PM +0700, Philippe Alcoy wrote: # HG changeset patch # User Philippe Alcoy phili...@alcoy.co.uk # Date 1340188944 -25200 # Node ID f6ce2daddba6234643f715c46b28d5562940eb1f # Parent add12b838ef461b20eec223e551bf508038dc640 Fix ha.cf node parsing in ha_propagate Someone is actually *using* this? and even with multiple nodes per node line in ha.cf? diff -r add12b838ef4 -r f6ce2daddba6 heartbeat/lib/ha_propagate.in --- a/heartbeat/lib/ha_propagate.in Mon Apr 09 17:50:27 2012 +0200 +++ b/heartbeat/lib/ha_propagate.in Wed Jun 20 17:42:24 2012 +0700 @@ -27,9 +27,8 @@ for line in f: if line.startswith(node): toks = line.split() - if (len(toks) == 2): - nodeName = toks[1] - nodes.append(nodeName) + if (len(toks) 0): I'd suggest to use 1. + nodes = toks[1:] f.close() thisnode = os.uname()[1] -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] Fix chkconfig heartbeat on flag missing in ha_propagate
On Thu, Jun 21, 2012 at 10:07:11AM +0700, Philippe Alcoy wrote: # HG changeset patch # User Philippe Alcoy phili...@alcoy.co.uk # Date 1340192740 -25200 # Node ID 8cf404c39c7e0c63ec4a86b967d8a3e5de60977b # Parent f6ce2daddba6234643f715c46b28d5562940eb1f Fix chkconfig heartbeat on flag missing in ha_propagate diff -r f6ce2daddba6 -r 8cf404c39c7e heartbeat/lib/ha_propagate.in --- a/heartbeat/lib/ha_propagate.in Wed Jun 20 17:42:24 2012 +0700 +++ b/heartbeat/lib/ha_propagate.in Wed Jun 20 18:45:40 2012 +0700 @@ -39,5 +39,5 @@ print Propagating HA configuration files to node + v + . res = os.system(scp + cfgfile + + authfile + root@ + v + : + cfgdir) print Setting HA startup configuration on node + v + . -res = os.system(ssh + root@ + v + chkconfig `chkconfig heartbeat`) +res = os.system(ssh + root@ + v + chkconfig `chkconfig heartbeat on`) Well, No. Is is supposed to propagate the setting of the current node to all nodes. There are valid use cases for having heartbeat *off* on system boot. Note the double chkconfig and the backticks. When it was implemented, chkconfig single-service-name apparently returned the current setting as on or off to stdout, at least on the distribution this was implemented on. I guess this is no longer true, or diverges between distributions. It is not even available on all distributions. If anything, I'm tempted to drop that ha_propagate thing altogether. People should be able to scp config files and runlevel settings themselves, really... -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] exportfs RA patch
On Wed, Jun 06, 2012 at 02:06:19PM +0300, Borislav Borisov wrote:
Ok. So we forgot about exportfs *:/path,
which is shown as /path world.
I'd like to have it anchored.
oes this still work for you?
+ sed -e '$! N; s/\n[[:space:]]\+/ /; t;
s/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/g; s/ world$/ */g; P;D;'
To some extent it does... it will catch the ones that end with world, but
the problem is the search and replaces used are appending a newline. Also,
I initially missed a replace after the first search and replace. Eventually
I ended up with something close to what you want:
sed -e '$! N; s/\n[[:space:]]\+/ /; s/ world$/ */g; t;
s/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/g; s/ world\(\n\|$\)/
*\1/g; P;D;'
That one, however, is bugging me a lot.
It is not even strictly correct, I think.
You need to change the order of the first too substitutions for the t;
to be correct. See below.
So I came up with another solution,
which of course isn't anchored either, but is more readable:
sed -n '1h; 1!H; ${g; s/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/g; s/
world/ */g; p}'
I don't like that this is first reading all of it,
then do a replace over all of it.
It also breaks if you have white space in path names (which is probably
a very bad practise, in case it is even legal ;-)
I do not consider myself a sed guru, maybe someone who is can figure a
better approach.
I think this one is correct, please see if you can break it.
sed -e '$! N;s/\n[[:space:]]\+world/ */; s/\n[[:space:]]\+/ /; t;
s/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/; s/ world\(\n\|$\)/ *\1/;
P;D;'
Note that I dropped the /g from the latter substitutions.
FMTYEWTK:
$! N; # append next line, unless end of file
s/\n[[:space:]]\+world/ */;
# join continuation lines and replace world, if present.
s/\n[[:space:]]\+/ /;
# join continuation lines. can not succeed if previous line succeeded.
t; # if one of the previous two join lines succeeded,
# print and start new cycle.
# else:
# we have one single line record,
# and a potentially partial record as second line
s/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/;
# squeeze spaces between path and client spec
# If we put a /g on there, it may also squeeze spaces
# within the path spec of a trailing partial record.
# Which is why I left it off now.
# Not sure why I put it in there in the first place.
s/ world\(\n\|$\)/ *\1/;
# if the client spec is world, substitute by *
P; # print up to the first newline
D; # delete up to the first newline and start new cycle
If we can be sure that exportfs output will always be \n terminated,
we could leave off the \(\n\|$\) ... \... parts.
Would it be easier to the eye if we break it up into single statements?
sed -e '$! N;' \
-e 's/\n[[:space:]]\+world/ */;' \
-e 's/\n[[:space:]]\+/ /;' \
-e 't;' \
-e 's/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/;' \
-e 's/ world\(\n\|$\)/ *\1/;' \
-e 'P;D;'
Again, please, everybody:
try to break this with any output exportfs may produce.
If only ... I mean, well, it is 2012, right?
ah well, never mind :-/
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] exportfs RA patch
On Sat, Jun 02, 2012 at 03:07:10PM +0300, Borislav Borisov wrote:
Hello,
I have patched the exportfs RA to allow anonymous usage. The patch is in
the following fork commit -
https://github.com/borislav-borisov/resource-agents/commit/be684a71c1fd1b5811df399101752078ac7c9e3e
.
I'm lazy.
If I am asked by email to look at patches,
I really like them to be inlined in that very email.
Saves so much time.
wget -nv -O-
https://github.com/borislav-borisov/resource-agents/commit/be684a71c1fd1b5811df399101752078ac7c9e3e.diff
diff --git a/heartbeat/exportfs b/heartbeat/exportfs
index 8bcc1ea..fb2a24e 100755
--- a/heartbeat/exportfs
+++ b/heartbeat/exportfs
@@ -188,7 +188,7 @@ exportfs_monitor ()
# We unwrap here with sed.
# We then do a literal match on the full line (grep -x -F)
exportfs |
- sed -e '$! N; s/\n[[:space:]]\+/ /; t;
s/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/g; P;D;' |
+ sed -e '$! N; s/\n[[:space:]]\+/ /; t;
s/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/g; s/world/*/g; P;D;' |
grep -q -x -F ${OCF_RESKEY_directory} ${OCF_RESKEY_clientspec}
#Adapt grep status code to OCF return code
Ok. So we forgot about exportfs *:/path,
which is shown as /path world.
I'd like to have it anchored.
oes this still work for you?
+ sed -e '$! N; s/\n[[:space:]]\+/ /; t;
s/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/g; s/ world$/ */g; P;D;' |
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Request to everybody adding tag
On Mon, May 28, 2012 at 04:40:04PM +0900, noza...@gmail.com wrote: Hi Everybody adding tag please add -a or -s option. Because the script uses git describe when I make rpm package , a package is made with a strange value. Even though many Pacemaker tags are neither signed nor annotated as well (afaics), given the timing, I assume you refer to the recent resource-agents v3.9.3 release. Signed tag has been added for that one. Did you know git describe --tags ? Thanks, -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Patch] The patch which revises memory leak.
On Wed, May 16, 2012 at 09:33:48AM +0900, renayama19661...@ybb.ne.jp wrote: Hi Lars, In the environment where we confirmed leak, I confirm your patch. Pushed to http://hg.linux-ha.org/glue Thanks, -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Patch] The patch which revises memory leak.
On Mon, May 14, 2012 at 05:44:55PM +0200, Lars Ellenberg wrote: By the way, I suspect Lars' suggestion would work fine. I would certainly explain what the better patch is in the comments when you apply this one. My suggestion would be: diff --git a/lib/clplumbing/GSource.c b/lib/clplumbing/GSource.c --- a/lib/clplumbing/GSource.c +++ b/lib/clplumbing/GSource.c @@ -1506,6 +1506,7 @@ Gmain_timeout_add_full(gint priority g_source_set_callback(source, function, data, notify); append-gsourceid = g_source_attach(source, NULL); + g_source_unref(source); return append-gsourceid; } Then recompile and reinstall libglue. Please see if that also fixes the leak for you (and does not explode). Hm. Looks like it *does* explode (aka segfault) on various occasions throughout at least heartbeat core and lrmd, probably when a timeout callback tries to remove itself explicitly before returning FALSE. Guess we need to stay bug-to-bug compatible here, unless the users of Gmain_timeout_* from clplumbing turn out to be less than a handful and can be audited. Which may well be the case. Though it would break the ABI, so it would be a lot of effort, and I frankly doubt this is worth it for clplumbing at this point in its lifecycle. :-( -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Patch] The patch which revises memory leak.
On Tue, May 15, 2012 at 11:14:53AM +0200, Lars Ellenberg wrote:
On Mon, May 14, 2012 at 05:44:55PM +0200, Lars Ellenberg wrote:
By the way, I suspect Lars' suggestion would work fine. I would
certainly explain what the better patch is in the comments when you
apply this one.
Hm. Looks like it *does* explode (aka segfault)
Continuing my monologue ...
it may just have been incomplete.
The patch below seems to work just fine.
I managed to occasionally trigger the
Attempt to remove timeout (%u) with NULL source
message, but I have seen that one without the patch as well,
so that may just be some other oddity somewhere:
double removal of timeout resources ;-)
We can find and drop those later,
they look harmless enough.
I do not see any memleak anywhere anymore with this patch applied.
Comments/review/testing welcome.
# HG changeset patch
# User Lars Ellenberg l...@linbit.com
# Date 1337066453 -7200
# Node ID e63dd41f46b7bd150a23a62303bde6be78305c9c
# Parent 63d968249025b245e38b1da6d0202438ec45ebf3
[mq]: potential-fix-for-timer-leak
diff --git a/lib/clplumbing/GSource.c b/lib/clplumbing/GSource.c
--- a/lib/clplumbing/GSource.c
+++ b/lib/clplumbing/GSource.c
@@ -1507,6 +1507,7 @@
g_source_set_callback(source, function, data, notify);
append-gsourceid = g_source_attach(source, NULL);
+ g_source_unref(source);
return append-gsourceid;
}
@@ -1517,14 +1518,12 @@
GSource* source = g_main_context_find_source_by_id(NULL,tag);
struct GTimeoutAppend* append = GTIMEOUT(source);
- g_source_remove(tag);
-
if (source == NULL){
cl_log(LOG_ERR, Attempt to remove timeout (%u)
with NULL source,tag);
}else{
g_assert(IS_TIMEOUTSRC(append));
- g_source_unref(source);
+ g_source_remove(tag);
}
return;
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Patch] The patch which revises memory leak.
On Wed, May 02, 2012 at 09:00:35PM -0600, Alan Robertson wrote: This is very interesting. My apologies for missing this memory leak :-(. The code logs memory usage periodically exactly to help notice such a thing. In my new open source project [http://assimmon.org], I am death on memory leaks. But I can assure you that back when that code was written, it was not at all clear who deleted what memory when - when it came to the glib. I'm not sure if valgrind was out back then, but I certainly didn't know about it. I confess that even on this new project I had a heck of a time making all the glib objects go away. I finally got them cleaned up - but it took weeks of running under valgrind before I worked out when to do what to make it throw the objects away - but not crash due to a bad reference. By the way, I suspect Lars' suggestion would work fine. I would certainly explain what the better patch is in the comments when you apply this one. My suggestion would be: diff --git a/lib/clplumbing/GSource.c b/lib/clplumbing/GSource.c --- a/lib/clplumbing/GSource.c +++ b/lib/clplumbing/GSource.c @@ -1506,6 +1506,7 @@ Gmain_timeout_add_full(gint priority g_source_set_callback(source, function, data, notify); append-gsourceid = g_source_attach(source, NULL); + g_source_unref(source); return append-gsourceid; } Then recompile and reinstall libglue. Please see if that also fixes the leak for you (and does not explode). Note: that is *instead* of your patch, not in addition to. Thanks, Lars -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Patch] The patch which revises memory leak.
On Wed, May 02, 2012 at 10:43:36AM +0900, renayama19661...@ybb.ne.jp wrote: Hi Lars, And when it passes more than a full day * node1 32126 ?SLs 79:52 0 182 71189 24328 0.1 heartbeat: master control process * node2 31928 ?SLs 77:01 0 182 70869 24008 0.1 heartbeat: master control process Oh, I see. This is a design choice (maybe not even intentional) of the Gmain_* wrappers used throughout the heartbeat code. The real glib g_timeout_add_full(), and most other similar functions, internally do id = g_source_attach(source, ...); g_source_unref(source); return id; Thus in g_main_dispatch, the need_destroy = ! dispatch (...) if (need_destroy) g_source_destroy_internal() in fact ends up destroying it, if dispatch() returns FALSE, as documented: The function is called repeatedly until it returns FALSE, at which point the timeout is automatically destroyed and the function will not be called again. Not so with the heartbeat/glue/libplumbing Gmain_timeout_add_full. It does not g_source_unref(), so we keep the extra reference around until someone explicitly calls Gmain_timeout_remove(). Talk about principle of least surprise :( Changing this behaviour to match glib's, i.e. unref'ing after g_source_attach, would seem like the correct thing to do, but is likely to break other pieces of code in subtle ways, so it may not be the right thing to do at this point. I'm going to take your patch more or less as is. If it does not show up soon, prod me again. Thank you for tracking this down. Best Regards, Hideo Yamauchi. --- On Tue, 2012/5/1, renayama19661...@ybb.ne.jp renayama19661...@ybb.ne.jp wrote: Hi Lars, We confirmed that this problem occurred with v1 mode of Heartbeat. * The problem happens with the v2 mode in the same way. We confirmed a problem in the next procedure. Step 1) Put a special device extinguishing a communication packet of Heartbeat in the network. Step 2) Between nodes, the retransmission of the message is carried out repeatedly. Step 3) Then the memory of the master process increases little by little. As a result of the ps command of the master process -- * node1 (start) 32126 ? SLs 0:00 0 182 53989 7128 0.0 heartbeat: master control process (One hour later) 32126 ? SLs 0:03 0 182 54729 7868 0.0 heartbeat: master control process (Two hour later) 32126 ? SLs 0:08 0 182 55317 8456 0.0 heartbeat: master control process (Four hours later) 32126 ? SLs 0:24 0 182 56673 9812 0.0 heartbeat: master control process * node2 (start) 31928 ? SLs 0:00 0 182 53989 7128 0.0 heartbeat: master control process (One hour later) 31928 ? SLs 0:02 0 182 54481 7620 0.0 heartbeat: master control process (Two hour later) 31928 ? SLs 0:08 0 182 55353 8492 0.0 heartbeat: master control process (Four hours later) 31928 ? SLs 0:23 0 182 56689 9828 0.0 heartbeat: master control process The state of the memory leak seems to vary according to a node with the quantity of the retransmission. The increase of this memory disappears by applying my patch. And the similar correspondence seems to be necessary in send_reqnodes_msg(), but this is like little leak. Best Regards, Hideo Yamauchi. --- On Sat, 2012/4/28, renayama19661...@ybb.ne.jp renayama19661...@ybb.ne.jp wrote: Hi Lars, Thank you for comments. Have you actually been able to measure that memory leak you observed, and you can confirm this patch will fix it? Because I don't think this patch has any effect. Yes. I really measured leak. I can show a result next week. #Japan is a holiday until Tuesday. send_rexmit_request() is only used as paramter to Gmain_timeout_add_full, and it returns FALSE always, which should cause the respective sourceid to be auto-removed. It seems to be necessary to release gsource somehow or other. The similar liberation seems to be carried out in lrmd. Best Regards, Hideo Yamauchi. --- On Fri, 2012/4/27, Lars Ellenberg lars.ellenb...@linbit.com wrote: On Thu, Apr 26, 2012 at 10:56:30AM +0900, renayama19661...@ybb.ne.jp wrote: Hi All, We gave test that assumed remote cluster environment. And we tested packet lost. The retransmission timer of Heartbeat causes memory leak. I donate a patch. Please confirm the contents of the patch. And please reflect a patch in a repository of Heartbeat. Have you actually been able to measure that memory leak you observed, and you can confirm this patch will fix it? Because I don't think
Re: [Linux-ha-dev] heartbeat gmain source priority inversion with rexmit and dead node detection
On Mon, Apr 30, 2012 at 12:23:56PM +1000, Andrew Beekhof wrote: On Sat, Apr 28, 2012 at 12:11 AM, Lars Ellenberg lars.ellenb...@linbit.com wrote: On Thu, Apr 26, 2012 at 10:56:30AM +0900, renayama19661...@ybb.ne.jp wrote: Hi All, We gave test that assumed remote cluster environment. And we tested packet lost. You may be interested in this patch I have lying around for ages. It may be incomplete for one corner case: On a seriously misconfigured and overloaded system, I have seen reports for a single send_local_status() (that is basically one single send_cluster_msg()) which took longer to execute than deadtime (without even returning to the mainloop!). This cornercase should be handled with a watchdog. But without a watchdog, and without stonith, the CCM was confused, because one node saw a leave then re-join after partition event, while the other node did not even notice it had left and rejoined the membership... and pacemaker ended up being DC on both :-/ A side effect of the ccm being really confused I assume? I guess so, yes. Not sure if pacemaker could have handled it differently, based on the input it was fed from ccm. At some point, Pacemaker complained about Another DC detected, but things never really recovered. If I can dig up the logs again, I'll show you some lines. But then, no stonith, no watchdog, and system overloaded to the point where processing a single mainloop dispatch callback takes longer than what is supposed to be the deadtime, and that is within the heartbeat communication main processes, which are supposed to be realtime... I don't think additional paranoia code would do much good, on any level. But thanks for your attention ;-) So I guess send_local_status() could do with an explicit call to check_for_timeouts(), but that may need recursion protection. I should really polish and push my queue some day soon... Cheers, diff --git a/heartbeat/hb_rexmit.c b/heartbeat/hb_rexmit.c ... -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Patch] The patch which revises memory leak.
On Thu, Apr 26, 2012 at 10:56:30AM +0900, renayama19661...@ybb.ne.jp wrote:
Hi All,
We gave test that assumed remote cluster environment.
And we tested packet lost.
The retransmission timer of Heartbeat causes memory leak.
I donate a patch.
Please confirm the contents of the patch.
And please reflect a patch in a repository of Heartbeat.
Have you actually been able to measure that memory leak you observed,
and you can confirm this patch will fix it?
Because I don't think this patch has any effect.
send_rexmit_request() is only used as paramter to
Gmain_timeout_add_full, and it returns FALSE always,
which should cause the respective sourceid to be auto-removed.
diff -r 106ca984041b heartbeat/hb_rexmit.c
--- a/heartbeat/hb_rexmit.c Thu Apr 26 19:28:26 2012 +0900
+++ b/heartbeat/hb_rexmit.c Thu Apr 26 19:31:44 2012 +0900
@@ -164,6 +164,8 @@
seqno_t seq = (seqno_t) ri-seq;
struct node_info* node = ri-node;
struct ha_msg* hmsg;
+ unsigned long sourceid;
+ gpointer value;
if (STRNCMP_CONST(node-status, UPSTATUS) != 0
STRNCMP_CONST(node-status, ACTIVESTATUS) !=0) {
@@ -196,11 +198,17 @@
node-track.last_rexmit_req = time_longclock();
- if (!g_hash_table_remove(rexmit_hash_table, ri)){
- cl_log(LOG_ERR, %s: entry not found in rexmit_hash_table
-for seq/node(%ld %s),
-__FUNCTION__, ri-seq, ri-node-nodename);
- return FALSE;
+ value = g_hash_table_lookup(rexmit_hash_table, ri);
+ if ( value != NULL) {
+ sourceid = (unsigned long) value;
+ Gmain_timeout_remove(sourceid);
+
+ if (!g_hash_table_remove(rexmit_hash_table, ri)){
+ cl_log(LOG_ERR, %s: entry not found in
rexmit_hash_table
+ for seq/node(%ld %s),
+ __FUNCTION__, ri-seq, ri-node-nodename);
+ return FALSE;
+ }
}
schedule_rexmit_request(node, seq, max_rexmit_delay);
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
[Linux-ha-dev] heartbeat gmain source priority inversion with rexmit and dead node detection
On Thu, Apr 26, 2012 at 10:56:30AM +0900, renayama19661...@ybb.ne.jp wrote:
Hi All,
We gave test that assumed remote cluster environment.
And we tested packet lost.
You may be interested in this patch I have lying around for ages.
It may be incomplete for one corner case:
On a seriously misconfigured and overloaded system,
I have seen reports for a single send_local_status()
(that is basically one single send_cluster_msg())
which took longer to execute than deadtime
(without even returning to the mainloop!).
This cornercase should be handled with a watchdog.
But without a watchdog, and without stonith,
the CCM was confused, because one node saw a
leave then re-join after partition event, while the other node did not
even notice it had left and rejoined the membership...
and pacemaker ended up being DC on both :-/
So I guess send_local_status() could do with an explicit call to
check_for_timeouts(), but that may need recursion protection.
I should really polish and push my queue some day soon...
Cheers,
diff --git a/heartbeat/hb_rexmit.c b/heartbeat/hb_rexmit.c
--- a/heartbeat/hb_rexmit.c
+++ b/heartbeat/hb_rexmit.c
@@ -168,6 +168,7 @@ send_rexmit_request( gpointer data)
if (STRNCMP_CONST(node-status, UPSTATUS) != 0
STRNCMP_CONST(node-status, ACTIVESTATUS) !=0) {
/* no point requesting rexmit from a dead node. */
+ g_hash_table_remove(rexmit_hash_table, ri);
return FALSE;
}
@@ -243,7 +244,7 @@ schedule_rexmit_request(struct node_info
ri-seq = seq;
ri-node = node;
- sourceid = Gmain_timeout_add_full(G_PRIORITY_HIGH - 1, delay,
+ sourceid = Gmain_timeout_add_full(PRI_REXMIT, delay,
send_rexmit_request, ri, NULL);
G_main_setall_id(sourceid, retransmit request,
config-heartbeat_ms/2, 10);
diff --git a/heartbeat/heartbeat.c b/heartbeat/heartbeat.c
--- a/heartbeat/heartbeat.c
+++ b/heartbeat/heartbeat.c
@@ -1585,7 +1585,7 @@ master_control_process(void)
send_local_status();
- if (G_main_add_input(G_PRIORITY_HIGH, FALSE,
+ if (G_main_add_input(PRI_POLL, FALSE,
polled_input_SourceFuncs) ==NULL){
cl_log(LOG_ERR, master_control_process: G_main_add_input
failed);
}
diff --git a/include/hb_api_core.h b/include/hb_api_core.h
--- a/include/hb_api_core.h
+++ b/include/hb_api_core.h
@@ -40,6 +40,12 @@
#definePRI_READPKT (PRI_SENDPKT+1)
#definePRI_FIFOMSG (PRI_READPKT+1)
+/* PRI_POLL is where the timeout checks on deadtime happen.
+ * Better be sure rexmit requests for lost packets
+ * from a now dead node do not preempt detecting it as being dead. */
+#define PRI_POLL (G_PRIORITY_HIGH)
+#define PRI_REXMIT PRI_POLL
+
#define PRI_CHECKSIGS (G_PRIORITY_DEFAULT)
#define PRI_FREEMSG(PRI_CHECKSIGS+1)
#definePRI_CLIENTMSG (PRI_FREEMSG+1)
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Occasionally heartbeat doesn't start...
On Fri, Mar 09, 2012 at 11:52:56AM -0700, Alan Robertson wrote: Hi, I've been investigating an HA configuration for a customer. One time in testing heartbeat didn't start, because rpcbind had stolen its reserved port. Restarting rpcbind made it choose a different random port. This is definitely an interesting problem - even if it doesn't happen very often. The best solution to this, AFAIK is to make a file /etc/portreserve/heartbeat with this one line in it: 694/udp and then add portrelease heartbeat to the init script. rpcbind used to be portmap. You would need the portreserve daemon available, installed, and started at the right time during your boot sequence. So that's only a hackish workaround. On Debian (Ubuntu, other derivatives) you'd simply add a line to /etc/bindresvport.blacklist. But that may fail as well, there have been reports where this was ignored for some reason. So that again is just a workaround. If you know exactly what will register with portmap (rpcbind), you can tell those services to request fixed ports instead. Typically you do, and those are just a few nfs related services. So just edit /etc/sysconfig/* or /etc/defaults/* to e.g. include -o and -p options for rpc.statd, and similar. This really is a fix, as long as you know all services that are started before heartbeat, and can tell them to use specific ports. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH 2/2] nfsserver: Support of multiple IP addresses
On Mon, Feb 27, 2012 at 08:55:25PM +0800, Gao,Yan wrote:
Hi,
This one is also from the effort of SGI and Xinwei Hu.
https://github.com/gao-yan/resource-agents/commit/7ae7560bb6e85f4870a1f1a2ed9c0fb2f6241c90
${x//,/ } is bad substitution in dash ...
Please check for bashisms, or switch to #!/bin/bash.
Other than that: :-)
Cheers,
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [resource-agents] `findif` could be rewritten in shell (#53)
On Mon, Feb 06, 2012 at 09:51:46PM +0900, nozawat wrote: Hi Lars I added IPv6 correspondence to a script of Lars. Which is nice, thank you ;-) but useless :( unless we also want to rewrite the currently written in C IPv6addr resource agent ... Which would also be nice, from my point of view. But I'm not sure if that is feasable. Cheers, -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH]Monitor failure and IPv6 support of apache-ra
On Wed, Feb 01, 2012 at 02:32:27PM +0900, Keisuke MORI wrote:
Hi Dejan,
2012/1/31 Dejan Muhamedagic de...@suse.de:
Hi Keisuke-san,
On Tue, Jan 31, 2012 at 09:52:24PM +0900, Keisuke MORI wrote:
Hi Dejan
2012/1/31 Dejan Muhamedagic de...@suse.de:
Hi Keisuke-san,
(...)
On Tue, Jan 31, 2012 at 08:46:35PM +0900, Keisuke MORI wrote:
The current RA will try to check the top page (http://localhost:80)
as the default behavior if you have not enabled server-status in
httpd.conf
and it would fail to start even for the apache's default test page:)
Hmm, the current RA would produce an error for that URL:
488 case $STATUSURL in
489 http://*/*) ;;
490 *)
491 ocf_log err Invalid STATUSURL $STATUSURL
492 exit $OCF_ERR_ARGS ;;
493 esac
Strange. That URL is generated by the RA itself.
apache-conf.sh:
119 buildlocalurl() {
120 [ x$Listen != x ]
121 echo http://${Listen}; ||
122 echo ${LOCALHOST}:${PORT}
apache-conf.sh actually does, in GetParams,
StatusURL=`FindLocationForHandler $1 server-status | tail -1`
STATUSURL=`buildlocalurl`$StatusURL
Where StatusURL, as found by FindLocationForHandler config file server-status,
is supposed to end up being /server-status (or similar),
so STATUSURL in fact should match http://*/*
Probably we should relax the validation pattern, as just 'http://*' ?
Agreed. I thought that the intention was to always use the status
Yes, I think it is intentional.
page, but obviously people figured out that they could skip that.
Just as well.
Thank you for your productive comments and discussions!
As the result of the discussion regarding to this topic,
I would suggest two patches as the pull request below:
https://github.com/ClusterLabs/resource-agents/pull/54
We can do that anyways, but why?
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH]Monitor failure and IPv6 support of apache-ra
On Mon, Jan 30, 2012 at 09:24:19PM +0100, Dejan Muhamedagic wrote: As for the regular expression like ^ or $, it looks like working as expected with -z option in my quick tests. Do you have any examples that it may break the configuration? For instance, what I see here in the status page is also a PID at the beginning of line: xen-d:~ # wget -q -O- -L --no-proxy --bind-address ::1 http://[::1]/server-status | grep ^PID PID Key: br / xen-d:~ # wget -q -O- -L --no-proxy --bind-address ::1 http://[::1]/server-status | grep -z ^PID xen-d:~ # echo $? 1 different versions of grep in various distributions seem to behave differently with -z and ^: distro; grep --version | head -n1; printf A\nB\nC\n | grep -q -z ^B ; echo $? etchgrep (GNU grep) 2.5.1 0 lenny GNU grep 2.5.3 0 squeeze GNU grep 2.6.3 1 breaks lucid GNU grep 2.5.4 0 rhel4 grep (GNU grep) 2.5.1 1 breaks rhel5 grep (GNU grep) 2.5.1 0 rhel6 GNU grep 2.6.3 1 breaks sles9 grep (GNU grep) 2.5.1 1 breaks sles10 grep (GNU grep) 2.5.1 1 breaks sles11 GNU grep 2.5.2 1 breaks sles11-sp1 GNU grep 2.5.2 1 breaks So neither distros nor grep version seem to agree if ^ is supposed to be equivalent to pcre /^/m or /\A/ ;-) Best practices for monitoring apache with pacemaker would be to have some dedicated page print out either a text/plain ALL OK only. Or, in case something is wrong, anything with arbitrary diagnostic output, if any, which reliably does not containing that string. But we could just as well reduce to the default regular expression to '/ *html *'. If nobody objects :) As the default regex only checks for basically any response that remotely looks like it may have something to do with html output, that should be fine. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [resource-agents] `findif` could be rewritten in shell (#53)
Taking this to the mailing list as well, to give it a wider audience ...
On Tue, Jan 31, 2012 at 07:56:56AM -0800, b-a-t wrote:
Referring to the #52 I got the idea that shell script for finding
interface by IP address could be better solution than parsing routing
table in C.
For (most) Linux distributions 'ip' command is a part of the standard
installation and:
pre
# ip route get 10.0.0.220
10.0.0.220 dev eth1 src 10.0.0.106
/pre
gives better and more portable results.
For FreeBSD(and Solaris I guess):
We can ignore non-Linux for IPaddr2, that is already linux specific.
Even parsing of '/proc/net/route' is easier in shell, so I see no good
reason for quite complex and not flexible C binary for these
purpose(except the speed, possibly).
If this idea gets support I can try to write such a replacement.
once upon a time I started something like that already,
just for fun. But then left it alone for quite some time,
because, well, if it ain't broken, don't fix it...
I'm not sure why we would care to specify an explicit broadcast address,
so we probably should not try to guess it (the kernel knows better, anyways)
and only put it into the command line if it really was passed in via
configuration parameters, in case there actually is a use case for that.
findif seems to have a few heuristics, which seem to override the input?
Not sure here.
There is also the hack-ish LVS support stuff in the script,
we need to make sure that does not break.
Nor any other aspect of unusual usage.
Anyways, it may be a starting point:
findif()
{
local match
# FIXME: if all is specified, why would we try to second guess?
# just use the input, and if it fails, it fails?
# non-local, return values
NIC= NETMASK=
BRDCAST=$OCF_RESKEY_broadcast
# FIXME just make sure you drop the brd $BRDCAST
# from the ip addr add command if $BRDCAST is empty,
# and the kernel will do the right thing. Or so me thinks...
# Also see below, where we try to second guess BRDCAST.
match=$OCF_RESKEY_ip
# ip actually does not care if the mask is cidr or dotted quad,
# as long as it is a mask.
# No mask implies /32 (for the match, only).
# The match in ip route list match ... will find the best (longest
prefix) route.
[ -n $OCF_RESKEY_cidr_netmask ]
match=$match/$OCF_RESKEY_cidr_netmask
# FIXME: what if the guessed nic,
# and the requested nic, do not match?
# Would tools/findif.c have ignored the input in that case?
# Should the RA return $OCF_ERR_CONFIGURED in that case?
### FIXME ### [ -n $OCF_RESKEY_nic ] match=$match dev
$OCF_RESKEY_nic
# Only routes with scope link.
set -- `ip -o -f inet route list match $match scope link`
if [ $# = 0 ] ; then
# possibly need to add special case for 127.x.y.z
# at least tools/findif.c has one
case $OCF_RESKEY_ip in
127.*)
set -- `ip -o -f inet route list match $match table
local scope host`
# prints local as first word
shift
esac
fi
# Still nothing? Too bad.
[ $# = 0 ] return 1
# paranoia
case $1 in
*/*): OK ;;
*)
# No slash should only happen for table local, no mask
# (or /32), and address already present, and even then
# it should not show up as first line of output...
# If we cannot guess the netmask, you need to specify it.
return 1 ;;
esac
NIC=$3
NETMASK=${1#*/}
: == DEBUG == $NIC / $NETMASK ==
# Do we really need to guess the broadcast? Why?
# We could try to guess it from the primary address on the nic
# but actually that seems pointless?
set -- `ip -o -f inet addr show dev $NIC primary`
# this really should have worked, always!?
# for 127.x.y.z, there usually won't be a broadcast address
[ $5 = brd ] BRDCAST=$6
: == DEBUG == brd $BRDCAST ==
return 0
}
# try it:
set -vx
OCF_RESKEY_ip=10.9.9.99 findif
OCF_RESKEY_ip=127.1.2.3/27 findif
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] new resource agents release
On Fri, Jan 27, 2012 at 03:25:32PM +0100, Dejan Muhamedagic wrote: Hi Keisuke-san, On Fri, Jan 27, 2012 at 09:37:18PM +0900, Keisuke MORI wrote: Hi Dejan, I would appreciate you for bringing up for the new release. I really look forward to see it very soon! Among other issues, I consider that the two issues below are critical and want to make it for the new release somehow: 1. tomcat RA: SEARCH_STR fix http://www.gossamer-threads.com/lists/linuxha/dev/75950#75950 We can reconsider the patch if you're not comfortable with it, but without fixing this, it just doesn't work with a typical catalina_opts configuration. I'm basically clueless about tomcat (at least this kind of tomcats :) So far nobody else volunteered an opinion on the matter and the person (Brett Delle Grazie) who seemed to be very involved with this RA vanished from the scene. So, I'll take your word for it and apply the patch. I think that this was about using the command line as pattern in pgrep -f $SEARCH_STR, but as according to pgrep man page, that is actually an extended regular expression, and command line options may contain special characters (e.g. '+'), that does not work. So one option would have been to properly regex-quote SEARCH_STR, before using it as argument to pgrep -f. 2. apache RA: testregex matching fix http://www.gossamer-threads.com/lists/linuxha/dev/77619#77619 This looks as a regression since heartbeat-2.1.4 from user's point of view; one of our customer reported that they had been using 2.1.4 and apache without problems and when they tried to upgrade to the recent Pacemaker without any changes in apache, it failed because of this issue. You're referring to apache-002.patch? Well, that's unfortunate as the two are incompatible, i.e. if the configuration has 'whatever-string$' in testregex and we reintroduce tr '\012' ' ' that would break such configurations. This changed a bit more than three years ago and so far nobody complained. So, perhaps better to leave it as it is and whoever wants to upgrade from a +3 old installation should anyway do some good testing. What do you think? As for the other issues posted from our team, although it would be great if they are also fixed as much as possible, I do not want to delay the release schedule any longer for them. There is the apache IPv6 support which is halfway ready, I'll see if I can finish that today. Cheers, Dejan Regards, Keisuke MORI 2012/1/27 Dejan Muhamedagic de...@suse.de: Hello, The resource agents release 3.9.2 was end of last June, quite a while ago. High time to do a new one. Any obstacles or release blockers? Or other opinions? Florian and Lars, is that fine with you in particular? Now, I know that there are some fixes and ammendments and improvements which have been posted to this list or elsewhere and never made it to the repository. If there are some which didn't get appropriate attention, please give us a heads up and then we can discuss the matter. Unfortunately, I cannot guarantee that all are going to get due attention, in particular those which would be deemed to jeopardize stability. Best regards, Dejan ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/ -- Keisuke MORI ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/ ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/ -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [resource-agents] exportfs change to support wildcard exports (#45)
On Fri, Jan 20, 2012 at 04:18:02AM +0100, Dejan Muhamedagic wrote:
Hi Lars,
On Tue, Dec 13, 2011 at 10:55:57PM +0100, Lars Ellenberg wrote:
Taking this to the mailing list to give it a wider audience.
On Tue, Dec 13, 2011 at 09:59:11AM -0800, acqant wrote:
--- a/heartbeat/exportfs
+++ b/heartbeat/exportfs
@@ -181,9 +181,11 @@ END
exportfs_monitor ()
{
+ local clientspec_re
# grep -z matches across newlines, which is necessary as
# exportfs output wraps lines for long export directory names
- exportfs | grep -zqs
${OCF_RESKEY_directory}[[:space:]]*${OCF_RESKEY_clientspec}
+ clientspec_re=`echo ${OCF_RESKEY_clientspec} | sed 's/*/[*]/'`
+ exportfs | grep -zqs
${OCF_RESKEY_directory}[[:space:]]*${clientspec_re}
#Adapt grep status code to OCF return code
case $? in
Or you can view, comment on it, or merge it online at:
https://github.com/ClusterLabs/resource-agents/pull/45
Thinking about it, I've got a problem with this whole grepping thing here.
grep -z does not just match accross newlines,
it matches records separated by NUL in that file
(which would be very unexpected).
So it matches the full file.
No anchors on the regex whatsoever.
Client spec will typically have dots in them,
both hostname and ip address form,
which would also need to be matched literally.
If you have two exports /bar and /foo/bar,
to the same (or similar enough, see above) client spec,
the grep for /bar will also match on /foo/bar.
The mount point may also contain dots or other special chars.
I don't like that, really :(
Suggestion:
Why not unwrap the exportfs output first,
so we get one record per line,
then match literal (grep -F)?
That sounds good to me. I wonder if the author of the patch is
subscribed here.
I first commented on his pull request on github, then basically
forwarded what I said there slightly edited to the list here.
That should cover most of these issues
(appart from multiple consecutive blanks, or tabs, or newlines,
in the mount point... would that even be legal?)
I don't think we'd need to support that.
exportfs | fmt -w 1000 -t -u |
grep -x -F ${OCF_RESKEY_directory} ${OCF_RESKEY_clientspec}
I'm not completely sure about the fmt trick:
Availability should not be a problem (coreutils).
But, is the exportfs output and fmt behaviour really consistent enough
to have that work on all platforms?
The original usage was probably just fmt -1000 but that won't
do. IIRC, fmt on AIX was just like that.
But since both exportfs and fmt predate linux, maybe that just works?
If necessary, we can pull off the unwrap with sed in a more controlled
fashion as well.
That'd be preferable (and you're a sed expert :) awk or perl
would also do.
As you wish ;-)
exportfs |
sed -e '$! N; s/\n[[:space:]]\+/ /; t;
s/[[:space:]]\+\([^[:space:]]\+\)\(\n\|$\)/ \1\2/g; P;D;' |
grep -x -F ${OCF_RESKEY_directory} ${OCF_RESKEY_clientspec}
(please someone double check that gobbledygook!)
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH]Monitor failure and IPv6 support of apache-ra
On Tue, Jan 17, 2012 at 11:07:09AM +0900, nozawat wrote: Hi Dejan and Lars, I send the patch which I revised according to indication of Lars. OK. I guess that this won't introduce a regression. And I guess that sometimes one may need a newline in the test string. I seemed to surely take such a step in the past. However, I thought that the tr processing was deleted that load became higher. Therefore I used the -z option. Thinking about it, maybe to reduce chance of regression, we can try both? I'm not sure about the default order, ipv4 or ipv6 first? for bind_address in 127.0.0.1 ::1 ; do wget ... ret=$? [ $ret = 0 ] break; # recent wget could [ $ret != 4 ] break, # Network error. But older wget return 1... done Dejan? -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH]Monitor failure and IPv6 support of apache-ra
On Fri, Jan 13, 2012 at 06:58:21PM +0100, Dejan Muhamedagic wrote:
Hi,
On Wed, Dec 14, 2011 at 02:27:56PM +0900, nozawat wrote:
Hi
1.apache.patch(Coping of monitor failure)
When a newline code is included, I fail in a monitor.
Therefore I coped with a newline code.
OK. I guess that this won't introduce a regression. And I guess
that sometimes one may need a newline in the test string.
2.http-mon.sh.patch(IPv6 support)
When I did wget, I assigned 127.0.0.1 to --bind-address optionally,
but this deleted it now for IPv4.
Even IPv6 works by doing so it.
But this may have adverse effect on existing configurations. For
instance, if the configuration specifies a URL which may be
accessed from outside, then suddenly a monitor will return
success on any node in the cluster. I think that we need a
different approach here.
Why exactly IPv6 doesn't work?
I think, because:
$ wget -O- -q -L --no-proxy --bind-address 127.0.0.1 http://[::1]/;
does not work.
strace reveals:
socket(PF_INET6, SOCK_STREAM, IPPROTO_IP) = 3
bind(3, {sa_family=AF_INET, sin_port=htons(0),
sin_addr=inet_addr(127.0.0.1)}, 16) = -1 EINVAL (Invalid argument)
However, this does work:
$ wget -O- -q -L --no-proxy --bind-address ::1 http://[::1]/; ; echo $?
So maybe bindaddress should become an option as well?
Or auto-detect:
if ip -o -f inet6 a s dev lo | grep -q ::1/; then
# alternatively, to not be linux specific, ifconfig lo | grep ...
# in case the output is actually reliably grep-able accross OSes.
bind_address=::1
else
bind_address=127.0.0.1
fi
Because, at least as long as net.ipv6.bindv6only = 0,
binding to ::1 works for http://127.0.0.1 as well.
Thanks,
Dejan
Regards,
Tomo
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [resource-agents] exportfs change to support wildcard exports (#45)
Taking this to the mailing list to give it a wider audience.
On Tue, Dec 13, 2011 at 09:59:11AM -0800, acqant wrote:
--- a/heartbeat/exportfs
+++ b/heartbeat/exportfs
@@ -181,9 +181,11 @@ END
exportfs_monitor ()
{
+ local clientspec_re
# grep -z matches across newlines, which is necessary as
# exportfs output wraps lines for long export directory names
- exportfs | grep -zqs
${OCF_RESKEY_directory}[[:space:]]*${OCF_RESKEY_clientspec}
+ clientspec_re=`echo ${OCF_RESKEY_clientspec} | sed 's/*/[*]/'`
+ exportfs | grep -zqs
${OCF_RESKEY_directory}[[:space:]]*${clientspec_re}
#Adapt grep status code to OCF return code
case $? in
Or you can view, comment on it, or merge it online at:
https://github.com/ClusterLabs/resource-agents/pull/45
Thinking about it, I've got a problem with this whole grepping thing here.
grep -z does not just match accross newlines,
it matches records separated by NUL in that file
(which would be very unexpected).
So it matches the full file.
No anchors on the regex whatsoever.
Client spec will typically have dots in them,
both hostname and ip address form,
which would also need to be matched literally.
If you have two exports /bar and /foo/bar,
to the same (or similar enough, see above) client spec,
the grep for /bar will also match on /foo/bar.
The mount point may also contain dots or other special chars.
I don't like that, really :(
Suggestion:
Why not unwrap the exportfs output first,
so we get one record per line,
then match literal (grep -F)?
That should cover most of these issues
(appart from multiple consecutive blanks, or tabs, or newlines,
in the mount point... would that even be legal?)
exportfs | fmt -w 1000 -t -u |
grep -x -F ${OCF_RESKEY_directory} ${OCF_RESKEY_clientspec}
I'm not completely sure about the fmt trick:
Availability should not be a problem (coreutils).
But, is the exportfs output and fmt behaviour really consistent enough
to have that work on all platforms?
But since both exportfs and fmt predate linux, maybe that just works?
If necessary, we can pull off the unwrap with sed in a more controlled
fashion as well.
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Sometimes not in our membership failure in 2-node cluster
: event=NEW MEMBERSHIP: Still it is unusual for heartbeat to send out NEW MEMBERSHIP events, if nothing actually changed: no new members, no members lost... ./ccm_testclient[27359]: 2011/11/17_12:53:52 info: instance=516 # ttl members=1, ttl_idx=0 # new members=0, new_idx=1 # out members=0, out_idx=3 ./ccm_testclient[27359]: 2011/11/17_12:53:52 info: NODES IN THE PRIMARY MEMBERSHIP ./ccm_testclient[27359]: 2011/11/17_12:53:52 info: nodeid=0, uname=debian60-clnode1, born=516 ./ccm_testclient[27359]: 2011/11/17_12:53:52 info: MY NODE IS A MEMBER OF THE MEMBERSHIP LIST ./ccm_testclient[27359]: 2011/11/17_12:53:52 info: NEW MEMBERS ./ccm_testclient[27359]: 2011/11/17_12:53:52 info: NONE ./ccm_testclient[27359]: 2011/11/17_12:53:52 info: MEMBERS LOST ./ccm_testclient[27359]: 2011/11/17_12:53:52 info: NONE ./ccm_testclient[27359]: 2011/11/17_12:53:52 info: --- (or lib64, may be in the -dev package) btw, as long as it can talk to ccm, that does not terminate by itself, you need to ctrl-c it... I don't think so. Anything about lost packets? I rephrase: any logs similar to WARN: [0-9]* lost packet(s) for ... If you can easily reproduce (with most recent heartbeat), a tcpdump may be useful from, say, 10 heartbeats before you disable, to half a minute minute after you re-enable the network link. (or just crank up debuggin so high that you even see message dumps in the logs...) I have to sniff. As I said: with a high enough debug level, you get insane amounts of logging, including heartbeat packet traces. But that may be too much logging, so it may be better to tcpdump/tshark/wireshark it, yes. In any case, at least use debug level 1. In this case when the nodes show each other offline, I disconnected and reconnected the interfaces again. After a few seconds they show each other online. After that ccm_testclient shows: ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: mem_handle_event: instance=574, nodes=2, new=2, lost=0, n_idx=0, new_idx=0, old_idx=4 ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: event=NEW MEMBERSHIP: ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: instance=574 # ttl members=2, ttl_idx=0 # new members=2, new_idx=0 # out members=0, out_idx=4 ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: NODES IN THE PRIMARY MEMBERSHIP ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: nodeid=1, uname=debian60-clnode2, born=1 ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: nodeid=0, uname=debian60-clnode1, born=574 ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: MY NODE IS A MEMBER OF THE MEMBERSHIP LIST ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: NEW MEMBERS ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: nodeid=1, uname=debian60-clnode2, born=1 ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: nodeid=0, uname=debian60-clnode1, born=574 clnode1 seems to have some serious problem there, it was much more often born... does it keep respawning the ccm process for some reason? --- look at the debug logs. Which means this one even lost _itself_ from the membership? (it sees two new nodes... it should only see one new node...) Any log message like No local heartbeat. Forcing restart. I suspect your test method is somehow invalid, you seem to test something that is not supposed to work... Do you have crm respawn in your ha.cf, by any chance? If so, does it behave better with crm on? (in which case, if the ccm terminates, it will not be respawned, but cause a reboot). ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: MEMBERS LOST ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: NONE ./ccm_testclient[27417]: 2011/11/17_12:56:48 info: --- I also checked times. They are in sync. Wall clock time is completely irrelevant for heartbeat, though ntp synced time certainly makes correlating logs much easier. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] ocf_run: sanitize output before logging?
On Mon, Nov 14, 2011 at 09:53:12PM +0100, Florian Haas wrote: Dejan, Lars, and other shell gurus in attendance, maybe I'm totally off my rocker, and one of you guys can set me straight. But to me this part of the ocf_run function seems a bit fishy: output=`$@ 21` rc=$? output=`echo $output` Am I gravely mistaken, or would any funny control characters produced by the wrapped command line totally mess up the content of output here as it is mangled by the backticks? What I'm noticing is the invocation of ocf_run sipsak -v -s uri, which we put into the asterisk RA as per Russell Bryant's suggestion, seems to totally garble the output. Compare this: $ sipsak -v -s sip:somenotexistantextens...@ekiga.net 21 SIP/2.0 200 OK Via: SIP/2.0/UDP 127.0.0.1:43665;branch=z9hG4bK.539207ad;rport=53485;alias;received=85.127.155.32 From: sip:sipsak@127.0.0.1:43665;tag=6dafacb9 To: sip:somenotexistantextens...@ekiga.net;tag=c64e1f832a41ec1c1f4e5673ac5b80f6.3109 Call-ID: 1840229561@127.0.0.1 CSeq: 1 OPTIONS Server: Kamailio (1.5.3-notls (i386/linux)) Content-Length: 0 To this: $ output=`sipsak -v -s sip:somenotexistantextens...@ekiga.net 21` $ echo $output Content-Length: 0(1.5.3-notls (i386/linux))tag=c64e1f832a41ec1c1f4e5673ac5b80f6.8ff585.127.155.32 In this case it appears to be due to carriage-return (0x0d, ^M) characters that sipsak injects into its output, which is annoying but relatively benign. But maybe we want to sanitize the ocf_run output before we hand it off to be written to the logs? output=`echo $output` is there to make it one line, I think. Any funny control characters should not be a problem. well, unless you echo them to a terminal. ASCII NUL may be fun, though, and will probably differ accross shel flavour and versions. The terminal is also the problem you are seing there. In a log file, it will all be one line, with some spurious ^M. But on a terminal, well, CR is carriage return: start again at column 0. so it just prints one line over the other, and the trailing garbage is simply from the previous, longer lines. Of course, you could do output=`echo $output | cat -v` instead, but it loses information. If you are only bothered by CR, do IFS=$' \t\r\n' output=`echo $output` where $'' is most likely a bashism, but you get the idea. A tr equivalent is obviously output=`echo $output | tr -s ' \r' ' '` (\t and \n have been squeezed already by the leaving off the around $output) -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Sometimes not in our membership failure in 2-node cluster
? If you can easily reproduce (with most recent heartbeat), a tcpdump may be useful from, say, 10 heartbeats before you disable, to half a minute minute after you re-enable the network link. (or just crank up debuggin so high that you even see message dumps in the logs...) -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Linux-HA] [RfC] Review request for ocf:heartbeat:asterisk (Asterisk OCF RA)
On Thu, Nov 10, 2011 at 04:11:16PM +0100, Florian Haas wrote: Hi Dejan, thanks for the feedback! We've worked in most of your suggested changes, see below: More direct would be: if [ $? -ne 0 ]; then $? in a test is almost always an error. Because you lose the actual value it had. So rather ex=$?, [ $ex ... ], and add $ex to the log message. Yo have that error also later in Asterisk start command failed: $?, that will always log 0. compare the output of the next two lines. ( (exit 7); if [ $? -ne 0 ]; then echo ALWAYS 0!! exit code: $?; fi ) ( (exit 7); ex=$?; if [ $ex -ne 0 ]; then echo exit code: $ex; fi ) Btw, User $OCF_RESKEY_user doesn't exit there is an s missing. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Patch to JBoss RA
On Fri, Nov 04, 2011 at 09:42:50AM -0500, David Gersic wrote: On 11/3/2011 at 11:20 AM, Dejan Muhamedagic de...@suse.de wrote: Hunks 2 and 3 fail, don't know if it's due to space being mangled or the jboss RA version you worked on is old: I started with the newest JBoss RA I could find, but that was a while ago. Where can I get the current one? Also, was there a reason removing \n in the export lines? To be honest, I don't remember. I think I had problems getting it working with them in. I'll re-check that though. They are wrong. They would try to execute some binary named n. Yes, please, remove these bogus \n. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Administrivia
On Wed, Nov 02, 2011 at 02:27:16PM +0100, Lars Ellenberg wrote: On Wed, Nov 02, 2011 at 08:39:31AM +0100, Dejan Muhamedagic wrote: Hi, On Mon, Oct 31, 2011 at 04:11:53PM -0500, David Gersic wrote: The list info page at (http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev) and the welcome email message both have links to (http://linux-ha.org/HATodo.html). Following this leads to a page that does not contain any actual to do list content. OK, that should be fixed. I removed those stale links for now, and reworded both messages slightly. If someone has other suggestions for wording, or for (re-) adding links again, just let us know. And thank you, you are the first one to notice *and* complain about those broken links in more than 6 years ;-) Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Problem Heartbeat Changing Time Server
On Thu, Oct 27, 2011 at 10:18:13AM -0200, gilmarli...@agrovale.com.br wrote: Hello! Anyone know if the heartbeat can change the time for a server? Because when it gets close to 30 days uptime simply migrate services to server1 server2. Reviewing the logs you can check that the time is wrong. This timetable was given before the problem occurred and was correct. I think because of the time shift caused by the heartbeat generated this error in the log. You are using which version of heartbeat, on which platform? bare metal or within some VM? Oct 27 10:22:09 inga heartbeat: [2313]: WARN: Gmain_timeout_dispatch: Dispatch function for send local status was delayed 2998210 ms ( 10010 ms) before being called (GSource: 0x1f11370)Oct 27 10:22:09 inga heartbeat: [2313]: info: Gmain_timeout_dispatch: started at 1965715369 should have started at 1965415548Oct Please learn to paste ;-) 27 10:22:09 inga heartbeat: [2313]: WARN: Late heartbeat: Node inga: interval 3018270 ms Those close to 30 days uptime may not be rather close to 49 days? And you are using a 1000 Hz kernel? Or are you running within some VM, and those 30 days are your monthly backup run, which decided to freeze the VM for as long as it takes, and it just took too long, like, 50 minutes? -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] attrd and repeated changes
On Thu, Oct 20, 2011 at 08:48:36AM -0600, Alan Robertson wrote: On 10/20/2011 03:41 AM, Philipp Marek wrote: Hello, when constantly sending new data via attrd the changes are never used. Example: while sleep 1 do attrd_updater -l reboot -d 5 -n rep_chg -U try$SECONDS cibadmin -Ql | grep rep_chg done This always returns the same value - the one that was given with more than 5 seconds delay afterwards, so that the dampen interval wasn't broken by the next change. I've attached two draft patches; one for allowing the _first_ value in a dampen interval to be used (effectively ignoring changes until this value is written), and one for using the _last_ value in the dampen interval (by not changing the dampen timer). [1] *** Note: they are for discussion only! *** I didn't test them, not even for compilation. What is the correct way to handle multiple updates within the dampen interval? Personally, I'd vote for the last value. I agree with you about this being a bug. If the attribute is used to check connectivity changes (ping resource agent), or similar, and we have a flaky, flapping connectivity, it would be useful to have a max or min consolidation function for incoming values during a dampen interval. Otherwise, I get + + - + + -|+ + + + and if the dampen interval just happened to expire where I put the | above, it would have pushed a - to the cib, where I'd rather kept it at +. We likely want to add an option to attrd_updater (and to the ipc messages it sends to attrd, and to the rest of the chain involved), which can specify the consolidation function to be used. The initial set I suggest would be generic: oldest latest (default?) for values assumed to be numeric: max (also a candidate for default behaviour) min avg (with a printf like template for rounding, %.2f or similar, so we could even average boolean values) I suggest this behaviour: * If different updates request a different consolidation function, the last one (within the respective dampen interval) wins. * update with the _same_ value: Do not start or modify any timer. If a timer is pending, still add the value to the list of values to be processed by the consolidation function (relevant for avg, possibly not yet listed others). * update with a different value: Start a new timer, unless one is pending already. Do not restart/modify an already pending timer. Add to the list of values for the consolidation function. * Flush message received: expire timer. See below. * Timer expires: Apply consolidation function to list of values. If list is empty (probably flush message without pending timer), use current value. Send that result to the cib. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] [glue] Fix compilation with GCC 4.6
On Thu, Oct 13, 2011 at 01:01:26PM +0900, nozawat wrote:
Hi
BTW, does this get rid of the compiler warning as well?
The warning did not change.
I build in gcc4.6.1 of Fedora15.
---
apcsmart.c: In function 'apcsmart_hostlist':
apcsmart.c:725:34: error: cast discards '__attribute__((const))' qualifier
from pointer target type [-Werror=cast-qual]
cc1: all warnings being treated as errors
Yeah, well, I meanwhile got me a setup where the gcc is new enough to
warn about this. Could be fixed with (const char **)(void*).
But that only masks an unclean interface.
I suggest three steps:
- Changing the signature of get_confignames, and CopyHostList.
This is a boring mechanical patch.
- move qsort to sort the copy, not the supposedly const char * const *
- Re-enable -Wcast-qual
see below.
==
Changing the signature of get_confignames, and CopyHostList.
diff -r 61fd9fb20b99 include/stonith/stonith.h
--- a/include/stonith/stonith.h Wed Oct 12 20:49:40 2011 +0200
+++ b/include/stonith/stonith.h Wed Oct 12 23:00:54 2011 +0200
@@ -95,7 +95,7 @@
Stonith*stonith_new(const char * type);
void stonith_delete(Stonith *);
-const char** stonith_get_confignames (Stonith* s);
+const char * const * stonith_get_confignames (Stonith* s);
/* static/global return */
/* Return number and list of valid s_names */
diff -r 61fd9fb20b99 include/stonith/stonith_plugin.h
--- a/include/stonith/stonith_plugin.h Wed Oct 12 20:49:40 2011 +0200
+++ b/include/stonith/stonith_plugin.h Wed Oct 12 23:00:54 2011 +0200
@@ -53,7 +53,7 @@
void (*destroy) (StonithPlugin*); /*(full) Destructor */
const char* (*get_info) (StonithPlugin*, int infotype);
- const char** (*get_confignames) (StonithPlugin*);
+ const char * const * (*get_confignames) (StonithPlugin*);
int (*set_config) (StonithPlugin*, StonithNVpair* list);
/* Finishes construction */
/*
@@ -104,7 +104,7 @@
const char* (*GetValue)(StonithNVpair*, const char * name);
int (*CopyAllValues) (StonithNamesToGet* out, StonithNVpair* in);
char **(*StringToHostList)(const char * hlstring);
- char **(*CopyHostList)(const char ** hlstring);
+ char **(*CopyHostList)(const char * const * hlstring);
void (*FreeHostList)(char** hostlist);
int (*TtyLock)(const char* tty);
int (*TtyUnlock)(const char* tty);
diff -r 61fd9fb20b99 lib/plugins/stonith/apcmaster.c
--- a/lib/plugins/stonith/apcmaster.c Wed Oct 12 20:49:40 2011 +0200
+++ b/lib/plugins/stonith/apcmaster.c Wed Oct 12 23:00:54 2011 +0200
@@ -65,7 +65,7 @@
static StonithPlugin * apcmaster_new(const char *);
static voidapcmaster_destroy(StonithPlugin *);
-static const char ** apcmaster_get_confignames(StonithPlugin *);
+static const char * const *apcmaster_get_confignames(StonithPlugin *);
static int apcmaster_set_config(StonithPlugin *, StonithNVpair *);
static const char *apcmaster_getinfo(StonithPlugin * s, int InfoType);
static int apcmaster_status(StonithPlugin * );
@@ -678,7 +678,7 @@
/*
* Get the configuration parameters names
*/
-static const char **
+static const char * const *
apcmaster_get_confignames(StonithPlugin * s)
{
static const char * ret[] = {ST_IPADDR, ST_LOGIN, ST_PASSWD, NULL};
diff -r 61fd9fb20b99 lib/plugins/stonith/apcsmart.c
--- a/lib/plugins/stonith/apcsmart.cWed Oct 12 20:49:40 2011 +0200
+++ b/lib/plugins/stonith/apcsmart.cWed Oct 12 23:00:54 2011 +0200
@@ -109,7 +109,7 @@
static StonithPlugin * apcsmart_new(const char *);
static voidapcsmart_destroy(StonithPlugin *);
-static const char**apcsmart_get_confignames(StonithPlugin*);
+static const char * const *apcsmart_get_confignames(StonithPlugin*);
static int apcsmart_set_config(StonithPlugin *, StonithNVpair*);
static const char *apcsmart_get_info(StonithPlugin * s, int InfoType);
static int apcsmart_status(StonithPlugin * );
@@ -621,7 +621,7 @@
ad-upsfd = -1;
}
}
-static const char**
+static const char * const *
apcsmart_get_confignames(StonithPlugin* sp)
{
static const char * names[] = {ST_TTYDEV, ST_HOSTLIST, NULL};
@@ -719,7 +719,7 @@
}
ERRIFNOTCONFIGED(s,NULL);
- return OurImports-CopyHostList((const char **)ad-hostlist);
+ return OurImports-CopyHostList((const char **)(void*)ad-hostlist);
}
static gboolean
diff -r 61fd9fb20b99 lib/plugins/stonith/baytech.c
--- a/lib/plugins/stonith/baytech.c Wed Oct 12 20:49:40 2011 +0200
+++ b/lib/plugins/stonith/baytech.c Wed Oct 12 23:00:54 2011 +0200
@@ -37,7 +37,7 @@
static StonithPlugin * baytech_new(const char *);
static voidbaytech_destroy(StonithPlugin *);
static
Re: [Linux-ha-dev] Patches for named and pgsql
On Thu, Oct 13, 2011 at 06:40:50AM -0600, Serge Dubrouski wrote: Hello - Attached are patches for named and pgsql patches that repace == with = when compare strings. Pushed, as well as a similar fix for LVM. /heartbeat/pgsql +++ b/heartbeat/pgsql @@ -644,7 +644,7 @@ esac pgsql_validate_all rc=$? -[ $1 == validate-all ] exit $rc +[ $1 = validate-all ] exit $rc if [ $rc -ne 0 ] then --- a/heartbeat/pgsql +++ b/heartbeat/pgsql @@ -644,7 +644,7 @@ esac pgsql_validate_all rc=$? -[ $1 == validate-all ] exit $rc +[ $1 = validate-all ] exit $rc if [ $rc -ne 0 ] then ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
[Linux-ha-dev] STONITH plugin vcenter, please check for regression
To fix unfullfillable rpm dependencies on the toolkit/sdk,
I recently pushed http://hg.linux-ha.org/glue/rev/508dd314f129
(also inline below; - use, + eval { require })
Anyone actually using this agent, please check and confirm
if that change breaks anything, or if it still works as intended.
Thanks,
# HG changeset patch
# User Lars Ellenberg l...@linbit.com
# Date 1316533223 -7200
# Node ID 508dd314f1291bdbd0e01d17b2c487820a79093b
# Parent 45b21f952b0fcf5f7ce2d490dd20a01a9ddc194c
Medium: fix stonith/external/vcenter to eval { require VMware::VIRuntime }
This should log a better error message, and as a side-effect, should cause
find-requires resp. perl.req to not RPM-require that module.
--- a/lib/plugins/stonith/external/vcenter
+++ b/lib/plugins/stonith/external/vcenter
@@ -114,7 +114,8 @@ Allowed values: 0, 1
# Command belongs to the group of commands that require connecting to VMware
vCenter
elsif ($command ~~ @netCommands) {
- use VMware::VIRuntime;
+ eval { require VMware::VIRuntime; }
+ or dielog(Missing perl module VMware::VIRuntime. Download and install
'VMware Infrastructure (VI) Perl Toolkit', available at
http://www.vmware.com/support/developer/viperltoolkit/ \n);
# A valid VI_CREDSTORE is required to avoid interactive prompt
( exists $ENV{'VI_CREDSTORE'} ) || dielog(VI_CREDSTORE not
specified\n);
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH 00 of 10] logd improvements v2
On Wed, Nov 10, 2010 at 08:23:42PM +0100, Bernd Schubert wrote: ... But: X-Mailman-Approved-At: Wed, 24 Aug 2011 05:32:24 -0600 So, ignore this. Has been dealt with 9 month ago... -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Dovecot OCF Resource Agent
On Fri, Aug 19, 2011 at 04:07:04PM +0200, jer...@intuxicated.org wrote:
On Wed, 3 Aug 2011 15:34:37 +0200, Dejan Muhamedagic de...@suse.de wrote:
Hi Jeroen,
On Fri, Jul 22, 2011 at 10:51:56AM +0200, jer...@intuxicated.org wrote:
On Fri, 15 Apr 2011 14:45:59 +0200, Raoul Bhatia [IPAX]
r.bha...@ipax.at
wrote:
On 04/15/2011 01:19 PM, Andrew Beekhof wrote:
On Fri, Apr 15, 2011 at 12:53 PM, Raoul Bhatia [IPAX]
r.bha...@ipax.at
wrote:
On 04/15/2011 11:10 AM, jer...@intuxicated.org wrote:
Yes, it does the same thing but contains some additional features,
like
logging into a mailbox.
first of all, i do not know how the others think about a ocf ra
implemented in c. i'll suggest waiting for comments from dejan or
fghass.
the ipv6addr agent was written in C too
the OCF standard does not dictate the language to be used - its
really
a matter of whether C is the best tool for this job
thank you andrew!
jeroen, can you please create a github fork off
https://github.com/ClusterLabs/ (it's really easy!)
and add your resource agent in the same fashion as IPv6addr.c [1] ?
thanks,
raoul
[1]
https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/IPv6addr.c
Hi,
I finally found some time to get the code on GitHub.
https://github.com/perrit/dovecot-ocf-resource-agent
As you can see it's kind of hard to merge the code in the same way as
IPv6addr.c as it currently spans multiple files. Would you like me to
just
put it in a directory? Maybe it's a good idea to split the dovecot part
and
the mailbox login part, so that there's a mailbox login resource agent
becomes more like the ping resource agent?
I really hate to say it, since you obviously invested quite a
bit of time to put together this agent, but C is arguably not
the best suited programming language for resource agents. I
guess that's why all init scripts are, well, shell scripts. And
all but one of our OCF resource agents. The code is around
4kloc, which is as big as some of our subsystems. That's a lot
of code to read and maintain.
Was there a good reason to choose C for the implementation?
Cheers,
Dejan
Hi Dejan,
Sorry for the late reply.
Well I the wanted to automatically decide which interfaces should be
monitored (POP3/IMAP). I had a lot of trouble parsing with sh/bash,
If trouble parsing the doveconf output was/is the main reason
for using C, maybe I can help.
That should be fairly easy, actually,
seeing that you only care about very few settings?
Especially if you change your doveconf invocation to
doveconf -c $config_path base_dir listen shutdown_clients \
service/imap-login/inet_listener/{address,port,ssl} \
service/pop3-login/inet_listener/{address,port,ssl}
I don't see too much complexity there,
but maybe I'm missing something.
so I
decided this would easier to do in C. Most of the code is needed to imitate
the ocf bash functions in C, if those functions could be provided in a
library (which others could also benefit from of course), the resource
agent itself can be much, much smaller.
Maybe the the POP3/IMAP login part can be split from the dovecot specific
part, and provide a seperate resource agent, much like the ping resource
agent. So that other POP3/IMAP servers like courier or perdition for
example can also be monitored. The resource agent for the dovecot service
itself can then simply be written in sh/bash.
If I ever want to write a new resource agent, is there a list of basic
requirements somewhere to get the resource agent included?
sh or bash is probably much faster to get included.
If you have a previous version of this in python or bash,
or could point out where your problems have been, maybe we can
revive/write it in python or bash more quickly than getting the C
version included?
I'm offering my help with doing it in bash.
If you don't want to go that way, that is ok with me, too.
It certainly won't hurt to have a set of C ocf library functions around
for cases where it really is easier do get the job done in C.
Lars
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [patch] conntrackd RA
On Thu, Aug 18, 2011 at 12:28:44PM +0200, Albéric de Pertat wrote:
Hi,
While playing with the conntrackd agent on Debian Squeeze, I found out the
method used in the monitor action is not accurate and can sometimes yield
false results, believing conntrackd is running when it is not.
As in when?
Could that be resolved?
I am instead checking the existence of the control socket and it has
so far proved more stable.
If you kill -9 conntrackd (or contrackd should crash for some reason),
it will leave behind that socket.
So testing on the existence of that socket is in no way more reliable
than looking through the process table.
Maybe there is some ping method in the conntrack-tools?
Or something that could be used as such?
If not, maybe try a connect to that socket, using netcat/socat?
Just checking if the socket exists will not detect conntrackd crashes.
--- conntrackd2011-08-18 12:12:36.807562142 +0200
+++ /usr/lib/ocf/resource.d/heartbeat/conntrackd 2011-08-18
12:25:20.0 +0200
@@ -111,8 +111,10 @@
conntrackd_monitor() {
rc=$OCF_NOT_RUNNING
- # It does not write a PID file, so check with pgrep
- pgrep -f $OCF_RESKEY_binary rc=$OCF_SUCCESS
+# It does not write a PID file, so check the socket exists after
+# extracting its path from the configuration file
+local conntrack_socket=$(awk '/^ *UNIX *{/,/^ *}/ { if ($0 ~ /^
*Path /) { print $2 } }' $OCF_RESKEY_config)
+[ -S $conntrack_socket ] rc=$OCF_SUCCESS
if [ $rc -eq $OCF_SUCCESS ]; then
# conntrackd is running
# now see if it acceppts queries
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [patch] conntrackd RA
On Thu, Aug 18, 2011 at 12:58:21PM +0200, Lars Ellenberg wrote:
On Thu, Aug 18, 2011 at 12:28:44PM +0200, Albéric de Pertat wrote:
Hi,
While playing with the conntrackd agent on Debian Squeeze, I found out the
method used in the monitor action is not accurate and can sometimes yield
false results, believing conntrackd is running when it is not.
As in when?
Could that be resolved?
I am instead checking the existence of the control socket and it has
so far proved more stable.
If you kill -9 conntrackd (or contrackd should crash for some reason),
it will leave behind that socket.
So testing on the existence of that socket is in no way more reliable
than looking through the process table.
Maybe there is some ping method in the conntrack-tools?
Or something that could be used as such?
Ah, my bad...
should have looked not at the patch only,
but at the RA in context.
That check if it accepts queries is done
immediately following this test.
So yes, your patch is good. Acked-by lars ;-)
--- conntrackd 2011-08-18 12:12:36.807562142 +0200
+++ /usr/lib/ocf/resource.d/heartbeat/conntrackd2011-08-18
12:25:20.0 +0200
@@ -111,8 +111,10 @@
conntrackd_monitor() {
rc=$OCF_NOT_RUNNING
- # It does not write a PID file, so check with pgrep
- pgrep -f $OCF_RESKEY_binary rc=$OCF_SUCCESS
+# It does not write a PID file, so check the socket exists after
+# extracting its path from the configuration file
+local conntrack_socket=$(awk '/^ *UNIX *{/,/^ *}/ { if ($0 ~ /^
*Path /) { print $2 } }' $OCF_RESKEY_config)
+[ -S $conntrack_socket ] rc=$OCF_SUCCESS
if [ $rc -eq $OCF_SUCCESS ]; then
# conntrackd is running
# now see if it acceppts queries
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [patch] conntrackd RA
On Thu, Aug 18, 2011 at 01:24:04PM +0200, Lars Ellenberg wrote:
On Thu, Aug 18, 2011 at 12:58:21PM +0200, Lars Ellenberg wrote:
On Thu, Aug 18, 2011 at 12:28:44PM +0200, Albéric de Pertat wrote:
Hi,
While playing with the conntrackd agent on Debian Squeeze, I found out
the
method used in the monitor action is not accurate and can sometimes yield
false results, believing conntrackd is running when it is not.
As in when?
Could that be resolved?
I am instead checking the existence of the control socket and it has
so far proved more stable.
If you kill -9 conntrackd (or contrackd should crash for some reason),
it will leave behind that socket.
So testing on the existence of that socket is in no way more reliable
than looking through the process table.
Maybe there is some ping method in the conntrack-tools?
Or something that could be used as such?
Ah, my bad...
should have looked not at the patch only,
but at the RA in context.
That check if it accepts queries is done
immediately following this test.
So yes, your patch is good. Acked-by lars ;-)
Oh, these monologues...
--- conntrackd2011-08-18 12:12:36.807562142 +0200
+++ /usr/lib/ocf/resource.d/heartbeat/conntrackd 2011-08-18
12:25:20.0 +0200
@@ -111,8 +111,10 @@
conntrackd_monitor() {
rc=$OCF_NOT_RUNNING
- # It does not write a PID file, so check with pgrep
- pgrep -f $OCF_RESKEY_binary rc=$OCF_SUCCESS
+# It does not write a PID file, so check the socket exists after
+# extracting its path from the configuration file
+local conntrack_socket=$(awk '/^ *UNIX *{/,/^ *}/ { if ($0 ~ /^
*Path /) { print $2 } }' $OCF_RESKEY_config)
Is space really the only allowed white space there?
I guess the regex has to be changed to use [[:space:]]*
local conntrack_socket=$(awk '/^[[:space:]]*UNIX[[:space:]]*{/,/^[[:space:]]*}/
{ if ($1 == Path) { print $2 } }' $OCF_RESKEY_config)
Maybe rather do it the other way round, see below.
+[ -S $conntrack_socket ] rc=$OCF_SUCCESS
if [ $rc -eq $OCF_SUCCESS ]; then
# conntrackd is running
# now see if it acceppts queries
(untested)
diff --git a/heartbeat/conntrackd b/heartbeat/conntrackd
--- a/heartbeat/conntrackd
+++ b/heartbeat/conntrackd
@@ -110,16 +110,17 @@ conntrackd_set_master_score() {
}
conntrackd_monitor() {
- rc=$OCF_NOT_RUNNING
- # It does not write a PID file, so check with pgrep
- pgrep -f $OCF_RESKEY_binary rc=$OCF_SUCCESS
- if [ $rc -eq $OCF_SUCCESS ]; then
- # conntrackd is running
- # now see if it acceppts queries
- if ! $OCF_RESKEY_binary -C $OCF_RESKEY_config -s /dev/null
21; then
+ # see if it acceppts queries
+ if ! $OCF_RESKEY_binary -C $OCF_RESKEY_config -s /dev/null 21; then
+ local conntrack_socket=$(awk
'/^[[:space:]]*UNIX[[:space:]]*{/,/^[[:space:]]*}/ { if ($1 == Path) { print
$2 } }' $OCF_RESKEY_config)
+ if test -S $conntrack_socket ; then
rc=$OCF_ERR_GENERIC
- ocf_log err conntrackd is running but not responding
to queries
+ ocf_log err conntrackd control socket exists, but not
responding to queries
+ else
+ rc=$OCF_NOT_RUNNING
fi
+ else
+ rc=$OCF_SUCCESS
if conntrackd_is_master; then
rc=$OCF_RUNNING_MASTER
# Restore master setting on probes
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] OCF RA for named
On Tue, Aug 16, 2011 at 08:51:04AM -0600, Serge Dubrouski wrote: On Tue, Aug 16, 2011 at 8:44 AM, Dejan Muhamedagic de...@suse.de wrote: Hi Serge, On Fri, Aug 05, 2011 at 08:19:52AM -0600, Serge Dubrouski wrote: No interest? Probably not true :) It's just that recently I've been away for a while and in between really swamped with my daily work. I'm trying to catch up now, but it may take a while. In the meantime, I'd like to ask you about the motivation. DNS already has a sort of redundancy built in through its primary/secondary servers. That redundancy doesn't work quite well. Yes you can have primary and secondary servers configured in resolv.conf but if primary is down resolver waits till request times out for the primary server till it sends a request to the secondary one. The dealy can be up to 30 seconds and impacts some applications pretty badly, This is standard behaviour for Linux, Solaris for example works differently and isn't impacted by this issue. Works around are having caching DNS server working locally or having primary DNS server highly available with using Pacemaker :-) Here is what man page for resolv.conf says: nameserver Name server IP address Internet address (in dot notation) of a name server that the resolver should query. Up to MAXNS (currently 3, see resolv.h) name servers may be listed, one per keyword. If there are multiple servers, the resolver library queries them in the order listed. If no nameserver entries are present, the default is to use the name server on the local machine. *(The algorithm used is to try a name server, and if the query times out, try the next, until out of name servers, then repeat trying all the name servers until a maximum number of retries are made.)* options timeout:2 attempts:5 rotate but yes, it is still a valid use case to have a clustered primary name server, and possibly multiple backups. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] [glue] Fix compilation with GCC 4.6
On Wed, Jun 29, 2011 at 12:55:08PM +0100, Pádraig Brady wrote:
On 29/06/11 11:56, Lars Ellenberg wrote:
On Tue, Jun 28, 2011 at 01:44:23AM +0100, Pádraig Brady wrote:
The attached patch fixes compilation -Werrors with GCC 4.6
cheers,
Pádraig.
Fix compilation with GCC 4.6
avoid -Werror=unused-but-set-variable issues
remove -Wcast-qual which caused issues with copyHostList()
Care to explain or show those issues?
All uses of copyHostList error like:
apcsmart.c: In function 'apcsmart_hostlist':
apcsmart.c:722:34: error: to be safe all intermediate pointers in cast from
'char **' to 'const char **' must be 'const' qualified [-Werror=cast-qual]
From the gcc 4.5 documentation:
-Wcast-qual
Warn whenever a pointer is cast so as to remove a type qualifier from
the target
type. For example, warn if a const char * is cast to an ordinary char *.
Also warn when making a cast which introduces a type qualifier in an
unsafe
way. For example, casting char ** to const char ** is unsafe, as in
this ex-
ample:
/* p is char ** value. */
const char **q = (const char **) p;
/* Assignment of readonly string to const char * is OK.
*q = string;
/* Now char** pointer points to read-only memory. */
**p = ’b’;
So apparently it needs to be
include/stonith/stonith_plugin.h:
struct StonithImports_s {
...
- char **(*CopyHostList)(const char ** hlstring);
+ char **(*CopyHostList)(const char * const * hlstring);
And the callers have to be adjusted accordingly?
Could you check if that works?
- nbytes=vsnprintf(buf, sizeof(buf)-1, fmt, ap);
+ (void) vsnprintf(buf, sizeof(buf)-1, fmt, ap);
What is that (void) supposed to achieve?
Just a personal preference to document
we're discarding the return on purpose.
I'll remove this extra syntax.
In that case, just leave it in, no problem there.
I just wondered it was to suppress yet an other warning,
and which one that might be ;-)
Updated patch attached.
Thanks, will have a look soon.
Dejan, anyone, what do you think?
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] [glue] Fix compilation with GCC 4.6
@@ op-user_data_len = strlen(op-user_data)+1; op-interval = 2000; op-target_rc=EVERYTIME; - call_id = rsc-ops-perform_op(rsc,op); + rsc-ops-perform_op(rsc,op); printf_op(op); lrm_free_op(op); @@ -180,7 +179,7 @@ op-user_data_len = strlen(op-user_data)+1; op-interval = 3000; op-target_rc=EVERYTIME; - call_id = rsc-ops-perform_op(rsc,op); + rsc-ops-perform_op(rsc,op); printf_op(op); lrm_free_op(op); diff -r 856ae1408ff9 lrm/test/callbacktest.c --- a/lrm/test/callbacktest.c Sun Jun 19 21:03:24 2011 +0200 +++ b/lrm/test/callbacktest.c Tue Jun 28 01:23:28 2011 +0100 @@ -44,7 +44,6 @@ lrm_op_t* op = NULL; const char* rid = ip248; GHashTable* param = NULL; - int call_id; lrm = ll_lrm_new(lrm); @@ -94,7 +93,7 @@ op-user_data_len = strlen(op-user_data)+1; op-interval = 1000; op-target_rc=EVERYTIME; - call_id = rsc-ops-perform_op(rsc,op); + rsc-ops-perform_op(rsc,op); printf_op(op); puts(perform_op(stop)...); ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/ -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Uniquness OCF Parameters
On Thu, Jun 16, 2011 at 03:52:37PM -0600, Alan Robertson wrote: On 06/16/2011 02:51 AM, Lars Ellenberg wrote: On Thu, Jun 16, 2011 at 09:48:20AM +0200, Florian Haas wrote: On 2011-06-16 09:03, Lars Ellenberg wrote: With the current unique=true/false, you cannot express that. Thanks. You learn something every day. :) Sorry that I left off the As you are well aware of, introductionary phrase. ;-) I just summarized the problem: Depending on what we chose the meaning to be, parameters marked unique=true would be required to either be all _independently_ unique, or be unique as a tuple. And made a suggestion how to solve it: If we want to be able to express both, we need a different markup. Of course, we can move the markup out of the parameter description, into an additional markup, that spells them out, likeunique params=foo,bar /unique params=bla /. But using unique=0 as the current non-unique meaning, then unique=small-integer-or-even-named-label-who-cares, would name the scope for this uniqueness requirement, where parameters marked with the same such label would form a unique tuple. Enables us to mark multiple tuples, and individual parameters, at the same time. If we really think it _is_ a problem. If one wanted to, one could say unique=1,3 or unique=1 unique=3 Then parameters which share the same uniqueness list are part of the same uniqueness grouping. Since RAs today normally say unique=1, if one excluded the unique group 0 from being unique, then this could be done in a completely upwards-compatible way for nearly all resources. That is what I suggested, yes. Where unique=0 is basically not mentioning the unique hint. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Uniquness OCF Parameters
On Wed, Jun 15, 2011 at 04:07:27PM +0200, Florian Haas wrote: On 2011-06-15 15:50, Alan Robertson wrote: On 06/14/2011 06:03 AM, Florian Haas wrote: On 2011-06-14 13:08, Dejan Muhamedagic wrote: Hi Alan, On Mon, Jun 13, 2011 at 10:32:02AM -0600, Alan Robertson wrote: On 06/13/2011 04:12 AM, Simon Talbot wrote: A couple of observations (I am sure there are more) on the uniqueness flag for OCF script parameters: Would it be wise for the for the index parameter of the SFEX ocf script to have its unique flag set to 1 so that the crm tool (and others) would warn if one inadvertantly tried to create two SFEX resource primitives with the same index? Also, an example of the opposite, the Stonith/IPMI script, has parameters such as interface, username and password with their unique flags set to 1, causing erroneous warnings if you use the same interface, username or password for multiple IPMI stonith primitives, which of course if often the case in large clusters? When we designed it, we intended that Unique applies to the complete set of parameters - not to individual parameters. It's like a multi-part unique key. It takes all 3 to create a unique instance (for the example you gave). That makes sense. Does it really? Then what would be the point of having some params that are unique, and some that are not? Or would the tuple of _all_ parameters marked as unique be considered unique? I don't know what you think I said, but A multi-part key to a database is a tuple which consists of all marked parameters. You just said what I said in a different way. So we agree. Jfyi, I was asking a question, not stating an opinion. Hence the use of a question mark. ;-) So then, if the uniqueness should be enforced for a unique key that is comprised of _all_ the parameters marked unique in a parameter set, then what would be the correct way to express required uniqueness of _individual_ parameters? In other words, if I have foo and bar marked unique, then one resource with foo=1 and bar=2, and another with foo=1, bar=3 does not violate the uniqueness constraint. What if I want both foo and bar to be unique in and of themselves, so any duplicate use of foo=1 should be treated as a uniqueness violation? With the current unique=true/false, you cannot express that. Depending on what we chose the meaning to be, parameters marked unique=true would be required to either be all _independently_ unique, or be unique as a tuple. If we want to be able to express both, we need a different markup. Of course, we can move the markup out of the parameter description, into an additional markup, that spells them out, like unique params=foo,bar /unique params=bla /. But using unique=0 as the current non-unique meaning, then unique=small-integer-or-even-named-label-who-cares, would name the scope for this uniqueness requirement, where parameters marked with the same such label would form a unique tuple. Enables us to mark multiple tuples, and individual parameters, at the same time. Question is: do we really want or need that. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Uniquness OCF Parameters
On Thu, Jun 16, 2011 at 09:48:20AM +0200, Florian Haas wrote: On 2011-06-16 09:03, Lars Ellenberg wrote: With the current unique=true/false, you cannot express that. Thanks. You learn something every day. :) Sorry that I left off the As you are well aware of, introductionary phrase. ;-) I just summarized the problem: Depending on what we chose the meaning to be, parameters marked unique=true would be required to either be all _independently_ unique, or be unique as a tuple. And made a suggestion how to solve it: If we want to be able to express both, we need a different markup. Of course, we can move the markup out of the parameter description, into an additional markup, that spells them out, like unique params=foo,bar /unique params=bla /. But using unique=0 as the current non-unique meaning, then unique=small-integer-or-even-named-label-who-cares, would name the scope for this uniqueness requirement, where parameters marked with the same such label would form a unique tuple. Enables us to mark multiple tuples, and individual parameters, at the same time. If we really think it _is_ a problem. Question is: do we really want or need that. That is a discussion for the updated OCF RA spec discussion, really. And the driver of that discussion is currently submerged. :) I guess this was @LMB? Hey there ... do you read? :) As to stood the test of time, well, no. Not these resource agent parameter hints. Not yet. Especially the unique and type hints have been mostly ignored until now, the type hints are still wrong for some resource agents last time I checked, and also mostly ignored, and the unique hint just starts to throw a warning in crm shell now. So, because these hints have been ignored so far, they have not been tested, not even by time... These hints are also not enforced by the cib (which does not know about them anyways), but only hints to some frontend. And because some frontend now started to at least consider these hints, we are having this discussion now... Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
[Linux-ha-dev] [Linux-HA] Announcement for Heartbeat 3.0.5
About two month ago, dealing with a bug report of some paying customer, I fixed some long standing bugs in the heartbeat communication layer that caused heartbeat to segfault, and other bad behaviour. These bugs where triggered by misbehaving API clients, respectively massive packet loss on the communication links, and have been present basically since inception. The changelog does not really look spectacular, but is supposed to very much improve the robustness of the heartbeat communication stack if you experience massive packet loss on all channels, for whatever reason. As these are fixes that affect the heartbeat messaging core, they are relevant for both Pacemaker and haresources style clusters. Changelog: - do not request retransmission of lost messages from dead members - fix segfault due to recursion in api_remove_client_pid - properly cleanup pending delayed rexmit requests before reset of seqtrack - create HA_RSCTMP on start, if necessary - improve detection of pacemaker clusters in init script Tarball: http://hg.linux-ha.org/heartbeat-STABLE_3_0/archive/STABLE-3.0.5.tar.bz2 Enjoy! -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] postfix's postfix_validate_all and initial monitor action
On Tue, Jun 14, 2011 at 05:45:21PM +0200, Raoul Bhatia [IPAX] wrote: this caused errors for the initial probe, so i did the following change: LSB_STATUS_STOPPED=3 if [ $ret -ne $OCF_SUCCESS ] || ocf_is_probe; then (see the new ocf_is_probe?) case $1 in stop) exit $OCF_SUCCESS ;; monitor) exit $OCF_NOT_RUNNING;; status) exit $LSB_STATUS_STOPPED;; *) exit $ret;; esac fi so we always enter this case in the event of a probe. this correctly handles the initial probe and returns OCF_NOT_RUNNING so that pacemaker can continue. *but* the command crm resource reprobe is also considered a ocf_is_probe. thus, this block will return a OCF_NOT_RUNNING on *every* node. the standby node *not* running postfix (which is ok) but also on the node which actually *is* running postfix. (and it would also return OCF_NOT_RUNNING if postfix was started at system bootup...) this lets the cluster believe the resource is not running and - because of my configuration - the resource will be (re)started on the last known location/node (which in fact is still running postfix). i hope i managed to explain it properly. :) Yep. That code is clearly broken. A probe (regardless of initial, manual or for whatever reason) has to correctly report the current status. Your probe always returns not running. Fix that ;-) Dejan: can we have an additional branch in ocf-tester, that checks something like probe if stopped returns $OCF_NOT_RUNNING, probe if started returns $OCF_SUCCESS Lars -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
[Linux-ha-dev] RC1 announcement for Heartbeat 3.0.5
About two month ago, dealing with a bug report of some paying customer, I fixed some long standing bugs in the heartbeat communication layer that caused heartbeat to segfault, and other bad behaviour. These bugs where triggered by misbehaving API clients, respectively massive packet loss on the communication links, and have been present basically since inception. The changelog does not really look spectacular, but is supposed to very much improve the robustness of the heartbeat communication stack if you experience massive packet loss on all channels, for whatever reason. As these are fixes affect the heartbeat messaging core, they are relevant for both Pacemaker and haresources style clusters. Changelog: - do not request retransmission of lost messages from dead members - fix segfault due to recursion in api_remove_client_pid - properly cleanup pending delayed rexmit requests before reset of seqtrack - create HA_RSCTMP on start, if necessary Tarball: http://hg.linux-ha.org/heartbeat-STABLE_3_0/archive/STABLE-3.0.5-rc1.tar.bz2 We expect no further changes. If you want to give it a whirl on your test clusters, please do so, and report any issues before Wednesday June 15, when we expect to cut the 3.0.5 final. Enjoy! -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Linux-HA] Hetzner server stonith agent
of zero, anyways... but see above, to have it explicit and obvious and robust, just leave it in. ;; getinfo-devid) echo Hetzner STONITH device exit 0 ;; getinfo-devname) echo Hetzner STONITH external device exit 0 ;; getinfo-devdescr) echo Hetzner host reset echo Manages the remote webservice for reset a remote server. exit 0 ;; getinfo-devurl) echo http://wiki.hetzner.de/index.php/Robot_Webservice_en; exit 0 ;; getinfo-xml) cat HETZNERXML parameters parameter name=hostname unique=1 content type=string / shortdesc lang=en Hostname /shortdesc longdesc lang=en The name of the host to be managed by this STONITH device. /longdesc /parameter parameter name=remote_ip unique=1 required=1 content type=string / shortdesc lang=en Remote IP /shortdesc longdesc lang=en The address of the remote IP that manages this server. /longdesc /parameter /parameters HETZNERXML exit 0 ;; *) maybe: echo 2 Don't know what to do for '$1' or display some usage or URL of wiki page or whatever exit 1 ;; esac Thank you! Lars -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Linux-HA] Hetzner server stonith agent
On Thu, May 26, 2011 at 10:50:52PM +0200, Lars Ellenberg wrote: case $1 in gethosts) echo $hostname exit 0 ;; on) # Can't really be implemented because Hetzner webservice cannot power on a system exit 1 ;; off) # Can't really be implemented because Hetzner webservice cannot power on a system BTW, is this true? in that wiki page, echo http://wiki.hetzner.de/index.php/Robot_Webservice_en; There is POST /boot/server-ip/linux Would that not be a power on? Not that this was important... Reset is perfect. Lars -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Linux-HA] Hetzner server stonith agent
On Thu, May 26, 2011 at 10:58:44PM +0200, Lars Ellenberg wrote: On Thu, May 26, 2011 at 10:50:52PM +0200, Lars Ellenberg wrote: case $1 in gethosts) echo $hostname exit 0 ;; on) # Can't really be implemented because Hetzner webservice cannot power on a system exit 1 ;; off) # Can't really be implemented because Hetzner webservice cannot power on a system BTW, is this true? in that wiki page, echo http://wiki.hetzner.de/index.php/Robot_Webservice_en; There is POST /boot/server-ip/linux Would that not be a power on? alright, seems to just be change boot settings, not boot per se. Oh well... Not that this was important... Reset is perfect. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [Linux-HA] Hetzner server stonith agent
On Tue, May 24, 2011 at 03:41:53PM +0200, Dejan Muhamedagic wrote:
On Tue, May 24, 2011 at 12:44:42PM +0200, RaSca wrote:
Il giorno Mar 24 Mag 2011 12:27:04 CET, Dejan Muhamedagic ha scritto:
Hi,
Hi Dejan,
[...]
# Read parameters
conf_file=/etc/hetzner.cfg
user=`cat /etc/hetzner.cfg | egrep ^user.*= | sed 's/^user.*=\ *//g'`
Better:
user=`sed -n 's/^user.*=\ *//p' /etc/hetzner.cfg`
Absolutely agree.
pass=`cat /etc/hetzner.cfg | egrep ^pass.*= | sed 's/^pass.*=\ *//g'`
hetzner_server=https://robot-ws.your-server.de;
I assume that this is a well-known URL which doesn't need to be
passed as a parameter.
As far as I know it is the only address, I hard-coded it for this
reason, but maybe should be a parameter...
OK.
is_host_up() {
if [ $1 != ]
then
status=`curl -s -u $user:$pass $hetzner_server/server/$1 | sed
's/.*status\:\([A-Za-z]*\),.*/\1/g'`
if [ $status = ready ]
then
return 0
else
return 1
fi
This if statement can be reduced to (you save 5 lines):
[ $status = ready ]
else
return 1
fi
}
You mean the statement should be:
[ $status = ready ] return 0
return 1
?
No, just [ $status = ready ]. The return is implied at the
end of the function and the code is the exit code of the last
command.
[...]
Again, better (is return code of is_host_up inverted?):
is_host_up $hostaddress
exit # this is actually also superfluous, but perhaps better left in
The action is reset, so if I had success then is_host_up must be NOT
ready. Or not?
Right, sorry, I should've turned on my brain beforehand. You can
try this:
exit $((! $?))
That is going to invert the code.
the shell has ! for that.
! is_host_up
Coffee?
;-)
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] New OCF RA: symlink
On Wed, May 04, 2011 at 03:06:27PM +0200, Florian Haas wrote: Coming back to this one, as the discussion seems to have died down. On 2011-04-20 19:00, Lars Ellenberg wrote: Oh, well, thinking about non-roots that may have cibadmin karma, they now can configure a resource that will remove /etc/passwd. I'm not sure if I like that. How about a staged system? Double symlinks? Similar to the alternatives system in Debian or others. The RA will force a single directory that will contain the indirection symlinks, and will sanitize (or force) link names to not contain slashes. The real symlinks will point to that indirection symlink, which will point to the end location. /etc/postfix/main.cf - /var/lib/wherever-indirection-dir/postfix_main.cf === - /mnt/somewhere/you/want/to/point/to/main.cf And === will be managed by the resource agent. Considering we have an anything resource agent which, well, lets us do anything, I consider this pointless cluttering of the resource agent which creates a false sense of security. Thoughts? Trying to think out loud... There are a few aspects. * if we don't have ACLs, anyone with write access to the CIB is de facto root on the cluster nodes. Which makes privilege escalation bugs non-existent by definition ;-) * potential privilege escalation Well, to make this an issue, we would need to establish that * the cib ACL system is in place, * and used, * and properly configured, which would probably also mean that access would be limited to only certain resource agent types. Most likely the ACL will be short as well: one role that can do cleanup and otherwise only view things, one role that can do everything (equivalent to root). Services running under Pacemaker control are probably critical, so a malicious person with even only stop access on the CIB can do a DoS. I guess we have to assume people with any write access at all to the CIB are trusted, and not malicious. Adding more insecure RAs would just keep the white list shorter. No problem with me. White list would probably be really short anyways. Still we may want to try and limit the potential damage that can be done by roles that have only limited rights (start/stop/reconfigure existing). /me not sure about all this ACL stuff anyways, I'd try to avoid it where ever possible ;-) * limit potential damage on accidental carelessness Well, yes, I think that is something we could aim for, too. And a some more thoughts, but I would probably need a few beers to make them appear coherent even to myself ;-) -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Crowd-sourceing request to review RAs for missing input sanitation/missing quotes/missing escapes etc. [Was: New OCF RA: symlink]
On Thu, Apr 21, 2011 at 03:19:10PM +0200, Florian Haas wrote: On 2011-04-20 19:00, Lars Ellenberg wrote: On Wed, Apr 20, 2011 at 06:49:48PM +0200, Lars Ellenberg wrote: [a lot] I know I'm paranoid. Am I too paranoid? Patches welcome. That phrase does work as reply to everything you don't want to hear about ;-) Just because we probably have resource agents in tree that don't do proper input sanitation, and some of them may even do things like eval, or forget to quote parameters that need to be quoted ... Just because we have such stuff in tree already, does not mean we must take more of the same. Or that we must ignore that it could be a problem. Or that x=some link name and then doing ln $x y instead of ln $x y is simply wrong code. If we can fix things when taking them in, we should do that. That's naturally the point in time when they get most attention. So that's also when all potential issues should be brought up. And no, just because someone spots a potential problem does not make it his job to fix it. Of course we should also crowd source a review for the resource agents we already have. Improper use of input parameters becomes more important with the cib supporting ACLs, as then it becomes a potential privilege escalation problem. Whereas as long as you assume anyone with access to the cib is basically equivalent to root on the cluster nodes anyways, it is only an annoyance, and should be fixed anyways. Those resource agents I have actually read (as opposed to quickly browsed over or not even looked at) at least have nothing obvious of the sort. As for the symlink RA, I still think it is a good idea to use an indirection scheme, instead of using the symlinks directly. /etc/cron.d - /etc/cluster-symlinks/cron.d - /mnt/somewhere/cron.d does not only prevent the RA from removing unrelated files unintentionally, but also has the nice propery to clearly show /etc/cron.d is managed by this system, and makes it very fast to get an overview which links currently are supposed to be managed by that system. Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] STONITH plugin for VMware vCenter
On Tue, Apr 19, 2011 at 02:21:38PM +0200, Dejan Muhamedagic wrote: Hi, On Fri, Apr 08, 2011 at 11:38:23AM +0200, Nhan Ngo Dinh wrote: Logging added Many thanks. Please see below for a few more comments, mainly about the meta-data. Lars, any more comments on from you? No time right now. I'd say take it and let users complain about whatever they find. Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] New OCF RA: symlink
On Wed, Apr 20, 2011 at 02:37:00PM +0200, Florian Haas wrote: On 2011-04-20 11:41, Dominik Klein wrote: Hi I wrote a new RA that can manage a symlink. Configuration: primitive mylink ocf:heartbeat:symlink \ params link=/tmp/link target=/tmp/target \ op monitor interval=15 timeout=15 This will basically ln -s /tmp/target /tmp/link hth Dominik There used to be something called drbdlinks. Never really looked at it, though, so I'm not sure what exactly it does or does not do. In any case I suggest using ln -nvsTf, and add comments as to why. the -n is in case it points to a directory already. try this: mkdir symlink-test cd symlink-test mkdir a mkdir b ln -sv b current # (outch) ln -nsvf b current # (better now) rm current mkdir current ln -nsvf b current # (outch) ln -nsvfT b current # (better now, at least cleanly errors out) Not sure about the portability of all those flags. Not sure about security implications (symlink races in world writeable directories) Maybe you want to at least allow (or even default) to enforce certain permissions (root:root 0755 as a minimum?) on the directory the symlink is located in? Maybe nothing can go wrong anyways? Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] New OCF RA: symlink
On Wed, Apr 20, 2011 at 06:49:48PM +0200, Lars Ellenberg wrote: On Wed, Apr 20, 2011 at 02:37:00PM +0200, Florian Haas wrote: On 2011-04-20 11:41, Dominik Klein wrote: Hi I wrote a new RA that can manage a symlink. Configuration: primitive mylink ocf:heartbeat:symlink \ params link=/tmp/link target=/tmp/target \ op monitor interval=15 timeout=15 This will basically ln -s /tmp/target /tmp/link hth Dominik There used to be something called drbdlinks. Never really looked at it, though, so I'm not sure what exactly it does or does not do. In any case I suggest using ln -nvsTf, and add comments as to why. the -n is in case it points to a directory already. try this: mkdir symlink-test cd symlink-test mkdir a mkdir b ln -sv b current # (outch) ln -nsvf b current # (better now) rm current mkdir current ln -nsvf b current # (outch) ln -nsvfT b current # (better now, at least cleanly errors out) Not sure about the portability of all those flags. Not sure about security implications (symlink races in world writeable directories) Maybe you want to at least allow (or even default) to enforce certain permissions (root:root 0755 as a minimum?) on the directory the symlink is located in? Maybe nothing can go wrong anyways? Oh, well, thinking about non-roots that may have cibadmin karma, they now can configure a resource that will remove /etc/passwd. I'm not sure if I like that. How about a staged system? Double symlinks? Similar to the alternatives system in Debian or others. The RA will force a single directory that will contain the indirection symlinks, and will sanitize (or force) link names to not contain slashes. The real symlinks will point to that indirection symlink, which will point to the end location. /etc/postfix/main.cf - /var/lib/wherever-indirection-dir/postfix_main.cf === - /mnt/somewhere/you/want/to/point/to/main.cf And === will be managed by the resource agent. I know I'm paranoid. Am I too paranoid? Lars ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] STONITH plugin for VMware vCenter
On Tue, Apr 05, 2011 at 06:27:19PM +0200, Nhan Ngo Dinh wrote: VMware perl library allows to use a .vimrc file in the user home, Ouch. I guess they all use emacs, then. Or notepad ;-) -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] STONITH plugin for VMware vCenter
there is some bugzilla on that issue (protecting
sensitive information in the cib from appearing in logs,
or rather keep them out of the cib completely).
Dejan?
/longdesc
/parameter
/parameters
};
$ret = 0;
}
else {
$ret = 1;
}
}
exit($ret);
Enough for today.
Thank you for this contribution.
If we now can have a few other Reviewed-by: or Tested-by: signatures,
that would be great.
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Patch : ocf/resource.d/heartbeat/Filesystem: Use /proc/mounts if available since /etc/mtab might be outdated
On Tue, Mar 22, 2011 at 02:52:05PM +0100, Corvus Corax wrote:
--- Filesystem2011-03-22 14:49:58.103845191 +0100
+++ Filesystem_lw 2011-03-22 14:49:32.396825985 +0100
@@ -186,7 +186,9 @@
# Take advantage of /etc/mtab if present, use portable mount command
# otherwise. Normalize format to dev mountpoint fstype.
list_mounts() {
- if [ -f /etc/mtab -a -r /etc/mtab ]; then
+ if [ -f /proc/mounts -a -r /proc/mounts ]; then
+ cut -d' ' -f1,2,3 /proc/mounts
+ elif [ -f /etc/mtab -a -r /etc/mtab ]; then
cut -d' ' -f1,2,3 /etc/mtab
else
$MOUNT | cut -d' ' -f1,3,5
IIRC, it was a concious decision to prefer /etc/mtab over /proc/mounts
because it is needed for bind mounts, and various other aspects.
The exact reasons probably deserve some comments in the script.
If someone can remember a bit more detail?
You are of course free to symlink /etc/mtab to /proc/mounts,
if you feel that this is useful for your setup.
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] new resource agents repository
On Fri, Feb 25, 2011 at 08:32:54AM +0100, Andrew Beekhof wrote: On Thu, Feb 24, 2011 at 4:10 PM, Dejan Muhamedagic deja...@fastmail.fm wrote: On Thu, Feb 24, 2011 at 03:56:27PM +0100, Andrew Beekhof wrote: On Thu, Feb 24, 2011 at 2:59 PM, Dejan Muhamedagic deja...@fastmail.fm wrote: Hello, There is a new repository for Resource Agents which contains RA sets from both Linux HA and Red Hat projects: git://github.com/ClusterLabs/resource-agents.git The purpose of the common repository is to share maintenance load and try to consolidate resource agents. There were no conflicts with the rgmanager RA set and both source layouts remain the same. It is only that autoconf bits were merged. The only difference is that if you want to get Linux HA set of resource agents installed, configure should be run like this: configure --with-ras-set=linux-ha ... The new repository is git but the existing history is preserved. People used to Mercurial shouldn't have hard time working with git. We need to retire the existing repository hg.linux-ha.org. Are there any objections or concerns that still need to be addressed? Might not hurt to leave it around - there might be various URLs that point there. Yes, it will definitely remain there. What I meant with retire, is that the developers then start using the git repository exclusively. A Yes, and making read-only on the server it probably a good idea (to avoid pushes). I just committed: http://hg.linux-ha.org/agents/rev/7a11934b142d and then added a hook: # hg push -r default ssh://h...@hg.linux-ha.org/agents pushing to ssh://h...@hg.linux-ha.org/agents searching for changes remote: remote: Repository moved! Please use remote: https://github.com/ClusterLabs/resource-agents remote: git://github.com/ClusterLabs/resource-agents.git remote: remote: abort: prechangegroup.superseeded hook exited with status 1 abort: unexpected response: empty string is that ok with everyone? -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] [PATCH] ocf-tester: redirect error messages to stderr
On Wed, Feb 16, 2011 at 12:45:48AM +0100, Michael Prokop wrote:
Hi,
agent's ocf-tester doesn't redirect error messages to stderr.
The attached patch addresses this issue.
regards,
-mika-
--
http://michael-prokop.at/ || http://adminzen.org/
http://grml-solutions.com/ || http://grml.org/
# HG changeset patch
# User Michael Prokop
# Date 1297775920 -3600
# Node ID a86054c616c67bdc36c86151a85587f114848951
# Parent 17f68c9e93cb76a411c8bc0da8917596cee95a55
ocf-tester: redirect error messages to stderr
The --help option isn't handled in the command line parsing and
therefore doesn't have the same option handling as the -h option.
As a result do not mention the --help option in the usage information.
diff -r 17f68c9e93cb -r a86054c616c6 tools/ocf-tester.in
--- a/tools/ocf-tester.in Mon Feb 14 11:46:18 2011 +0100
+++ b/tools/ocf-tester.in Tue Feb 15 14:18:40 2011 +0100
@@ -37,15 +37,22 @@
num_errors=0
usage() {
-echo Tool for testing if a cluster resource is OCF compliant
-echo
-echo Usage: ocf-tester [-Lh] -n resource_name [-o name=value]*
/full/path/to/resource/agent
-echo
-echo Options:
-echo -h,--helpThis text
-echo -n name Name of the resource
-echo -o name=valueName and value of any parameters
required by the agent
-echo -L Use lrmadmin/lrmd for tests
+# make sure to output errors on stderr
+if [ $1 -ne 0 ] ; then
+usage_echo() { echo $@ 2 ; }
+else
+usage_echo() { echo $@ ; }
+fi
+
+usage_echo Tool for testing if a cluster resource is OCF compliant
+usage_echo
+usage_echo Usage: ocf-tester [-Lh] -n resource_name [-o name=value]*
/full/path/to/resource/agent
+usage_echo
+usage_echo Options:
+usage_echo -h This text
+usage_echo -n nameName of the resource
+usage_echo -o name=value Name and value of any
parameters required by the agent
+usage_echo -L Use lrmadmin/lrmd for tests
exit $1
}
hm.
how about
[ x$1 = x0 ] || exec 2
leave echo ... in place as is
exit $1
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] New master/slave resource agent for DB2 databases in HADR (High Availability Disaster Recovery) mode
On Wed, Feb 09, 2011 at 02:43:17PM +0100, Andrew Beekhof wrote: Are you going to change the name of every agent that gets a rewrite? IPaddr2-ng-ng-again-and-one-more-plus-one I don't think it is going to happen that often. It happens often enough - its just normally by a core developer. And realistically, almost every RA is going to get similar treatment (over time) as they're merged with the Red Hat ones. Solicit feedback, like was done for kliend's new agent, and replace the existing one it if/when people respond positively. Its not like the old one disappears from the face of the earth after you merge the new one. wget -o /usr/lib/ocf/resource.d/heartbeat/db2 http://hg.linux-ha.org/agents/file/agents-1.0.3/heartbeat/db2 What do you suggest? That we add to the release announcement: The db2 RA has been rewritten and didn't get yet a lot of field testing. Please help test it. So don't do that :-) Put up a wiki page with instructions for how to download+use the new agent and give feedback. How about a staging area? /usr/lib/ocf/resource.d/staging/ we can also add a /usr/lib/ocf/resource.d/deprecated/ The thing in .../heartbeat/ can become a symlink, and be given config file status by the package manager? Something like that. So we have it bundled with the release, it is readily available without much go to that web page and download and save to there and make executable and then blah. It would simply pop up in crm shell and DRBD-MC and so on. We can add please give feedback to the description, and this will replace the current RA with release + 2 unless we get veto-ing feedback to the release notes. Once settled, we copy over the staging one to the real directory, replacing the original one, and add a please fix your config to the thing that remains in staging/, so we will be able to start a further rewrite with the next merge window. * does not break existing setups * new RAs and rewrites are readily available -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] New master/slave resource agent for DB2 databases in HADR (High Availability Disaster Recovery) mode
On Wed, Feb 09, 2011 at 02:03:04PM +0100, Dejan Muhamedagic wrote:
hadr_vars=$(echo $output |\
awk '$0 ~/HADR database role/ {printf HADR_ROLE=%s; , $NF;}
$0 ~/HADR_TIMEOUT/ {printf HADR_TIMEOUT=%s; , $NF;}
$0 ~/First active log file/ {printf FIRST_ACTIVE_LOG=%s\n, $NF;}
$0 ~/HADR_PEER_WINDOW/ {printf HADR_PEER_WINDOW=%s\n, $NF;}')
You can leave out $0 ~. It is going to work this way too,
but otherwise it does not look very awk :)
Also, since it goes to eval below, better add single quotes around
/HADR_PEER_WINDOW/ {printf HADR_PEER_WINDOW='%s'\n ...
# sets HADR_ROLE HADR_TIMEOUT HADR_PEER_WINDOW
eval $hadr_vars
Hmm. Since both shell and awk by default do word split on white space,
how about this, without doing eval:
set -- $(echo $output |
awk '/HADR database role/ { role=$NF; }
/HADR_TIMEOUT/ { timeout=$NF; }
/First active log file/{ log=$NF; }
/HADR_PEER_WINDOW/ { window=$NF; }
END { printf .%s\t.%s\t.%s\t.%s\n,
role, timeout, log, window; }')
HADR_ROLE=${1#.}
HADR_TIMEOUT=${2#.}
FIRST_ACTIVE_LOG=${3#.}
HADR_PEER_WINDOW=${4#.}
/me tries to avoid eval wherever possible ;-)
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
___
Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev
Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] New master/slave resource agent for DB2 databases in HADR (High Availability Disaster Recovery) mode
On Wed, Feb 09, 2011 at 06:22:09PM +0100, Dejan Muhamedagic wrote: Hi Holger, On Wed, Feb 09, 2011 at 06:08:51PM +0100, Holger Teutsch wrote: Guys, coming back from a 3 hour break I'm a bit shocked about the very *active* discussion and it is difficult to me to find an entry point. No worries, I found it very difficult to follow to. There were like a few thousand subthreads. ;-) Right now, I think the best option would be to put your agent in /usr/lib/ocf/resource.d/testing and ask people to help test it and log that as well from the old db2 resource agent (sth like this db2 RA is about to be deprecated, please try ocf:testing:db2). Actually, because it is db2, I expect anyone running db2 to have a test system where he will excercise any and all changes to the system, be it config changes, tuning parameters, or package upgrades. So I'm all for doing an inplace replace this time. What can possibly go wrong? * test cluster failing during pre-deployment upgrade tests. Great. That's _exactly_ the feedback we'd need. I know it's somewhat heavy to put the burden of tests on to the new contributor (it'd be better if we already had some to verify what we already ship), but we desperately need test coverage for our RAs, since they provide crucial and essential integration code. To make up for that, I'll personally invite everyone who contributes a reasonably comprehensive, non-trivial test suite for an RA to lunch. This lunch thing sounds very attractive. Aha, so here's that lunch offer :-) Who is cooking? -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] slapd ocf resource agent
On Tue, Feb 08, 2011 at 01:10:19PM +0100, Dejan Muhamedagic wrote: On Fri, Feb 04, 2011 at 11:14:59PM +0100, Lars Ellenberg wrote: On Fri, Feb 04, 2011 at 04:16:23PM +0100, jer...@intuxicated.org wrote: Hi, I've updated the resource agent so that it tries to establish an LDAP connection. Most of the issues where resolved, but I haven't had the time yet to test with other OSes so I changed /bin/sh to /bin/bash. I'll try to remove bashisms as soon as possible. Jeroen Thanks. This is no actual review, Hmm, what would an actual review look like me wonders ;-) Well, I only reviewed that validate function, and completely ignored the rest of the code. Jeroen, did you try to contribute this RA to the openldap distribution? That would definitely be for the best. Yes please. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] slapd ocf resource agent
On Fri, Feb 04, 2011 at 04:16:23PM +0100, jer...@intuxicated.org wrote:
Hi,
I've updated the resource agent so that it tries to establish an LDAP
connection. Most of the issues where resolved, but I haven't had the time
yet to test with other OSes so I changed /bin/sh to /bin/bash. I'll try to
remove bashisms as soon as possible.
Jeroen
Thanks. This is no actual review,
but only a suggestion to cut down on the unwieldy for loop
Dejan complained about.
#
# slapd_validate_dirs: retrieves all slapd database directories from the
# configuration file(s) and verifies existence and
# permissions.
#
slapd_validate_dirs()
{
local dir
local dirs
local groups=`groups $user | sed -ne 's/^[^:]\+: \(.*\)$/\1/p' | sed 's/
\+/\\\|/' 2/dev/null`
local perms
local result
local state
[ $user = root ] return $OCF_SUCCESS
Why would a world writeable data directory not be a problem
if the service runs as root?
IFS=$NEWLINE
if [ -d $config_file ]; then
for dir in `find $config_file/'cn=config' -type f -name olcDatabase*
-exec \
Careful, unescaped *, don't do that.
Will break in completely non-obvious ways
if for some obscure reason in the current workingdirectory
there are files named olcDatabase-something.
sed -ne 's/^olcDbDirectory:[[:space:]]\+\(.\+\)/\1/p' {} \;`
find | xargs sed ?
Sure that there is no leading space allowed?
/me is outing himself has ignorant to ldap configuration syntax.
do
dir=${dir#\*}
no * necessary, actually the * is misleading here.
dir=${dir%\*}
the same
dirs=$dirs$NEWLINE$dir
what do you $NEWLINE for, there?
do you want to allow for spaces in directories,
but no newline?
done
why not strip the quotes in sed as well,
so you can say dirs=$(whatever) ?
this should be equivalent to this for loop:
dirs=$(find $config_file/cn=config -type f -name olcDatabase* -print0 |
xargs -0 sed -n -e '/^olcDbDirectory:[[:space:]]/!d;' \
-e 's/^[^:]*:[[:space:]]*//;s/^//;s/$//;p')
elif [ -f $config_file ]; then
for dir in `sed -ne 's/^directory[[:space:]]\+\(.\+\)/\1\n/p'
$config_file`
do
dir=${dir#\*}
dir=${dir%\*}
dirs=$dirs$NEWLINE$dir
done
the same,
dirs=$(sed -n -e '/^directory[[:space:]]/!d;' \
-e 's/^[a-z]*[[:space:]]*//;s/^//;s/$//;p' \
$config_file)
what about the else ?
no -d, and no -f either,
probably not existing (ignore specials for a second).
That's not good either?
fi
state=$OCF_SUCCESS
for dir in $dirs; do
IFS=$ORIG_IFS
In case you actually wanted to allow spaces (but no newlines)
in directory names, as I believe you attempted, with bash you'd do
ORIG_IFS=$IFS
IFS=$'\n'
dirs=($( sed ... ))
IFS=$ORIG_IFS
and then later again
for dir in ${dirs[@]}; do
...
But see below.
perms=`stat -c %U:%G:%a $dir`; result=$?
if [ $result -eq 0 ]; then
echo $perms | grep $user:[^:]\+:7.. /dev/null 21; result=$?
if [ $result -ne 0 ]; then
if [ -n $groups ]; then
echo $perms | grep [^:]\+:\($group\|$groups\):.7. /dev/null
21; result=$?
else
echo $perms | grep [^:]\+:$group:.7. /dev/null 21; result=$?
fi
if [ $result -ne 0 ]; then
echo $perms | grep :..7 /dev/null 21; result=$?
if [ $result -ne 0 ]; then
state=$OCF_ERR_GENERIC
ocf_log err slapd data directory '$dir' is not writable.
else
ocf_log warn slapd data directory '$dir' is world writable. Mode
0700 recommended.
fi
else
ocf_log warn slapd data directory '$dir' is group writable. Mode
0700 recommended.
fi
else
ocf_log warn slapd data directory '$dir' is accessible by others.
Mode 0700 recommended.
fi
fi
IFS=$NEWLINE
done
IFS=$ORIG_IFS
return $state
}
If I read that correctly, it checks if some directories are
* in some way writeable by $user
* complain, if dir is not mode 0700 and owned by $user
* does not care if it exists (was that intentional?)
dirs=$( find $dirs -maxdepth 0 -not \( -type d -user $user -perm 0700 \) )
for dir in $dirs; do
if ! su $user -s /bin/sh -c test -w '$dir' ; then
state=$OCF_ERR_GENERIC
ocf_log err ... $dir not writable by $user ...
else
ocf_log warn ... consider to 'chmod 0700 $dir; chown $user
$dir;'
fi
done
Note about spaces in path names... gets more ugly, but doable.
You need to decide wether you want to allow something that
no-one is actually doing, ever ;-)
and simply refuse it, in step 1, when gathering the
list of directories to check.
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com
DRBD® and LINBIT
Re: [Linux-ha-dev] Add real monitoring capabilities to IPaddr2 resource agent
+ + # check arping_ip_list + if [ $CURRENT_CHECK_LEVEL -le 19 ]; then + ocf_log debug check arping_ip_list + if [ $OCF_RESKEY_arping_ip_list != ]; then No need to check for != , the for loop will just not execute if it is. + for ip in $OCF_RESKEY_arping_ip_list; do + do_arping $ip return $OCF_SUCCESS + done + fi + fi + + # check arping ARP cache entries + if [ $CURRENT_CHECK_LEVEL -le 20 ]; then + ocf_log debug check arping ARP cache entries + if [ $OCF_RESKEY_arping_ip_list != ]; then no need here either, probably even wrong. You want this even if there is no arping_ip_list explicitly set, right? + for ip in `get_arp_list`; do + do_arping $ip return $OCF_SUCCESS + done + fi + fi + + # watch for packet counter changes in promiscios mode + if [ $CURRENT_CHECK_LEVEL -le 30 ]; then + ocf_log debug watch for packet counter changes in promiscios mode + # be sure switch off promiscios mode in any case + trap $IP2UTIL link set dev $NIC promisc off; exit INT TERM EXIT I don't like this. You assume that the dev does not operate in promisc mode. This will break the (presumably minority of) cases that actually run promisc intentionally during normal operation. + $IP2UTIL link set dev $NIC promisc on + watch_pkt_counter return $OCF_SUCCESS + $IP2UTIL link set dev $NIC promisc off + trap - INT TERM EXIT + fi How about you remember the packet stats first thing, then do all the other checks, finally you do a broadcast ping or something, then go again waiting for packets, comparing with the first remembered count? If you really want the promisc check, maybe first check if it was on, and if so, leave it on. Or make that an advanced option: keep_promisc_on... Maybe I'm just wrong here, and it's ok as you have it. + # looks like it's not working (for whatever reason) + # remove address from NIC + return $OCF_NOT_RUNNING +} + Enough for today... Thanks again for this valuable contribution. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
Re: [Linux-ha-dev] Patch: Add master/slave support to RA Dummy
On Fri, Jan 28, 2011 at 10:37:52AM +0100, Holger Teutsch wrote: Hi, for exploring master/slave dynamics in pacemaker I augmented RA Dummy accordingly. Others might be interested as well. There is the Stateful RA for that ;-) -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. ___ Linux-HA-Dev: Linux-HA-Dev@lists.linux-ha.org http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
