Hi, On Tue, Jan 12, 2010 at 04:54:43PM +0800, Jiaju Zhang wrote: > On Mon, Jan 11, 2010 at 9:21 PM, Lars Ellenberg > <[email protected]> wrote: > > On Mon, Jan 11, 2010 at 06:56:03PM +0800, Jiaju Zhang wrote: > >> # HG changeset patch > >> # User Jiaju Zhang <[email protected]> > >> # Date 1263205796 -28800 > >> # Node ID 96ffc17dafd253e71f916b4d90ce701e086e2927 > >> # Parent c76b4a6eb576feb3b39852aa2349a0716bda1078 > >> Dev: portblock: Tickle ACK to TCP connections > >> > >> diff -r c76b4a6eb576 -r 96ffc17dafd2 heartbeat/portblock > >> --- a/heartbeat/portblock Mon Jan 04 14:42:10 2010 +0100 > >> +++ b/heartbeat/portblock Mon Jan 11 18:29:56 2010 +0800 > > > > btw, there is dead code: > > iptables_spec() { ... } > > > > > >> @@ -14,6 +14,8 @@ > >> # OCF_RESKEY_portno > >> # OCF_RESKEY_action > >> # OCF_RESKEY_ip > >> +# OCF_RESKEY_tickle_dir > >> +# OCF_RESKEY_sync_script > >> ####################################################################### > >> # Initialization: > >> > >> @@ -26,6 +28,7 @@ > >> : ${OCF_RESKEY_ip=${OCF_RESKEY_ip_default}} > >> ####################################################################### > >> CMD=`basename $0` > >> +TICKLETCP=$HA_BIN/tickle_tcp > >> > >> usage() > >> { > >> @@ -58,6 +61,34 @@ > >> the server. > >> > >> NOTE: iptables is linux-specific... > >> + > >> + An additional feature in the portblock RA is the tickle ACK function > >> + which triggers the clients to faster reconnect their TCP > >> connections > >> + to the fail-overed server. > > > > either add here: > > enabled by specifying the tickle_dir parameter. > > > > or move the following paragraph below the "When using the tickle ACK > > function ..." > > > >> + > >> + Please note that this feature is often used for the floating IP fail- > >> + over scenario where the long-lived TCP connections need to be > >> tickled. > >> + It doesn't support the cluster alias IP scenario. And if you want to > >> + tickle the TCP connections to _one_ floating IP(maybe the connections > >> + are to different ports), you only need _one_ portblock resource. > > > > ... you should *enable* tickles for _one_ portblock resource only, > > and of course for the action=unblock instance (otherwise that > > won't work; maybe enfoce this?). > > > > Of course you may have more multiple portblocks per IP. > > You may occasionally need several ports (maybe even > > more than iptables accepts for one multiport commandline), or udp as > > well as tcp blocking, or have other reasons for multiple portblocks. > > And typically, you have one portblock action: block, and one portblock > > action: unblock, each. > > Yeah, I'll modify the usage specification to try to clarify these :) > > > > >> + > >> + When using the tickle ACK function, in addition to the normal usage > >> + of portblock RA, the parameter tickle_dir must be specified! The > >> + tickle_dir is a location which stores the established TCP > >> connections. > >> + It can be a shared directory which is cluster-visible to all nodes. > >> + But if you don't have a shared directory, you could also use a local > >> + directory with cysnc2 pre-configured. > >> + For example, if you use the local directory /tmp/tickle as > >> tickle_dir, > >> + you could configure your /etc/csync2/csync2.cfg like: > >> + group ticklegroup { > >> + host node1; > >> + host node2; > >> + key /etc/csync2/ticklegroup.key; > >> + include /etc/csync2/csync2.cfg; > >> + include /tmp/tickle; > >> + auto younger; > >> + } > >> + Then specify the parameter sync_script as "csync2 -xv". > >> > >> END > >> } > >> @@ -100,6 +131,25 @@ > >> <content type="string" default="${OCF_RESKEY_ip_default}" /> > >> </parameter> > >> > >> +<parameter name="tickle_dir" unique="0" required="0"> > >> +<longdesc lang="en"> > >> +The shared or local directory(_must_ be absolute path) which > >> +stores the established TCP connections. > >> +</longdesc> > >> +<shortdesc lang="en">Tickle directory</shortdesc> > >> +<content type="string" default="" /> > >> +</parameter> > >> + > >> +<parameter name="sync_script" unique="0" required="0"> > >> +<longdesc lang="en"> > >> +The script used for synchronizing TCP connection state file, such as > >> +csync2, some wrapper of rsync, or whatever. > >> +If you used local directory as tickle_dir, you must specify this > >> parameter. > >> +</longdesc> > >> +<shortdesc lang="en">File sync script</shortdesc> > >> +<content type="string" default="csync2 -xv" /> > >> +</parameter> > >> + > >> <parameter name="action" unique="0" required="1"> > >> <longdesc lang="en"> > >> The action (block/unblock) to be done on the protocol::portno. > >> @@ -149,6 +199,33 @@ > >> { > >> PAT=`active_grep_pat "$1" "$2" "$3"` > >> $IPTABLES -n -L INPUT | grep "$PAT" >/dev/null > >> +} > >> + > >> +save_tcp_connections() > >> +{ > >> + [ -z "$OCF_RESKEY_tickle_dir" ] && return > >> + statefile=$OCF_RESKEY_tickle_dir/$OCF_RESKEY_ip > >> + if [ -z "$OCF_RESKEY_sync_script" ]; then > >> + netstat -tn |awk -F '[:[:space:]]+' ' > >> + $8 == "ESTABLISHED" && $4 == "'$OCF_RESKEY_ip'" \ > >> + {printf "%s:%s\t%s:%s\n", $4,$5, $6,$7}' | > >> + dd of="$statefile".new conv=fsync && > >> + mv "$statefile".new "$statefile" > >> + else > >> + netstat -tn |awk -F '[:[:space:]]+' ' > >> + $8 == "ESTABLISHED" && $4 == "'$OCF_RESKEY_ip'" \ > >> + {printf "%s:%s\t%s:%s\n", $4,$5, $6,$7}' \ > >> + > $statefile > >> + $OCF_RESKEY_sync_script $statefile > /dev/null 2>&1 & > >> + fi > >> +} > >> + > >> +run_tickle_tcp() > >> +{ > >> + [ -z "$OCF_RESKEY_tickle_dir" ] && return > >> + echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle > >> + f=$OCF_RESKEY_tickle_dir/$OCF_RESKEY_ip > >> + [ -f $f ] && cat $f | $TICKLETCP -n 3 > >> } > >> > >> SayActive() > >> @@ -195,8 +272,9 @@ > >> ;; > >> > >> *) > >> - if ha_pseudo_resource "${OCF_RESOURCE_INSTANCE}" status; then > >> + if ha_pseudo_resource "${OCF_RESOURCE_INSTANCE}" status; then > >> SayActive $* > >> + save_tcp_connections > > > > rather, > > ocf_is_probe || save_tcp_connections > > because we do not want to save_tcp_connections if this is a probe > > (crm verifying on hosts where this should not run that it is not running) > > > > though that ha_pseudo_resource thing should only return true if this is > > not a probe, right? > > Yeah, the combination of "chain_isactive" and "ha_pseudo_resource" can > differentiate if it is a probe or real monitor. > > > > > so maybe leave it as is, but add a comment indicating that this is only > > run on real monitor events. > > OK, I'll add the comment. > > > > >> rc=$OCF_SUCCESS > >> else > >> SayInactive $* > >> @@ -243,7 +321,10 @@ > >> ha_pseudo_resource "${OCF_RESOURCE_INSTANCE}" start > >> case $4 in > >> block) IptablesBLOCK "$@";; > >> - unblock) IptablesUNBLOCK "$@";; > >> + unblock) > >> + IptablesUNBLOCK "$@" > >> + run_tickle_tcp > > > > problem: > > exit code now contains something other than the return code of > > IptablesUNBLOCK > > > > rather, > > IptablesUNBLOCK "$@" > > rc=$? > > run_tickle_tcp > > # ignore run_tickle_tcp exit code! > > return $rc > > Thanks for pointing out this :) > > > > >> + ;; > >> *) usage; return 1; > >> esac > >> > >> @@ -256,7 +337,10 @@ > >> ha_pseudo_resource "${OCF_RESOURCE_INSTANCE}" stop > >> case $4 in > >> block) IptablesUNBLOCK "$@";; > >> - unblock) IptablesBLOCK "$@";; > >> + unblock) > >> + save_tcp_connections > >> + IptablesBLOCK "$@" > >> + ;; > >> *) usage; return 1;; > >> esac > >> > >> @@ -269,14 +353,7 @@ > >> CheckPort() { > >> # Examples of valid port: "1080", "1", "0080" > >> # Examples of invalid port: "1080bad", "0", "0000", "" > >> - case "$1" in > >> - *[!0-9]*) #got invalid char > >> - false;; > >> - *[1-9]*) #no invalid char, and has non-zero digit, so is a good port > >> - true;; > >> - *) #empty string, or string of 0's > >> - false;; > >> - esac > >> + echo $1 |egrep -qx '[0-9]+(:[0-9]+)?(,[0-9]+(:[0-9]+)?)*' > > > > thanks ;-) > > though this now would allow the all zero port, which the old one did not > > allow. > > but I don't care, and neither does iptables (at least on my box...) > > > >> } > >> > >> IptablesValidateAll() > >> @@ -296,6 +373,13 @@ > >> else > >> ocf_log err "Invalid port number $portno!" > >> exit $OCF_ERR_ARGS > >> + fi > >> + > >> + if [ -n "$OCF_RESKEY_tickle_dir" ]; then > >> + if [ ! -d "$OCF_RESKEY_tickle_dir" ]; then > >> + ocf_log err "The tickle dir doesn't exist!" > >> + exit $OCF_ERR_ARGS > >> + fi > > > > > > hmm. > > as tickle_dir is only used on the unblock action, > > maybe enforce that: > > $action = unblock || "tickles are only useful with > > action=unblock", ERR_ARGS ... > > OK, will enforce that. > > > > > > > I think we should include this now. > > Thanks! New patch attached :)
Many thanks! I'll move some of the text from usage into the metadata. Nobody's going to read usage, let's hope some will take a look at the metadata :) After another review, I'll apply the patch. Then if there are more issues, we can take it from there. Cheers, Dejan _______________________________________________________ Linux-HA-Dev: [email protected] http://lists.linux-ha.org/mailman/listinfo/linux-ha-dev Home Page: http://linux-ha.org/
