On 01/10/2018 03:27 PM, Patrick Boutilier wrote:
> What I would do is download the BIOS update file once and then create a
> script to rsync it to the 1500 hosts and run the update. I presume you would
> have a script to run dsu on the 1500 hosts anyway?
First: i gave up on using Dell's enterprise management tools due to constant
heartache/headache/frustration. It definitely makes me sad that these tools
continually change, but never actually get much better. (okay, it looks like
they are finally attacking the BASHisms in some of their scripts that borked
Debian/Ubuntu systems badly, but the continual lack of correctness/current-cy,
etc just pains me).
Second:
FYI, LLNL's 'pdsh' works great for this. requires ssh public key trust for
'root' in my following examples to do the following (running DUPs requires
'root'):
** note that your public key should be offline until loaded in your ssh-agent
(oof, Meltdown/Spectre, sigh)
WARNING: as any tool that allows mass-execution, if you screw up, you've now
multiplied that screwup to a large number of systems, so always be careful.
If you have a file of hostnames already:
pdsh -lroot -g {file} 'command-to-run-on-all-remote-systems'
(file is usually dshgroup module selected so ~/.dsh/group/{file} or
/etc/dsh/group/{file})
If you wanna hit everything in /etc/hosts, instead:
awk '$0!~/^(#.*| *)$/{print$2}' /etc/hosts | WCOLL=- pdsh -Rexec scp
FOO-2.7.0.BIN root@%h:/dev/shm/
explained: all non-comment/blank lines, print hostname (field 2), setup pdsh's
WCOLL envvar as the file containing hostnames to stdin (-), use the exec module
to scp your DUP.BIN file, substituting %h for each hostname successively.
awk '$0!~/^(#.*| *)$/{print$2}' /etc/hosts | WCOLL=- pdsh -lroot
'/dev/shm/FOO-2.7.0.BIN -q'
now, issue 'ssh' session to all hosts to run the DUP.BIN with '-q'. ('-q'
doesn't display changelog or prompt to run, also won't reboot after completion
automatically)
Note that 'pdsh' fans-out commands, running "N" jobs simultaneously (default
32). I limit mine to 8 so i can use some special gateway-hop syntax with
custom ./ssh/config rules to bounce past the admin nodes on clusters into the
backend compute nodes. This avoids the default 'sshd' connection throttling
limits (usually 10 simultaneous connections) (e.g. cluster1-admin!!node3)
using the ssh-config rule:
Host *!!*
GatewayPorts no
ProxyCommand $(h="%h";p="%p" ; echo ssh -W ${h##*\!\!}:%p -l root
${h%%\!\!*})
It's much easier to use libgenders or dshgroup style files for this kind of
thing (than /etc/hosts and awk, etc), so you can use attribute selectors
(genders) like:
gpsh -lroot -g 'model=poweredge_r730' 'do-something'
(it's up to you to create a genders file with the right attributes filled in)
records in my genders file, as created from a scripted MySQL asset database
extraction look like:
host99
name=host99,manu=dell,model=precision_t3400,hwtype=desktop,sn=XXXXXXX,os=debian_linux,status=in_use,user=godot,responsible=godot,purpose=user_room_linux,sa1=sdowdy,project=unknown,location=fl2-2094
Unfortunately, 'genders' doesn't support REGEX :-( but you can use regex
selection on hostnames in pdsh (just not attributes), like:
pdsh -lroot -g 'hwtype=desktop' -w '/engr-.*/' ...
to only hit the systems that are desktops and filter-down to only names with
"engr-" in them.
--stephen
--
Stephen Dowdy - Systems Administrator - NCAR/RAL
303.497.2869 - [email protected] - http://www.ral.ucar.edu/~sdowdy/
_______________________________________________
Linux-PowerEdge mailing list
[email protected]
https://lists.us.dell.com/mailman/listinfo/linux-poweredge
