Thanks for sharing this. Makes me want to spent time and try out OME. I used the 64bit EXEs in iDRAC just recently for the first time wondering how the iDRAC can handle EXEs?!?! :) It worked though. Is OME available for GNU/Linux?
On Fri, Apr 20, 2018 at 12:37 PM, Cameron Smith <[email protected]> wrote: > For localizing the repo you can look into Dell Repo Manager: > > https://www.dell.com/support/home/us/en/04/Drivers/DriversDe > tails?driverId=2GY7P > > You can customize the repo if you need to hold back a version for anything > and it's a great tool. Runs on Linux now!!! > > For firmware updates I used to use .BIN files in the OS. I then moved to > 64-bit .exe files through DRAC. I now use OME to DRAC for almost everything > for managing about 600 servers (11/12/13Gen). > > OME has gotten much better. Based on your numbers though I believe you > would need to run multiple installs of OME as I "think" it maxes out at > close to 2000 devices. It's is also good to batch update smaller groups of > devices at a time (20 or so) rather than trying to update 1000 at a time. > Just something to think about if you need to end up going this route. > > Cameron > > On Fri, Apr 20, 2018 at 6:32 AM, Prashant Sun <[email protected]> > wrote: > >> Thanks RS & Florian for your suggestions. >> >> I intend to use Catalog.xml file as my primary tool to approve updates >> that are applied by iDRAC or dsu. I plan to download this Catalog.xml and >> publish via ftp/http internally and do a phased roll-out. Say dev servers >> get first.I understand the firmware behaves differently even on same model >> at different times but that's a risk we are willing to take. >> >> Couple of more questions: >> >> Q1) Does anyone here use iDrac or dsu based updates? Do you mirror the >> upstream repo locally and point to it somehow? Please respond to this list >> or directly so we can talk further. >> >> Q2) Any other strategies for updating large(3000+) servers that is OS >> agnostic? I am keeping OME as last option. >> >> >> Thanks all >> >> On Thu, Apr 19, 2018 at 2:40 AM, Florian Haller-Casagrande < >> [email protected]> wrote: >> >>> Hi, >>> >>> >>> Another solution is to setup an OpenManage Essential server (or "OME", >>> available for free on Dell website), let is scan your network and find all >>> you iDRAC (no need to go further, like OS-level or with OMSA agents). It >>> will then display you all the available updates for your machines, and you >>> will be able to schedule them (or apply them immediately), through the >>> iDRAC (and the LC). >>> >>> >>> That is clearly, IMHO, the easiest way to go with dozens/hundreds of >>> servers. >>> >>> >>> But, these are some limitations I have with this solution : >>> >>> - OME is heavy, requiring a SQL server (embedded) and eating a lot of >>> CPU/RAM when you have hundreds of machines ; >>> >>> - OME offers many features, such as managing iDRAC/BIOS/etc >>> configurations, licenses, hardwares issues and so on, but it is not easy to >>> handle, and to be honest I only use it to update my firmwares ; >>> >>> - 80% of my servers are pretty well detected, but for some of them the >>> inventory task fail, and they are not listed (so I can't update them with >>> OME, I still need to go with a Dell ISO or whatever) ; >>> >>> - As Rene Shuster said, BIOS and LC updates are (almost) the first to >>> run. Personally, I first update all the iDRACs, as OME will go through it >>> to push updates to the LC. So : iDRAC, then BIOS+LC, then everything else ; >>> >>> - I still have many iDRAC6, and the iDRAC update is strangely not >>> "reboot-less" (if you upgrade through its webUI, no need to reboot the >>> server, only the iDRAC). With OME, the update is loaded (into the LC ?), >>> and waiting for server reboot to be applied... >>> >>> >>> I was previously using the ISO solution, but having to connect to every >>> single iDRAC, reboot and then go to PXE boot is time-consuming. And, most >>> of the time, you have to reboot twice with the ISO, as some updates fail >>> the first time because of some dependences (the Dell support teams are very >>> insistent >>> on this point). >>> >>> >>> As we have various Linux/*BSD systems, we can't rely on DSU or such >>> tools (Dell still doesn't support Debian 9...), and that is why I focus on >>> out-of-band solutions. >>> >>> >>> My 2cts. >>> >>> [image: Logo] <http://www.smile.fr/> >>> >>> 107 Boulevard de Stalingrad >>> 69100 Lyon Villeurbanne >>> www.smile.fr >>> *Florian HALLER-CASAGRANDE* >>> Ingénieur Infrastructures >>> Email : [email protected] >>> Tel : +33 4 26 29 12 25 >>> >>> [image: Facebook] <https://www.facebook.com/smileopensource> [image: >>> Google%2B] <https://plus.google.com/u/0/+SmileFrOpenSource/posts> [image: >>> LinkedIn] <https://www.linkedin.com/in/florianhc> [image: Twitter] >>> <https://twitter.com/GroupeSmile> >>> >>> >>> >>> [image: eco]Pour la planète, n'imprimez ce mail que si c'est nécessaire. >>> On 04/18/2018 09:27 PM, R S wrote: >>> >>> I recommend to apply BIOS update and LC update separately from all other >>> updates and do them first with whatever route you choose. They go together >>> is what DELL documentation says. BIOS first, then LC, then reboot and hope >>> for the best. >>> >>> Here are the pitfalls I encountered: >>> * updating the LC controller will result that all other updates chained >>> behind the LC update cannot be applied when using for example an ISO that >>> has been created with DELL Repo Manager. >>> * You might loose KVM capability when updating LC >>> * There is a high chance that a LC update will render your iDRAC/LC into >>> a brick >>> * replacing a bricked iDRAC used to be swapping out the iDRAC card >>> (available used for $60), starting with iDRAC7 DELL decided to solder it on >>> the mainboard. >>> * Check the warranty of all 3000 servers first as you will be opening >>> tickets with DELL to get your mainboard replaced due to bricked iDRAC/LC if >>> they are still under warranty. >>> * a lot of PSU updates are not listed in the catalog and you will need >>> to apply them in a different way. I do them last as they need up to 30 >>> minutes to apply to both PSU. Don't make the mistake and get impatient and >>> power the server on during the firmware update. The FW update will fail and >>> you will need to start over >>> * NIC updates sometimes fail to apply. Sometimes they need stepped >>> updates, for example to fix the underlying issue of not beeing able to >>> update to a more recent FW >>> * a lot of HDD/SSD updates are not listed in the catalog either and need >>> to be installed in a different way. >>> * iDSDM update is not listed in catalog. >>> >>> All of the above depends on a lot of factors. You could have two servers >>> with the same FW level and one fails and the other applies all FW fine. >>> Even heavily outdated servers might apply the latest FW updates just fine, >>> but then again a server just one month behind might fail updating to the >>> latest. >>> >>> >>> On Wed, Apr 18, 2018 at 2:10 PM, Prashant Sun <[email protected] >>> > wrote: >>> >>>> Greetings! >>>> >>>> I am taking up a project to consolidate the bios/LC/idrac/hw firmware >>>> updates for powerEdge 12G+ servers and would appreciate if you can answer >>>> few questions noted below. >>>> >>>> >>>> Environment: 3000+ Linux servers(RHEL6, 7) all running in multiple >>>> sites. Primarily PE R600 & 700 series with idrac enterprise 7,8,9. >>>> >>>> Update Plan: Create a local mirror of the upstream repo and use it in >>>> some fashion. >>>> >>>> I narrowed down my update strategy to following options. >>>> >>>> A. Install using yum repo (os-independent & os-dependent) >>>> >>>> B. Install using DSU by passing catalog.xml(update definitions) & >>>> location of .BIN files(using config.xml) >>>> >>>> C. Create an iso using DSU by passing Catalog.xml & >>>> config.xml(pointing to local .BIN repo). Then PXE boot to this iso to >>>> patch. >>>> >>>> D. Setup iDrac scheduled updates using local copy of repo and use >>>> multiple Catalog.xml to roll-out in phased manner. >>>> >>>> __Questions__: >>>> >>>> Q1. I like option:D as it is OS agnostic and uses iDRAC/LC to apply >>>> patches in a scheduled way. Has anyone encountered issues where certain >>>> category of updates fail for some reason? Will probably make windows server >>>> team happy too with this. :) >>>> >>>> Q2. I can also deal with option:C which involves creating iso and pxe >>>> booting servers into it. This has historically worked well for me using >>>> Dell Repo Mnager but the nv is too large and I'd like to avoid manual work >>>> having to do this. So curious to know if folks here prefer this over >>>> option:C. >>>> >>>> Q3. In order to go with option C or D ), is there a .BIN repo that I >>>> can mirror locally? Sorry I may not have google'd hard enough. If you have >>>> the link handy, please share. Thx. I found the Catalog.xml file from ' >>>> https://downloads.dell.com/catalog/' but don't see fw files there. >>>> >>>> Q4. I have never used RPM based updates(option A), but curious to know >>>> your experiences? Are all updates available via DRM typically also packaged >>>> into rpms or only a subset? >>>> >>>> Q5. Option B sounds like a custom tailored updates for each server but >>>> I have heard from fellow admins that it is a hit or miss. Do you agree >>>> with this? Do you recommend even looking at this? >>>> >>>> Any other ideas to fully automate bios/lc/idrac/hw firmware updates is >>>> welcome. >>>> >>>> >>>> Cheers >>>> P >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Linux-PowerEdge mailing list >>>> [email protected] >>>> https://lists.us.dell.com/mailman/listinfo/linux-poweredge >>>> >>>> >>> >>> >>> -- >>> Tech III * AppControl * Endpoint Protection * Server Maintenance >>> Buncombe County Schools Technology Department Network Group >>> ComicSans Awareness Campaign <http://comicsanscriminal.com> >>> >>> >>> _______________________________________________ >>> Linux-PowerEdge mailing >>> [email protected]https://lists.us.dell.com/mailman/listinfo/linux-poweredge >>> >>> >>> >>> _______________________________________________ >>> Linux-PowerEdge mailing list >>> [email protected] >>> https://lists.us.dell.com/mailman/listinfo/linux-poweredge >>> >>> >> >> _______________________________________________ >> Linux-PowerEdge mailing list >> [email protected] >> https://lists.us.dell.com/mailman/listinfo/linux-poweredge >> >> > > _______________________________________________ > Linux-PowerEdge mailing list > [email protected] > https://lists.us.dell.com/mailman/listinfo/linux-poweredge > > -- Tech III * AppControl * Endpoint Protection * Server Maintenance Buncombe County Schools Technology Department Network Group ComicSans Awareness Campaign <http://comicsanscriminal.com>
_______________________________________________ Linux-PowerEdge mailing list [email protected] https://lists.us.dell.com/mailman/listinfo/linux-poweredge
