In a word? Exausting. I arrived at the old location at 8:40 and started the final bits like DNS changes. We (4 of us) were unracked and out by around 10:45, after a little fun shutting down a few servers, minor hassles with Oracle mostly.
I arrived in-suite at the new location by 11:15 and then the fun started. 3 of us were involved in this stage, and one of the biggest hassles we had was with rack-mounting rails. It strikes me that the people who design these instruments of torture never actually use them. We're primarily a Dell house, and its long been the policy of Dell to change the rails at any oppirtunity. It's Sunday? better change the rails again... the R600s were easy. *click**click* done. The rest involved physical pain and torture. Worst of all was the IBM one for an old server we have. We broke past 2:30 due to all that fun. Racking and cabling was relatively straightforward after that, though we took lots of time to do it as neatly as possible. We'd premade a whole load of cables of different lengths to give us as much flexibility as possible. Starting up servers was interesting & a real hassle. Someone preferred hard instead of soft binding ldap auth, bothi main LDAP servers took 30+m to boot. I ended up wasting an hour and a half trying to figure out what was wrong with our firewalls only to discover my idea of in/out is different fron the sysadmin who labelled them. Itmay be a quirk from an ISP I worked at, but I think "inbound" & "outbound" not "inside" and "outside". Unfortunatley almost every server had the LDAP problem and seemed to be having fun binding to either LDAP server. I ended up booting servers up off a livecd, mounting and eduting the ldap.conf and rebooting. >From there it was mostly just sorting out services that didnt start cleanly, reverting DNS changes and then tidying up some odds and ends. Still, I left the new location at 11:55pm, exhausted and hungry (boss brought us lunch but we worked straight through dinner) I was back again today to sort out serial console cables and will have to go back later this week to sort out the 3 that failed. What would I do differently if I did it again? Tough to say. Should have paid more attention to the firewall cabling. Could possibly have done experimental reboots out of hours of certain srvers to see how they'd behave. Next steps: investigate and remove any system/service accounts that were put in LDAP. They really shouldn't be there. No idea why someone thought it would be a good idea. Fix LDAP settings. Investigate why some services wouldn't run on start-up. Sleep??? Paul _______________________________________________ Discuss mailing list [email protected] https://lists.lopsa.org/cgi-bin/mailman/listinfo/discuss This list provided by the League of Professional System Administrators http://lopsa.org/
