You must make sure all 5 DCs for all domains are shutdown together, before
taking any of the images. (as they're all replicas of the config NC,
being they're in the same forest)
And obviously during restore you need to make sure you keep them from
talking to (i.e. trying to replicate w/) the existing DCs (b/c it's
unrealistic to get 190 disseperate DCs shutdown). There is guidance in
the AD forest recovery paper for this.
Sooo in my somewhat sleeply state, I see nothing wrong with your method.
But do not take me saying I don't see an issue off the top of my head, as
any sort of Microsoft buy off. Restating the disclaimer now:
This posting is provided "AS IS" with no warranties, and confers
no rights.
It's not technically performing any aspect of a stated plan that usually
makes me nervous, it's human nature that makes me nervous ...
Somewhere on one of the previous USN rollback threads, we discussed this
idea, what happens if you (who understand the semantics of this) get hit
by a bus, is your procedure well enough documented that a less astute
admin would not misunderstand the constraints of your restore system, and
make a significant misstep?
Human Nature aspect at issue:
We disregard rules that don't make immediate sense.
One last thing that makes me queasy, is I know what happens in an IT
meltdown, esp. in bigger environments, the junior admin on duty, will
usually DO ANYTHING to get the server back online. You could come in, in
the morning only to discover one of the VM DCs was brought back up from
the image, and (I'm sure the quote will go exactly like this) "there still
seems to be some replication issues, things are not syncing right, but at
least we got the server back up!!!"
Human Nature aspect at issue:
Panicing, creates poor choices.
You should view putting in place mechanisms to insure against such
missteps by your staff, as part of your resposibility as an IT admin.
Cheers,
-BrettSh [msft]
Disclaimer2: Good luck.
On Thu, 6 Oct 2005, Carroll Frank USGR wrote:
> Brett,
>
> My plan for the VMWare images is really for the ultimate DR scenario
> where I have already lost the entire forest. In this case, I would use
> the 5 images to completely restart from scratch (god help me ;-). The
> theroy is that if I shut them down gracefully and then shoot the now
> closed image file off to tape I would have a much better shot with the
> image file on different hardware, etc. The images together would be a
> consistent point in time backup. The images would only be used if we
> decide that the entire forest is already dead.
>
> I have a total of about 190 +/- dedicated DCs for the entire forest. Of
> those, about 30 of them are spread across three backbone nodes and those
> 30 are the ones that I send to tape daily (full system state). In the
> case of losing a given DC (backbone or site level) the SOP is to remove
> the remnants of the dead DC from the AD, rebuild/replace the server and
> promote it again.
>
> The goal was that I want to have an ace in the hole so I don't orphan
> 20K clients, 1500 servers and the rest of the AD objects (user accounts,
> groups, mail info, etc).
>
> Have I missed something here???
>
> Thanks
> Frank
>
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Brett Shirley
> Sent: Thursday, October 06, 2005 9:51 AM
> To: [email protected]
> Subject: RE: [ActiveDir] AD Restore Problem
>
> If you have any replicas of those servers, when you restore those VMWare
> images, you will have corrupted your forest during restore.
>
> -BrettSh [msft]
>
> This posting is provided "AS IS" with no warranties, and confers no
> rights.
>
>
> On Thu, 6 Oct 2005, Carroll Frank USGR wrote:
>
> > I am working my way down the VMWare path also for my ultimate DR "ace
> in
> > the hole". The environment is a TLD with 4 child domains. I am
> planning
> > on running a single VMWare server that has virtual DCs for all 5
> > domains. I am going to peel off a dedicated site/vlan and put the
> > physical VMWare server and all of the DC virt servers in that site.
> None
> > of the virtual DCs are going to be GCs. The reason for the dedicated
> > site is so I can keep people from using them for validation in
> > production.
> >
> > Once I have them running, I plan to use the VM scripting to gracefully
> > shut them down once a day and then shoot the image file of the
> shutdown
> > DC off to tape, which then goes off-site. After the backup completes I
> > then restart the virtual servers.
> >
> > This plays into the different hardware scenario since I can use VMWare
> > to abstract the hardware.
> >
> > Of course, this whole process is the backup to the normal system state
> > backup of all my backbone DCs.
> >
> > FWIW - Frank
> >
> > ________________________________
> >
> > From: [EMAIL PROTECTED]
> > [mailto:[EMAIL PROTECTED] On Behalf Of Coleman,
> Hunter
> > Sent: Wednesday, October 05, 2005 5:37 PM
> > To: [email protected]
> > Subject: RE: [ActiveDir] AD Restore Problem
> >
> >
> > You will still need to abandon the snapshot/image approach. Go to
> > http://www.mail-archive.com/[email protected]/ and search
> for
> > "usn rollback". You can get the same information by searching
> > support.microsoft.com, but without the colorful and enlightening
> > commentary that the list provides.
> >
> > Hunter
> >
> > ________________________________
> >
> > From: [EMAIL PROTECTED]
> > [mailto:[EMAIL PROTECTED] On Behalf Of CHIANESE,
> DAVID
> > Sent: Wednesday, October 05, 2005 2:09 PM
> > To: [email protected]
> > Subject: RE: [ActiveDir] AD Restore Problem
> >
> >
> > I should clarify we don't actually use a laptop anymore as we have a
> HOT
> > DR site defined and replicating live to Sungard. Basically we have a
> > vmware server in the DR site and replicate from that. It greatly
> > reduces post DR test administration in that we can revert back to the
> > machine state previous to the test and not worry about metadata clean
> > up. The laptop always served us fine in a DR test with varying
> hardware
> > at varying DR sites & tests. Of course what I forgot to mention is
> that
> > a good backup tape of your directory should be in the DR kit just in
> > case the laptop comes up corrupt. At least then you can restore
> vmware
> > to the laptop and then the backup of AD to a vmware DC and go from
> > there.
> >
> >
> > Regards,
> >
> > David Chianese
> >
> > -----Original Message-----
> > From: [EMAIL PROTECTED]
> > [mailto:[EMAIL PROTECTED] On Behalf Of Coleman,
> Hunter
> > Sent: Wednesday, October 05, 2005 3:19 PM
> > To: [email protected]
> > Subject: RE: [ActiveDir] AD Restore Problem
> >
> >
> >
> > There have been lots of discussions on this list about the
> > perils of imaging DCs and introducing them back into your production
> > environment. Avoid that like the plague.
> >
> > However, since VMWare/Virtual Server abstracts the hardware, it
> > eliminates the restore-to-different-hardware problems. Build a DC on a
> > virtual server and use NTBackup or your favorite 3rd party utility to
> > back up the virtual server just as if it were a physical DC. Load up
> > VMWare/Virtual Server on the alternate hardware and then restore your
> > backup to a guest virtual machine.
> >
> > Besides, relying on a laptop in the DR kit means that you're
> > putting a lot of faith in the laptop's hardware. Dicey proposition,
> IMO.
> >
> > Hunter
> >
> > ________________________________
> >
> > From: [EMAIL PROTECTED]
> > [mailto:[EMAIL PROTECTED] On Behalf Of CHIANESE,
> DAVID
> > Sent: Wednesday, October 05, 2005 12:58 PM
> > To: [email protected]
> > Subject: RE: [ActiveDir] AD Restore Problem
> >
> >
> > You hit the nail on the head with VmWare. Simply make a vmware
> > laptop and dcpromo it to a DC/GC. Place that laptop in a DR kit
> > offsite. Recall the kit and laptop once every 30 days and plug it
> into
> > production to allow it to catch up on replication. Place it back in
> > your DR kit and ship it off site. You can now contend with 2 DR
> > scenarios:
> >
> > 1.) A Real DR where a regional or national disaster occurs.
> > 2.) A DR test where you do not want to affect production by
> > seizing FSMO roles, making DNS changes, etc.
> >
> > In a real DR situation, you would simply plug in your DR laptop
> > and build a new Windows server, dcpromo and replicate from the laptop.
> > In fact, if you actually only had a regional outage you would be able
> to
> > build a new server and replicate with whatever DC(s) were left in
> > production that are reachable.
> >
> > In a test with VMware you can snapshot the image (Prior to
> > declaring the test). This insures you have a valid up to date image
> > prior to making changes. Perform your test by building a DC from
> > Sungard hardware and allowing it to replicate from your DR laptop.
> When
> > the test completes simply destroy the Sungard DC and revert your
> laptop
> > image back to the pre-test snapshot. It will then (When you plug it
> > back in at home office) catch up on replication. Place it back in the
> > DR box for next time. The reason you would need to update the laptop
> > monthly is to avoid the tombstone life of objects (default 60 days).
> So
> > by replicating the laptop once a month you overcome this obstacle as
> > well.
> >
> > I hope this helps. If you have any questions don't hesitate to
> > ask.
> >
> >
> > Regards,
> >
> > David Chianese RHCE, MCSE+I, CNE, CNA
> > Network Engineer
> > Philadelphia Insurance Companies
> > o: 610 538-2970
> > c: 267 549-4777
> > e: [EMAIL PROTECTED]
> > w: http://www.phly.com <http://www.phly.com/>
> >
> > -----Original Message-----
> > From: [EMAIL PROTECTED]
> > [mailto:[EMAIL PROTECTED] On Behalf Of Carerros,
> > Charles
> > Sent: Wednesday, October 05, 2005 9:05 AM
> > To: '[email protected]'
> > Subject: [ActiveDir] AD Restore Problem
> >
> >
> > I'm having a problem restoring my AD to different
> > hardware. I know there are some issues but I hear that people have
> been
> > able to follow some MS docs and get it done but I can't seem to pull
> it
> > off.
> >
> > I working with a HP server to Dell hardware and in the
> > next week I will be going from HP to Compaq at our DR test site and I
> > kinda need to get this working.
> >
> > I have included my documentation on how to do this DR
> > restore below and they are the steps that I went through and when I
> got
> > to the end I still get the blue screen and reboot. Can someone tell
> me
> > where I'm going wrong?
> >
> > We are running W2K3 fully patched with the exception of
> > SP1. DCs are all GCs, DNS and WINS servers.
> >
> > Thanks,
> >
> > Charlie
> >
> > Active Directory Disaster Recovery
> >
> > Company Name
> >
> > April 18, 2005, Revision 4
> >
> >
> >
> >
> >
> > The ability to recover from a catastrophic disaster is
> > one of the goals of the Network Team. With Active Directory quickly
> > becoming the core technology for items such as e-mail, Citrix and
> local
> > workstation security, it is imperative that in the case of a disaster
> a
> > quick recovery can be had. This process will outline the
> > non-authoritative active directory restore process. [The authoritative
> > process is used to restore a portion of the Active Directory while
> > leaving parts intact.]
> >
> >
> >
> > Resources:
> >
> > To conduct a successful restore you must have the
> > correct toolset. In conducting restores the following items must be
> > had. It is also important to note that all of this must be accessible
> > without access to network data storage. In the case of a disaster,
> > there will not be a network data storage to access.
> >
> >
> >
> > q Tested backup
> >
> > q Software that was used to take the backup
> >
> > q Server installation CDs (to include hardware
> > drivers)
> >
> > q Documentation on how the server was installed
> >
> > q Hardware to test the server on (if different
> > hardware, you must have drivers)
> >
> > q Workstation hardware
> >
> > q Separate VLAN that is not connected to production
> >
> > q Restore plan
> >
> > q All passwords, recovery and administrative
> >
> >
> >
> > If any of these items are not present then a restore
> > will not be able to be undertaken with success.
> >
> >
> >
> > The current backup strategy of the PRIMARYDC and
> > SECONDARYDC is:
> >
> >
> >
> > Daily backup using NTBackup to
> > BACKUPSERVER\d$\NetAdmin\AD Backup
> >
> > This backup captures the system state and
> > SYSVOL and Net Logon folders
> >
> > The server name is used as the backup file
> >
> > This is then backed up with the process that
> > backs up BACKUPSERVER
> >
> > No automated alert is currently configured
> > to monitor this backup process
> >
> >
> >
> > Process:
> >
> >
> >
> > 1. Review the resources to ensure that all are
> > present. Once all of the items are gathered then the process may move
> > forward.
> >
> >
> >
> > 2. Install Windows 2003 server on the server
> > hardware using the documentation that outlines the procedure that was
> > taken during the creation of the initial box. Be sure that you use
> disk
> > space equal to or larger than the original server and the drive
> letters
> > MUST be the same or the databases will not be properly restored. If
> you
> > do not use the appropriate volume sizes the restore may fail with a
> blue
> > screen.
> >
> >
> >
> > 3. Patch the server up to the same level of
> > patching that the original server had. If the original server did not
> > have Windows 2003 SP1, then DO NOT apply that patch until after the
> > restoration process is complete. The dll and security changes that
> > occur during OS patching can change the system state setup and
> therefore
> > render your backup useless.
> >
> >
> >
> > 4. Ensure that you install DNS and WINS servers.
> > (If you do not install DNS and WINS they may not restore correctly and
> > DNS and WINS will then need to be restored manually).
> >
> >
> >
> > 5. Start the computer in Directory Services Restore
> > Mode.
> >
> >
> >
> > a. Restart the computer
> > b. After the BIOS information is displayed,
> > press F8.
> > c. Use the Down Arrow to select "Directory
> > Services Restore Mode (Windows Server 2003 domain controllers only)
> > d. Use the Up and Down Arrows to select the
> > Windows Server 20003 operating system, and then press ENTER.
> > e. Log on with your administrative account
> > and password.
> >
> >
> >
> > 6. Start the Windows Server 2003 backup utility:
> >
> > a. Click Start
> > b. Point to "All Programs" => "Accessories"
> > => "System Tools" then click "Backup".
> >
> >
> >
> > 7. This procedure provides steps for restoring from
> > backup in Wizard Mode. By default, the Always Start in Wizard Mode
> check
> > box is selected in the Backup or Restore Wizard. If the Welcome to the
> > Backup Utility Advanced Mode page appears, click Wizard Mode to open
> the
> > Backup or Restore Wizard.
> >
> >
> >
> > 8. On the "Welcome to the Backup or Restore Wizard"
> > page, click Next.
> >
> >
> >
> > 9. Click Restore files and settings, and then click
> > Next.
> >
> >
> >
> > 10. Select the files that you want to restore (you
> > should have them on the local server), and then click Next.
> >
> >
> >
> > 11. On the Completing the Backup or Restore Wizard
> > page, click Advanced.
> >
> >
> >
> > 12. In Restore files to, click Original Location,
> > and then click Next.
> >
> >
> >
> > 13. Click Leave existing files (Recommended), and
> > then click Next.
> >
> >
> >
> > 14. In Advanced Restore Options, select the
> > following check boxes, and then click Next:
> >
> >
> >
> > a. Restore security settings
> >
> > b. Restore junction points, but not the folders
> > and file data they reference
> >
> > c. Preserve existing volume mount points
> >
> > d. For a primary restore of SYSVOL, also select
> > the following check box: When restoring replicated data sets, mark the
> > restored data as the primary data for all replicas.
> >
> >
> >
> > [A primary restore is required only if the domain
> > controller that you are restoring is the only domain controller in the
> > domain. A primary restore is required on the first domain controller
> > that is being restored in a domain if you are restoring the entire
> > domain or forest.]
> >
> >
> >
> > 15. Click Finish.
> >
> >
> >
> > 16. When the restore process is complete, click
> > Close, and then do one of the following:
> >
> >
> >
> > a. Change the BurFlags value to d4. [If the
> > restored domain controller's BurFlags value is not changed to d4,
> sysvol
> > does not share out.]
> >
> > * Click Start, and then Run
> >
> > * In the Open box, type regedit, and then click
> > OK
> >
> > * In the left pane, expand My Computer
> >
> > * Expand HKEY_LOCAL_MACHINE, SYSTEM,
> > CurrentControlSet, Services, NtFrs, Parameters, Backup/Restore,
> Process
> > at Startup
> >
> > * In the right pane, right-click BurFlags and
> > then click Modify
> >
> > * In the Value data box, type d4 and then click
> > OK
> >
> >
> >
> >
> >
> > b. If you do not need to authoritatively
> > restore any objects, click Yes to restart the computer. The system
> will
> > restart and replicate any new information that is received since the
> > last backup with its replication partners.
> >
> >
> >
> > c. If you need to authoritatively restore
> > any objects or if you need to create an LDAP Data Interchange Format
> > (LDIF) file to restore back-links on this domain controller, click No
> to
> > remain in Directory Services Restore Mode. For information about how
> to
> > proceed with authoritative restore, see Performing an Authoritative
> > Restore of Active Directory Objects.
> >
> >
> >
> > 17. If the server fails to boot properly:
> >
> > a. Boot the computer off the Windows 2003
> > server CD
> > b. The repair operation begins after you
> > accept the license agreement and after the Setup program searches for
> > previous installations of Windows to repair
> > c. When the Setup program finds the damaged
> > installation, press R to repair the installation (DO NOT USE THE
> > RECOVERY CONSOLE)
> > d. Following the onscreen steps to complete
> > the repair.
> > e. When the repair completes, reboot the
> > server.
> >
> >
> >
> > 18. If the server fails to boot past BIOS:
> >
> > a. Book the computer off the Windows 2003
> > server CD.
> > b. Select the appropriate HAL option for
> > you computer hardware.
> > c. After the HAL loads, select "R" for the
> > Recovery Console.
> > d. Logon to the Windows directory that you
> > need to repair by selection the appropriate number (default of 1).
> > e. Logon using the DSRM password.
> > f. At the command prompt type "disable
> > acpi" and hit enter
> > g. Make a note of the registry change.
> > h. Type "exit" and hit "enter" to reboot
> > the machine.
> > i. When the machine boots, follow step 17
> > to complete the HAL recreation.
> >
> >
> >
> > 19. Install the Windows 2003 Admin Pack. (You do
> > not need to install this prior to this point as the dlls will be
> > overwritten if you are forced to follow step 17).
> >
> >
> >
> > 20. If you run ADUC and receive an error connecting
> > to the active directory. Reboot the server. During the initial
> reboot
> > some installation process have not yet completed so the Active
> Directory
> > does not fully execute. The secondary reboot will correct this issue.
> >
> >
> >
> >
> >
> > Verification
> >
> >
> > After a restore is completed verification must be done
> > to ensure that it is functioning correctly. The easiest way to
> conduct
> > the verification is to use a laptop that was on the network before the
> > backup was taken. Simply connect the laptop to the switch that server
> > is on and attempt to authenticate and access resources on the server
> (a
> > file share could be placed on the restored server to ensure that the
> > authentication process is working correction). The greatest test
> would
> > be to down the server that is being restored and plug in the current
> > machine. Although this will allow the best functional test, if
> > something in the backup went wrong then you could possibly corrupt the
> > production sever.
> >
> >
> >
> > You will want to test the logon scripts and a number of
> > different users (to include administrative user accounts, delegated
> > security user accounts and service accounts). Once you are fully
> > satisfied with the restore process, this document should be updated
> and
> > forwarded to the bank for safekeeping.
> >
> >
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive:
> http://www.mail-archive.com/activedir%40mail.activedir.org/
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
>
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.mail-archive.com/activedir%40mail.activedir.org/
