>From the "Q" you have, it sounds like someone's marketing people have been
there! EMC doesn't necessarily replicate the entire array. Whether it is
"nearly instantaneous" or not depends on a lot of factors - transaction
load, distance, etc. I would also contend that any talk about using
geographically dispersed SRDF-maintained arrays for "operating system
fail-over" is basically over-generalized marketing propaganda. EMC doesn't
sell clustering solutions, they sell storage solutions! Don't get me wrong,
I have recommended and implemented EMC solutions of the type you describe,
but, depending on your needs, storage replication alone may or may not be an
appropriate solution. For one thing, considering that EMC uses pure
physical disk replication, most corruptions in the primary will be
religiously propagated to the secondary. You might not want this to happen
"nearly instantaneously"!
Without delving into all the tangential availability issues (network, power,
application restart, ad infinitum), when considering Oracle availability,
there are essentially two areas of concern - instance and database.
Consider these two basic classes of failure, their modes of failure, and how
to handle them. Consider also that there are preventative and restorative
measures for each.
Instance failure is the case where the database itself is intact, but the
instance running against it is inaccessible or down. One common cause
of this sort of failure is hardware failure on a particular system - CPU
failure,
mainboard failure, NIC failure, etc. Also consider things like memory
leaks that might crash the running system or make it unusable.
Preventative measures include hardware redundancy, proper testing of patches
(OS, Oracle, application, ...), etc.
The most basic cure is to fix the root problem and then restart the
instance. There are essentially three other restorative measures for this
sort of failure - all similar in concept. All involve redundancy of the
node and the software. In increasing order of complexity, sophistication,
and level of availability, they are:
1) Spare server - sitting idle, configured identically to the primary
server, with identical software installed, etc. When the primary server or
instance dies, the external storage housing the database is attached to the
spare and the spare is brought up in place of the primary. This is
obviously a very simple approach.
2) "HA" solutions such as Veritos HA, Sun HA Cluster, etc. These
essentially automate the "spare server" solution. Shared disk houses the
database so no physical movement of array connections is required. The HA
software may be configured to automatically detect an outage and initiate
the failover. At any given time, only one of these nodes is active - only
one instance is running against the database.
3) Oracle parallel server (OPS) [in 9i it is called Real Application
Clusters (RAC)] is multiple instances running against a common database. In
just the availability context, OPS provides much faster failover capability
than HA since a surviving instance is already running against the same
database. There are two "minor modes" for OPS/RAC. If it is being used
strictly for availability, one can run Oracle parallel fail safe. This is
*NOT* the same thing as NT-only Oracle Fail-Safe. It is a special
configuration for OPS where multiple (usually two) instances are running,
but only one of them is doing application work at any given time. In the
event of a failure, the other takes over. The "normal" mode for OPS is to
have multiple instances performing application work.
Note that none of the above, by themselves, provide protection against a
database failure. They are all solutions where, in the event of an
instance/system failure, another instance/system takes over for the failed
one - running against the same database.
Database failure is the case where there is corruption or loss of datafiles,
redo logs, or control files. The number of modes of failure is greater
here - many are quite survivable, some are not. For example, the loss of a
single control file should not be a major disaster! The loss of some
datafiles, in some systems, may be non-critical in the near term. The loss
of a single redo log file may not be critical.
Preventative measures include multiple members per redo log group, multiple
copies of the control file, RAID, multiple controllers, and such - all
familiar to DBAs.
General classification of restorative solutions is a bit more difficult and
the potential solutions are greater in number. At the most basic level is
simple backup and recovery. Other, more sophisticated, solutions fall into
essentially two classes - physical (disk) replication and logical
(transaction) replication. However the distinctions are not always clear
and hybrids abound.
Physical data replication involves making, perhaps periodic, physical copies
of the database and/or mirroring writes to the database files. Simple
backups fall into this category. Some other technologies, such as EMC's
Timefinder, BCVs, and SRDF and also fall into this category since they copy
the physical structure of the database and changes to it without knowledge
of or regard to logical transactions. They simply mirror physical
(controller or OS) writes and may require some additional action, such as
applying archive log files, before the "replicated" database can reach a
consistent and usable state. For example, BCVs are essentially Oracle
backups to disk, with greatly enhanced recovery time since the data doesn't
need to be completely copied back to the original location before processing
can resume. Standby databases, in the most elemental form at least, also
fall into this category and are yet another variation on the backup &
recovery theme.
Logical replication is different in that the physical structure of the
database may be different - the datafiles may be sized differently, they may
have different names, etc. Oracle advanced replication is a solution of
this type. but there are others. The idea is that you are replicating
logical transactions or writes, not physical disk writes. Some log sniffing
technologies are essentially this class of solution also, but may have some
"physical" constraints (e.g. is ROWID critical?). Another seldom mentioned
solution of this type is to use a TP layer mechanism to replicate
transactions to a backup system. This can be a very complex, but flexible
solution. The "redundant" systems in the TP replication model can even be
heterogeneous. In the most complex cases, the primary could be DB2 on a
mainframe while the "backup" is Oracle on Unix.
In the data/transaction/database replication options, whether physical or
logical, there are also considerations of currency. Should it be
synchronous or asynchronous? SRDF, Oracle advanced replication, and other
such technologies offer this choice. Careful consideration should be given
to this decision. Too many times, I have seen people automatically assume
that synchronous is better. Even aside from performance implications, it
isn't always so. The question that usually stops them dead in their tracks
is "Should corruptions be applied immediately also?" For example, if all
your solutions are synchronous, then "drop table ORDERS" is applied
synchronously everywhere. Is this desirable? I have seen this kind of
thing happen, where one outage exceeded the professed "five 9s" goal - for
the next decade! One option, for example, might be to maintain a lag
between the primary database and the standby database. By delaying the
application of archived redo log files to the standby by some time delta, it
might be possible to prevent some types of corruption propagation. An ideal
solution using standby databases might be to maintain two standbys - one
kept as current as possible and one delayed by some well-considered delta.
Note that some database recovery solutions also cover instance recovery.
For example, a standby database could be used as a "standby instance" also
in the event of a failure of the primary server or instance. It may,
however, not be a appropriate instance failure solution for some systems.
Since OPS and HA would not need to perform any of the database recovery
operations that a standby database would, they would likely be available
much more quickly. Since the standby database might be in a remote
location, there might be a large number of other additional tasks that need
to be performed to use the standby.
In disaster recovery scenarios, like the catastrophic loss of an entire data
center, both modes come into play. HA and OPS typically require the nodes
of the cluster to be in close proximity. If you lose the entire data
center, the entire cluster would be inaccessible.
The key considerations in either instance or database availability are
essentially similar to the classic considerations for backup and recovery:
How much data can you afford to lose?
How much downtime can you afford before recovery?
How much can you afford to spend for a solution?
[The traditional "How much downtime can you afford for backups?" is moot in
the context of true high availability. Cold backups are simply not an
option for most systems where it is desired.]
The best solution for "high availability" for any system depends on how it
is defined in context. What is the probability of an instance failure?
What is the probability of a database failure? What is the probability of a
complete disaster? How much does each of these cost the company? What are
the costs associated with different potential solutions? How much does each
save? For some, a single standby database alone may suffice. For others,
hybrid solutions are essential. For example, one project I did recently had
extremely high availability and performance requirements and a suitably
large budget. The designed solution was to use 8i OPS for scalability and
instance availability, EMC's Timefinder and BCVs for quick database recovery
on the primary and also to enable offloading database backups from the
primary systems, and a standby database in a remote data center utilizing
SRDF to maintain synchronous copies of the online redo logs and the
archives log files - for a "no data loss" standby database. However, the
application of the archive logs to the standby was deferred for thirty
minutes. Thus the standby could be brought up from any state between "30
minutes before failure time" and "failure time - with no data loss".
Striking a good balance between cost, complexity, and availability is
difficult - at best. It isn't rocket science - it is harder! By far the
toughest part of the job isn't the technology, it is in getting accurate
requirements and sufficient budget. Too often the (supposed) requirements
are simply a stern mandate from on high for "No data loss and instantaneous
automatic failover - an economical and immediate solution". One of the
biggest problems is that CEOs/CIOs and the like rarely have a good grip on
even the basics. They read each other's wildly optimistic and often
outrageously exaggerated statements claiming 99.999% availability and think
that everybody else actually has it (when they don't), so it can't be that
difficult.
-Don Granaman
[certifiable Orasaurus]
----- Original Message -----
To: "Multiple recipients of list ORACLE-L" <[EMAIL PROTECTED]>
Sent: Monday, September 03, 2001 3:26 AM
MY Qs. IN CAPITALS BELOW :-
Storage-Array/OS fail-over :-
--------------------------
Storage-array vendors, most notably EMC, have taken the concept of
operating system fail-over one step further. Noting the drawbacks of
clustered hardware in the area of disaster recovery and rolling
upgrades, they have taken the basic fail-over mechanism at the operating
system level, and replaced the clustered hardware concept with the idea
of replicated storage-arrays.
So, instead of clustered nodes sharing the data by sharing the same
physical disk drives, vendors such as EMC introduced storage-array
replication.
All transactions written to one storage-array would be replicated almost
instantaneously to another storage-array.
Qs. IF THE DATABASE CRASHES IN THE PRIMARY STORAGE ARRAY BOX , WILL IT
BE COPIED TO THE 2nd REPLICATION ARRAY THEREBY CAUSING IT TOO HOLD A
CRASHED COPY OF THE DATABASE TOO ?
These two storage-arrays can be geographically separated, though the
network between them must be extremely large (i.e. OC3 or higher). The
computers, then, are not clustered, and do not require the additional
hardware and operating system software to achieve clustering.
Products include EMC Symmetrix Data Replication Facility (SDRF), and
Oracle Support has white papers describing certification testing
performed using SDRF at Oracle.
> -----Original Message-----
> From: VIVEK_SHARMA
> Sent: Sunday, September 02, 2001 3:12 PM
> To: '[EMAIL PROTECTED]'; '[EMAIL PROTECTED]';
> '[EMAIL PROTECTED]'
> Subject: Disaster Recovery Fallback Sites
>
>
> Disaster Recovery Fallback Sites :-
> --------------------------------
> What are the Different Solutions/methodologies implemented ?
> Is Standby Database the best Solution ?
> What are the Harware/O.S. Solutions available ?
> Which would be better using Oracle or O.S. Solutions ?
> Any papers , Links , best practices ?
>
--
Please see the official ORACLE-L FAQ: http://www.orafaq.com
--
Author: VIVEK_SHARMA
INET: [EMAIL PROTECTED]
Fat City Network Services -- (858) 538-5051 FAX: (858) 538-5051
San Diego, California -- Public Internet access / Mailing Lists
--------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from). You may
also send the HELP command for other information (like subscribing).
--
Please see the official ORACLE-L FAQ: http://www.orafaq.com
--
Author: Don Granaman
INET: [EMAIL PROTECTED]
Fat City Network Services -- (858) 538-5051 FAX: (858) 538-5051
San Diego, California -- Public Internet access / Mailing Lists
--------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from). You may
also send the HELP command for other information (like subscribing).
