Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite
> I believe that this is only a limitation for RHEL4. RHEL5 should have a > fix that allows dm-multipath to properly pass ioctls to all devices. Hello, One problem is registration, but another problem is un-registration fe. when there is failover from one HBA to another and failback. Third problem is active-standby LB for two or more HBA, and how to handle this. Best Regards Maciej Bogucki -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster
Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite
Tomasz Sucharzewski napisał(a): > Hello, > > Ryan O'Hara wrote: >>> 4 - Limitations > ... >>> - Multipath devices are not currently supported. > > What is the reason - it is strongly required to use at least two HBA in a SAN > network, which is useless when using scsi reservation. > Hello, There is need to use multipath in production environments, and it is the main drawback of SCSI Reservations for Red Hat Cluster Suite. Here [1] You can read more about the problems with SCSI Reservations http://www.mail-archive.com/[email protected]/msg00524.html Best Regards Maciej Bogucki -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster
Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite
True. Any solution for auto discovery? I've no problem with statically defining a device, but that would be pretty nice if possible. Alex Kompel wrote: > On Fri, Mar 28, 2008 at 8:15 AM, Ryan O'Hara <[EMAIL PROTECTED]> wrote: >> The reason for the cluster LVM2 requirement is for device discovery. The >> scripts use LVM commands to find cluster volumes and then gets a list of >> devices that make up those volumes. Consider the alternative -- users >> would have to manually define a list of devices that need >> registrations/reservations. This would have to be defined on each node. >> What make this even more problematic is that each node may have >> different device names for shared storage devices (ie. what may be >> /deb/sdb on one node may be /deb/sdc on another). Furthermore, those >> device names could change between reboots. The general solution is to >> query clvmd for a list of cluster volumes and get a list of devices for >> those volumes. > > You can also use symbolic links under /dev/disk/by-id/ which are > persistent across nodes/reboots. > > -Alex > > -- > Linux-cluster mailing list > [email protected] > https://www.redhat.com/mailman/listinfo/linux-cluster > > ___ > > Inbound Email has been scanned by Nexa Technologies Email Security Systems. > ___ -- Jeff Macfarland ([EMAIL PROTECTED]) Nexa Technologies - 972.747.8879 Systems Administrator GPG Key ID: 0x5F1CA61B GPG Key Server: hkp://wwwkeys.pgp.net -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster
Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite
On Fri, Mar 28, 2008 at 8:15 AM, Ryan O'Hara <[EMAIL PROTECTED]> wrote: > > The reason for the cluster LVM2 requirement is for device discovery. The > scripts use LVM commands to find cluster volumes and then gets a list of > devices that make up those volumes. Consider the alternative -- users > would have to manually define a list of devices that need > registrations/reservations. This would have to be defined on each node. > What make this even more problematic is that each node may have > different device names for shared storage devices (ie. what may be > /deb/sdb on one node may be /deb/sdc on another). Furthermore, those > device names could change between reboots. The general solution is to > query clvmd for a list of cluster volumes and get a list of devices for > those volumes. You can also use symbolic links under /dev/disk/by-id/ which are persistent across nodes/reboots. -Alex -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster
Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite
Tomasz Sucharzewski wrote: Hello, Ryan O'Hara wrote: 4 - Limitations ... - Multipath devices are not currently supported. What is the reason - it is strongly required to use at least two HBA in a SAN network, which is useless when using scsi reservation. It has to do with how dm-multipath handles ioctls. In brief, the SCSI registration/reservation commands use ioctls, and those need to be passed to all devices within a dm-multipath target. This is required for getting SCSI reservations to work, since all the I/O paths need to be registered correctly. I believe that this is only a limitation for RHEL4. RHEL5 should have a fix that allows dm-multipath to properly pass ioctls to all devices. -Ryan -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster
Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite
Hello, Ryan O'Hara wrote: >>4 - Limitations ... >>- Multipath devices are not currently supported. What is the reason - it is strongly required to use at least two HBA in a SAN network, which is useless when using scsi reservation. Regards, Tomasz Sucharzewski On Fri, 28 Mar 2008 09:20:53 -0500 "Ryan O'Hara" <[EMAIL PROTECTED]> wrote: > 4 - Limitations > > In addition to these requirements, fencing by way of SCSI persistent > reservations also some limitations. > > - Multipath devices are not currently supported. -- Tomasz Sucharzewski <[EMAIL PROTECTED]> -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster
Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite
Jeff Macfarland wrote: Nice overview. Wish I had this a few weeks ago :-) I am curious as to why LVM2 is required? The reason for the cluster LVM2 requirement is for device discovery. The scripts use LVM commands to find cluster volumes and then gets a list of devices that make up those volumes. Consider the alternative -- users would have to manually define a list of devices that need registrations/reservations. This would have to be defined on each node. What make this even more problematic is that each node may have different device names for shared storage devices (ie. what may be /deb/sdb on one node may be /deb/sdc on another). Furthermore, those device names could change between reboots. The general solution is to query clvmd for a list of cluster volumes and get a list of devices for those volumes. With simple modification of the scsi_reserve (and maybe fence_scsi), using an msdos partitioned disk seems to work fine. This is only in testing but I haven't seen any issues as of yet. Ryan O'Hara wrote: Attached is the latest version of the "Using SCSI Persistent Reservations with Red Hat Cluster Suite" document for review. Feel free to send questions and comments. -Ryan -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster
Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite
Nice overview. Wish I had this a few weeks ago :-) I am curious as to why LVM2 is required? With simple modification of the scsi_reserve (and maybe fence_scsi), using an msdos partitioned disk seems to work fine. This is only in testing but I haven't seen any issues as of yet. Ryan O'Hara wrote: > Attached is the latest version of the "Using SCSI Persistent > Reservations with Red Hat Cluster Suite" document for review. > > Feel free to send questions and comments. > > -Ryan > -- Jeff Macfarland ([EMAIL PROTECTED]) Nexa Technologies - 972.747.8879 Systems Administrator GPG Key ID: 0x5F1CA61B GPG Key Server: hkp://wwwkeys.pgp.net -- Linux-cluster mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-cluster
[Linux-cluster] SCSI Reservations & Red Hat Cluster Suite
Attached is the latest version of the "Using SCSI Persistent Reservations with Red Hat Cluster Suite" document for review. Feel free to send questions and comments. -Ryan Using SCSI Persistent Reservations with Red Hat Cluster Suite Ryan O'Hara <[EMAIL PROTECTED]> January 2008 -- 1 - Introduction When cluster nodes share storage devices, it is necessary to control access to the storage devices. In the event of a node failure, the failed not should not have access to the underlying storage devices. SCSI persistent reservations provide the capability to control the access of each node to shared storage devices. Red Hat Cluster Suite employs SCSI persistent reservations as a fencing methods through the use of the fence_scsi agent. The fence_scsi agent provides a method to revoke access to shared storage devices, provided that the storage support SCSI persistent reservations. Using SCSI reservations as a fencing method is quite different from traditional power fencing methods. It is very important to understand the software, hardware, and configuration requirements prior to using SCSI persistent reservations as a fencing method. 2 - Overview In order to understand how Red Hat Cluster Suite is able to use SCSI persistent reservations as a fencing method, it is helpful to have some basic knowledge of SCSI persistent reservations. There are two important concepts withing SCSI persistent reservations that should be made clear: registrations and reservations. 2.1 - Registrations A registration occurs when a node registers a unique key with a device. A device can have many registrations. For our purposes, each node will create a registration on each device. 2.2 - Reservations A reservation dictates how a device can be accessed. In contrast to registrations, there can be only one reservation on a device at any time. The node that holds the reservation is know as the "reservation holder". The reservation defines how other nodes may access the device. For example, Red Hat Cluster Suite uses a "Write Exclusive, Registrants Only" reservation. This type of reservation indicates that only nodes that have registered with that device may write to the device. 2.3 - Fencing Red Hat Cluster Suite is able to perform fencing via SCSI persistent reservations by simply removing a node's registration key from all devices. When a node failure occurs, the fence_scsi agent will remove the failed node's key from all devices, thus preventing it from being able to write to those devices. More information can be found in Section x.x of this document. 3 - Requirements 3.1 - Software Requirements In order to use SCSI persistent reservations as a fencing methods, several requirements must be met/ - Red Hat Cluster Suite 4.5 or greater - Red Hat Cluster Suite 5.0 or greater The sg3_utils package must also be installed. This package provides the tools needed by the various scripts to manage SCSI persistent reservations. 3.2 - Storage Requirements In order to use SCSI persistent reservations as a fencing method, all shared storage must use LVM2 cluster volumes. In addition, all devices within these volumes must be SPC-3 compliant. If you are unsure if your cluster and shared storage environment meets these requirements, a script is available to determine if your shared storage devices are capable of using SCSI persistent reservations. See section x.x. 4 - Limitations In addition to these requirements, fencing by way of SCSI persistent reservations also some limitations. - Multipath devices are not currently supported. - All nodes in the cluster must have a consistent view of storage. In other words, all nodes in the cluster must register with the same devices. This limitation exists for the simple reason that each node must be able to remove another node's registration key from all the devices that it registered with. I order to do this, the node performing the fencing operation must be aware of all devices that other nodes are registered with. If all cluster nodes have a consistent view of storage, this requirement is met. - Devices used for the cluster volumes should be a complete LUN, not partitions. SCSI persistent reservations work on an entire LUN, meaning that access is controlled to each LUN, not individual partitions. To assist with finding and detecting devices which are (or are not) suitable for use with fence_scsi, a tool has been provided. The fence_scsi_test script will find devices visible to the node and report whether or not they are compatible with SCSI persistent reservations. A full description of this tool can be found in Section x.x of this document. 4 - Components Red Hat Cluster Suite provides three components (scripts) to be used in conjunction with SCSI persistent reservations. The fence_scsi_test script provides a means to discover and test devices and report whether or not they are capable of support SCSI persistent reservations. The scsi_reserve init script, if
