[Linux-cluster] Re: fence_manual missing /tmp/fence_manual.fifo

[denis]

Maciej Bogucki wrote:
>> Are you certain you want to continue? [yN] y
>> can't open /tmp/fence_manual.fifo: No such file or directory
> Hello,
> 
> Do You have the latest version of fence package?
> How does look like Your cluster.conf?

Hi,

I have installed the patched cman package from Lon H. to fix the broken
rgmanager communications bug :
https://bugzilla.redhat.com/show_bug.cgi?id=327721

Version : 2.0.73Vendor: Red Hat, Inc.
Release : 1.6.el5.test.bz327721  Build Date: Mon 26 Nov 2007
07:22:55 PM CET
Install Date: Thu 27 Mar 2008 03:28:20 PM CET  Build Host:
hs20-bc1-6.build.redhat.com
Group   : System Environment/Base   Source RPM:
cman-2.0.73-1.6.el5.test.bz327721.src.rpm
Size: 1164641  License: GPL
Signature   : (none)
Packager: Red Hat, Inc. 
URL : http://sources.redhat.com/cluster/
Summary : cman - The Cluster Manager


cluster.conf @ pastebin : http://pastebin.com/m5f8787a2

cluster.conf :



































































Regards
--
Denis

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] Unformatting a GFS cluster disk

[Lombard, David N]

On Thu, Mar 27, 2008 at 03:26:55PM -0400, christopher barry wrote:
> On Wed, 2008-03-26 at 13:58 -0700, Lombard, David N wrote:
> ...
> > > Can you point me at any docs that describe how best to implement snaps
> > > against a gfs lun?
> > 
> > FYI, the NetApp "snapshot" capability is a result of their "WAFL" filesystem
> > .  Basically, they use a
> > copy-on-write mechanism that naturally maintains older versions of disk 
> > blocks.
> > 
> > A fun feature is that the multiple snapshots of a file have the identical
> > inode value
> > 
> 
> fun as in 'May you live to see interesting times' kinda fun? Or really
> fun?

The former.  POSIX says that two files with the identical st_dev and
st_ino must be the *identical* file, e.g., hard links.  On a snapshot,
they could be two *versions* of a file with completely different
contents.  Google suggests that this contradiction also exists
elsewhere, such as with the virtual FS provided by ClearCase's VOB.

-- 
David N. Lombard, Intel, Irvine, CA
I do not speak for Intel Corporation; all comments are strictly my own.

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

[Linux-cluster] SCSI Reservations & Red Hat Cluster Suite

[Ryan O'Hara]

Attached is the latest version of the "Using SCSI Persistent 
Reservations with Red Hat Cluster Suite" document for review.


Feel free to send questions and comments.

-Ryan

Using SCSI Persistent Reservations with Red Hat Cluster Suite

Ryan O'Hara <[EMAIL PROTECTED]>

January 2008

--

1 - Introduction

When cluster nodes share storage devices, it is necessary to control
access to the storage devices. In the event of a node failure, the
failed not should not have access to the underlying storage
devices. SCSI persistent reservations provide the capability to
control the access of each node to shared storage devices. Red Hat
Cluster Suite employs SCSI persistent reservations as a fencing
methods through the use of the fence_scsi agent. The fence_scsi agent
provides a method to revoke access to shared storage devices, provided
that the storage support SCSI persistent reservations.

Using SCSI reservations as a fencing method is quite different from
traditional power fencing methods. It is very important to understand
the software, hardware, and configuration requirements prior to using
SCSI persistent reservations as a fencing method.

2 - Overview

In order to understand how Red Hat Cluster Suite is able to use SCSI
persistent reservations as a fencing method, it is helpful to have
some basic knowledge of SCSI persistent reservations.

There are two important concepts withing SCSI persistent
reservations that should be made clear: registrations and
reservations. 

2.1 - Registrations

A registration occurs when a node registers a unique key with a
device. A device can have many registrations. For our purposes, each
node will create a registration on each device.

2.2 - Reservations

A reservation dictates how a device can be accessed. In contrast to
registrations, there can be only one reservation on a device at any
time. The node that holds the reservation is know as the "reservation
holder". The reservation defines how other nodes may access the
device. For example, Red Hat Cluster Suite uses a "Write Exclusive,
Registrants Only" reservation. This type of reservation indicates that
only nodes that have registered with that device may write to the
device.

2.3 - Fencing

Red Hat Cluster Suite is able to perform fencing via SCSI persistent
reservations by simply removing a node's registration key from all
devices. When a node failure occurs, the fence_scsi agent will remove
the failed node's key from all devices, thus preventing it from being
able to write to those devices. More information can be found in
Section x.x of this document.

3 - Requirements

3.1 - Software Requirements

In order to use SCSI persistent reservations as a fencing methods,
several requirements must be met/

- Red Hat Cluster Suite 4.5 or greater
- Red Hat Cluster Suite 5.0 or greater

The sg3_utils package must also be installed. This package provides
the tools needed by the various scripts to manage SCSI persistent
reservations.

3.2 - Storage Requirements

In order to use SCSI persistent reservations as a fencing method, all
shared storage must use LVM2 cluster volumes. In addition, all devices
within these volumes must be SPC-3 compliant. If you are unsure if
your cluster and shared storage environment meets these requirements,
a script is available to determine if your shared storage devices are
capable of using SCSI persistent reservations. See section x.x.

4  - Limitations

In addition to these requirements, fencing by way of SCSI persistent
reservations also some limitations.

- Multipath devices are not currently supported.

- All nodes in the cluster must have a consistent view of storage. In
  other words, all nodes in the cluster must register with the same
  devices. This limitation exists for the simple reason that each node
  must be able to remove another node's registration key from all the
  devices that it registered with. I order to do this, the node
  performing the fencing operation must be aware of all devices that
  other nodes are registered with. If all cluster nodes have a
  consistent view of storage, this requirement is met.

- Devices used for the cluster volumes should be a complete LUN, not
  partitions. SCSI persistent reservations work on an entire LUN,
  meaning that access is controlled to each LUN, not individual
  partitions.

To assist with finding and detecting devices which are (or are not)
suitable for use with fence_scsi, a tool has been provided. The
fence_scsi_test script will find devices visible to the node and
report whether or not they are compatible with SCSI persistent
reservations. A full description of this tool can be found in Section
x.x of this document.

4 - Components

Red Hat Cluster Suite provides three components (scripts) to be used
in conjunction with SCSI persistent reservations. The fence_scsi_test
script provides a means to discover and test devices and report
whether or not they are capable of support SCSI persistent
reservations. The scsi_reserve init script, if

[Linux-cluster] About GFS1 and I/O barriers.

[Mathieu Avila]

Hello GFS team,

Some recent kernel developements have brought IO barriers into the
kernel to prevent corruptions that could happen when blocks are being
reordered before write, by the kernel or the block device itself, just
before an electrical power failure.
(on high-end block devices with UPS or NVRAM, those problems cannot
happen)
Some file systems implement them, notably ext3 and XFS. It seems to me
that GFS1 has no such thing.

Do you plan to implement it ? If so, could the attached patch do the
work ? It's incomplete : it would need a global tuning like fast_stafs,
and a mount option like it's done for ext3. The code is mainly a
copy-paste from JBD, and does a barrier only for journal meta-data.
(should i do it for other meta-data ?)

Thanks,

--
Mathieu

Index: cluster/gfs-kernel/src/gfs/dio.c
===
--- cluster/gfs-kernel/src/gfs/dio.c	(révision 26309)
+++ cluster/gfs-kernel/src/gfs/dio.c	(copie de travail)
@@ -917,8 +917,29 @@
 int
 gfs_logbh_wait(struct gfs_sbd *sdp, struct buffer_head *bh)
 {
-	wait_on_buffer(bh);
+static int allow_barrier = 1;
+int ret;
 
+	if (allow_barrier)
+	  {
+	set_buffer_dirty(bh);
+	set_buffer_ordered(bh);
+	ret = sync_dirty_buffer(bh);
+	if (ret == -EOPNOTSUPP)
+	  {
+		printk(KERN_WARNING
+		   "barrier-based sync failed. Disabling barriers\n");
+		allow_barrier = 0;
+		clear_buffer_ordered(bh);
+		set_buffer_uptodate(bh);
+		set_buffer_dirty(bh);
+	  }
+	  }
+	if (!allow_barrier)
+	  {
+	wait_on_buffer(bh);
+	  }
+
 	if (!buffer_uptodate(bh) || buffer_dirty(bh)) {
 		gfs_io_error_bh(sdp, bh);
 		return -EIO;
--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

[Linux-cluster] GFS reports withdraw from cluster

[Ewald Beekman]

We have a RHEL4 2 node cluster running on
HP Integrity servers (rx4640, 4 Itanium's, 32GB RAM each).
The GFS is running on top of a SAN (EMC Cx380's).

We get these kinds of errors:
Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0: fatal: invalid 
metadata block
Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0:   bh = 
16659526 (type: exp=5, found=4)
Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0:   function = 
gfs_get_meta_buffer
Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0:   file = 
/builddir/build/BUILD/gfs-kernel-2.6.9-75/up/src/gfs/dio.c, line = 1223
Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0:   time = 
1206652995 Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0: 
about to withdraw from the cluster
Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0: waiting for 
outstanding I/O
Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0: telling LM to 
withdraw
Mar 27 22:23:15 idefix kernel: lock_dlm: withdraw abandoned memory
and then the filesystem is gone :-(

We are up2date with current patch levels:
GFS-6.1.15-1
GFS-kernel-2.6.9-75.12
cman-kernel-2.6.9-53.9
cman-1.0.17-0.el4_6.5

Not sure whether this is a SAN issue (that would be bad, 'cause
a lot of systemen are dependant of the SAN) or OS or inside
the GFS?

Any help is greatly appreciated.

Ewald...

-- 
Ewald Beekman, Security Engineer, Academic Medical Center,
dept. ADICT/AD  Server  en  Netwerkbeheer, The Netherlands
## Your mind-mint is:
Blessed is the man who, having nothing to say, abstains from giving
wordy evidence of the fact.
-- George Eliot

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite

[Jeff Macfarland]

Nice overview. Wish I had this a few weeks ago :-)

I am curious as to why LVM2 is required? With simple modification of the
scsi_reserve (and maybe fence_scsi), using an msdos partitioned disk
seems to work fine.

This is only in testing but I haven't seen any issues as of yet.

Ryan O'Hara wrote:
> Attached is the latest version of the "Using SCSI Persistent
> Reservations with Red Hat Cluster Suite" document for review.
> 
> Feel free to send questions and comments.
> 
> -Ryan
>

-- 
Jeff Macfarland ([EMAIL PROTECTED])
Nexa Technologies - 972.747.8879
Systems Administrator
GPG Key ID: 0x5F1CA61B
GPG Key Server: hkp://wwwkeys.pgp.net

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite

[Ryan O'Hara]

Jeff Macfarland wrote:

Nice overview. Wish I had this a few weeks ago :-)

I am curious as to why LVM2 is required?


The reason for the cluster LVM2 requirement is for device discovery. The 
scripts use LVM commands to find cluster volumes and then gets a list of 
devices that make up those volumes. Consider the alternative -- users 
would have to manually define a list of devices that need 
registrations/reservations. This would have to be defined on each node. 
What make this even more problematic is that each node may have 
different device names for shared storage devices (ie. what may be 
/deb/sdb on one node may be /deb/sdc on another). Furthermore, those 
device names could change between reboots. The general solution is to 
query clvmd for a list of cluster volumes and get a list of devices for 
those volumes.



With simple modification of the
scsi_reserve (and maybe fence_scsi), using an msdos partitioned disk
seems to work fine.

This is only in testing but I haven't seen any issues as of yet.

Ryan O'Hara wrote:

Attached is the latest version of the "Using SCSI Persistent
Reservations with Red Hat Cluster Suite" document for review.

Feel free to send questions and comments.

-Ryan





--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

RE: [Linux-cluster] RHCS / DRBD / MYSQL

[Lon Hohberger]

On Fri, 2008-03-28 at 17:24 +1300, Arjuna Christensen wrote:

> === Resource Tree ===
> service {
>   name = "asteriskcluster";
>   domain = "asterisk";
>   autostart = "1";
>   ip {
> address = "192.168.111.1";
> monitor_link = "1";
> script {
>   name = "drbdcontrol";
>   file = "/etc/init.d/drbdcontrol";
> fs {
> name = "mysqlfs";
> mountpoint = "/var/lib/mysql";
> device = "/dev/drbd0";
> fstype = "ext3";
> force_unmount = "0";
> self_fence = "1";
> fsid = "11607";
> force_fsck = "0";
> options = "";
> script {
>   name = "asterisk";
>   file = "/etc/init.d/asterisk";
>   }
> script {
>   name = "mysql";
>   file = "/etc/init.d/mysql";
>   }
>   }
> }
>   }
> }

This looks fine.  The only thing I can think of is:

 * that your version of rg_test doesn't match clurgmgrd for some reason.
I don't know why that would happen.

 * you have an old version of ccs installed with a new version of
rgmanager.  rg_test doesn't use ccs - so the query it uses to find
children works even if ccs breaks.

I'd pull the most recent snapshot of rgmanager for the version you're
running (e.g. from the RHEL4 or RHEL5 branch; Fabio seemed to think
yours was pretty old).

-- Lon

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite

[Tomasz Sucharzewski]

Hello,

Ryan O'Hara wrote:
>>4  - Limitations
...
>>- Multipath devices are not currently supported.

What is the reason - it is strongly required to use at least two HBA in a SAN 
network, which is useless when using scsi reservation.

Regards,
Tomasz Sucharzewski

On Fri, 28 Mar 2008 09:20:53 -0500
"Ryan O'Hara" <[EMAIL PROTECTED]> wrote:

> 4  - Limitations
> 
> In addition to these requirements, fencing by way of SCSI persistent
> reservations also some limitations.
> 
> - Multipath devices are not currently supported.

-- 
Tomasz Sucharzewski <[EMAIL PROTECTED]>

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite

[Ryan O'Hara]

Tomasz Sucharzewski wrote:

Hello,

Ryan O'Hara wrote:

4  - Limitations

...

- Multipath devices are not currently supported.


What is the reason - it is strongly required to use at least two HBA in a SAN 
network, which is useless when using scsi reservation.



It has to do with how dm-multipath handles ioctls. In brief, the SCSI 
registration/reservation commands use ioctls, and those need to be 
passed to all devices within a dm-multipath target. This is required for 
getting SCSI reservations to work, since all the I/O paths need to be 
registered correctly.


I believe that this is only a limitation for RHEL4. RHEL5 should have a 
fix that allows dm-multipath to properly pass ioctls to all devices.


-Ryan

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite

[Alex Kompel]

On Fri, Mar 28, 2008 at 8:15 AM, Ryan O'Hara <[EMAIL PROTECTED]> wrote:
>
> The reason for the cluster LVM2 requirement is for device discovery. The
> scripts use LVM commands to find cluster volumes and then gets a list of
> devices that make up those volumes. Consider the alternative -- users
> would have to manually define a list of devices that need
> registrations/reservations. This would have to be defined on each node.
> What make this even more problematic is that each node may have
> different device names for shared storage devices (ie. what may be
> /deb/sdb on one node may be /deb/sdc on another). Furthermore, those
> device names could change between reboots. The general solution is to
> query clvmd for a list of cluster volumes and get a list of devices for
> those volumes.

You can also use symbolic links under /dev/disk/by-id/ which are
persistent across nodes/reboots.

-Alex

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] SCSI Reservations & Red Hat Cluster Suite

[Jeff Macfarland]

True. Any solution for auto discovery? I've no problem with statically
defining a device,  but that would be pretty nice if possible.

Alex Kompel wrote:
> On Fri, Mar 28, 2008 at 8:15 AM, Ryan O'Hara <[EMAIL PROTECTED]> wrote:
>> The reason for the cluster LVM2 requirement is for device discovery. The
>> scripts use LVM commands to find cluster volumes and then gets a list of
>> devices that make up those volumes. Consider the alternative -- users
>> would have to manually define a list of devices that need
>> registrations/reservations. This would have to be defined on each node.
>> What make this even more problematic is that each node may have
>> different device names for shared storage devices (ie. what may be
>> /deb/sdb on one node may be /deb/sdc on another). Furthermore, those
>> device names could change between reboots. The general solution is to
>> query clvmd for a list of cluster volumes and get a list of devices for
>> those volumes.
> 
> You can also use symbolic links under /dev/disk/by-id/ which are
> persistent across nodes/reboots.
> 
> -Alex
> 
> --
> Linux-cluster mailing list
> Linux-cluster@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
> 
> ___
> 
> Inbound Email has been scanned by Nexa Technologies Email Security Systems.
> ___

-- 
Jeff Macfarland ([EMAIL PROTECTED])
Nexa Technologies - 972.747.8879
Systems Administrator
GPG Key ID: 0x5F1CA61B
GPG Key Server: hkp://wwwkeys.pgp.net

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

RE: [Linux-cluster] RHCS / DRBD / MYSQL

[Fabio M. Di Nitto]

On Fri, 28 Mar 2008, Lon Hohberger wrote:


On Fri, 2008-03-28 at 17:24 +1300, Arjuna Christensen wrote:


=== Resource Tree ===
service {
  name = "asteriskcluster";
  domain = "asterisk";
  autostart = "1";
  ip {
address = "192.168.111.1";
monitor_link = "1";
script {
  name = "drbdcontrol";
  file = "/etc/init.d/drbdcontrol";
fs {
name = "mysqlfs";
mountpoint = "/var/lib/mysql";
device = "/dev/drbd0";
fstype = "ext3";
force_unmount = "0";
self_fence = "1";
fsid = "11607";
force_fsck = "0";
options = "";
script {
  name = "asterisk";
  file = "/etc/init.d/asterisk";
  }
script {
  name = "mysql";
  file = "/etc/init.d/mysql";
  }
  }
}
  }
}


This looks fine.  The only thing I can think of is:

* that your version of rg_test doesn't match clurgmgrd for some reason.
I don't know why that would happen.

* you have an old version of ccs installed with a new version of
rgmanager.  rg_test doesn't use ccs - so the query it uses to find
children works even if ccs breaks.

I'd pull the most recent snapshot of rgmanager for the version you're
running (e.g. from the RHEL4 or RHEL5 branch; Fabio seemed to think
yours was pretty old).


He is using an ubuntu package made up by a CVS snapshot from 20070315. All 
the versions of the binaries are coming from the same snapshot.


Fabio

--
I'm going to make him an offer he can't refuse.

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] GFS reports withdraw from cluster

[Bob Peterson]

On Fri, 2008-03-28 at 15:49 +0100, Ewald Beekman wrote:
> We get these kinds of errors:
> Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0: fatal: 
> invalid metadata block
> Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0:   bh = 
> 16659526 (type: exp=5, found=4)
> Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0:   function = 
> gfs_get_meta_buffer
> Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0:   file = 
> /builddir/build/BUILD/gfs-kernel-2.6.9-75/up/src/gfs/dio.c, line = 1223
> Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0:   time = 
> 1206652995 Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0: 
> about to withdraw from the cluster
> Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0: waiting for 
> outstanding I/O
> Mar 27 22:23:15 idefix kernel: GFS: fsid=alpha_cluster:orabck.0: telling LM 
> to withdraw
> Mar 27 22:23:15 idefix kernel: lock_dlm: withdraw abandoned memory
> and then the filesystem is gone :-(

Hi Ewald,

This likely means you have file system corruption of some kind.
There are many ways a GFS file system can get corrupted through no
fault of GFS.  Please see the following entry in the FAQ:

http://sources.redhat.com/cluster/wiki/FAQ/GFS#gfs_consistency

At any rate, I recommend unmounting it from all nodes and
running gfs_fsck (from one node).

Regards,

Bob Peterson
Red Hat GFS


--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] Unformatting a GFS cluster disk

[christopher barry]

On Fri, 2008-03-28 at 07:42 -0700, Lombard, David N wrote:
> On Thu, Mar 27, 2008 at 03:26:55PM -0400, christopher barry wrote:
> > On Wed, 2008-03-26 at 13:58 -0700, Lombard, David N wrote:
> > ...
> > > > Can you point me at any docs that describe how best to implement snaps
> > > > against a gfs lun?
> > > 
> > > FYI, the NetApp "snapshot" capability is a result of their "WAFL" 
> > > filesystem
> > > .  Basically, they use a
> > > copy-on-write mechanism that naturally maintains older versions of disk 
> > > blocks.
> > > 
> > > A fun feature is that the multiple snapshots of a file have the identical
> > > inode value
> > > 
> > 
> > fun as in 'May you live to see interesting times' kinda fun? Or really
> > fun?
> 
> The former.  POSIX says that two files with the identical st_dev and
> st_ino must be the *identical* file, e.g., hard links.  On a snapshot,
> they could be two *versions* of a file with completely different
> contents.  Google suggests that this contradiction also exists
> elsewhere, such as with the virtual FS provided by ClearCase's VOB.
> 

So, I'm trying to understand what to takeaway from this thread:
* I should not use them?
* I can use them, but having multiple snapshots introduces a risk that a
snap-restore could wipe files completely by potentially putting a
deleted file on top of a new file?
* I should use them - but not use multiples.
* something completely different ;)

Our primary goal here is to use snapshots to enable us to backup to tape
from the snapshot over FC - and not have to pull a massive amount of
data over GbE nfs through our NAT director from one of our cluster nodes
to put it on tape. We have thought about a dedicated GbE backup network,
but would rather use the 4Gb FC fabric we've got.

If anyone can recommend a better way to accomplish that, I would love to
hear about how other people are backing up large-ish (1TB) GFS
filesystems to tape.

Regards,
-C

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] Unformatting a GFS cluster disk

[Wendy Cheng]

christopher barry wrote:

On Fri, 2008-03-28 at 07:42 -0700, Lombard, David N wrote:
  

On Thu, Mar 27, 2008 at 03:26:55PM -0400, christopher barry wrote:


On Wed, 2008-03-26 at 13:58 -0700, Lombard, David N wrote:
...
  

Can you point me at any docs that describe how best to implement snaps
against a gfs lun?
  

FYI, the NetApp "snapshot" capability is a result of their "WAFL" filesystem
.  Basically, they use a
copy-on-write mechanism that naturally maintains older versions of disk blocks.

A fun feature is that the multiple snapshots of a file have the identical
inode value



fun as in 'May you live to see interesting times' kinda fun? Or really
fun?
  

The former.  POSIX says that two files with the identical st_dev and
st_ino must be the *identical* file, e.g., hard links.  On a snapshot,
they could be two *versions* of a file with completely different
contents.  Google suggests that this contradiction also exists
elsewhere, such as with the virtual FS provided by ClearCase's VOB.




So, I'm trying to understand what to takeaway from this thread:
* I should not use them?
* I can use them, but having multiple snapshots introduces a risk that a
snap-restore could wipe files completely by potentially putting a
deleted file on top of a new file?
* I should use them - but not use multiples.
* something completely different ;)
  


Wait ! First, the "multiple snapshots sharing one inode" interpretation 
about WAFL is not correct.  Second, there are plenty documents talking 
about how to do snapshots with Linux filesystems (e.g. ext3) on Netapp 
NOW web site where its customers can get accesses. Third, doing snapshot 
on GFS is easier than ext3 (since ext3 journal can be on different volume).


Will do a draft write-up as soon as I'm off my current task (sometime 
over this weekend).


-- Wendy

Our primary goal here is to use snapshots to enable us to backup to tape
from the snapshot over FC - and not have to pull a massive amount of
data over GbE nfs through our NAT director from one of our cluster nodes
to put it on tape. We have thought about a dedicated GbE backup network,
but would rather use the 4Gb FC fabric we've got.

If anyone can recommend a better way to accomplish that, I would love to
hear about how other people are backing up large-ish (1TB) GFS
filesystems to tape.

Regards,
-C

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster
  



--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] Unformatting a GFS cluster disk

[Lombard, David N]

On Fri, Mar 28, 2008 at 03:51:54PM -0400, christopher barry wrote:
> On Fri, 2008-03-28 at 07:42 -0700, Lombard, David N wrote:
> > On Thu, Mar 27, 2008 at 03:26:55PM -0400, christopher barry wrote:
> > > 
> > > fun as in 'May you live to see interesting times' kinda fun? Or really
> > > fun?
> > 
> > The former.  POSIX says that two files with the identical st_dev and
> > st_ino must be the *identical* file, e.g., hard links.  On a snapshot,
> > they could be two *versions* of a file with completely different
> > contents.  Google suggests that this contradiction also exists
> > elsewhere, such as with the virtual FS provided by ClearCase's VOB.
> > 
> 
> So, I'm trying to understand what to takeaway from this thread:
> * I should not use them?

I'm not, in any way shape, or form, suggesting you avoid snapshots!
It has saved me from misery more than once.

> * I can use them, but having multiple snapshots introduces a risk that a
> snap-restore could wipe files completely by potentially putting a
> deleted file on top of a new file?
> * I should use them - but not use multiples.
> * something completely different ;)

I have no information on this.

-- 
David N. Lombard, Intel, Irvine, CA
I do not speak for Intel Corporation; all comments are strictly my own.

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] Unformatting a GFS cluster disk

[Lombard, David N]

On Fri, Mar 28, 2008 at 04:54:22PM -0500, Wendy Cheng wrote:
> christopher barry wrote:
> >On Fri, 2008-03-28 at 07:42 -0700, Lombard, David N wrote:
> >  
> >>On Thu, Mar 27, 2008 at 03:26:55PM -0400, christopher barry wrote:
> >>
> >>>On Wed, 2008-03-26 at 13:58 -0700, Lombard, David N wrote:
> A fun feature is that the multiple snapshots of a file have the 
> identical
> inode value
> 
> Wait ! First, the "multiple snapshots sharing one inode" 
> interpretation about WAFL is not correct.

Same inode value.  I've experienced this multiple times, and, as
I noted, is a consequence of copy-on-write.

I've also had to help other people understand why various utilities
didn't work as expected, like gnu diff, which immediately reported
identical files as soon as it saw the identical values for st_dev
and st_ino in the two files it was asked to compare.

>From the current diffutils (2.8.1) source:

  /* Do struct stat *S, *T describe the same file?  Answer -1 if unknown.  */
  #ifndef same_file
  # define same_file(s, t) \
  s)->st_ino == (t)->st_ino) && ((s)->st_dev == (t)->st_dev)) \
   || same_special_file (s, t))
  #endif

>Second, there are plenty 
> documents talking about how to do snapshots with Linux filesystems 
> (e.g. ext3) on Netapp NOW web site where its customers can get 
> accesses.

I didn't say snapshots don't work on Linux.  I've used NetApp on
Linux and directly benefitted from snapshots.

-- 
David N. Lombard, Intel, Irvine, CA
I do not speak for Intel Corporation; all comments are strictly my own.

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

[Linux-cluster] SCSI reservation conflicts after update

[Sajesh Singh]

After updating my GFS cluster to the latest packages (as of 3/28/08) on 
an Enterprise Linux 4.6 cluster (kernel version 2.6.9-67.0.7.ELsmp)  I 
am receiving scsi reservation errors whenever the nodes are rebooted. 
The node is then subsequently rebooted at varying intervals without any 
intervention. I have tried to disable the scsi_reserve script from 
startup, but it does not seem to have any effect. I have also tried to 
use the sg_persist command to clear all reservations with the -C option 
to no avail. I first noticed something was wrong when the 2nd node of 
the 2 node cluster was being updated. That was the first sign of the 
scsi reservation errors on the console.


From my understanding persistent SCSI reservations are only needed if I 
am using the fence_scsi module.


I would appreciate any guidance.

Regards,

Sajesh Singh

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] SCSI reservation conflicts after update

[christopher barry]

On Fri, 2008-03-28 at 21:03 -0400, Sajesh Singh wrote:
> After updating my GFS cluster to the latest packages (as of 3/28/08) on 
> an Enterprise Linux 4.6 cluster (kernel version 2.6.9-67.0.7.ELsmp)  I 
> am receiving scsi reservation errors whenever the nodes are rebooted. 
> The node is then subsequently rebooted at varying intervals without any 
> intervention. I have tried to disable the scsi_reserve script from 
> startup, but it does not seem to have any effect. I have also tried to 
> use the sg_persist command to clear all reservations with the -C option 
> to no avail. I first noticed something was wrong when the 2nd node of 
> the 2 node cluster was being updated. That was the first sign of the 
> scsi reservation errors on the console.
> 
>  From my understanding persistent SCSI reservations are only needed if I 
> am using the fence_scsi module.
> 
> I would appreciate any guidance.
> 
> Regards,
> 
> Sajesh Singh
> 
> --
> Linux-cluster mailing list
> Linux-cluster@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster


Sajesh, start here:
http://www.mail-archive.com/linux-cluster@redhat.com/msg01029.html

I went through this too, and Ryan helped me out a lot!

Good Luck!
-C

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

Re: [Linux-cluster] SCSI reservation conflicts after update

[Sajesh Singh]

christopher barry wrote:

On Fri, 2008-03-28 at 21:03 -0400, Sajesh Singh wrote:
  
After updating my GFS cluster to the latest packages (as of 3/28/08) on 
an Enterprise Linux 4.6 cluster (kernel version 2.6.9-67.0.7.ELsmp)  I 
am receiving scsi reservation errors whenever the nodes are rebooted. 
The node is then subsequently rebooted at varying intervals without any 
intervention. I have tried to disable the scsi_reserve script from 
startup, but it does not seem to have any effect. I have also tried to 
use the sg_persist command to clear all reservations with the -C option 
to no avail. I first noticed something was wrong when the 2nd node of 
the 2 node cluster was being updated. That was the first sign of the 
scsi reservation errors on the console.


 From my understanding persistent SCSI reservations are only needed if I 
am using the fence_scsi module.


I would appreciate any guidance.

Regards,

Sajesh Singh

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster




Sajesh, start here:
http://www.mail-archive.com/linux-cluster@redhat.com/msg01029.html

I went through this too, and Ryan helped me out a lot!

Good Luck!
-C
  

Christopher,
  I have read through the entire posting and a bit of 
information seems to be missing. Did you fix it by simply disabling the 
scsi_reserve script and clearing the stale reservations ?


Thanks,

Sajesh
--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster