Yes the DRBD 0.8 supports 2 I think or even three mounts. I want drbd and
ocfs2 as an active/active disk cluster.
I have these versions of things
-rw-r--r-- 1 root root 482489 May 16 11:09 drbd-8.0.3.tar.gz
-rw-r--r-- 1 root root 5126020 May 16 11:08
kernel-devel-2.6.15-1.2054_FC5.x86_
4.rpm
-rw-r--r-- 1 root root 1173119 May 15 16:18 ocfs2-tools-1.2.1-1.x86_64.rpm
"/etc/ocfs2/cluster.conf"
cluster:
node_count = 2
name = input
node:
ip_port = 7777
ip_address = 10.10.10.81
number = 1
name = input2a
cluster = input
node:
ip_port = 7777
ip_address = 10.10.10.82
number = 2
name = input2b
cluster = input
global {
usage-count yes;
}
common { syncer { rate 10M; } }
resource drbd0 {
protocol C;
net {
cram-hmac-alg sha1;
shared-secret "blabla";
allow-two-primaries;
after-sb-0pri discard-least-changes;
after-sb-1pri discard-secondary;
after-sb-2pri disconnect;
rr-conflict disconnect;
}
on input2a {
device /dev/drbd0;
disk /dev/mapper/VolGroup00-data_vol;
address 10.10.10.81:7789;
meta-disk internal;
}
on input2b {
device /dev/drbd0;
disk /dev/mapper/VolGroup00-data_vol;
address 10.10.10.82:7789;
meta-disk internal;
}
}
I do not know how to answer your OCFS2 userspace comment:
By 'running the file system unmanaged' I meant that having an active/active
disk on a two node clusters adds complexity. I think it is better to start
you an active/active file system from the init scripts rather then from
inside heartbeat, otherwise you are linking the filesystem to HA when it
does not need to be. It is nice that heartbeat is aware of the file system,
but when they are tied together you can not stop heartbeat without stopping
the disk.
My post was more of a rant then anything else. I was looking for 50 people
or so to read my post and say. "You must be doing something wrong. I run
DRBD with OCFS2 multi node actice/active and MySQL and its super fast and
never crashes on two 100 mhz laptops"
The concept is great an active/active disk partition and heartbeat with no
fancy SAN's. It works for stretches 3 or 4 weeks. But then I run into a
weird locked directory that I cant delete or a file owned by '?'. Or the
partition unmounts and the system will not reboot.
On 8/7/07, Andrew Beekhof <[EMAIL PROTECTED]> wrote:
>
> On 8/3/07, Eddie C <[EMAIL PROTECTED]> wrote:
> > I see a lot of people on this list talking about Linux HA, DRDB, and
> OCFS2 I
> > thought I would share some of my experiences with it.
>
> Is this even possible?
> I didn't think you could mount more than one node with drbd... is this
> a new feature in drbd8?
> If not, running OCFS2 on top would be guaranteed to fsck your data.
>
> Its also not clear to me that you ran OCFS2 in userspace-heartbeat
> mode which is also quite important.
>
> > We took two dell 860 servers out of the box. We setup FC5. Because we
> are
> > using software striping we ended up creating LVM partitions.
> > We decided to leave the a huge slice of the disk
> unallocated/unformattted
> > for /opt and run DRBD. Our software running and working inside opt can
> only
> > exist on one node at a time, but I decided to use DRBD two primaries.
> This
> > way I can look at the data on both nodes, and it seemed really cool.
> >
> > DRDB and OCFS2 did not work correctly on the base FC5 Kernel so I ran
> 'yum
> > update' and got the newest one.
> > Ok so this is the first sticky point. Yes FC5 is a development line, but
> FC5
> > is also IMHO just as stable as any other OS. Lets agree that it cant be
> > 'much less stable' if that makes any sense.
>
> Sure.
> But what versions of drbd/heartbeat/ocfs2 were you running?
> Thats the relevant question.
>
> > I grabbed the lastest DBRD. Finding an OCFS2 rpm for FC5 took searching
> but
> > I found it.
> >
> > I did get the system setup in a testing environment. I created a file on
> one
> > node. Then I edited on the other. So the clustered file system works.
> > Failover worked. All was well. I was suprised with how well things were
> > going.
> >
> > I tried to implement the File system in with heartbeat. At the time HA
> did
> > not have a DRBD two primary resource RA. I settled on making CLONED
> > resources out of the /etc/init.d files.
> >
> > I think this might be a bad idea in an active/active setup. You might be
> > better making your storage unmanaged. Because of the way I set this up
> > stopping heartbeat would stop the cluster disk.
>
> Uh, yeah.
> Having cluster resources running on non-cluster members (any node
> without fully active/running cluster software is NOT a cluster member)
> is _insanely_ dangerous.
>
> > This freaked out users that
> > may have been just looking at a file on the passive node while heartbeat
> was
> > restarting.
> >
> > Also I have found that adding resources into and out of groups or
> changing
> > the order can cause resources in HA to restart unexpectedly. I felt like
> my
> > setup was a minefield. A bug in one init script might cause a casade, or
> if
> > something in a colocation died it was going to cause a cluster
> > wide failover.
>
> This is not exactly surprise.
> Even in a non-clustered environment, a filesystem failure will cascade
> upwards to any databases, web/mail servers etc that may be running on
> it.
>
> So if the init script is broken and creates/invents a failure... the
> same thing will happen (or at least we'll try to recover them before
> they fail).
>
> Likewise for colocation... if you tell us A must run with B and B is
> not running...
> consider s/A/database/ and s/B/filesystem/
>
>
> > It was hard for me to enforce the business rules that I
> > really wanted to. Restarting something in a colocation caused a cascade.
> > This may be more or less due to a lack of heartbeat knowledge on my
> side.
> >
> > Everything was grand in the backroom. BUT
> > Our production configuration opens more files for logging and our
> software
> > opens a lot of files in general. OCFS2 and DRBD have a noticable lag in
> > starting our application. Could be the time to aquire file locks,etc.
> > Whatever the case it was slower. This had a bad effect on my timeouts in
> my
> > status and monitor functions. Life lesson, timeouts need to be set
> higher.
> > Still this caused headaches because I had already handing the system off
> to
> > the next implementer and they could not understand why configurations
> > changes was causing cluster wide failover from node1,
> node2,node1,node2.....
> > and repeatedly kicking you out of ssh as the VIP IP address kept failing
> > back and forth.
> >
> > My final straws were:
> >
> > That a system went up to high IO wait it needed a reboot.
> >
> > Corrupt files that can not be deleted.
> >
> > Unable to reboot machine after OCFS2 and DRDB failed. This was really
> > annoying might have been more of an OCFS problem then DRBD. Regardless I
> > could not unmount the disk in drbd, OCFS2 and o2cb stop, unload , etc
> would
> > not do it. Killing processes did not handle it neither did modprobing.
> Had
> > to call out datacenter. They rebooted the wrong node.
> >
> > I finally gave up on the disk cluster. We have a plan to move it all
> back to
> > a single server. I shut off one end of the node. It ran for about a
> > week. The system crashed today I got this message. 'OCFS timeout
> > accessing DRBD. OCFS2 is sorry to be fencing the system'
> >
> > Not as sorry as I am.
> > _______________________________________________
> > Linux-HA mailing list
> > [email protected]
> > http://lists.linux-ha.org/mailman/listinfo/linux-ha
> > See also: http://linux-ha.org/ReportingProblems
> >
> _______________________________________________
> Linux-HA mailing list
> [email protected]
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>
_______________________________________________
Linux-HA mailing list
[email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
