Re: FreeBSD HA file cluster possibilities
RIck van der Zwet wrote:
I been (re)searching and reading what the options are with regards to
H(igh) A(vailablility) file storage using FreeBSD, but cannot yet find a
proper working solution. Any advice welcome!
I like to be able to mirror a full identical disk between two server. So
in case of hardware failure of server A (Master). Server B (Slave)
immediately takes over, without any loss of data. The Network
configuration is easy using ucarp/vppr. But the file system is the hard
part. Paths I have investigated:
a) ggate gmirror: Export system on Server B to Server A. Use gmirror
on Server A to keep identical disks. When the ggated on Server B
actually goes down, the whole setup freezes, until the ggated is back up
again. Second on network delays gmirror looses, having to sync all over
again. Leaving the machine at risk.
The freezing has come to a end, with the patch attached, but is the
patch the right way to go (as C coding is not my strongest point)?
To test:
# Create backup filesystem export it
serverB$ truncate -s100m /root/ha-slave.img
serverB$ echo 192.168.33.41 RW /root/ha-slave.img /etc/gg.exports
serverB$ ggated
# Apply attached patch
serverA$ cd /usr/src/sbin/ggate/ggatec
serverA$ patch %%ATTACHED_FILE%%
serverA$ make clean install
# Local file image
serverA$ truncate -s 100m /root/ha-master.img
serverA$ mdconfig -t vnode -f /root/ha-master.img
#Remote file image
serverA$ ggatec create 192.168.33.42 /root/ha-slave.img
# Mirror building
serverA$ gmirror label hamirror ggate0 md0
serverA$ newfs /dev/mirror/hamirror
serverA$ mount /dev/mirror/hamirror /mnt
Note: if you have _not_ applied the patch and you kill ggated on
machineB you will notice machineA freeze when trying to write to
something on /mnt or call `gmirror status'. Same applies if you kill
ggatec on machineA without patch.
Using net/ucarp I detect failures on serverA and terminate ggated and
mount the image on serverB.
/Rick
--- ggatec.c.orig 2009-07-09 18:27:12.0 +0200
+++ ggatec.c 2009-07-14 10:15:34.0 +0200
@@ -156,7 +156,7 @@
break;
if (data != sizeof(hdr)) {
g_gate_log(LOG_ERR, Lost connection 1.);
- reconnect = 1;
+ reconnect = 0;
pthread_kill(recvtd, SIGUSR1);
break;
}
@@ -168,7 +168,7 @@
break;
if (data != ggio.gctl_length) {
g_gate_log(LOG_ERR, Lost connection 2 (%zd != %zd)., data, (ssize_t)ggio.gctl_length);
-reconnect = 1;
+reconnect = 0;
pthread_kill(recvtd, SIGUSR1);
break;
}
@@ -177,6 +177,7 @@
}
}
g_gate_log(LOG_DEBUG, %s: Died., __func__);
+ g_gate_destroy(unit, 1);
return (NULL);
}
@@ -203,7 +204,7 @@
if (data == -1 errno == EAGAIN)
continue;
g_gate_log(LOG_ERR, Lost connection 3.);
- reconnect = 1;
+ reconnect = 0;
pthread_kill(sendtd, SIGUSR1);
break;
}
@@ -223,7 +224,7 @@
g_gate_log(LOG_DEBUG, Received data packet.);
if (data != ggio.gctl_length) {
g_gate_log(LOG_ERR, Lost connection 4.);
-reconnect = 1;
+reconnect = 0;
pthread_kill(sendtd, SIGUSR1);
break;
}
@@ -235,6 +236,7 @@
g_gate_ioctl(G_GATE_CMD_DONE, ggio);
}
g_gate_log(LOG_DEBUG, %s: Died., __func__);
+ g_gate_destroy(unit, 1);
pthread_exit(NULL);
}
@@ -410,8 +412,7 @@
static void
signop(int sig __unused)
{
-
- /* Do nothing. */
+ g_gate_destroy(unit,1);
}
static void
@@ -420,6 +421,7 @@
struct g_gate_ctl_cancel ggioc;
signal(SIGUSR1, signop);
+ signal(SIGINT, signop);
for (;;) {
g_gatec_start();
g_gate_log(LOG_NOTICE, Disconnected [%s %s]. Connecting...,
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD HA file cluster possibilities
On Tue, Jul 07, 2009 at 03:54:23PM -0500, Adam Vande More typed: On Tue, Jul 7, 2009 at 7:17 AM, RIck van der Zwet rvdz...@transip.nlwrote: d) drbd: http://www.drbd.org/ - Does anyone has a working (alternative) setup on FreeBSD? Any other solution/setup I am missing? Using external hardware/software is all-right as long I am able to connect to FreeBSD with it and it leaves no single point of failure. Thanks in advance! /Rick freebsd-questions@freebsd.org Is there a problem with the DRBD and Heartbeat config? Ehm... It doesn't exist in FreeBSD? regards, Ruben ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD HA file cluster possibilities
Hi Rick, On Tue, Jul 07, 2009 at 02:17:46PM +0200, RIck van der Zwet typed: I been (re)searching and reading what the options are with regards to H(igh) A(vailablility) file storage using FreeBSD, but cannot yet find a proper working solution. Any advice welcome! I've sought for proper working solutions as well, and unfortunately found much the same obstacles as you have. a DRBD port to FreeBSD would be great, but I think it would be a lot of work. The way it can recover from a broken mirror makes it a killer app IMO. This is what I ended up doing: - Use application level mirroring where possible - Use rsync where (small) data loss is acceptable - buy dedicated hardware where HA storage is mandatory regards, Ruben ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD HA file cluster possibilities
On Wed, Jul 8, 2009 at 2:06 AM, Ruben de Groot mai...@bzerk.org wrote: On Tue, Jul 07, 2009 at 03:54:23PM -0500, Adam Vande More typed: On Tue, Jul 7, 2009 at 7:17 AM, RIck van der Zwet rvdz...@transip.nl wrote: d) drbd: http://www.drbd.org/ - Does anyone has a working (alternative) setup on FreeBSD? Any other solution/setup I am missing? Using external hardware/software is all-right as long I am able to connect to FreeBSD with it and it leaves no single point of failure. Thanks in advance! /Rick freebsd-questions@freebsd.org Is there a problem with the DRBD and Heartbeat config? Ehm... It doesn't exist in FreeBSD? regards, Ruben Sry, my bad. the ggated setup looks promising though, and you could tie it into carp. -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
FreeBSD HA file cluster possibilities
I been (re)searching and reading what the options are with regards to H(igh) A(vailablility) file storage using FreeBSD, but cannot yet find a proper working solution. Any advice welcome! I like to be able to mirror a full identical disk between two server. So in case of hardware failure of server A (Master). Server B (Slave) immediately takes over, without any loss of data. The Network configuration is easy using ucarp/vppr. But the file system is the hard part. Paths I have investigated: a) ggate gmirror: Export system on Server B to Server A. Use gmirror on Server A to keep identical disks. When the ggated on Server B actually goes down, the whole setup freezes, until the ggated is back up again. Second on network delays gmirror looses, having to sync all over again. Leaving the machine at risk. b) SAN/iSCSI targets/gmirror: [Brainstorm] Have 2 iSCSI storage boxes namely C and D. Have Server A run gmirror on the C and D. And let B pickup on failure? Does this work? c) ssync/rsync: This does not suit the needs as it does not provide realtime mirroring. Causing data-loss on failure. As rsync/ssync does not run in realtime. d) drbd: http://www.drbd.org/ - Does anyone has a working (alternative) setup on FreeBSD? Any other solution/setup I am missing? Using external hardware/software is all-right as long I am able to connect to FreeBSD with it and it leaves no single point of failure. Thanks in advance! /Rick ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD HA file cluster possibilities
Hi, I'm curious about your solution. Let me know if you have one :-) I'm wondering if d) is possible in FreeBSD as from my view it would be the best solution. Wether you use synchronous (assuming you have sufficient bandwith) or asynchronous writes. A wonderful solution would be if AFS would provide in-time replication. However at the moment replication is hand-task in AFS. Another solution would be possible but I'm not sure that it has decent performance. You could use subversion with autoversioning and let it write into a mysql DB (or any other FS that writes into a database) and configure database replication between the two fileservers. Has anyone setup this and is capable of providing information on data throughput? Regards, --- Mr. Olli On Tue, 2009-07-07 at 14:17 +0200, RIck van der Zwet wrote: I been (re)searching and reading what the options are with regards to H(igh) A(vailablility) file storage using FreeBSD, but cannot yet find a proper working solution. Any advice welcome! I like to be able to mirror a full identical disk between two server. So in case of hardware failure of server A (Master). Server B (Slave) immediately takes over, without any loss of data. The Network configuration is easy using ucarp/vppr. But the file system is the hard part. Paths I have investigated: a) ggate gmirror: Export system on Server B to Server A. Use gmirror on Server A to keep identical disks. When the ggated on Server B actually goes down, the whole setup freezes, until the ggated is back up again. Second on network delays gmirror looses, having to sync all over again. Leaving the machine at risk. b) SAN/iSCSI targets/gmirror: [Brainstorm] Have 2 iSCSI storage boxes namely C and D. Have Server A run gmirror on the C and D. And let B pickup on failure? Does this work? c) ssync/rsync: This does not suit the needs as it does not provide realtime mirroring. Causing data-loss on failure. As rsync/ssync does not run in realtime. d) drbd: http://www.drbd.org/ - Does anyone has a working (alternative) setup on FreeBSD? Any other solution/setup I am missing? Using external hardware/software is all-right as long I am able to connect to FreeBSD with it and it leaves no single point of failure. Thanks in advance! /Rick ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD HA file cluster possibilities
On Tue, Jul 7, 2009 at 7:17 AM, RIck van der Zwet rvdz...@transip.nlwrote: I been (re)searching and reading what the options are with regards to H(igh) A(vailablility) file storage using FreeBSD, but cannot yet find a proper working solution. Any advice welcome! I like to be able to mirror a full identical disk between two server. So in case of hardware failure of server A (Master). Server B (Slave) immediately takes over, without any loss of data. The Network configuration is easy using ucarp/vppr. But the file system is the hard part. Paths I have investigated: a) ggate gmirror: Export system on Server B to Server A. Use gmirror on Server A to keep identical disks. When the ggated on Server B actually goes down, the whole setup freezes, until the ggated is back up again. Second on network delays gmirror looses, having to sync all over again. Leaving the machine at risk. b) SAN/iSCSI targets/gmirror: [Brainstorm] Have 2 iSCSI storage boxes namely C and D. Have Server A run gmirror on the C and D. And let B pickup on failure? Does this work? c) ssync/rsync: This does not suit the needs as it does not provide realtime mirroring. Causing data-loss on failure. As rsync/ssync does not run in realtime. d) drbd: http://www.drbd.org/ - Does anyone has a working (alternative) setup on FreeBSD? Any other solution/setup I am missing? Using external hardware/software is all-right as long I am able to connect to FreeBSD with it and it leaves no single point of failure. Thanks in advance! /Rick freebsd-questions@freebsd.org Is there a problem with the DRBD and Heartbeat config? -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
