Sven Willenberger wrote:
On Thu, 2007-04-05 at 17:38 +0100, Tom Judge wrote:
Dmitriy Kirhlarov wrote:
On Thu, Apr 05, 2007 at 10:58:56AM -0400, Sven Willenberger wrote:
I am trying to set up a HA type system involving two identical boxes and
have gone through the following to set up the systems:

Slave server: ggated -R 196608 -S 196608
(exporting /dev/amrd1 )
net.inet.tcp.sendspace: 65536
net.inet.tcp.recvspace: 131072
Try
net.local.stream.recvspace=65535
net.local.stream.sendspace=65535

Also, try increase this sysctls with
net.inet.tcp.rfc1323=1

I use it on FreeBSD 5.x with:
net.inet.tcp.sendspace=131072
net.inet.tcp.recvspace=131072
net.local.stream.recvspace=65535
net.local.stream.sendspace=65535

ggated -R 1048576 -S 1048576
ggatec -R 1048576 -S 1048576

WBR.
Dmitriy
_______________________________________________
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

I have seen sustained writes of 30Mb/s using the following configuration:

cat /boot/loader.conf
kern.ipc.nmbclusters="32768"

cat /etc/sysctl.conf
net.inet.tcp.sendspace=1048576
net.inet.tcp.recvspace=1048576

Server:
/sbin/ggated -S 1310720 -R 1310720 -a 172.31.0.18 /etc/gg.exports

Client:
/sbin/ggatec create -q 2048 -t 5 -S 1310720 -R 1310720 172.31.0.18 /dev/amrd0s2

The raid array is a RAID 1 volume on a dell PERC4 (Dell PE1850) with adaptive read ahead and write back caching.

Tom

I have tried both the settings ideas suggested above but I cannot even
get out of the gate with those. Setting net.inet.tcp.{send,recv}space to
anything higher that 131072 results in ggated bailing with the error:
# ggated -v -a 10.10.0.19
info: Reading exports file (/etc/gg.exports).
debug: Added 10.10.0.0/24 /dev/amrd1 RW to exports list.
debug: Added 10.10.0.0/24 /dev/amrd3 RW to exports list.
info: Exporting 2 object(s).
error: Cannot open stream socket: No buffer space available.
error: Exiting.

setting net.inet.tcp.{send,recv}space to 131072 allows me to start
ggated with the default R and S values of 131072; anything higher
results in "no buffer space" errors. At 131072 ggated starts but then I
cannot even open a new connection (like ssh) to the server as the ssh
client bails with "no buffer space available".

Did you also set kern.ipc.nmbclusters="32768" in /boot/loader.conf and reboot? It sounds like you did not as this is the exact same problem I came across before adjusting that value.

<<SNIP>>

This is on a FreeBSD 6.2-RELENG box i386 SMP using the amr driver (SATA
Raid using LSiMegaRaid.

Do you have the cache BBU fitted (Batery Backup Unit) and the array caching set to write back? Also have you tested writing to the array locally without ggate to test the write speed?


The odd thing is that even after I set the send and recvspace down to
values like 65536, I continue to get the no buffer error when trying to
connect to it remotely again.


I found that the easyest way to fix this was to reboot the system with good values for net.inet.tcp.{send,recv}space.



_______________________________________________
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Reply via email to