Re: misc. questions

2013-03-14 Thread Roger Pau Monné 
On 07/03/13 15:00, Jay West wrote:
> It was written
>> I think there were several fixes for the RTC clock recently, so trying the
> Xen 4.2 branch might bring some improvements.
> I'm using Citrix Xenserver 6.1, so stuff from the Xen 4.2 branch isn't
> available for me

Just tried under Xen 4.3 (-unstable), and the problem is fixed, the
clock seems to be stable and not drifting any more. I don't know if the
patches that fixed that problem have been pulled to either 4.1 or 4.2.

___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

RE: misc. questions

2013-03-07 Thread Jay West 
It was written
>I think there were several fixes for the RTC clock recently, so trying the
Xen 4.2 branch might bring some improvements.
I'm using Citrix Xenserver 6.1, so stuff from the Xen 4.2 branch isn't
available for me

>Also, it is possible to use the PV clock from HVM (PVHVM) domains, which
will probably bring better stability. I'm going to look into this when I
have some spare cycles.
That I would very much like to know how to do so that I can give it a shot.

J



___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

Re: misc. questions

2013-03-07 Thread Colin Percival 
On 03/07/13 04:35, Mark Felder wrote:
> On Wed, 06 Mar 2013 16:40:38 -0600, Colin Percival  
> wrote:
>> You'll want to turn off tso, since it produces long mbuf chains which most
>> xn netbacks choke on.  (I have a very ugly workaround patch for this which I
>> use on EC2, but simply turning off tso is enough unless you need Gbps+ 
>> speeds).
> 
> Can you link me to this patch? I have an environment that might warrant using 
> it
> for now.

Attached.

And remember that I said it was a *very ugly* workaround... :-)

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
--- sys/kern/uipc_mbuf.c(revision 223824)
+++ sys/kern/uipc_mbuf.c(working copy)
@@ -525,12 +525,14 @@
  * only their reference counts are incremented.
  */
 struct mbuf *
-m_copym(struct mbuf *m, int off0, int len, int wait)
+m_copy_nbufs(struct mbuf *m, int off0, int len, int wait, long * outlen,
+int nbufmax)
 {
struct mbuf *n, **np;
int off = off0;
struct mbuf *top;
int copyhdr = 0;
+   int len_orig = len;
 
KASSERT(off >= 0, ("m_copym, negative off %d", off));
KASSERT(len >= 0, ("m_copym, negative len %d", len));
@@ -546,7 +548,7 @@
}
np = ⊤
top = 0;
-   while (len > 0) {
+   while (len > 0 && nbufmax-- > 0) {
if (m == NULL) {
KASSERT(len == M_COPYALL, 
("m_copym, length > size of mbuf chain"));
@@ -584,6 +586,9 @@
if (top == NULL)
mbstat.m_mcfail++;  /* XXX: No consistency. */
 
+   if (outlen)
+   *outlen = len_orig - len;
+
return (top);
 nospace:
m_freem(top);
@@ -591,6 +596,13 @@
return (NULL);
 }
 
+struct mbuf *
+m_copym(struct mbuf *m, int off0, int len, int wait)
+{
+
+   return (m_copy_nbufs(m, off0, len, wait, NULL, INT_MAX));
+}
+
 /*
  * Returns mbuf chain with new head for the prepending case.
  * Copies from mbuf (chain) n from off for len to mbuf (chain) m
--- sys/netinet/tcp_output.c(revision 228872)
+++ sys/netinet/tcp_output.c(working copy)
@@ -183,6 +183,7 @@
int sack_rxmit, sack_bytes_rxmt;
struct sackhole *p;
int tso;
+   int max_mbuf_chain_len = 16;/* XXX Set this based on interface? */
struct tcpopt to;
 #if 0
int maxburst = TCP_MAXBURST;
@@ -806,16 +807,6 @@
struct mbuf *mb;
u_int moff;
 
-   if ((tp->t_flags & TF_FORCEDATA) && len == 1)
-   TCPSTAT_INC(tcps_sndprobe);
-   else if (SEQ_LT(tp->snd_nxt, tp->snd_max) || sack_rxmit) {
-   tp->t_sndrexmitpack++;
-   TCPSTAT_INC(tcps_sndrexmitpack);
-   TCPSTAT_ADD(tcps_sndrexmitbyte, len);
-   } else {
-   TCPSTAT_INC(tcps_sndpack);
-   TCPSTAT_ADD(tcps_sndbyte, len);
-   }
MGETHDR(m, M_DONTWAIT, MT_DATA);
if (m == NULL) {
SOCKBUF_UNLOCK(&so->so_snd);
@@ -847,7 +838,8 @@
mtod(m, caddr_t) + hdrlen);
m->m_len += len;
} else {
-   m->m_next = m_copy(mb, moff, (int)len);
+   m->m_next = m_copy_nbufs(mb, moff, len, M_DONTWAIT,
+   &len, max_mbuf_chain_len);
if (m->m_next == NULL) {
SOCKBUF_UNLOCK(&so->so_snd);
(void) m_free(m);
@@ -856,6 +848,18 @@
}
}
 
+   /* Update stats here as m_copy_nbufs may have adjusted len. */
+   if ((tp->t_flags & TF_FORCEDATA) && len == 1)
+   TCPSTAT_INC(tcps_sndprobe);
+   else if (SEQ_LT(tp->snd_nxt, tp->snd_max) || sack_rxmit) {
+   tp->t_sndrexmitpack++;
+   TCPSTAT_INC(tcps_sndrexmitpack);
+   TCPSTAT_ADD(tcps_sndrexmitbyte, len);
+   } else {
+   TCPSTAT_INC(tcps_sndpack);
+   TCPSTAT_ADD(tcps_sndbyte, len);
+   }
+
/*
 * If we're sending everything we've got, set PUSH.
 * (This will keep happy those implementations which only
--- sys/sys/mbuf.h  (revision 223824)
+++ sys/sys/mbuf.h  (working copy)
@@ -849,6 +849,7 @@
int, int, int, int);
 struct mbuf*m_copypacket(struct mbuf *, int);
 voidm_copy_pkthdr(struct mbuf *, struct mbuf *);
+struct mbuf*m_copy_nbufs(struct mbuf *, int, int, int, long *, int);
 struct mbuf*m_copyup(struct mbuf *n, int len, int dstoff);
 struct mbuf*m_defrag(struct mbuf *, int);
 voidm_demote(struct mbuf *, int);
___

Re: misc. questions

2013-03-07 Thread Mark Felder 
On Wed, 06 Mar 2013 16:40:38 -0600, Colin Percival   
wrote:


You'll want to turn off tso, since it produces long mbuf chains which  
most
xn netbacks choke on.  (I have a very ugly workaround patch for this  
which I
use on EC2, but simply turning off tso is enough unless you need Gbps+  
speeds).


Can you link me to this patch? I have an environment that might warrant  
using it for now.



Thanks!
___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

Re: misc. questions

2013-03-07 Thread Roger Pau Monné 
On 07/03/13 05:38, Jay West wrote:
> Mark wrote...
>> You're having success with Xenserver 6.1? That's good news. I wonder why
> XCP 1.6 is unable to boot FreeBSD if there's an emulated DVDROM? :(
> 
> Yes. Stock install 64bit 9.1-R as HVM, recompile kernel using supplied
> XENHVM config file. Then there's a few things to tweak before rebooting, one
> is fstab because of the different naming convention for disk devices or you
> choke on reboot after install (you can still do it after by entering correct
> device in boot string). The other is the cd drive, you have to remove the cd
> device (see http://support.citrix.com/article/CTX132411). There may be
> another step I'm forgetting, but that's the basic gist. Oh, and fix rc.conf
> to use the xen (xn0) network device. We are still having one really nagging
> issue - timekeeping. The clocks are way off and fluctuate wildly. FreeBSD is
> selecting the clock source (from several possibilities) that already has the
> highest stability rating, so I haven't tried setting the sysctl to use a
> different clock source. I see no way to fix this and it's pretty crippling
> in my environment anyways. Anyone have a fix for that?

I think there were several fixes for the RTC clock recently, so trying
the Xen 4.2 branch might bring some improvements.

Also, it is possible to use the PV clock from HVM (PVHVM) domains, which
will probably bring better stability. I'm going to look into this when I
have some spare cycles.

___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

RE: misc. questions

2013-03-06 Thread Chris Bachmann 
I haven't seen any solutions to the time problems. I have a ton of debian vm
clients as well and they have the same issue. I've just gotten used to
installing ntp to keep the clocks current.

-Original Message-
From: owner-freebsd-...@freebsd.org [mailto:owner-freebsd-...@freebsd.org]
On Behalf Of Jay West
Sent: Wednesday, March 06, 2013 11:38 PM
To: 'Mark Felder'; freebsd-xen@freebsd.org
Subject: RE: misc. questions

Mark wrote...
> You're having success with Xenserver 6.1? That's good news. I wonder 
> why
XCP 1.6 is unable to boot FreeBSD if there's an emulated DVDROM? :(

Yes. Stock install 64bit 9.1-R as HVM, recompile kernel using supplied
XENHVM config file. Then there's a few things to tweak before rebooting, one
is fstab because of the different naming convention for disk devices or you
choke on reboot after install (you can still do it after by entering correct
device in boot string). The other is the cd drive, you have to remove the cd
device (see http://support.citrix.com/article/CTX132411). There may be
another step I'm forgetting, but that's the basic gist. Oh, and fix rc.conf
to use the xen (xn0) network device. We are still having one really nagging
issue - timekeeping. The clocks are way off and fluctuate wildly. FreeBSD is
selecting the clock source (from several possibilities) that already has the
highest stability rating, so I haven't tried setting the sysctl to use a
different clock source. I see no way to fix this and it's pretty crippling
in my environment anyways. Anyone have a fix for that?

>If you're using pf, you will certainly need to set net.inet.tcp.tso=0
By pf I assume you mean the freebsd firewall? We don't use that, all the
firewall stuff is via a frontend pfsense install (non-vm), so I assume the
above setting isn't required?

>As far as the other offloading -- I put this on my NICs:
>ifconfig_xn0="(your stuff here) -txcsum -rxcsum -lro -tso"
Ah, good to know. Thanks!

>We usually install the same version on the VM and rsync the entire OS over.
Then replace the kernel with the XENHVM one, do a bit of tweaking (rc.conf,
pf.conf, fstab), and move on.
The "best" way I found to do this (WRT speed & functionality) was to just
use "dump" on the old machine, create the vm, then use "restore". That being
said, I can understand the commercial citrix xenserver being slow to
implement "XenConvert" on freebsd... but I'm surprised that the open source
Xen doesn't have a tool that turns a bare metal freebsd install into an
OVF/VDI file. Surely someone is working on this. Fortunately, all our
freebsd boxes are now virtualized, but I'm sure others are wanting
"XenConvert" to work on FreeBSD.

Thanks a ton for the input folks!

Jay



___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

RE: misc. questions

2013-03-06 Thread Jay West 
Mark wrote...
> You're having success with Xenserver 6.1? That's good news. I wonder why
XCP 1.6 is unable to boot FreeBSD if there's an emulated DVDROM? :(

Yes. Stock install 64bit 9.1-R as HVM, recompile kernel using supplied
XENHVM config file. Then there's a few things to tweak before rebooting, one
is fstab because of the different naming convention for disk devices or you
choke on reboot after install (you can still do it after by entering correct
device in boot string). The other is the cd drive, you have to remove the cd
device (see http://support.citrix.com/article/CTX132411). There may be
another step I'm forgetting, but that's the basic gist. Oh, and fix rc.conf
to use the xen (xn0) network device. We are still having one really nagging
issue - timekeeping. The clocks are way off and fluctuate wildly. FreeBSD is
selecting the clock source (from several possibilities) that already has the
highest stability rating, so I haven't tried setting the sysctl to use a
different clock source. I see no way to fix this and it's pretty crippling
in my environment anyways. Anyone have a fix for that?

>If you're using pf, you will certainly need to set net.inet.tcp.tso=0
By pf I assume you mean the freebsd firewall? We don't use that, all the
firewall stuff is via a frontend pfsense install (non-vm), so I assume the
above setting isn't required?

>As far as the other offloading -- I put this on my NICs:
>ifconfig_xn0="(your stuff here) -txcsum -rxcsum -lro -tso"
Ah, good to know. Thanks!

>We usually install the same version on the VM and rsync the entire OS over.
Then replace the kernel with the XENHVM one, do a bit of tweaking (rc.conf,
pf.conf, fstab), and move on.
The "best" way I found to do this (WRT speed & functionality) was to just
use "dump" on the old machine, create the vm, then use "restore". That being
said, I can understand the commercial citrix xenserver being slow to
implement "XenConvert" on freebsd... but I'm surprised that the open source
Xen doesn't have a tool that turns a bare metal freebsd install into an
OVF/VDI file. Surely someone is working on this. Fortunately, all our
freebsd boxes are now virtualized, but I'm sure others are wanting
"XenConvert" to work on FreeBSD.

Thanks a ton for the input folks!

Jay



___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

Re: misc. questions

2013-03-06 Thread Colin Percival 
On 03/06/13 12:17, Jay West wrote:
> 1)One or two of our freebsd 9.1 HVM (with PVM drivers) under
> Xenserver 6.1 advanced are fairly frequently generating this message on the
> console: xn_txeof: WARNING: response is -1! Any ideas what this may be and
> what should be done? It does seem to only occur on the machines that have
> higher network load than the others.

I see this on EC2 (running under HVM) as well.  It doesn't seem to interfere
with anything and only occurs occasionally (maybe five times per GB of network
traffic?) so I haven't worried too much about it -- but if anyone has a clue
about this I'd love to fix it from the "tidy up loose ends" perspective.

> 2)We did a pilot project of about 8 VM's (the above mentioned
> environment) and all went well. Now that we've moved it into production with
> many more VM's, I'm wondering about recommended tuning. I seem to recall
> from watching this list that there are a few sysctl's and the like that are
> highly recommended, I think they had to do with network settings and turning
> off "offloading" or somesuch. Does anyone have a quick & dirty list of
> "here's the first things you should always change" with regards to FreeBSD
> HVM (pvm drivers) under XenServer?

You'll want to turn off tso, since it produces long mbuf chains which most
xn netbacks choke on.  (I have a very ugly workaround patch for this which I
use on EC2, but simply turning off tso is enough unless you need Gbps+ speeds).

There's a separate issue affecting pf+tso+xn (possibly pf+tso generally) which
I haven't gotten around to tracking down.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

Re: misc. questions

2013-03-06 Thread Mark Felder 

On Wed, 06 Mar 2013 14:17:02 -0600, Jay West  wrote:


1)One or two of our freebsd 9.1 HVM (with PVM drivers) under
Xenserver 6.1 advanced are fairly frequently generating this message on  
the
console: xn_txeof: WARNING: response is -1! Any ideas what this may be  
and

what should be done? It does seem to only occur on the machines that have
higher network load than the others.


You're having success with Xenserver 6.1? That's good news. I wonder why  
XCP 1.6 is unable to boot FreeBSD if there's an emulated DVDROM? :(



2)We did a pilot project of about 8 VM's (the above mentioned
environment) and all went well. Now that we've moved it into production  
with

many more VM's, I'm wondering about recommended tuning. I seem to recall
from watching this list that there are a few sysctl's and the like that  
are
highly recommended, I think they had to do with network settings and  
turning

off "offloading" or somesuch. Does anyone have a quick & dirty list of
"here's the first things you should always change" with regards to  
FreeBSD

HVM (pvm drivers) under XenServer?


If you're using pf, you will certainly need to set net.inet.tcp.tso=0

As far as the other offloading -- I put this on my NICs:

ifconfig_xn0="(your stuff here) -txcsum -rxcsum -lro -tso"

3)When migrating bare-metal non-VM FreeBSD machines  
(primarily

webservers and mailservers) to the above Xenserver environment, we have
always just created VM's from an ISO, installed apache, sendmail, etc.  
and
then migrated the websites, mailboxes, etc. manually "across the wire"  
from

the non-VM machines to the replacement VM's. Xenconvert does not seem to
support FreeBSD/gpart/ufs. Does anyone know of a way (software or  
procedure)
to take a bare-metal FreeBSD 9.1 install and turn it into a VDI or  
OVF/OVA

that can be imported to Xenserver?


We usually install the same version on the VM and rsync the entire OS  
over. Then replace the kernel with the XENHVM one, do a bit of tweaking  
(rc.conf, pf.conf, fstab), and move on.


FYI, when we build kernels we do:

make buildkernel KERNCONF=XENHVM
make installkernel KERNCONF=XENHVM DESTDIR=/boot/ KODIR=9.1-XENHVM

This puts the kernel and modules in /boot/9.1-XENHVM and leaves the  
GENERIC kernel alone (as an emergency fallback)


/boot/loader.conf then contains kernel="9.1-XENHVM"

Note: for 9.0 it doesn't install the modules when you do it this way. We  
just set module_path="/boot/kernel;/boot/modules" and piggyback on the  
GENERIC kernel's modules.

___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"

misc. questions

2013-03-06 Thread Jay West 
1)One or two of our freebsd 9.1 HVM (with PVM drivers) under
Xenserver 6.1 advanced are fairly frequently generating this message on the
console: xn_txeof: WARNING: response is -1! Any ideas what this may be and
what should be done? It does seem to only occur on the machines that have
higher network load than the others.

 

2)We did a pilot project of about 8 VM's (the above mentioned
environment) and all went well. Now that we've moved it into production with
many more VM's, I'm wondering about recommended tuning. I seem to recall
from watching this list that there are a few sysctl's and the like that are
highly recommended, I think they had to do with network settings and turning
off "offloading" or somesuch. Does anyone have a quick & dirty list of
"here's the first things you should always change" with regards to FreeBSD
HVM (pvm drivers) under XenServer?

 

3)When migrating bare-metal non-VM FreeBSD machines (primarily
webservers and mailservers) to the above Xenserver environment, we have
always just created VM's from an ISO, installed apache, sendmail, etc. and
then migrated the websites, mailboxes, etc. manually "across the wire" from
the non-VM machines to the replacement VM's. Xenconvert does not seem to
support FreeBSD/gpart/ufs. Does anyone know of a way (software or procedure)
to take a bare-metal FreeBSD 9.1 install and turn it into a VDI or OVF/OVA
that can be imported to Xenserver? 

 

Thanks in advance for any tips, pointers, or insight!

 

 

Jay West

EZwind.net

PO Box 460474

Saint Louis, MO 63146

Voice: (314) 262-4143 ext 1000

Toll Free: (866) 343-2589

Fax: (314) 558-9284

  jw...@ezwind.net

 



This e-mail transmission may contain information that is proprietary,
privileged and/or confidential

and is intended exclusively for the person(s) to whom it is addressed. Any
use, copying, retention

or disclosure by any person other than the intended recipient or the
intended recipient's designees

is strictly prohibited. If you are not the intended recipient or their
designee, please notify the

sender immediately by return e-mail and delete all copies.

 

___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"