Re: FreeBSD 10.0-R as Xen 'guest' - clarification?
On Jan 29, 2014, at 8:28 AM, Karl Pielorz kpielorz_...@tdx.co.uk wrote: Hi, With FreeBSD 10 being out now (Great!) - GENERIC appears now has everything needed for Xen to run in PVHVM for amd64. The man page for xen (man 4 xen) states you should have: options NO_ADAPTIVE_MUTEXES options NO_ADAPTIVE_RWLOCKS options NO_ADAPTIVE_SX The “NO_ADAPTIVE” settings are an optimization when running in environments where different guests run on the same physical CPU. However, many cloud providers seem to statically pin CPUs to VMs, which means the adaptive lock optimization works as expected. — Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD Alpha5 amd64 - Citrix Xen 6.2 problem
On Oct 10, 2013, at 3:09 AM, Shanker Balan m...@shankerbalan.net wrote: On 10-Oct-2013, at 1:37 PM, Shanker Balan m...@shankerbalan.net wrote: On 09-Oct-2013, at 8:06 PM, Mark Felder f...@freebsd.org wrote: On Wed, Oct 9, 2013, at 8:53, Roger Pau Monné wrote: On 09/10/13 13:49, Mark Felder wrote: On Wed, Oct 9, 2013, at 2:46, Roger Pau Monné wrote: On 09/10/13 08:18, Shanker Balan wrote: On 08-Oct-2013, at 9:19 PM, Mark Felder f...@freebsd.org wrote: On Tue, Oct 8, 2013, at 9:45, Josias L.G wrote: Problem with Citrix Xen 6.2 and install from ISO. The solution was remove cd-rom drive from virtual machine. Not possible now with xen default in GENERIC kernel. Message error: run_interrupt_driven_hooks - still waiting after 300 seconds for xenbusb_nop_confighook_cb panic: run_interrupt_driven_config_hooks: waited too long I was going to test this soon... but you're right -- you probably can't install FreeBSD 10 from ISO on Citrix XenServer because of this bug. Can someone working on the xen bits test and maybe find a workaround? The xenbusb_nop_confighook_cb issue is the only issue which I am aware of that prevents CloudStack/XenServer IaaS private clouds from offering FreeBSD 10 as a supported OS template. The vbd-destroy workaround is not possible as the ISO is attached to the VM instance during the installation. A please pretty please request to @citrix RD for the hopefully last fix to get FreeBSD 10 running on XenServer+CloudStack. The earlier HyperV related panic on XenServer has been fixed in ALPHA5. Hello, I've taken a look into this and I'm afraid there's no easy way to workaround it from FreeBSD. When Xen is detected all IDE devices are disconnected, and there's no fine grained way to only disable IDE disks and not cdrom devices. Could you please contact your XenServer representative, and/or submit this bug to xs-devel (xs-de...@lists.xenserver.org) mailing lists in order to get this fixed on XenServer. Citrix is aware of this as I've contacted several people there and this has been discussed both here and on the xs-devel list. There has to be something FreeBSD can do to work around this issue since Linux and NetBSD have no issues. Linux and NetBSD have no issues because you probably only tried them on PV mode, which doesn't exhibit this issue (also NetBSD doesn't have PVHVM support, so it's quite clear it won't have this issue). As far as I'm aware the issue has been tracked down to badly behaving qemu in XenServer -- they don't use upstream qemu in XenServer (yet), and instead have their own fork. A future release is supposed to merge with upstream qemu. The main problem here is that XenServer announces a PV block device on xenstore (the cdrom), but then it seems like there's no backend to handle it, so it hangs on the connection phase. IMHO the problem is not with the device model (Qemu), but with the backend that should handle this PV device. Xen only allows you to either disable all IDE devices or none, so the only possible solution I can think of is to not disable anything at all and use the emulated devices, which will leave us with very poor performance (unless I'm missing something, there's no way to only disable disks but not cdroms). But the fact remains that this is a non-issue on Linux and NetBSD who handle this buggy virtual CDROM without any problems. There has to be some way we can add a quirk on our side so this device doesn't stop the entire boot process. If FreeBSD 10 is released without out-of-the-box support on the premier commercial Xen platform we'll be shooting ourselves in the foot and all of this work will be for naught. Amazon isn't the only Xen platform people use. You can always use the pre-build VM images I guess (I have not tested those, but I expect they should work fine under Xen). ftp://ftp.nl.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/20131007/10.0-ALPHA5/amd64/ This is way better. Having ready to import Xen images beats ISO installation any day. Will try on my cloudstack setup later today. I converted the QCOV image to VHD and have imported it into CloudStack. Instance created from the template also have the xenbusb issue at bootup. Oh well. Thanks for all the help. Regards. @shankerbalan The CDROM issue should be resolved in FreeBSD/head. These fixes will be include in 10.0-BETA2 as well as the first 11-CURRENT snapshot that should pop up in the next day or two. The snapshots are announced on the freebsd-snapshots mailing list. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Another blkback issue related to flush
On Sep 4, 2013, at 11:07 AM, Roger Pau Monné roger@citrix.com wrote: Hello, I've found another issue with blkback handling of flush operations, it was incorrectly setting one of the bio parameters when using a block device as backend. The attached patch fixes it, and also includes a small fix to correctly set the operation when writing the response on the ring (all responses written by FreeBSD blkback were of type BLKIF_OP_READ, because nreq-operation was not set). It seems that the req's pendcnt is unused and can be culled. Otherwise, looks good to me. Here's the patch I have in my tree now. -- Justin blkback.diff Description: Binary data ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: blkback making assumptions about the id of the requests
On Sep 3, 2013, at 3:39 AM, Roger Pau Monné roger@citrix.com wrote: On 02/09/13 22:03, Justin T. Gibbs wrote: On Sep 2, 2013, at 11:58 AM, Roger Pau Monné roger@citrix.com wrote: Hello, While playing with driver domains using FreeBSD I've found out that blkback in FreeBSD makes assumptions about the id of a request instead of actually using the id of the request on the shared ring. This seems wrong to me, since a frontend might choose whatever ids it like for the requests (like using 100-131 instead of 0-31). The patch attached fixes it by copying the id from the request on the ring to blkback internal request structure. Roger. It looks to me like the id is set in xbb_dispatch_io(). Why it is done there and not earlier, I don't recall. Sorry, I've missed to spot that the id is set there, the problem is that requests of type BLKIF_OP_FLUSH_DISKCACHE never reach that point, so they end up having incorrect requests ids. I've reworded the commit message and removed the late setting of the request id, now it is set earlier so requests of type flush also have a valid id when writing the response on the ring. Makes sense. Committed. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: blkback making assumptions about the id of the requests
On Sep 2, 2013, at 11:58 AM, Roger Pau Monné roger@citrix.com wrote: Hello, While playing with driver domains using FreeBSD I've found out that blkback in FreeBSD makes assumptions about the id of a request instead of actually using the id of the request on the shared ring. This seems wrong to me, since a frontend might choose whatever ids it like for the requests (like using 100-131 instead of 0-31). The patch attached fixes it by copying the id from the request on the ring to blkback internal request structure. Roger. It looks to me like the id is set in xbb_dispatch_io(). Why it is done there and not earlier, I don't recall. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
[CFR] Xen IPI patch
In my continuing effort to get all of Roger's Xen enhancements into FreeBSD, I'm just about finished reviewing the next patch in his series. The current status of the patch can be found here: http://people.freebsd.org/~gibbs/xen_ipi.diff The main, and late breaking, wrinkle for this patch is the newly committed PCID support form AMD64. I've done my best to translate the updated assembly IPI handlers into the C equivalents needed for Xen. But, I'm by no means an x86 assembly expert, and my current Xen installation doesn't advertise PCID, so all of the new branches I've added have yet to be tested. I also wonder if PVHVM guests are supposed to perform the PCID work natively or use some other Hypervisor functionality. Other things I'm not happy about with the patch: - hvm.c wants to use invltlb_globpcid(), but it is private to pmap.c. I've moved it to cpufunc.h to be next to invltlb(), but the definition of CR4_PGE (from specialreg.h) isn't visible in all of the contexts that include cpufunc.h. So I hackishly define it if necessary. Ick. - With the divergence of IPI handling between i386 and amd64, x86/xen/hvm.c is growing too many ifdefs. Should we delegate the definitions of the ipi functions to amd64/xen/hvm.c and i386/xen/hvm.c? - There's too much code duplication in xen_invlrng() and some of the logic in the other handlers can probably be tightened up. Unfortunately, I'm out of time for tonight. Hopefully the list will have some great suggestions/fixes/improvements for this patch before I pick it up again tomorrow. Thanks, Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
[CFR] Event channel Interrupts and unified Xen interrupt infrastructure.
I've been working to get the next chunk of Spectra/Roger Pau Monné Xen work into head. The latest version of the patch I'm working on can be found here: http://people.freebsd.org/~gibbs/xen_intr.diff I will continue my testing today and commit it tonight unless I hear complaints. Comments, corrections, improvements, and bug reports welcome. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: [CFR] Event channel Interrupts and unified Xen interrupt infrastructure.
On Aug 29, 2013, at 1:15 PM, Colin Percival cperc...@freebsd.org wrote: On 08/29/13 09:32, Justin T. Gibbs wrote: I've been working to get the next chunk of Spectra/Roger Pau Monné Xen work into head. The latest version of the patch I'm working on can be found here: http://people.freebsd.org/~gibbs/xen_intr.diff I will continue my testing today and commit it tonight unless I hear complaints. Comments, corrections, improvements, and bug reports welcome. Could this patch be split into functional and non-functional components? It's distracting having everything mixed up together in a single megapatch. Even committing the s/unlikely/__predict_false/ separately would help. It could, but it would delay an already long delayed process in getting this work into the tree. How do you intend to have Xen HVM work in the GENERIC kernel configuration? Will you be adding 'options XENHVM' to GENERIC? Yes. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: [CFR] Event channel Interrupts and unified Xen interrupt infrastructure.
On Aug 29, 2013, at 1:36 PM, Colin Percival cperc...@freebsd.org wrote: In sys/x86/xen/hvm.c: +/** + * If non-zero, the hypervisor has been configured to use a direct + * IDT event callback to the + */ This comment seems to be missing part of its Fixed. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Difference in event channel implementation for Xen PV vs HVM guests
On Mar 21, 2013, at 6:50 AM, Roger Pau Monné roger@citrix.com wrote: On 21/03/13 13:43, Roger Pau Monné wrote: On 18/03/13 14:08, Justin T. Gibbs wrote: Hi Roger, I know of no reasons why XENHVM cannot use the full event channel interface. In fact, Spectra Logic implemented PV timers and a general cleanup of the HVM event channel interface. I haven't merged it back yet because I know the changes break PV and I haven't found time to clean up the PV code, merge the HVM and PV event channel systems, and move IPI/MSI delivery to event channels. I've uploaded Spectra's changes here: http://people.freebsd.org/~gibbs/xen_ev/ Hi Justin, There's at least one file which seems to be missing: Ops, there at least two files missing: sys/x86/xen/hvm.c sys/xen/evtchn/evtchnvar.h I've regenerated the archive with those two files added. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Difference in event channel implementation for Xen PV vs HVM guests
On Mar 18, 2013, at 6:35 AM, Roger Pau Monné roger@citrix.com wrote: Hello, While working on improving XENHVM (I've been looking at adding PV timers), I've realized that the event channel implementation in PV vs HVM mode differs greatly. Xen PV port uses sys/xen/evtchn/evtchn.c while Xen HVM uses sys/dev/xenpci/evtchn.c, and the Xen HVM implementation is greatly reduced (only contains the necessary functions to operate backends/frontends). To implement PV timers I need to expand the event channel interface for XENHVM, and I was wondering why FreeBSD choose to have two different implementations, the main difference between PV and HVM is the event callback, but I guess this can be abstracted between the two different implementations, and then everything else could be reused. Am I missing something obvious? Is there any known technical problem in modifying XENHVM to use the full event channel implementation present in sys/xen/evtchn/evtchn.c that prevented XENHVM from using it in the first place? (Sorry if I've Cc'ed someone not related) Thanks, Roger. ___ freebsd-virtualizat...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org Hi Roger, I know of no reasons why XENHVM cannot use the full event channel interface. In fact, Spectra Logic implemented PV timers and a general cleanup of the HVM event channel interface. I haven't merged it back yet because I know the changes break PV and I haven't found time to clean up the PV code, merge the HVM and PV event channel systems, and move IPI/MSI delivery to event channels. I've uploaded Spectra's changes here: http://people.freebsd.org/~gibbs/xen_ev/ The diffs file provides the history of the original checkins to our Perforce repository. The tar file includes all the files that have been modified and reflects our efforts to keep our code base in sync with stable/9. Apart from the PV issues outlined above, I would expect the code provided to just drop right in to stable/9. Unfortunately, Xen support is not a current priority for Spectra so I don't have a lot of day job time to focus on getting this code back into FreeBSD. However, if this code looks like it would suite your needs, and you have resources for testing i386/PV, I'd be happy to collaborate with you and will make the time to help get this committed. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: amd64 xen hvm shutdown
On Feb 2, 2012, at 12:31 PM, Richard Kojedzinszky wrote: Dear users, I am trying to compare the 8.2 i386 PV mode and 9.0 amd64 hvm mode, regarding $ xm shutdown from outside. While 8.2 PV shuts down well, 9.0 hvm does not. Is it an issue of the xen hypervisor, or of freebsd? What experience do others have? If you are using a FreeBSD amd64 kernel with Xen PV drivers installed, shutdown behavior is controlled by the PV control driver. This driver notices updates to the control/shutdown node in the XenStore and is supposed to act accordingly. For a reboot or power off event, it simply calls shutdown_nice(RB_POWEROFF|RB_HALT), or shutdown_nice(0) as appropriate. If you can capture the value of the shutdown node in the XenStore, it should be pretty easy to debug this. -- Justin___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: [CFT][PATCH] Netfront fixes for FreeBSD-head [and an OT issue]
On 9/15/11 4:35 AM, Hugo Silva wrote: On 09/15/11 05:43, Janne Snabb wrote: On Mon, 12 Sep 2011, Justin T. Gibbs wrote: I'm planning to ask RE for permission to merge the following netfront fixes (listed below) into 9.0/head. Assuming this is the same patch that I posted on the list after someone sent it to me, it seems to solve the problem (xn* works). It is similar, but not identical. If you have the resources to test my patch, I would appreciate it. A major problem (and after the netdiff fix, the only one that comes to mind in FreeBSD/Xen, from a SA PoV) right now is the VMs losing massive amounts of time (just the other day I manually ran ntpdate, for a 1h adjustment); started a thread last week or so, but it died down. It may not happen 100% of the time (calcru: runtime went backwards + losing system time), as I remember that a couple of years ago several people on the list had this problem while I wasn't experiencing it. Perhaps it depends on the dom0 OS, some system hardware.. I really have no idea.. If someone else is running amd64 HVM+PV, let us know if you see the same? We run it at Spectra Logic. I can replicate the calcru: runtime went backwards issue if I do not pin our FreeBSD VM such that it exclusively owns the CPUs it uses. I have not had time to dig into this yet, since it is not a blocking bug for our product. I hope to have some time to work on this in a few weeks. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
[CFT/CFR] Improved Xen Suspend/Resume Support
This work was sponsored by BQ Internet Corporation and they have
successfully migrated 100+ FreeBSD VMs with these changes (slightly
modified for use in 8.x). I'd like to have feedback from other
developers and FreeBSD Xen users before submitting this and/or
asking for approval to push them into 9.0.
Thanks,
Justin
==
Change 501527 by justing@justing-ns1 on 2011/08/14 14:12:39
Properly handle suspend/resume events in the Xen device
framework.
sys/xen/xenbus/xenbusb.c:
o In xenbusb_resume(), publish the state transition of the
resuming device into XenbusStateIntiailising so that the
remote peer can see it. Recording the state locally is
not sufficient to trigger a re-connect sequence.
o In xenbusb_resume(), defer new-bus resume processing until
after the remote peer's XenStore address has been updated.
The drivers may need to refer to this information during
resume processing.
sys/xen/xenbus/xenbusb_back.c:
sys/xen/xenbus/xenbusb_front.c:
Register xenbusb_resume() rather than bus_generic_resume()
as the handler for device_resume events.
sys/xen/xenstore/xenstore.c:
o Fix grammer in a comment.
o In xs_suspend(), pass suspend events on to the child
devices (e.g. xenbusb_front/back, that are attached
to the XenStore.
Affected files ...
... //depot/SpectraBSD/head/sys/xen/xenbus/xenbusb.c#9 edit
... //depot/SpectraBSD/head/sys/xen/xenbus/xenbusb_back.c#7 edit
... //depot/SpectraBSD/head/sys/xen/xenbus/xenbusb_front.c#6 edit
... //depot/SpectraBSD/head/sys/xen/xenstore/xenstore.c#5 edit
Differences ...
//depot/SpectraBSD/head/sys/xen/xenbus/xenbusb.c#9 (text)
@@ -773,7 +773,7 @@
ivars = device_get_ivars(kids[i]);
xs_unregister_watch(ivars-xd_otherend_watch);
- ivars-xd_state = XenbusStateInitialising;
+ xenbus_set_state(kids[i], XenbusStateInitialising);
/*
* Find the new backend details and
@@ -783,16 +783,16 @@
if (error)
return (error);
- DEVICE_RESUME(kids[i]);
-
statepath = malloc(ivars-xd_otherend_path_len
+ strlen(/state) + 1, M_XENBUS, M_WAITOK);
sprintf(statepath, %s/state, ivars-xd_otherend_path);
free(ivars-xd_otherend_watch.node, M_XENBUS);
ivars-xd_otherend_watch.node = statepath;
+
+ DEVICE_RESUME(kids[i]);
+
xs_register_watch(ivars-xd_otherend_watch);
-
#if 0
/*
* Can't do this yet since we are running in
//depot/SpectraBSD/head/sys/xen/xenbus/xenbusb_back.c#7 (text)
@@ -292,7 +292,7 @@
DEVMETHOD(device_detach,bus_generic_detach),
DEVMETHOD(device_shutdown, bus_generic_shutdown),
DEVMETHOD(device_suspend, bus_generic_suspend),
- DEVMETHOD(device_resume,bus_generic_resume),
+ DEVMETHOD(device_resume,xenbusb_resume),
/* Bus Interface */
DEVMETHOD(bus_print_child, xenbusb_print_child),
//depot/SpectraBSD/head/sys/xen/xenbus/xenbusb_front.c#6 (text)
@@ -171,7 +171,7 @@
DEVMETHOD(device_detach,bus_generic_detach),
DEVMETHOD(device_shutdown, bus_generic_shutdown),
DEVMETHOD(device_suspend, bus_generic_suspend),
- DEVMETHOD(device_resume,bus_generic_resume),
+ DEVMETHOD(device_resume,xenbusb_resume),
/* Bus Interface */
DEVMETHOD(bus_print_child, xenbusb_print_child),
//depot/SpectraBSD/head/sys/xen/xenstore/xenstore.c#5 (text)
@@ -721,8 +721,8 @@
/*
* The count of transactions drops if we attempted
* to end a transaction (even if that attempt fails
-* in error), we receive a transaction end acknowledgement
-* or if our attempt to begin a transactionfails.
+* in error), we receive a transaction end acknowledgement,
+* or if our attempt to begin a transaction fails.
*/
if (request_msg_type == XS_TRANSACTION_END
|| (request_reply_error == 0 reply_msg_type == XS_TRANSACTION_END)
@@ -1194,9 +1194,15 @@
* all transactions and individual requests have completed.
*/
static int
-xs_suspend(device_t dev __unused)
+xs_suspend(device_t dev)
{
+ int error;
+ /* Suspend child Xen devices. */
+ error = bus_generic_suspend(dev);
+
Re: [CFT/CFR] Improved Xen Suspend/Resume Support
On Sep 1, 2011, at 11:40 AM, Sean Bruno sean...@yahoo-inc.com wrote: On Thu, 2011-09-01 at 09:55 -0700, Justin T. Gibbs wrote: This work was sponsored by BQ Internet Corporation and they have successfully migrated 100+ FreeBSD VMs with these changes (slightly modified for use in 8.x). I'd like to have feedback from other developers and FreeBSD Xen users before submitting this and/or asking for approval to push them into 9.0. Thanks, Justin Firing these up on ref9-xen64/32 momentarily. What's the magic foo to get patch to understand p4 stuffs? Sean Only some p4 commands generate 100% patch friendly output and I neglected to use those here. You can either just feed the file name into path when it asks or I can regenerate the doffs when I get back to my desk. Sorry about that. :-( -- Justin___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: [CFT/CFR] Improved Xen Suspend/Resume Support
On Sep 1, 2011, at 12:05 PM, Sean Bruno sean...@yahoo-inc.com wrote: On Thu, 2011-09-01 at 09:55 -0700, Justin T. Gibbs wrote: This work was sponsored by BQ Internet Corporation and they have successfully migrated 100+ FreeBSD VMs with these changes (slightly modified for use in 8.x). I'd like to have feedback from other developers and FreeBSD Xen users before submitting this and/or asking for approval to push them into 9.0. Thanks, Justin hrm ... am I doing it right? [root@xen1 xen]# xm suspend ref9-xen64 Error: Domain is not managed by Xend lifecycle support. Usage: xm suspend DomainName Suspend a Xend managed domain [root@xen1 xen]# rpm -q -a|grep xen xen-libs-3.0.3-120.el5_6.2 xen-3.4.3-5.el5 xen-libs-3.4.3-5.el5 kernel-xen-2.6.18-238.12.1.el5 [root@xen1 xen]# Sean The domain must be managed. You can use xm new config_file to create a managed domain. -- Justin___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: breakage in blkfront with ring_pages 1
On 7/5/11 10:14 PM, Colin Percival wrote: On 07/05/11 19:42, Colin Percival wrote: On 07/05/11 19:04, Justin T. Gibbs wrote: On 7/5/11 7:14 PM, Colin Percival wrote: Maybe the right option is to have a loader tunable dev.xn.linuxback to control which version of the protocol is used? What a mess. Yep. Mess or not, shall I go ahead with having a loader tunable control this, or can you think of a better solution? Does anyone object to the attached patch? It keeps the differing behaviour to a minimum -- we MUST set ring-ref with a FreeBSD blkback, and we MUST NOT set it with a linux blkback -- but otherwise errs in the direction of setting more variables than are needed, to maximize the possibility of a future blkback being compatible with both blkback_is_linux=0 and blkback_is_linux=1. It would be better to just change the FreeBSD blkback driver to be compatible with the RedHat convention. I'm still unclear on why the current FreeBSD blkfront driver believes that it can use more than one page in your configuration given that the RedHat blkfront doesn't advertise this capability in a way that the FreeBSD blkfront understands (max-ring-pages isn'te set by blkback). Did you do something to force blkfront to use more than one page? -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: breakage in blkfront with ring_pages 1
On 7/5/11 7:14 PM, Colin Percival wrote: On 06/10/11 13:30, Justin T. Gibbs wrote: On 6/9/11 9:26 PM, Colin Percival wrote: Has anyone seen anything like this? Is it possible that there's a bug in how our blkfront negotiates the request ring? Does anyone have ring_pages 1 in use? The only backend driver I know of that can support more than one ring page is FreeBSD. So, the problem likely is that blkfront is getting the negotiation wrong and only 1 page is in use. Turns out that the linux backend in question really does support multiple pages. Can you provide a source or spec reference for this driver? There was a proposal (in 2009?) to add multiple page support to the XenSource provided blkfront/back, but it didn't get committed to the Xen repository. It also doesn't use the same XenStore variables that you list. Is this something Amazon developed in house? There's also an inconsistency about how multiple rings are negotiated: We set: * ring-pages to the number of pages blkfront wants to use * ring-ref to the reference for the first page * ring-refXX to the references for other pages while linux sets: * num-ring-pages to the number of pages blkfront wants to use * ring-refXX to the page references. You forgot to mention that the FreeBSD driver allows request size to be negotiated and explicitly supports chained requests to break the 44k request size limit imposed by the original interface. The 2009 proposal did not allow larger requests which is why I didn't follow it when I implemented the FreeBSD extension. Allowing larger requests both improves performance and removes the need to do costly split/coalesce operations on both sides of the blkif interface. It seems to be impossible to be compatible with both, since Linux interprets having a value set for ring-ref to indicate that the single-ring protocol is being used and doesn't check anything else. It's not Linux. There is no multi-page support in either the kernel.org hosted drivers or those in the Linux tree hosted in the official Xen repository. Is there any official source which defines the blkback/front protocol? Only the source code in the Xen repository. Both the FreeBSD extensions that I did and those for the Linux system you are using are likely backwards compatible to the original interface, but unfortunately are not compatible with each other. Maybe the right option is to have a loader tunable dev.xn.linuxback to control which version of the protocol is used? What a mess. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: breakage in blkfront with ring_pages 1
On 6/9/11 9:26 PM, Colin Percival wrote: Hi all, I'm seeing breakage with ring_pages 1 on a blkfront; more precisely, request #32 has a garbage response (resulting in a panic) exactly as if the dom0 thinks we're only using a single page for the ring. Forcing ring_pages to 1 makes the problem go away. Has anyone seen anything like this? Is it possible that there's a bug in how our blkfront negotiates the request ring? Does anyone have ring_pages 1 in use? The only backend driver I know of that can support more than one ring page is FreeBSD. So, the problem likely is that blkfront is getting the negotiation wrong and only 1 page is in use. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: [FreeBSD 8.2 amd64 XENHVM] DomU terrible network performance trought NAT
On 5/12/11 11:48 AM, Janne Snabb wrote: On Thu, 12 May 2011, Laurent Cligny wrote: All FreeBSD VM are 8.2 amd64 with XENHVM kernel anf the Linux VM is a Paravirtualized Debian amd64. My suggestion would be to try out the same setup with GENERIC kernel and the rtl driver (or even better e1000 if your Xen allows it) which is easy to do to make a simple comparison. In one of my recent benchmarks the FreeBSD Xen PV network driver performed very well in one direction, but very badly in the other. Using rtl or e1000 gave much better TCP throughput if you care equally about both directions. This benchmark was done only for internal TCP traffic between a Linux dom0 and FreeBSD dumU, the traffic never entered a real NIC. Do you recall which path was slow (rx or tx from the perspective of the FreeBSD driver) and what the relative difference in performance was between the two approaches? -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: xn0: Error 2 parsing device/vif/0/mac [PATCH]
On 1/15/2011 3:12 AM, Janne Snabb wrote: On Sat, 15 Jan 2011, Janne Snabb wrote: It appears that that the netfront driver fails to get the vif mac address which leads to panic shortly afterwards. The patch at the bottom of this message solves the problem for me. After that the current 8.2RC2 system works fine on amd64 with XENHVM kernel with Xen 4.0.1 (have not tested other versions). If the mac node does not appear in the front-end vif directory (does it ever appear there? in my experience no), we fetch a link to the backend directory for the same vif and try to get the mac node from there. I am not sure if this is the proper way to fix this, but it works for me. As with most things Xen, there is no official specification for what xenstore nodes are where. If we assume the Linux driver is the definitive reference, then the current FreeBSD driver behavior is correct: http://xenbits.xensource.com/linux-2.6.18-xen.hg?file/5e08fff8dc05/drivers/xen/netfront/netfront.c See talk_to_backend() and the xen_net_read_mac() function. There are two environments that setup the xenstore nodes: the Xen cloud platform, and the more common phython tool stack (aka xend). I haven't reviewed XCP, but Xend populates the MAC node in the frontend's tree unless the ioemu tag is in the VIF configuration line for that interface. Why this is the case, I have no idea - again the behavior is not documented. However, it seems reasonable to do what you've done in this patch and to rely on the backend's copy if the frontend's doesn't exist. It would also be good to add support to netfront to maintain the mac node in the frontend tree and to allow it to be set just like a real device. I believe netback will pick up and use the updated MAC value if you bounce the Xenbus connection after making the change. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/152818: [panic][xen] disk driver data cannot cross a page boundary
The following reply was made to PR kern/152818; it has been noted by GNATS. From: Justin T. Gibbs gi...@scsiguy.com To: Colin Percival cperc...@freebsd.org Cc: bug-follo...@freebsd.org Subject: Re: kern/152818: [panic][xen] disk driver data cannot cross a page boundary Date: Sat, 04 Dec 2010 15:49:11 -0700 On 12/4/2010 3:35 PM, Colin Percival wrote: It looks like this bug was fixed in busdma on amd64 by r204214 but never merged to i386. Justin, is there a technical reason why this wasn't merged, or should I try to do this myself? Oh. That bug. At the time I had difficulty getting an i386 image up and running to test. If you have a test system, feel free to merge the change. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Xen PV drivers for FreeBSD
On 12/1/2010 4:52 AM, Yuriy Kohut wrote: Hi, Now your commit in place and I have built XENHVM on RELENG_8. Some things are changed for me, but the issue still exists. I will try to replicate your issues again later today. One notable difference between my configuration and yours is that I am not passing through any cdrom devices. I have no idea how well block front deals with cdroms. My testing is on OpenSuSE 11.2 with Xen 4.0.0. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Xen PV drivers for FreeBSD
On 11/26/2010 8:56 AM, Yuriy Kohut wrote: Hi, I have just built XENHVM from RELANG_8_1. RELENG_8_1 doesn't have the latest Xen PV code in it. stable/8 (RELENG_8 if checking out of CVS instead of SVN?) does. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Xen PV drivers for FreeBSD
On 11/24/2010 10:03 AM, Yuriy Kohut wrote: Hi, It is for amd64 only, correct ? I have only tested it on amd64, but I don't know of anything inherent in the PV driver framework that would prevent it from operating on other architectures. Do you know is it stable enough ? It has been quite stable for me. There are still some issues with clock interrupt processing (time going backwards messages on the console at times), but nothing that brings down the system. BTW, my testing has been primarily on -current, but I just merged the latest Xen PV driver support into stable/8, so it should work fine there too. -- Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: XenTools development for FreeBSD
On 1/8/2010 1:23 AM, James Smith wrote: ... We want to use XenServer (iscsi SAN storage) by Citrix and FreeBSD. We've got a testing setup in place, so we can play around with the features. FreeBSD runs and installs quite well within this environment, along with Linux distros, Windows etc. There is something called XenTools (commercial software from Citrix), amongst other things its main job is to facilitate VM migration between nodes when using XenServer (it is awesome, no ping drops, at worst higher latency for two pings and then it's done). This runs on Linux ... but there is no FreeBSD version. Is this part of the XenServer-Essentials package? Can you point to a URL that describes XenTools in more detail? Thanks, Justin ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
