Re: Bug in XEN support with IPIs?
On 10/19/18 9:40 AM, McCarthy, Brian wrote: > it appears, that for FreeBSD XEN guests with SMP support, The IPIs get messed > up by intr_shuffle_irqs. The symptom is a smp_ipi_mtx > timeout during TLB shootdown. I've seen this mentioned in e-mails, with the > solution (without explanation) being to enable EARLY_AP_STARTUP. > The reason this works in BSD11 is that int_shuffle_irqs is conditionalixed > out on EARLY_AP_STARTUP. The real solution seems to be that > xi_cpu and/or ie_cpu should be set. I'm trying to figure out where that would > be (for our somewhat modified FreeBSD kernel). Which version are you looking at? We fixed a bug along these lines just about a month ago -- see r338624-338631. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: amazon/xen... any way at all to pass a message/signal/semaphoere/morse-code to the boot loader?
[CCing freebsd-cloud, which is the right place for discussions of FreeBSD/EC2] On 04/11/17 21:03, Julian Elischer wrote: > In Amazon ec2 they have no console access (though I heard rumors that it was > available I have not seen any sign of it) so I'd like to put a "recovery > partition" into an AMI. > The trick is how to convince it to boot to that instead of the regular action. Can you get what you want via gptboot's support for selecting the partition to boot via "bootonce" and "bootme" flags? > The ideal thing would be if there was way to 'influence' one of the smbios > values in some way, and have the boot code see it, but I'm open to any > suggestions. > I really need only 1 bit of information to get through. > > Possibilties include "changing the VM to have only 2G of ram" (we'd never do > that in a real machine). > or maybe temporarily removing all the disks other than the root drive? Almost > anything I could do to signal the boot code to behave differently. I don't think adding/removing disks will be useful, since the extra disks will be Xen blkfront devices; AFAIK the boot loader doesn't know anything about these. (The boot device is also a blkfront device but gets ATA emulation for the benefit of boot loaders.) Maybe you can repurpose some of the logic used for booting over NFS? I've never heard of people booting over NFS when the initial bootstrap comes from disk rather than PXE, but I assume it's possible...? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: svn commit: r310177 - in head/sys: amd64/conf i386/conf
This commit breaks the Xen boot: > panic: NULL pcpu device_t > cpuid = 0 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0x82233a20 > vpanic() at vpanic+0x186/frame 0x82233aa0 > kassert_panic() at kassert_panic+0x126/frame 0x82233b10 > xen_setup_cpus() at xen_setup_cpus+0x5b/frame 0x82233b50 > mi_startup() at mi_startup+0x118/frame 0x82233b70 > btext() at btext+0x2c I'm assuming this means xenpvcpu_attach isn't running early enough; does anyone have time to track this down and fix it? On 12/16/16 13:10, John Baldwin wrote: > Author: jhb > Date: Fri Dec 16 21:10:37 2016 > New Revision: 310177 > URL: https://svnweb.freebsd.org/changeset/base/310177 > > Log: > Enable EARLY_AP_STARTUP on amd64 and i386 kernels by default. > > PR: 199321, 203682 > MFC after: 2 months > Sponsored by: Netflix > > Modified: > head/sys/amd64/conf/GENERIC > head/sys/amd64/conf/MINIMAL > head/sys/i386/conf/GENERIC > > Modified: head/sys/amd64/conf/GENERIC > == > --- head/sys/amd64/conf/GENERIC Fri Dec 16 20:49:50 2016 > (r310176) > +++ head/sys/amd64/conf/GENERIC Fri Dec 16 21:10:37 2016 > (r310177) > @@ -98,6 +98,7 @@ options MALLOC_DEBUG_MAXZONES=8 # Separ > # Make an SMP-capable kernel by default > options SMP # Symmetric MultiProcessor Kernel > options DEVICE_NUMA # I/O Device Affinity > +options EARLY_AP_STARTUP > > # CPU frequency control > device cpufreq > > Modified: head/sys/amd64/conf/MINIMAL > == > --- head/sys/amd64/conf/MINIMAL Fri Dec 16 20:49:50 2016 > (r310176) > +++ head/sys/amd64/conf/MINIMAL Fri Dec 16 21:10:37 2016 > (r310177) > @@ -94,6 +94,7 @@ options MALLOC_DEBUG_MAXZONES=8 # Separ > # Make an SMP-capable kernel by default > options SMP # Symmetric MultiProcessor Kernel > options DEVICE_NUMA # I/O Device Affinity > +options EARLY_AP_STARTUP > > # CPU frequency control > device cpufreq > > Modified: head/sys/i386/conf/GENERIC > == > --- head/sys/i386/conf/GENERICFri Dec 16 20:49:50 2016 > (r310176) > +++ head/sys/i386/conf/GENERICFri Dec 16 21:10:37 2016 > (r310177) > @@ -96,6 +96,7 @@ options MALLOC_DEBUG_MAXZONES=8 # Separ > # To make an SMP kernel, the next two lines are needed > options SMP # Symmetric MultiProcessor Kernel > device apic# I/O APIC > +options EARLY_AP_STARTUP > > # CPU frequency control > device cpufreq > > > -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: xn0 systat no egress data?
On 10/31/16 14:57, Pete Wright wrote: > On 10/31/16 2:08 PM, Colin Percival wrote: >> On 10/31/16 13:44, Pete Wright wrote: >>> hey all - this is regarding the following bug i've opened: >>> >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213814 >>> >>> tl;dr version is that on 11.0-RELEASE on EC2 when i invoke "systat -ifstat" >>> the egress data flows for the xn0 device do not update. verified it works as >>> expected on 10.3-RELEASE. tried reading through code but am pretty confused >>> as to how xn0 is created and what code is used to update the sysctl MIBs. >> >> I think this might be fixed by r308126. Can you try applying that patch and >> report back? > > hrm doesn't look like it has changed anything on my end. > > i applied this patch against the 11.0.1 svn source: > https://svnweb.freebsd.org/base/head/sys/dev/xen/netfront/netfront.c?view=patch=308126=308125=308126 > > then built/installed a kernel after applying this patch with no change. I > don't think i need to rebuild world for this, right? Hmm, that should have worked. (Assuming you rebooted into the new kernel, of course.) I'm out of ideas... maybe someone on the -xen list will know more... -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: number of xen block devices
On 09/20/16 16:33, Julian Elischer wrote: > Is there a maximum number of block devices supported for an AMI type > installation? Does "AMI type installation" mean EC2? > Our tests a while back showed a limit of 14. > > Has that changed in 10.3? > > If not is it an artificial limit we can get past by simply recompiling > something? There was a bug a while back which resulted in high-numbered xbd devices not showing up; I fixed that three years ago in r255051, shortly before stable/10 branched. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
recent disk-related breakage
Some time in early December, disk I/O broke. The EC2 image built from r291495 works fine; the EC2 image built from r292413 is broken. Symptoms: 1. growfs reports "superblock not recognized" 2. fsck reports lots of "UNKNOWN FILE TYPE"s and after a few seconds provokes "panic: XEN disk driver data cannot cross a page boundary" xbd_mksegarray() at xbd_mksegarray+0x4b/frame 0xfe0f4dadb420 xbd_queue_cb() at xbd_queue_cb+0x1e8/frame 0xfe0f4dadb490 bus_dmamap_load_bio() at bus_dmamap_load_bio+0xad/frame 0xfe0f4dadb4f0 xbd_startio() at xbd_startio+0x194/frame 0xfe0f4dadb530 xbd_strategy() at xbd_strategy+0x6a/frame 0xfe0f4dadb560 g_disk_start() at g_disk_start+0x37c/frame 0xfe0f4dadb5d0 g_io_request() at g_io_request+0x39d/frame 0xfe0f4dadb630 g_part_start() at g_part_start+0x2b5/frame 0xfe0f4dadb6b0 g_io_request() at g_io_request+0x39d/frame 0xfe0f4dadb710 g_io_request() at g_io_request+0x39d/frame 0xfe0f4dadb770 g_dev_strategy() at g_dev_strategy+0x171/frame 0xfe0f4dadb7b0 physio() at physio+0x440/frame 0xfe0f4dadb850 devfs_read_f() at devfs_read_f+0xe7/frame 0xfe0f4dadb8b0 dofileread() at dofileread+0x98/frame 0xfe0f4dadb900 kern_readv() at kern_readv+0x68/frame 0xfe0f4dadb950 sys_read() at sys_read+0x60/frame 0xfe0f4dadb9a0 Does anyone remember touching any relevant bits of code in that timeframe? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: recent disk-related breakage
I'm rather suspicious of r292255 here -- royger, hps, kib, can any of you comment on whether it would be responsible for making non-page-aligned I/Os no longer get split on page boundaries? The commit message is suggestive, but I don't know the code very well. (If I don't hear back I'll determine if it's responsible experimentally, but testing a "system no longer boots" bug in EC2 is painfully slow and it's getting late here.) Colin Percival On 01/10/16 22:28, Colin Percival wrote: > Some time in early December, disk I/O broke. The EC2 image built from r291495 > works fine; the EC2 image built from r292413 is broken. > > Symptoms: > 1. growfs reports "superblock not recognized" > > 2. fsck reports lots of "UNKNOWN FILE TYPE"s and after a few seconds provokes > "panic: XEN disk driver data cannot cross a page boundary" > xbd_mksegarray() at xbd_mksegarray+0x4b/frame 0xfe0f4dadb420 > > xbd_queue_cb() at xbd_queue_cb+0x1e8/frame 0xfe0f4dadb490 > > bus_dmamap_load_bio() at bus_dmamap_load_bio+0xad/frame 0xfe0f4dadb4f0 > > xbd_startio() at xbd_startio+0x194/frame 0xfe0f4dadb530 > > xbd_strategy() at xbd_strategy+0x6a/frame 0xfe0f4dadb560 > > g_disk_start() at g_disk_start+0x37c/frame 0xfe0f4dadb5d0 > > g_io_request() at g_io_request+0x39d/frame 0xfe0f4dadb630 > > g_part_start() at g_part_start+0x2b5/frame 0xfe0f4dadb6b0 > > g_io_request() at g_io_request+0x39d/frame 0xfe0f4dadb710 > > g_io_request() at g_io_request+0x39d/frame 0xfe0f4dadb770 > > g_dev_strategy() at g_dev_strategy+0x171/frame 0xfe0f4dadb7b0 > > physio() at physio+0x440/frame 0xfe0f4dadb850 > > devfs_read_f() at devfs_read_f+0xe7/frame 0xfe0f4dadb8b0 > > dofileread() at dofileread+0x98/frame 0xfe0f4dadb900 > > kern_readv() at kern_readv+0x68/frame 0xfe0f4dadb950 > > sys_read() at sys_read+0x60/frame 0xfe0f4dadb9a0 > > > Does anyone remember touching any relevant bits of code in that timeframe? > -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: Networking under Xen
On 07/26/15 06:16, Kristof Provost wrote: On 2015-07-15 00:54:25 (-0700), Colin Percival cperc...@freebsd.org wrote: In my tests, deleting these lines from pf_ioctl.c 3570 /* We need a proper CSUM befor we start (s. OpenBSD ip_output) */ [...] unbreaks pf+TSO on EC2 instances. I'm not entirely sure why these lines are there in the first place, which is why I didn't want to simply go in and remove them -- but it may be that wrapping those lines in something like if ((csum_flags CSUM_TSO) == 0) would solve the problem without breaking anything else. I think the reason for this checksum calculation is that pf sometimes modifies the packet, so it also updates the checksum. Aha, this is exactly the sort of thing I was worried about. I'm glad you understand this stuff better than me. It doesn't work on Xen TSO interfaces because (I assume) it expects to get the pseudo header checksum, not the full checksum. It's not entirely clear to my why it's not broken on my hardware (which claims TSO support), but perhaps Xen is more picky than actual hardware. I'm not 100% certain about this, but I don't think Xen is doing anything with the checksum; rather, everything is being passed through to the underlying hardware, and some NICs are pickier than others. I'll refrain from commenting on your plans for fixing this since, as I mentioned above, you understand how pf and the network stack work far better than I do. :-) -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Networking under Xen
On 07/14/15 09:15, Mark Felder wrote: On Tue, Jul 14, 2015, at 07:36, Sydney Meyer wrote: - Dropped/Stalled Connections with TCP Segmentation Offload and pf enabled. TSO is a known issue. I've been turning it off for years to get FreeBSD to play nice on Xen. More precisely *pf* is a known issue. It's inserting TCP checksums into TSO packets. I reported this a long time ago and I don't know why pf still hasn't been fixed. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Fwd: New list: freebsd-cl...@freebsd.org
The freebsd-xen list has been the de facto forum for discussing FreeBSD/EC2 for many years; but we have a new list, so I encourage everybody interested in using FreeBSD on clouds (as opposed to, say, writing Xen kernel code) to migrate over to the freebsd-cloud list. Colin Percival Forwarded Message Subject: New list: freebsd-cl...@freebsd.org Date: Fri, 21 Nov 2014 20:22:42 -0800 From: Colin Percival cperc...@freebsd.org To: freebsd-cl...@freebsd.org This is a new mailing list created for discussion of using FreeBSD on cloud platforms like Amazon EC2, Google Compute Engine, and Microsoft Azure. Discussion of both creating and using FreeBSD images on these platforms is welcome here -- I'm hoping we'll have both, so that creators of such images can learn lessons from people building images for other clouds, and so that we can all learn from the experiences (and complaints) of the people using these images. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-cl...@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-cloud To unsubscribe, send any mail to freebsd-cloud-unsubscr...@freebsd.org ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Xen/HVM support in stable/9: GENERIC + xenhvm.ko
Hi all, As of r266269, stable/9 can run in Xen/HVM environments using the GENERIC kernel configuration plus a new xenhvm.ko. This will allow FreeBSD 9.3 to run in Xen using official release binaries as long as xenhvm_load=YES is placed in /boot/loader.conf. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Is ixgbe usable with SR-IOV?
On 12/05/13 00:18, Jack Vogel wrote: Unfortunately its a feature that was implemented, and worked, but it has not had a lot of use I suspect, and it has been in a limited number of setups. I know nothing about this Amazon provided hosting, is it linux/kvm or xen? I can see if we can get something set up and check it here. This is Xen, running in this case using HVM: http://aws.amazon.com/ec2/ -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Is ixgbe usable with SR-IOV?
Hi Jack, On 12/04/13 10:21, Jack Vogel wrote: Yes, the ixv code was specifically written to work in a guest with SRIOV, however our internal testing did not use DHCP, what if you assign it an address, will it work then? Hard-coding an IP address doesn't work either. Or rather, I can assign it an address, but it continues to not pass packets. Are there any diagnostic tunables/sysctls I should try turning on? Since this is in EC2 I'm unfortunately limited to printf-style debugging. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Is ixgbe usable with SR-IOV?
Hi Xen people jfv,
Amazon's new C3 instance type has support for SR-IOV using ixgbe hardware;
our driver looks like it should support this, but when I turn it on it doesn't
seem to work. Boot dmesg: http://pastebin.com/1Kkpfzbi
The virtual hardware is recognized:
ix0: Intel(R) PRO/10GbE Virtual Function Network Driver, Version - 1.1.4
mem 0xf300-0xf3003fff,0xf3004000-0xf3007fff at device 3.0 on pci0
ix0: Using MSIX interrupts with 2 vectors
ix0: Ethernet address: 02:91:23:76:38:77
And we seem to be able to send packets, but never see any responses:
ix0: link state changed to UP
Starting Network: lo0 ix0.
lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384
options=63RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff00
nd6 options=21PERFORMNUD,AUTO_LINKLOCAL
ix0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
options=401bbRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO
ether 02:91:23:76:38:77
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL
media: Ethernet autoselect
status: active
Starting devd.
Starting dhclient.
DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 8
DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 12
DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 12
DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 9
DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 10
DHCPDISCOVER on ix0 to 255.255.255.255 port 67 interval 10
No DHCPOFFERS received.
Under exactly the same Xen configuration except with SR-IOV turned off and
EC2 presenting a Xen netback to us instead, the DHCP works just fine.
Has anyone managed to use {FreeBSD, SR-IOV, ixgbe, Xen}? Was any magic
necessary in order to make it work?
--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Please test the new Xen code in EC2
Hi all, Earlier today I published EC2 images for 10.0-ALPHA4, which contains all of the latest FreeBSD/Xen code, including the merge from XENHVM into GENERIC. If you're ever going to test things before the release, now's the time. :-) AMIs are listed at http://www.daemonology.net/freebsd-on-ec2/ -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Current problem reports assigned to freebsd-xen@FreeBSD.org
On 09/03/13 07:38, Adam McDougall wrote: o kern/154428 xen[xen] xn0 network interface and PF - Massive performan I had similar issues, some testing I did last week makes me satisfied with r252781 in 9-stable (MFC of r291296, r291297, r291393) but I can't speak for everyone. f kern/135178 xen[xen] Xen domU outgoing data transfer stall when TSO i Looks like feedback timeout. I'd say that generally any PRs talking about performance problems or packet loss with TSO turned on can be closed. Also any PRs talking about FreeBSD/i386 Xen/PV can probably be closed with yes, that code is horribly broken and needs a complete rewrite, don't even try to use it for now. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: [CFR] Event channel Interrupts and unified Xen interrupt infrastructure.
On 08/29/13 09:32, Justin T. Gibbs wrote: I've been working to get the next chunk of Spectra/Roger Pau Monné Xen work into head. The latest version of the patch I'm working on can be found here: http://people.freebsd.org/~gibbs/xen_intr.diff I will continue my testing today and commit it tonight unless I hear complaints. Comments, corrections, improvements, and bug reports welcome. Could this patch be split into functional and non-functional components? It's distracting having everything mixed up together in a single megapatch. Even committing the s/unlikely/__predict_false/ separately would help. How do you intend to have Xen HVM work in the GENERIC kernel configuration? Will you be adding 'options XENHVM' to GENERIC? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
CFT: replacing XENHVM kernel config with GENERIC + xenhvm.ko
Hi all,
I've attached a patch which eliminates the XENHVM kernel configuration and
instead allows FreeBSD to run under Xen/HVM with PV drivers by loading a
new xenhvm.ko module from the boot loader. This will mean that FreeBSD
virtual machines running under Xen/HVM will be able to run straight off
the ISO binaries; this will also mean they will be able to use FreeBSD
Update to update the kernel.
I have spent about 10 minutes testing this in Amazon EC2. Please help me
out by doing some more testing. ;-)
The fine details:
1. I've created a new kernel module for i386 and amd64, xenhvm, with all
of the source files which were added via options XENHVM and device xenpci.
2. I have removed those from sys/conf/files and sys/conf/options.*; those
options are now meaningless.
3. I moved the detect Xen and disable QEMU emulated devices code from
sys/amd64/amd64/machdep.c to the kernel module MOD_LOAD event handler.
4. I have made the PCPU values required by Xen/HVM support -- two unsigned
ints -- unconditionally compiled in. This is the only change to the GENERIC
kernel.
5. I have removed the XENHVM kernel configuration files.
Depending on feedback from freebsd-xen@ I hope to send this to freebsd-current
for wider review later this week and then commit it before the FreeBSD 10.0
code freeze starts on September 7th.
--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
Index: sys/amd64/amd64/machdep.c
===
--- sys/amd64/amd64/machdep.c (revision 254941)
+++ sys/amd64/amd64/machdep.c (working copy)
@@ -1910,14 +1910,6 @@
if (env != NULL)
strlcpy(kernelname, env, sizeof(kernelname));
-#ifdef XENHVM
- if (inw(0x10) == 0x49d2) {
- if (bootverbose)
- printf(Xen detected: disabling emulated block and
network devices\n);
- outw(0x10, 3);
- }
-#endif
-
cpu_probe_amdc1e();
#ifdef FDT
Index: sys/amd64/conf/XENHVM
===
--- sys/amd64/conf/XENHVM (revision 254941)
+++ sys/amd64/conf/XENHVM (working copy)
@@ -1,22 +0,0 @@
-#
-# XENHVM -- Xen HVM kernel configuration file for FreeBSD/amd64
-#
-# $FreeBSD$
-#
-includeGENERIC
-ident XENHVM
-
-#
-# Adaptive locks rely on a lock-free pointer read to determine the run state
-# of the thread holding a lock when under contention; under a virtualisation
-# system, the thread run state may not accurately reflect whether the thread
-# (or rather its host VCPU) is actually executing. As such, disable this
-# optimisation.
-#
-optionsNO_ADAPTIVE_MUTEXES
-optionsNO_ADAPTIVE_RWLOCKS
-optionsNO_ADAPTIVE_SX
-
-# Xen HVM support
-optionsXENHVM
-device xenpci
Index: sys/amd64/include/pcpu.h
===
--- sys/amd64/include/pcpu.h(revision 254941)
+++ sys/amd64/include/pcpu.h(working copy)
@@ -42,14 +42,10 @@
#endif
#endif
-#ifdef XENHVM
#define PCPU_XEN_FIELDS
\
; \
unsigned int pc_last_processed_l1i; \
unsigned int pc_last_processed_l2i
-#else
-#define PCPU_XEN_FIELDS
-#endif
/*
* The SMP parts are setup in pmap.c and locore.s for the BSP, and
Index: sys/conf/files
===
--- sys/conf/files (revision 254941)
+++ sys/conf/files (working copy)
@@ -2490,16 +2490,14 @@
clean wpi.fw
dev/xe/if_xe.c optional xe
dev/xe/if_xe_pccard.c optional xe pccard
-dev/xen/balloon/balloon.c optional xen | xenhvm
-dev/xen/blkfront/blkfront.coptional xen | xenhvm
-dev/xen/blkback/blkback.c optional xen | xenhvm
+dev/xen/balloon/balloon.c optional xen
+dev/xen/blkfront/blkfront.coptional xen
+dev/xen/blkback/blkback.c optional xen
dev/xen/console/console.c optional xen
dev/xen/console/xencons_ring.c optional xen
-dev/xen/control/control.c optional xen | xenhvm
-dev/xen/netback/netback.c optional xen | xenhvm
-dev/xen/netfront/netfront.coptional xen | xenhvm
-dev/xen/xenpci/xenpci.coptional xenpci
-dev/xen/xenpci/evtchn.c optional xenpci
+dev/xen/control/control.c optional xen
+dev/xen/netback/netback.c optional xen
+dev/xen/netfront/netfront.coptional xen
dev/xl/if_xl.c optional xl pci
dev/xl/xlphy.c optional xl pci
fs/deadfs/dead_vnops.c standard
@@ -3813,18 +3811,18 @@
vm/vm_unix.c standard
vm/vm_zeroidle.c standard
vm/vnode_pager.c standard
-xen/gnttab.c optional xen
Re: svn commit: r251297 - head/sys/dev/xen/netfront
On 06/03/13 06:00, Andre Oppermann wrote: Log: Specify a maximum TSO length limiting the segment chain to what the Xen host side can handle after defragmentation. This prevents the driver from throwing away too long TSO chains and improves the performance on Amazon AWS instances with 10GigE virtual interfaces to the normally expected throughput. Thanks! For the benefit of the list: This (and the r251296, which provides the network stack infrastructure which this commit uses) was the last of the make FreeBSD behave well in Amazon EC2 patches I've been carrying around (and building into my EC2 images) for the past two years. Now the only step remaining to make FreeBSD/EC2 pure straight off the RELEASE ISO FreeBSD is to get Xen HVM functionality into GENERIC. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: [Xen-devel] FreeBSD PVHVM call for testing
On 05/28/13 12:18, Matt Wilson wrote: VCPUOP_set_singleshot_timer returns -EINVAL when: 1) the specified vCPU ID is out of range (0 or MAX_VIRT_CPUS) 2) the specified vCPU ID doesn't match the running vCPU. It seems that there is a confusion between the logical vCPU ID and the local APIC physical ID. [...] (XEN) Domain 1 (vcpu#16) VCPUOP_set_singleshot_timer specified vcpuid 1 [...] APIC: CPU 1 has ACPI ID 16 Thanks Matt! Looks like we need to pass our acpi_id to the Xen hypercall instead of our cpuid. Roger, changing the line int cpu = PCPU_GET(cpuid); to int cpu = PCPU_GET(acpi_id); in xentimer_et_start and xentimer_et_stop fixes this panic and gets me slightly further; the following lines are now added to the console output prior to the system appearing to hang: ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 1 vector 48 ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 2 vector 48 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 3 vector 48 ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 4 vector 48 ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 5 vector 48 ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 6 vector 48 ioapic0: routing intpin 28 (PCI IRQ 28) to lapic 7 vector 48 TSC timecounter discards lower 1 bit(s) Timecounter TSC-low frequency 1300024860 Hz quality -100 WARNING: WITNESS option enabled, expect reduced performance. On a cc2.8xlarge EC2 instance, the lines which come after this are GEOM: new disk xbd1 GEOM: new disk xbd2 GEOM: new disk xbd3 GEOM: new disk xbd4 Trying to mount root from ufs:/dev/ad0a [rw]... start_init: trying /sbin/init and then the userland boot process; have you made any bug fixes after your pvhvm_v7 which would explain why tasting disks was hanging? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: tcp offloading
On 05/23/13 07:02, Jay West wrote: Environment is Citrix Xenserver Advanced, with (mostly) freebsd 9.1 64bit HVM with PV drivers. I've heard much mention around the net that in pretty much all cases, TCP offloading should be disabled inside the VM. I have also seen other posts that TCP offloading should be disabled on the Dom0. So my question is; for best performance should TCP offloading be disabled in the VM, or the Xenserver (dom0), or should it be disabled in BOTH? There's an incompatibility between FreeBSD's TCP stack, TSO, and the Linux Xen netback driver: With TSO enabled on netfront, we can generate longer mbuf chains than the Linux netback is able to handle. I have an ugly patch for this on EC2, but andre will be fixing this better in the near future. For now I recommend # ifconfig xn0 -tso unless you need multi-Gbps networking performance. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
On 05/22/13 04:45, Roger Pau Monné wrote: On 18/05/13 17:44, Colin Percival wrote: That seems to work. dmesg is attached. Are there any particular tests you'd like me to run? I have not tested ZFS, that might be a good one. If you are running this on Xen 3.4 the behaviour should be the same as without this patches, so there shouldn't be many differences. I don't use ZFS personally, so I'm not sure exactly what tests to run on it; hopefully someone else can take care of that. If you could try that on Xen 4.0 at least (if I remember correctly that's when the vector callback was introduced), you should see the PV timer getting attached, and a performance increase. Testing on a cr1.8xlarge EC2 instance, I get Xen 4.2, but it ends up with a panic -- console output below. I can get a backtrace and possibly even a dump if those would help. Booting [/boot/kernel/kernel]... -\|/-\|GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb SMAP type=01 base= len=0009e000 SMAP type=02 base=0009e000 len=2000 SMAP type=02 base=000e len=0002 SMAP type=01 base=0010 len=eff0 SMAP type=02 base=fc00 len=0400 SMAP type=01 base=0001 len=003c1900 Table 'FACP' at 0xfc014980 Table 'APIC' at 0xfc014a80 APIC: Found table at 0xfc014a80 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 2 ACPI ID 1: enabled SMP: Added CPU 2 (AP) MADT: Found CPU APIC ID 4 ACPI ID 2: enabled SMP: Added CPU 4 (AP) MADT: Found CPU APIC ID 6 ACPI ID 3: enabled SMP: Added CPU 6 (AP) MADT: Found CPU APIC ID 8 ACPI ID 4: enabled SMP: Added CPU 8 (AP) MADT: Found CPU APIC ID 10 ACPI ID 5: enabled SMP: Added CPU 10 (AP) MADT: Found CPU APIC ID 12 ACPI ID 6: enabled SMP: Added CPU 12 (AP) MADT: Found CPU APIC ID 14 ACPI ID 7: enabled SMP: Added CPU 14 (AP) MADT: Found CPU APIC ID 32 ACPI ID 8: enabled SMP: Added CPU 32 (AP) MADT: Found CPU APIC ID 34 ACPI ID 9: enabled SMP: Added CPU 34 (AP) MADT: Found CPU APIC ID 36 ACPI ID 10: enabled SMP: Added CPU 36 (AP) MADT: Found CPU APIC ID 38 ACPI ID 11: enabled SMP: Added CPU 38 (AP) MADT: Found CPU APIC ID 40 ACPI ID 12: enabled SMP: Added CPU 40 (AP) MADT: Found CPU APIC ID 42 ACPI ID 13: enabled SMP: Added CPU 42 (AP) MADT: Found CPU APIC ID 44 ACPI ID 14: enabled SMP: Added CPU 44 (AP) MADT: Found CPU APIC ID 46 ACPI ID 15: enabled SMP: Added CPU 46 (AP) MADT: Found CPU APIC ID 1 ACPI ID 16: enabled SMP: Added CPU 1 (AP) MADT: Found CPU APIC ID 3 ACPI ID 17: enabled SMP: Added CPU 3 (AP) MADT: Found CPU APIC ID 5 ACPI ID 18: enabled SMP: Added CPU 5 (AP) MADT: Found CPU APIC ID 7 ACPI ID 19: enabled SMP: Added CPU 7 (AP) MADT: Found CPU APIC ID 9 ACPI ID 20: enabled SMP: Added CPU 9 (AP) MADT: Found CPU APIC ID 11 ACPI ID 21: enabled SMP: Added CPU 11 (AP) MADT: Found CPU APIC ID 13 ACPI ID 22: enabled SMP: Added CPU 13 (AP) MADT: Found CPU APIC ID 15 ACPI ID 23: enabled SMP: Added CPU 15 (AP) MADT: Found CPU APIC ID 33 ACPI ID 24: enabled SMP: Added CPU 33 (AP) MADT: Found CPU APIC ID 35 ACPI ID 25: enabled SMP: Added CPU 35 (AP) MADT: Found CPU APIC ID 37 ACPI ID 26: enabled SMP: Added CPU 37 (AP) MADT: Found CPU APIC ID 39 ACPI ID 27: enabled SMP: Added CPU 39 (AP) MADT: Found CPU APIC ID 41 ACPI ID 28: enabled SMP: Added CPU 41 (AP) MADT: Found CPU APIC ID 43 ACPI ID 29: enabled SMP: Added CPU 43 (AP) MADT: Found CPU APIC ID 45 ACPI ID 30: enabled SMP: Added CPU 45 (AP) MADT: Found CPU APIC ID 47 ACPI ID 31: enabled SMP: Added CPU 47 (AP) MADT: Found CPU APIC ID 0 ACPI ID 32: disabled MADT: Found CPU APIC ID 0 ACPI ID 33: disabled MADT: Found CPU APIC ID 0 ACPI ID 34: disabled MADT: Found CPU APIC ID 0 ACPI ID 35: disabled MADT: Found CPU APIC ID 0 ACPI ID 36: disabled MADT: Found CPU APIC ID 0 ACPI ID 37: disabled MADT: Found CPU APIC ID 0 ACPI ID 38: disabled MADT: Found CPU APIC ID 0 ACPI ID 39: disabled MADT: Found CPU APIC ID 0 ACPI ID 40: disabled MADT: Found CPU APIC ID 0 ACPI ID 41: disabled MADT: Found CPU APIC ID 0 ACPI ID 42: disabled MADT: Found CPU APIC ID 0 ACPI ID 43: disabled MADT: Found CPU APIC ID 0 ACPI ID 44: disabled MADT: Found CPU APIC ID 0 ACPI ID 45: disabled MADT: Found CPU APIC ID 0 ACPI ID 46: disabled MADT: Found CPU APIC ID 0 ACPI ID 47: disabled MADT: Found CPU APIC ID 0 ACPI ID 48: disabled MADT: Found CPU APIC ID 0 ACPI ID 49: disabled MADT: Found CPU APIC ID 0 ACPI ID 50: disabled MADT: Found CPU APIC ID 0 ACPI ID 51: disabled MADT: Found CPU APIC ID 0 ACPI ID 52: disabled MADT: Found CPU APIC ID 0 ACPI ID 53: disabled MADT: Found CPU APIC ID 0 ACPI ID 54: disabled MADT: Found CPU APIC ID 0 ACPI ID 55: disabled MADT: Found CPU APIC ID 0 ACPI ID 56: disabled MADT: Found CPU APIC ID 0 ACPI ID 57: disabled MADT: Found CPU APIC ID 0 ACPI ID 58: disabled MADT: Found CPU APIC ID 0 ACPI ID 59: disabled MADT: Found CPU APIC ID 0 ACPI
Re: FreeBSD PVHVM call for testing
On 05/18/13 02:50, Roger Pau Monné wrote: On 17/05/13 05:07, Colin Percival wrote: On 05/16/13 17:43, Roger Pau Monné wrote: Thanks for testing this on EC2, could you post the full dmesg? So I can see the hypervisor version and if the PV timer is loaded or not. Here's what I get on a cc2.8xlarge with boot_verbose=YES: I've pushed a new branch to my repository, pvhvm_v7 that should work, there was a bug with PCI event channel interrupt set up. I've tested with 3.4 and seems OK, but of course it doesn't support the vector callback injection. That seems to work. dmesg is attached. Are there any particular tests you'd like me to run? If anyone else wants to play with this, you can launch ami-e75c358e in the EC2 us-east-1 region. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid Table 'FACP' at 0xfc005ee0 Table 'APIC' at 0xfc005fe0 APIC: Found table at 0xfc005fe0 APIC: Using the MADT enumerator. MADT: Found CPU APIC ID 0 ACPI ID 0: enabled SMP: Added CPU 0 (AP) MADT: Found CPU APIC ID 2 ACPI ID 1: enabled SMP: Added CPU 2 (AP) MADT: Found CPU APIC ID 4 ACPI ID 2: enabled SMP: Added CPU 4 (AP) MADT: Found CPU APIC ID 6 ACPI ID 3: enabled SMP: Added CPU 6 (AP) MADT: Found CPU APIC ID 8 ACPI ID 4: enabled SMP: Added CPU 8 (AP) MADT: Found CPU APIC ID 10 ACPI ID 5: enabled SMP: Added CPU 10 (AP) MADT: Found CPU APIC ID 12 ACPI ID 6: enabled SMP: Added CPU 12 (AP) MADT: Found CPU APIC ID 14 ACPI ID 7: enabled SMP: Added CPU 14 (AP) MADT: Found CPU APIC ID 32 ACPI ID 8: enabled SMP: Added CPU 32 (AP) MADT: Found CPU APIC ID 34 ACPI ID 9: enabled SMP: Added CPU 34 (AP) MADT: Found CPU APIC ID 36 ACPI ID 10: enabled SMP: Added CPU 36 (AP) MADT: Found CPU APIC ID 38 ACPI ID 11: enabled SMP: Added CPU 38 (AP) MADT: Found CPU APIC ID 40 ACPI ID 12: enabled SMP: Added CPU 40 (AP) MADT: Found CPU APIC ID 42 ACPI ID 13: enabled SMP: Added CPU 42 (AP) MADT: Found CPU APIC ID 44 ACPI ID 14: enabled SMP: Added CPU 44 (AP) MADT: Found CPU APIC ID 46 ACPI ID 15: enabled SMP: Added CPU 46 (AP) MADT: Found CPU APIC ID 1 ACPI ID 16: enabled SMP: Added CPU 1 (AP) MADT: Found CPU APIC ID 3 ACPI ID 17: enabled SMP: Added CPU 3 (AP) MADT: Found CPU APIC ID 5 ACPI ID 18: enabled SMP: Added CPU 5 (AP) MADT: Found CPU APIC ID 7 ACPI ID 19: enabled SMP: Added CPU 7 (AP) MADT: Found CPU APIC ID 9 ACPI ID 20: enabled SMP: Added CPU 9 (AP) MADT: Found CPU APIC ID 11 ACPI ID 21: enabled SMP: Added CPU 11 (AP) MADT: Found CPU APIC ID 13 ACPI ID 22: enabled SMP: Added CPU 13 (AP) MADT: Found CPU APIC ID 15 ACPI ID 23: enabled SMP: Added CPU 15 (AP) MADT: Found CPU APIC ID 33 ACPI ID 24: enabled SMP: Added CPU 33 (AP) MADT: Found CPU APIC ID 35 ACPI ID 25: enabled SMP: Added CPU 35 (AP) MADT: Found CPU APIC ID 37 ACPI ID 26: enabled SMP: Added CPU 37 (AP) MADT: Found CPU APIC ID 39 ACPI ID 27: enabled SMP: Added CPU 39 (AP) MADT: Found CPU APIC ID 41 ACPI ID 28: enabled SMP: Added CPU 41 (AP) MADT: Found CPU APIC ID 43 ACPI ID 29: enabled SMP: Added CPU 43 (AP) MADT: Found CPU APIC ID 45 ACPI ID 30: enabled SMP: Added CPU 45 (AP) MADT: Found CPU APIC ID 47 ACPI ID 31: enabled SMP: Added CPU 47 (AP) Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #0 r+9b25356: Sat May 18 14:46:16 UTC 2013 root@ip-10-140-132-115:/usr/obj/usr/src/sys/XENHVM amd64 FreeBSD clang version 3.3 (trunk 178860) 20130405 WARNING: WITNESS option enabled, expect reduced performance. XEN: Hypervisor version 3.4 detected. XEN: Disabling emulated block and network devices Preloaded elf kernel /boot/kernel/kernel at 0x81912000. Hypervisor: Origin = XenVMMXenVMM Calibrating TSC clock ... TSC clock: 2593802768 Hz CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2593.80-MHz K8-class CPU) Origin = GenuineIntel Id = 0x206d6 Family = 0x6 Model = 0x2d Stepping = 6 Features=0x1781fbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x9c982201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,XSAVE,OSXSAVE,AVX,HV AMD Features=0x20100800SYSCALL,NX,LM AMD Features2=0x1LAHF real memory = 65011712000 (62000 MB) Physical memory chunk(s): 0x1000 - 0x0009bfff, 634880 bytes (155 pages) 0x0010 - 0x001f, 1048576 bytes (256 pages) 0x01972000 - 0xbfff, 3194544128 bytes (779918 pages) 0x0001 - 0x000efeb95fff, 60108136448 bytes (14674838 pages) avail memory = 60563271680 (57757 MB) Event timer LAPIC quality 400 ACPI APIC Table: Xen HVM INTR: Adding local APIC 1 as a target INTR: Adding local APIC 2 as a target INTR: Adding local APIC 3 as a target INTR: Adding local APIC 4 as a target INTR: Adding local APIC 5 as a target INTR
Re: FreeBSD PVHVM call for testing
) for rid 0 of orm0 pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0 pcib0: allocated type 3 (0xbf800-0xb) for rid 0 of orm0 ahc_isa_identify 0: ioport 0xc00 alloc failed ahc_isa_identify 1: ioport 0x1c00 alloc failed ahc_isa_identify 2: ioport 0x2c00 alloc failed ahc_isa_identify 3: ioport 0x3c00 alloc failed ahc_isa_identify 4: ioport 0x4c00 alloc failed ahc_isa_identify 5: ioport 0x5c00 alloc failed ahc_isa_identify 6: ioport 0x6c00 alloc failed ahc_isa_identify 7: ioport 0x7c00 alloc failed ahc_isa_identify 8: ioport 0x8c00 alloc failed ahc_isa_identify 9: ioport 0x9c00 alloc failed ahc_isa_identify 10: ioport 0xac00 alloc failed ahc_isa_identify 11: ioport 0xbc00 alloc failed ahc_isa_identify 12: ioport 0xcc00 alloc failed ahc_isa_identify 13: ioport 0xdc00 alloc failed ahc_isa_identify 14: ioport 0xec00 alloc failed isa_probe_children: disabling PnP devices atkbdc: atkbdc0 already exists; skipping it atrtc: atrtc0 already exists; skipping it attimer: attimer0 already exists; skipping it sc: sc0 already exists; skipping it uart: uart0 already exists; skipping it isa_probe_children: probing non-PnP devices sc0: System console at flags 0x100 on isa0 sc0: VGA 16 virtual consoles, flags=0x100 sc0: fb0, kbd1, terminal emulator: scteken (teken terminal) vga0: Generic ISA VGA at port 0x3c0-0x3df iomem 0xa-0xb on isa0 pcib0: allocated type 4 (0x3c0-0x3df) for rid 0 of vga0 pcib0: allocated type 3 (0xa-0xb) for rid 0 of vga0 fdc0: No FDOUT register! fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0 ppc0: cannot reserve I/O port range ppc0 failed to probe at irq 7 on isa0 pcib0: allocated type 4 (0x2f8-0x2ff) for rid 0 of uart1 uart1 failed to probe at port 0x2f8-0x2ff irq 3 on isa0 wbwd0 failed to probe on isa0 isa_probe_children: probing PnP devices Device configuration finished. procfs registered lapic: Divisor 2, Frequency 5453 Hz Timecounters tick every 10.000 msec vlan: initialized, using hash tables with chaining tcp_init: net.inet.tcp.tcbhashsize auto tuned to 524288 lo0: bpf attached hptrr: no controller detected. hpt27xx: no controller detected. xenbusb_front0: Xen Frontend Devices on xenstore0 ata0: reset tp1 mask=03 ostat0=00 ostat1=00 ata0: stat0=0x00 err=0x00 lsb=0x00 msb=0x00 ata0: stat1=0x00 err=0x00 lsb=0x00 msb=0x00 ata0: reset tp2 stat0=00 stat1=00 devices=0x0 ata1: reset tp1 mask=03 ostat0=00 ostat1=00 ata1: stat0=0x00 err=0x00 lsb=0x00 msb=0x00 ata1: stat1=0x00 err=0x00 lsb=0x00 msb=0x00 ata1: reset tp2 stat0=00 stat1=00 devices=0x0 -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
On 05/13/13 12:08, Michael Sierchio wrote: The Windoze tax is unacceptable for a number of reasons - the primary reason is that I'm not running Windows. I don't think the licensing scheme is unfair for those actually running Windows, mind you. Right, it's definitely annoying having to pay more -- I just wanted to point out that the ability does exist, if you're willing to pay the price. At the AWS Summit, an assertion was made that HVM support might be coming soon for all instance types for *NIX OSes. I hope that's true. Was it indeed? I must not have been present for that... it certainly would be good news. Certainly all the new instance types they've released in the past few years have had UNIX HVM support. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: [Xen-users] Trouble booting FreeBSD i386 PV DomU
On 03/26/13 03:10, Roger Pau Monné wrote: On 26/03/13 10:38, Colin Percival wrote: On 03/26/13 02:31, Roger Pau Monné wrote: Is Xen i386 PV broken? Not completely broken, but it's certainly not in a good state. I believe it's broken with SMP, for example -- if the crashed on cpu#7 in your output means cpu#7 from the guest, it would certainly explain things. My guest only has one vcpu (vcpu#0): Ok, I wasn't sure how to parse that output. HVM is the way to go with FreeBSD/Xen. Yes, I'm already working on that, and got vector callbacks working on both i386 and amd64 HVM guests, thanks to Justin T. Gibbs patch. Now I was trying to boot a PV guest to see how much breakage this change introduced to PV, but I'm not able to make it work, even without my patches. I've replied to this xen-users thread because the author seem to have a working FreeBSD DomU PV guest, and I was wondering how he did it. From my POV it seems like PV guests hasn't been working for a long time, since Xen 3.3 dropped support for non-PAE guests, and the FreeBSD kernel is detected as non-PAE. I had FreeBSD 8.2-RELEASE and a 9.0-CURRENT @ January 2011 running with PV in EC2 (http://www.daemonology.net/freebsd-on-ec2/, look for t1.micro instances only) and that used PAE. But it's entirely likely that something got broken in the past two years and nobody noticed because nobody ever uses PV... -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: misc. questions
On 03/07/13 04:35, Mark Felder wrote:
On Wed, 06 Mar 2013 16:40:38 -0600, Colin Percival cperc...@freebsd.org
wrote:
You'll want to turn off tso, since it produces long mbuf chains which most
xn netbacks choke on. (I have a very ugly workaround patch for this which I
use on EC2, but simply turning off tso is enough unless you need Gbps+
speeds).
Can you link me to this patch? I have an environment that might warrant using
it
for now.
Attached.
And remember that I said it was a *very ugly* workaround... :-)
--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
--- sys/kern/uipc_mbuf.c(revision 223824)
+++ sys/kern/uipc_mbuf.c(working copy)
@@ -525,12 +525,14 @@
* only their reference counts are incremented.
*/
struct mbuf *
-m_copym(struct mbuf *m, int off0, int len, int wait)
+m_copy_nbufs(struct mbuf *m, int off0, int len, int wait, long * outlen,
+int nbufmax)
{
struct mbuf *n, **np;
int off = off0;
struct mbuf *top;
int copyhdr = 0;
+ int len_orig = len;
KASSERT(off = 0, (m_copym, negative off %d, off));
KASSERT(len = 0, (m_copym, negative len %d, len));
@@ -546,7 +548,7 @@
}
np = top;
top = 0;
- while (len 0) {
+ while (len 0 nbufmax-- 0) {
if (m == NULL) {
KASSERT(len == M_COPYALL,
(m_copym, length size of mbuf chain));
@@ -584,6 +586,9 @@
if (top == NULL)
mbstat.m_mcfail++; /* XXX: No consistency. */
+ if (outlen)
+ *outlen = len_orig - len;
+
return (top);
nospace:
m_freem(top);
@@ -591,6 +596,13 @@
return (NULL);
}
+struct mbuf *
+m_copym(struct mbuf *m, int off0, int len, int wait)
+{
+
+ return (m_copy_nbufs(m, off0, len, wait, NULL, INT_MAX));
+}
+
/*
* Returns mbuf chain with new head for the prepending case.
* Copies from mbuf (chain) n from off for len to mbuf (chain) m
--- sys/netinet/tcp_output.c(revision 228872)
+++ sys/netinet/tcp_output.c(working copy)
@@ -183,6 +183,7 @@
int sack_rxmit, sack_bytes_rxmt;
struct sackhole *p;
int tso;
+ int max_mbuf_chain_len = 16;/* XXX Set this based on interface? */
struct tcpopt to;
#if 0
int maxburst = TCP_MAXBURST;
@@ -806,16 +807,6 @@
struct mbuf *mb;
u_int moff;
- if ((tp-t_flags TF_FORCEDATA) len == 1)
- TCPSTAT_INC(tcps_sndprobe);
- else if (SEQ_LT(tp-snd_nxt, tp-snd_max) || sack_rxmit) {
- tp-t_sndrexmitpack++;
- TCPSTAT_INC(tcps_sndrexmitpack);
- TCPSTAT_ADD(tcps_sndrexmitbyte, len);
- } else {
- TCPSTAT_INC(tcps_sndpack);
- TCPSTAT_ADD(tcps_sndbyte, len);
- }
MGETHDR(m, M_DONTWAIT, MT_DATA);
if (m == NULL) {
SOCKBUF_UNLOCK(so-so_snd);
@@ -847,7 +838,8 @@
mtod(m, caddr_t) + hdrlen);
m-m_len += len;
} else {
- m-m_next = m_copy(mb, moff, (int)len);
+ m-m_next = m_copy_nbufs(mb, moff, len, M_DONTWAIT,
+ len, max_mbuf_chain_len);
if (m-m_next == NULL) {
SOCKBUF_UNLOCK(so-so_snd);
(void) m_free(m);
@@ -856,6 +848,18 @@
}
}
+ /* Update stats here as m_copy_nbufs may have adjusted len. */
+ if ((tp-t_flags TF_FORCEDATA) len == 1)
+ TCPSTAT_INC(tcps_sndprobe);
+ else if (SEQ_LT(tp-snd_nxt, tp-snd_max) || sack_rxmit) {
+ tp-t_sndrexmitpack++;
+ TCPSTAT_INC(tcps_sndrexmitpack);
+ TCPSTAT_ADD(tcps_sndrexmitbyte, len);
+ } else {
+ TCPSTAT_INC(tcps_sndpack);
+ TCPSTAT_ADD(tcps_sndbyte, len);
+ }
+
/*
* If we're sending everything we've got, set PUSH.
* (This will keep happy those implementations which only
--- sys/sys/mbuf.h (revision 223824)
+++ sys/sys/mbuf.h (working copy)
@@ -849,6 +849,7 @@
int, int, int, int);
struct mbuf*m_copypacket(struct mbuf *, int);
voidm_copy_pkthdr(struct mbuf *, struct mbuf *);
+struct mbuf*m_copy_nbufs(struct mbuf *, int, int, int, long *, int);
struct mbuf*m_copyup(struct mbuf *n, int len, int dstoff);
struct mbuf*m_defrag(struct mbuf *, int);
voidm_demote(struct mbuf *, int
Re: misc. questions
On 03/06/13 12:17, Jay West wrote: 1)One or two of our freebsd 9.1 HVM (with PVM drivers) under Xenserver 6.1 advanced are fairly frequently generating this message on the console: xn_txeof: WARNING: response is -1! Any ideas what this may be and what should be done? It does seem to only occur on the machines that have higher network load than the others. I see this on EC2 (running under HVM) as well. It doesn't seem to interfere with anything and only occurs occasionally (maybe five times per GB of network traffic?) so I haven't worried too much about it -- but if anyone has a clue about this I'd love to fix it from the tidy up loose ends perspective. 2)We did a pilot project of about 8 VM's (the above mentioned environment) and all went well. Now that we've moved it into production with many more VM's, I'm wondering about recommended tuning. I seem to recall from watching this list that there are a few sysctl's and the like that are highly recommended, I think they had to do with network settings and turning off offloading or somesuch. Does anyone have a quick dirty list of here's the first things you should always change with regards to FreeBSD HVM (pvm drivers) under XenServer? You'll want to turn off tso, since it produces long mbuf chains which most xn netbacks choke on. (I have a very ugly workaround patch for this which I use on EC2, but simply turning off tso is enough unless you need Gbps+ speeds). There's a separate issue affecting pf+tso+xn (possibly pf+tso generally) which I haven't gotten around to tracking down. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
FreeBSD in AWS Marketplace
Hi all, Another small piece of progress in the ongoing FreeBSD/EC2 saga -- I've managed to get FreeBSD into the curated Amazon Web Services Marketplace for cluster compute and high-I/O instance types. These run HVM normally and thus don't need you to pay the windows tax which applies to running FreeBSD on most other instance types. https://aws.amazon.com/marketplace/pp/B00AA25MLK For now it's just 9.0-RELEASE; I didn't want to bother the Amazon folks with the BETA and RC images, but I'll make sure 9.1-RELEASE also gets into the AWS Marketplace when it's released. I'm continuing to nag at EC2 folks about finding a way for FreeBSD to run on all the other instance types without paying the Windows tax. I would not be surprised if seeing more people using the existing FreeBSD options (either on HPC instances via the Marketplace, or on other instance types via the images listed on my website) would help to convince Amazon that this should be a priority. ;-) -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: anyone using FreeBSD/EC2 in production?
On 09/11/12 19:10, I wrote: If anyone here is using FreeBSD/EC2 in production, please let me know. I'm trying to get FreeBSD into the Amazon Web Services Marketplace, and they want me to provide user references. Thanks to everyone who replied -- it turns out that Amazon doesn't need these after all. (I'm guessing the people who run the AWS Marketplace just realized that porting a new OS to EC2 is not exactly the same as the usual selling a software package stories they see.) -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
anyone using FreeBSD/EC2 in production?
Hi all, If anyone here is using FreeBSD/EC2 in production, please let me know. I'm trying to get FreeBSD into the Amazon Web Services Marketplace, and they want me to provide user references. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
building kernel modules with XENHVM kernel
Hi all, Ever since the XENHVM kernel config file was added, it has included makeoptionsMODULES_OVERRIDE= I don't see any reason for doing this, and I'd like to remove it; I took it out of the config file I use for EC2 instances a long time ago, and there are people using e.g., zfs.ko without problems. Any objections? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: HEADS UP: Xen merge coming to stable/8
On 02/17/12 02:00, Richard Kojedzinszky wrote: Dear Colin, Will the merges affect 8.2 as well? No, only security fixes and critical bug fixes get merged back to the old release branches. On Tue, 14 Feb 2012, Colin Percival wrote: Date: Tue, 14 Feb 2012 17:10:17 -0800 From: Colin Percival cperc...@freebsd.org To: Kenneth D. Merry k...@freebsd.org Cc: freebsd-xen@freebsd.org Subject: Re: HEADS UP: Xen merge coming to stable/8 On 02/14/12 15:36, Kenneth D. Merry wrote: I'm planning to merge pretty much all of the Xen changes from head into stable/8 soon. (i.e. before the stable/8 code freeze) Oh good, I've been meaning to do some merges but wasn't sure if I'd be able to find time before the freeze starts. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Q: speed and duplex of xn in amd64 XENHVM
On 02/16/12 04:06, moto kawasaki wrote: Could you please advice me of the speed and duplex mode of the xn interface in XENHVM kernel (amd64)? With what command can I determine it ? I am testing XenServer 5.6 SP2, and FreeBSD 8.2-RELEASE-p6 (amd64, XENHVM), and ifconfig command shows me media: Ethernet manual, instead of something like media: Ethernet autoselect (100baseTX full-duplex). The xen virtual network interface is just that -- virtual. It can run at several Gbps but you're probably going to be limited by the host's network interface; and there's no way to find out what that is. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: amd64 xen hvm shutdown
On 01/25/12 02:09, Richard Kojedzinszky wrote: I am looking for a solution to be able to shutdown my freebsd 9.0 amd64 xen hvm domain from outside, from the host. Precisely, when the host is halted, it issues an # xm shutdown domain --halt --wait to the domain, and the domain does not poweroff. It behaves as if I issue a halt inside of it, and not a poweroff. Is there a way to have it power down itself? That sounds like a Xen configuration issue (poweroff behaviour?) rather than a FreeBSD issue to me. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD now available on all EC2 instance types
On 01/17/12 13:06, Carsten Heesch wrote: There's a folder ec2-bits in the root directory, which contains a bunch of patches. From your blog post I gather that they will not (much longer) be required if one was to build their own kernel? What gave you that idea? The patches in that directory are the differences between the 9.0-RELEASE source tree and the one I used to build the AMI. On the same subject: The kernel running in that AMI is apparently i386/XENHVM: FreeBSD ip-10-30-30-11 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Mon Jan 16 18:25:55 UTC 2012 root@ip-10-17-42-118:/usr/obj/i386.i386/usr/src/sys/XENHVM i386 The csup'ed source tree does not contain such configuration. It does for amd64. And there's a XEN configuration for i386. But no XENHVM configuration for i386. Yep, that's why there's a patch in /root/ec2-bits which adds XENHVM. :-) -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD now available on all EC2 instance types
On 01/17/12 14:49, Carsten Heesch wrote: What gave you that idea? The patches in that directory are the differences between the 9.0-RELEASE source tree and the one I used to build the AMI. Oh, must have been wishful thinking then that it might go into FreeBSD and not remain a bunch of separate patches... :) Working on it. :-) The files rXX.patch are commits I've already made to HEAD but which weren't in 9.0-RELEASE. The others I hope to merge into FreeBSD in some form: * blkfront.patch makes us compatible with the multi-page request ring protocol used by Amazon's blkback and I'm waiting to confirm that I reverse-engineered that protocol correctly; * uart.patch is a workaround for a bug in the version of Xen which Amazon is using, but it's a really ugly hack so I'm hoping some more talented device driver hackers can improve it; * tcp_mbuf_chain_limit.patch limits the length of mbuf chains sent via TCP TSO due to limits in the linux netback driver, but currently it unconditionally limits all TCP connections -- I need to talk to network stack people about how the max-chain-length value should be passed from the network interface up the stack to the TCP code; * ec2.patch is just tweaking some configuration files, so that doesn't need to be merged and isn't really a patch anyway. There's also some new rc.d scripts which get installed and more configuration files; I'm not sure if it makes sense to bring those rc.d scripts into FreeBSD proper since they're only relevant to the EC2 environment. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
FreeBSD now available on all EC2 instance types
Details at http://www.daemonology.net/blog/2012-01-16-FreeBSD-now-on-all-EC2-instance-types.html -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
i386/XENHVM now works
Hi all, i386/XENHVM now works on HEAD. I'll MFC to stable/9 in a week or two. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: PV i386 patch
On 12/20/11 10:49, Alan Cox wrote: Do either of you know if there is a PR in gnats for this 768 MB limitation bug that I should mention in the commit log? The only one I'm aware of is kern/153789. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: PV i386 patch
On 12/17/11 16:56, Sean Bruno wrote: This seems happy on our ref9 VMs. I don't suppose this means I can go above 768M of Ram now? Can't hurt to try... whatever the problem is with our code and large amounts of RAM, the fact that it's an insta-panic during paging setup suggests that it's something at a similar level of fail. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD PV i386 panics
On 07/26/11 11:35, Sean Bruno wrote: I assume then, that the i386 PV instance cannot do more than 1 CPU at this time? Correct. SMP support was clearly started but never completed. I note that if I scale the i386 PV Xen kernel past 855MB, I get an insta panic at boot. I assume that there's a hard limit on the amount of RAMS we can use here? There's a problem somewhere in FreeBSD's PV machine init code, but I haven't had an opportunity to track it down yet. I hope to look at this some time soon (if I can access the xen box in the cluster?) but I've been swamped lately. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD 8, PV i386 VNC fails?
On 07/26/11 14:48, Sean Bruno wrote: Probably not doing it right. When I switched from pure HVM to PV on the i386 VMs in the cluster, VNC stopped working all together. Did I do it wrong? Is it possible to use PV with VNC? I thought under PV there wasn't any emulated console, just the Xen paravirtual console which lets you read and write characters as on a serial console. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: Xen save/resume with XENHVM kernel
On 07/20/11 15:50, Scott Burns wrote: I see in sys/dev/xen/blkfront/blkfront.c that the blkfront_resume() function is not implemented, and there are also some notes about it in blkif_recover(). Is anyone currently working on this? Is there a hack I could use to get live migration working, even if it's not under ideal conditions? I'm not aware of any ongoing work on this. Most of the places where FreeBSD is being used with Xen don't use live migration. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
how to get max # of mbufs in a packet from xn to the tcp stack?
Hi all,
I've attached a patch which fixes the nfrags 18, netback won't be able to
handle it problem with xen netfront when TSO is enabled.
It's not finished, though:
+ int max_mbuf_chain_len = 16;/* XXX Set this based on interface? */
I'm not sure what the right way is to feed a value from the interface up into
tcp_output; can someone advise?
--
Colin Percival
Security Officer, FreeBSD | freebsd.org | The power to serve
Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid
Index: kern/uipc_mbuf.c
===
--- kern/uipc_mbuf.c (revision 223824)
+++ kern/uipc_mbuf.c (working copy)
@@ -525,12 +525,14 @@
* only their reference counts are incremented.
*/
struct mbuf *
-m_copym(struct mbuf *m, int off0, int len, int wait)
+m_copy_nbufs(struct mbuf *m, int off0, int len, int wait, long * outlen,
+int nbufmax)
{
struct mbuf *n, **np;
int off = off0;
struct mbuf *top;
int copyhdr = 0;
+ int len_orig = len;
KASSERT(off = 0, (m_copym, negative off %d, off));
KASSERT(len = 0, (m_copym, negative len %d, len));
@@ -546,7 +548,7 @@
}
np = top;
top = 0;
- while (len 0) {
+ while (len 0 nbufmax-- 0) {
if (m == NULL) {
KASSERT(len == M_COPYALL,
(m_copym, length size of mbuf chain));
@@ -584,6 +586,9 @@
if (top == NULL)
mbstat.m_mcfail++; /* XXX: No consistency. */
+ if (outlen)
+ *outlen = len_orig - len;
+
return (top);
nospace:
m_freem(top);
@@ -591,6 +596,13 @@
return (NULL);
}
+struct mbuf *
+m_copym(struct mbuf *m, int off0, int len, int wait)
+{
+
+ return (m_copy_nbufs(m, off0, len, wait, NULL, INT_MAX));
+}
+
/*
* Returns mbuf chain with new head for the prepending case.
* Copies from mbuf (chain) n from off for len to mbuf (chain) m
Index: netinet/tcp_output.c
===
--- netinet/tcp_output.c (revision 223824)
+++ netinet/tcp_output.c (working copy)
@@ -183,6 +183,7 @@
int sack_rxmit, sack_bytes_rxmt;
struct sackhole *p;
int tso;
+ int max_mbuf_chain_len = 16; /* XXX Set this based on interface? */
struct tcpopt to;
#if 0
int maxburst = TCP_MAXBURST;
@@ -806,16 +807,6 @@
struct mbuf *mb;
u_int moff;
- if ((tp-t_flags TF_FORCEDATA) len == 1)
- TCPSTAT_INC(tcps_sndprobe);
- else if (SEQ_LT(tp-snd_nxt, tp-snd_max) || sack_rxmit) {
- tp-t_sndrexmitpack++;
- TCPSTAT_INC(tcps_sndrexmitpack);
- TCPSTAT_ADD(tcps_sndrexmitbyte, len);
- } else {
- TCPSTAT_INC(tcps_sndpack);
- TCPSTAT_ADD(tcps_sndbyte, len);
- }
MGETHDR(m, M_DONTWAIT, MT_DATA);
if (m == NULL) {
SOCKBUF_UNLOCK(so-so_snd);
@@ -847,7 +838,8 @@
mtod(m, caddr_t) + hdrlen);
m-m_len += len;
} else {
- m-m_next = m_copy(mb, moff, (int)len);
+ m-m_next = m_copy_nbufs(mb, moff, len, M_DONTWAIT,
+ len, max_mbuf_chain_len);
if (m-m_next == NULL) {
SOCKBUF_UNLOCK(so-so_snd);
(void) m_free(m);
@@ -856,6 +848,18 @@
}
}
+ /* Update stats here as m_copy_nbufs may have adjusted len. */
+ if ((tp-t_flags TF_FORCEDATA) len == 1)
+ TCPSTAT_INC(tcps_sndprobe);
+ else if (SEQ_LT(tp-snd_nxt, tp-snd_max) || sack_rxmit) {
+ tp-t_sndrexmitpack++;
+ TCPSTAT_INC(tcps_sndrexmitpack);
+ TCPSTAT_ADD(tcps_sndrexmitbyte, len);
+ } else {
+ TCPSTAT_INC(tcps_sndpack);
+ TCPSTAT_ADD(tcps_sndbyte, len);
+ }
+
/*
* If we're sending everything we've got, set PUSH.
* (This will keep happy those implementations which only
Index: sys/mbuf.h
===
--- sys/mbuf.h (revision 223824)
+++ sys/mbuf.h (working copy)
@@ -849,6 +849,7 @@
int, int, int, int);
struct mbuf *m_copypacket(struct mbuf *, int);
void m_copy_pkthdr(struct mbuf *, struct mbuf *);
+struct mbuf *m_copy_nbufs(struct mbuf *, int, int, int, long *, int);
struct mbuf *m_copyup(struct mbuf *n, int len, int dstoff);
struct mbuf *m_defrag(struct mbuf *, int);
void m_demote(struct mbuf *, int);
___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: breakage in blkfront with ring_pages 1
On 07/06/11 15:55, Justin T. Gibbs wrote: On 7/5/11 10:14 PM, Colin Percival wrote: On 07/05/11 19:42, Colin Percival wrote: Yep. Mess or not, shall I go ahead with having a loader tunable control this, or can you think of a better solution? Does anyone object to the attached patch? It keeps the differing behaviour to a minimum -- we MUST set ring-ref with a FreeBSD blkback, and we MUST NOT set it with a linux blkback -- but otherwise errs in the direction of setting more variables than are needed, to maximize the possibility of a future blkback being compatible with both blkback_is_linux=0 and blkback_is_linux=1. It would be better to just change the FreeBSD blkback driver to be compatible with the RedHat convention. Fine with me, but that will of course break compatibility between pre- and post- patch versions of FreeBSD. Aside from you, how many people use the FreeBSD blkback driver? I'm still unclear on why the current FreeBSD blkfront driver believes that it can use more than one page in your configuration given that the RedHat blkfront doesn't advertise this capability in a way that the FreeBSD blkfront understands (max-ring-pages isn'te set by blkback). Did you do something to force blkfront to use more than one page? I'm seeing max-ring-pages set to 4. I don't know what tree EC2 is using on their Dom0 -- I've heard rumours that there's a lot of RedHat going on behind the scenes, but given Amazon's size it's entirely possible that they got a blkback which hasn't been released publicly. I'm sent an email asking where their blkback came from; I'll report back if/when I know something. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: breakage in blkfront with ring_pages 1
On 06/10/11 13:30, Justin T. Gibbs wrote: On 6/9/11 9:26 PM, Colin Percival wrote: Has anyone seen anything like this? Is it possible that there's a bug in how our blkfront negotiates the request ring? Does anyone have ring_pages 1 in use? The only backend driver I know of that can support more than one ring page is FreeBSD. So, the problem likely is that blkfront is getting the negotiation wrong and only 1 page is in use. Turns out that the linux backend in question really does support multiple pages. There's also an inconsistency about how multiple rings are negotiated: We set: * ring-pages to the number of pages blkfront wants to use * ring-ref to the reference for the first page * ring-refXX to the references for other pages while linux sets: * num-ring-pages to the number of pages blkfront wants to use * ring-refXX to the page references. It seems to be impossible to be compatible with both, since Linux interprets having a value set for ring-ref to indicate that the single-ring protocol is being used and doesn't check anything else. Is there any official source which defines the blkback/front protocol? Maybe the right option is to have a loader tunable dev.xn.linuxback to control which version of the protocol is used? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: breakage in blkfront with ring_pages 1
[oops, let's try sending this again with reply-all instead of reply...] On 07/05/11 19:04, Justin T. Gibbs wrote: On 7/5/11 7:14 PM, Colin Percival wrote: On 06/10/11 13:30, Justin T. Gibbs wrote: On 6/9/11 9:26 PM, Colin Percival wrote: Has anyone seen anything like this? Is it possible that there's a bug in how our blkfront negotiates the request ring? Does anyone have ring_pages 1 in use? The only backend driver I know of that can support more than one ring page is FreeBSD. So, the problem likely is that blkfront is getting the negotiation wrong and only 1 page is in use. Turns out that the linux backend in question really does support multiple pages. Can you provide a source or spec reference for this driver? Ah, here it is, dating from 2008: http://xen.1045712.n5.nabble.com/attachment/2527534/0/big_ring.patch Sorry, I had several windows open and thought I was looking at kernel.org tree commits rather than a separate patch. There was a proposal (in 2009?) to add multiple page support to the XenSource provided blkfront/back, but it didn't get committed to the Xen repository. It also doesn't use the same XenStore variables that you list. Is this something Amazon developed in house? I don't know what hand if any Amazon had in writing this, but it looks like they've convinced at least Ubuntu and RedHat to use this version. There's also an inconsistency about how multiple rings are negotiated: We set: * ring-pages to the number of pages blkfront wants to use * ring-ref to the reference for the first page * ring-refXX to the references for other pages while linux sets: * num-ring-pages to the number of pages blkfront wants to use * ring-refXX to the page references. You forgot to mention that the FreeBSD driver allows request size to be negotiated and explicitly supports chained requests to break the 44k request size limit imposed by the original interface. I was trying to highlight the compatibility issues, not list all the ways that FreeBSD is superior to Linux. ;-) It seems to be impossible to be compatible with both, since Linux interprets having a value set for ring-ref to indicate that the single-ring protocol is being used and doesn't check anything else. It's not Linux. There is no multi-page support in either the kernel.org hosted drivers or those in the Linux tree hosted in the official Xen repository. Agreed. But it's multiple Linux distributions (and EC2). Is there any official source which defines the blkback/front protocol? Only the source code in the Xen repository. Both the FreeBSD extensions that I did and those for the Linux system you are using are likely backwards compatible to the original interface, but unfortunately are not compatible with each other. Yes, both versions are definitely backwards compatible. Maybe the right option is to have a loader tunable dev.xn.linuxback to control which version of the protocol is used? What a mess. Yep. Mess or not, shall I go ahead with having a loader tunable control this, or can you think of a better solution? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: breakage in blkfront with ring_pages 1
On 07/05/11 19:42, Colin Percival wrote:
On 07/05/11 19:04, Justin T. Gibbs wrote:
On 7/5/11 7:14 PM, Colin Percival wrote:
Maybe the right option is to have a loader tunable dev.xn.linuxback to
control which version of the protocol is used?
What a mess.
Yep. Mess or not, shall I go ahead with having a loader tunable control this,
or can you think of a better solution?
Does anyone object to the attached patch? It keeps the differing behaviour to
a minimum -- we MUST set ring-ref with a FreeBSD blkback, and we MUST NOT set
it with a linux blkback -- but otherwise errs in the direction of setting more
variables than are needed, to maximize the possibility of a future blkback
being compatible with both blkback_is_linux=0 and blkback_is_linux=1.
--
Colin Percival
Security Officer, FreeBSD | freebsd.org | The power to serve
Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid
Index: sys/dev/xen/blkfront/blkfront.c
===
--- sys/dev/xen/blkfront/blkfront.c (revision 223815)
+++ sys/dev/xen/blkfront/blkfront.c (working copy)
@@ -616,7 +616,16 @@
if (setup_blkring(sc) != 0)
return;
+ /* Different backends use different names for this variable. */
error = xs_printf(XST_NIL, node_path,
+ num-ring-pages,%u, sc-ring_pages);
+ if (error) {
+ xenbus_dev_fatal(sc-xb_dev, error,
+ writing %s/num-ring-pages,
+ node_path);
+ return;
+ }
+ error = xs_printf(XST_NIL, node_path,
ring-pages,%u, sc-ring_pages);
if (error) {
xenbus_dev_fatal(sc-xb_dev, error,
@@ -673,6 +682,9 @@
xenbus_set_state(sc-xb_dev, XenbusStateInitialised);
}
+static int blkback_is_linux = 0;
+TUNABLE_INT(dev.xbd.blkback_is_linux, blkback_is_linux);
+
static int
setup_blkring(struct xb_softc *sc)
{
@@ -702,14 +714,16 @@
return (error);
}
}
- error = xs_printf(XST_NIL, xenbus_get_node(sc-xb_dev),
- ring-ref,%u, sc-ring_ref[0]);
- if (error) {
- xenbus_dev_fatal(sc-xb_dev, error, writing %s/ring-ref,
- xenbus_get_node(sc-xb_dev));
- return (error);
+ if (!blkback_is_linux || sc-ring_pages == 1) {
+ error = xs_printf(XST_NIL, xenbus_get_node(sc-xb_dev),
+ ring-ref,%u, sc-ring_ref[0]);
+ if (error) {
+ xenbus_dev_fatal(sc-xb_dev, error, writing %s/ring-ref,
+ xenbus_get_node(sc-xb_dev));
+ return (error);
+ }
}
- for (i = 1; i sc-ring_pages; i++) {
+ for (i = 0; i sc-ring_pages; i++) {
char ring_ref_name[]= ring_refXX;
snprintf(ring_ref_name, sizeof(ring_ref_name), ring-ref%u, i);
___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD/EC2 lives!
On 06/21/11 22:34, Chris Fordham wrote: On Wed, 22 Jun 2011 15:12:55 +1000, Colin Percival wrote: The current status of FreeBSD/EC2 is at http://www.daemonology.net/freebsd-on-ec2/ I am an engineer at RightScale and a couple of us that are FreeBSD users are looking at creating RightImages for the 5 regions. We'd like to offer our help and support if needed. I've released 8.2-RELEASE AMIs for t1.micro instances for all 5 regions. There's also an 8.2ish AMI for cc1.4xlarge in us-east-1. Thank you kindly Colin. We are wondering about the limited instance types, the issues on why this is so and if there any plans to support more? Im doing my best. ;-) In addition, is it possible to get the build source or otherwise to build the AMIs from scratch? The t1.micro instances were built from 8.2-RELEASE. The source tree used for cc1.4xlarge was slightly modified and is included in the AMI. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD/EC2 lives!
On 06/21/11 18:15, Chris Fordham wrote: On Tue, 14 Dec 2010 06:00:34 +1100, Colin Percival wrote: Thanks to a month of bug fixing and a lot of help from Amazon, FreeBSD is now available on EC2: http://www.daemonology.net/blog/2010-12-13-FreeBSD-on-EC2.html Only for 9.0-CURRENT and only t1.micro instances so far, though -- but we're moving in the right direction. :-) A belated big thanks again for this Colin! Just wondering if there are any updates or developments post December 2010's initial release? Yes. The current status of FreeBSD/EC2 is at http://www.daemonology.net/freebsd-on-ec2/ I am an engineer at RightScale and a couple of us that are FreeBSD users are looking at creating RightImages for the 5 regions. We'd like to offer our help and support if needed. I've released 8.2-RELEASE AMIs for t1.micro instances for all 5 regions. There's also an 8.2ish AMI for cc1.4xlarge in us-east-1. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
breakage in blkfront with ring_pages 1
Hi all, I'm seeing breakage with ring_pages 1 on a blkfront; more precisely, request #32 has a garbage response (resulting in a panic) exactly as if the dom0 thinks we're only using a single page for the ring. Forcing ring_pages to 1 makes the problem go away. Has anyone seen anything like this? Is it possible that there's a bug in how our blkfront negotiates the request ring? Does anyone have ring_pages 1 in use? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/154428: xn0 network interface and PF - Massive performance drop
On 02/12/11 20:18, Alex wrote: Fixed by net.inet.tcp.tso: 1 - 0 but why?? found this by trial an error. setting net.inet.tcp.tso to 0 with pf enabled gives good performance, if i set it to 1, speeds plumet to below dialup! There have been problems with Xen and TSO in the past relating to how much data gets handed off to the hypervisor at once... why this would cause issues only with PF, I have no idea, though. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/154428: xn0 network interface and PF - Massive performance drop
The following reply was made to PR kern/154428; it has been noted by GNATS. From: Colin Percival cperc...@freebsd.org To: Alex joo...@joovke.com Cc: freebsd-xen@freebsd.org, bug-follo...@freebsd.org Subject: Re: kern/154428: xn0 network interface and PF - Massive performance drop Date: Sat, 12 Feb 2011 20:31:07 -0800 On 02/12/11 20:18, Alex wrote: Fixed by net.inet.tcp.tso: 1 - 0 but why?? found this by trial an error. setting net.inet.tcp.tso to 0 with pf enabled gives good performance, if i set it to 1, speeds plumet to below dialup! There have been problems with Xen and TSO in the past relating to how much data gets handed off to the hypervisor at once... why this would cause issues only with PF, I have no idea, though. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: does anyone have Xen running on an AMD system?
On 01/13/11 14:46, Gót András wrote: It boots fine with machdep.idle_mwait=0 Ok, so to be clear: The AMD system which needs my patch in order to boot 8.2 can boot the latest HEAD without any patches? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: does anyone have Xen running on an AMD system?
On 01/13/11 14:35, Gót András wrote: Colin Percival wrote: This is a different panic (PR kern/152228). To fix this one, set machdep.idle_mwait=0 in the loader. er, how could we set it from the Xen config side (or any side)? With Linux, in PV mode Xen just boots the kernel with some params. I think putting it into the extra=... line in a xen domain config file works. I'm using pv-grub, and with that I just add the config options after the kernel path (e.g., kernel /boot/kernel/kernel machdep.idle_mwait=0,boot_verbose=1). -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: does anyone have Xen running on an AMD system?
On 01/12/11 05:33, Gót András wrote: kernel trap 9 with interrupts disabled Fatal trap 9: general protection fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer= 0x21:0xc0321241 stack pointer= 0x29:0xc21ffc8c frame pointer= 0x29:0xc21ffc94 code segment= base 0x0, limit 0xf, type 0x1b = DPL 1, pres 1, def32 1, gran 1 processor eflags= resume, IOPL = 0 current process= 11 (idle: cpu0) [thread pid 11 tid 13 ] Stopped at spinlock_exit+0xe1: hlt Yep, this is what I was looking for. Can you try the attached patch (under the same conditions to avoid the pmap_init panic) and let me know if this problem goes away? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid Index: i386/i386/machdep.c === --- i386/i386/machdep.c (revision 217272) +++ i386/i386/machdep.c (working copy) @@ -2690,8 +2690,10 @@ thread0.td_pcb-pcb_fsd = PCPU_GET(fsgs_gdt)[0]; thread0.td_pcb-pcb_gsd = PCPU_GET(fsgs_gdt)[1]; +#if defined(XEN_PRIVILEGED) if (cpu_probe_amdc1e()) cpu_idle_fn = cpu_idle_amdc1e; +#endif } #else ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/153789: REGRESSION: FreeBSD 8.2-RC1 crashes under Solaris/Xen while FreeBSD 8.1-p2 does not.
The following reply was made to PR kern/153789; it has been noted by GNATS. From: Colin Percival cperc...@freebsd.org To: bug-follo...@freebsd.org, g...@avalon-island.ru Cc: Subject: Re: kern/153789: REGRESSION: FreeBSD 8.2-RC1 crashes under Solaris/Xen while FreeBSD 8.1-p2 does not. Date: Sat, 08 Jan 2011 05:27:49 -0800 pmap_init(): address is out of range You sure about that message? I can't find it anywhere in the source tree. Is it possible that the message was pmap_init(): page table page is out of range instead? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/153674: i386/XEN idle thread shows wrong percentages
The following reply was made to PR kern/153674; it has been noted by GNATS. From: Colin Percival cperc...@freebsd.org To: bug-follo...@freebsd.org, f...@freebsd.org Cc: Subject: Re: kern/153674: i386/XEN idle thread shows wrong percentages Date: Tue, 04 Jan 2011 07:31:16 -0800 FWIW, this sounds to me like the kernel scheduler is getting confused in its accounting; possibly due to stolen time (when the hypervisor is off running other instances) and possibly related to the clock-interrupts-getting-lost bug (kern/153620). -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/153620: Xen guest system clock drifts in AWS EC2 (FreeBSD 9.0-CURRENT i386 T1-micro)
The following reply was made to PR kern/153620; it has been noted by GNATS. From: Colin Percival cperc...@freebsd.org To: bug-follo...@freebsd.org, fbsd-9.0-aws-ec2-1293964...@holmberg.to Cc: Subject: Re: kern/153620: Xen guest system clock drifts in AWS EC2 (FreeBSD 9.0-CURRENT i386 T1-micro) Date: Sun, 02 Jan 2011 03:16:52 -0800 This is interesting -- I thought I had squashed all the clock drift bugs. Can you tell me: 1. Did the clock run ahead, or behind? 2. Can you reproduce this? 3. Did the clock _drift_, or _jump_? The 2200 seconds mentioned is almost exactly the 2^41 ns period of the Xen timecounter, so if the clock jumped it's probably safe to guess that it's involved somehow... -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/143069: [xen] [panic] Xen Kernel Panic - Memory modified after free (8.0-RELEASE)
The following reply was made to PR kern/143069; it has been noted by GNATS. From: Colin Percival cperc...@freebsd.org To: bug-follo...@freebsd.org, fbs...@s.rhythm.cx Cc: Subject: Re: kern/143069: [xen] [panic] Xen Kernel Panic - Memory modified after free (8.0-RELEASE) Date: Sun, 26 Dec 2010 22:08:56 -0800 Hi Steve, I've fixed several bugs in the FreeBSD/Xen code over the past couple of months, including a few which could have caused the corruption you reported here; can you try with a recent (today's 9-CURRENT, or 8.2-RC1) tree and let me know if you can still reproduce this? Thanks, -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/140313: [xen] [panic] FreeBSD8 RC2 as PV domU crashes during configure of security/libgcrypt
The following reply was made to PR kern/140313; it has been noted by GNATS.
From: Colin Percival cperc...@freebsd.org
To: bug-follo...@freebsd.org, mister.o...@googlemail.com
Cc:
Subject: Re: kern/140313: [xen] [panic] FreeBSD8 RC2 as PV domU crashes during
configure of security/libgcrypt
Date: Sat, 25 Dec 2010 02:06:45 -0800
Test case:
#include sys/mman.h
int main()
{
int i;
mlock(i, 4);
return (0);
}
This appears to have been partially fixed by SVN r204160, but pmap_pte_release
needs to be updated the same way as pmap_pte. I'll commit this once I've had
a chance to test further.
--
Colin Percival
Security Officer, FreeBSD | freebsd.org | The power to serve
Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid
___
freebsd-xen@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-xen
To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD 8.2 PR PANIC PVM on Citrix XenServer
On 12/18/10 05:28, Tobiasz Tworek wrote: I have build XEN kernel from RELANG_8 on Citrix XenServer 5.6 and got panic after reboot :-( My screen output: [snip] Any sugestions ?? What i have done wrong or what can i do to run freebsd in xenserver in pvm mode ? Try running HEAD. I have a lot of bug fixes in there which I haven't MFCed yet. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD/EC2 lives!
On 12/17/10 01:29, Robert Watson wrote: On Mon, 13 Dec 2010, Aleksey Ovcharenko wrote: Any plans of using ZFS there? I don't know about Amazon's live disk resizing policy, but at least in our local Xen deployment of FreeBSD at Cambridge, ZFS seems like the sensible thing so that we can do live disk resizing during provision of VMs through cloning. Amazon EBS doesn't allow disks to be resized -- this is something I asked for a long time ago, but I'm guessing it isn't something a very large number of people are demanding. Of course, with EBS you can always create a new volume, sync everything across, then destroy the old volume[*]; so it's not as large a problem as on a local Xen deployment where the amount of disk space you have available might limit your ability to temporarily duplicate everything. [*] In theory, at least. Last time I tried I got a FreeBSD kernel panic when I detached an EBS volume from a running instance. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
FreeBSD/EC2 lives!
Hi freebsd-xen people and wannabe FreeBSD/EC2 users, Thanks to a month of bug fixing and a lot of help from Amazon, FreeBSD is now available on EC2: http://www.daemonology.net/blog/2010-12-13-FreeBSD-on-EC2.html Only for 9.0-CURRENT and only t1.micro instances so far, though -- but we're moving in the right direction. :-) -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD/EC2 lives!
On 12/13/10 11:58, Vasiliy G Tolstov wrote: Great! Can You say, does freebsd support live migration and memory hotplug? I don't think so. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/152818: [panic][xen] disk driver data cannot cross a page boundary
The following reply was made to PR kern/152818; it has been noted by GNATS. From: Colin Percival cperc...@freebsd.org To: bug-follo...@freebsd.org, Justin T. Gibbs gi...@scsiguy.com Cc: Subject: Re: kern/152818: [panic][xen] disk driver data cannot cross a page boundary Date: Sat, 04 Dec 2010 14:35:57 -0800 It looks like this bug was fixed in busdma on amd64 by r204214 but never merged to i386. Justin, is there a technical reason why this wasn't merged, or should I try to do this myself? -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/141328: [xen] [panic] gstat exit causes kernel panic from unmanaged virtual address
The following reply was made to PR kern/141328; it has been noted by GNATS. From: Colin Percival cperc...@freebsd.org To: bug-follo...@freebsd.org, t...@hur.st Cc: Subject: Re: kern/141328: [xen] [panic] gstat exit causes kernel panic from unmanaged virtual address Date: Sun, 28 Nov 2010 07:30:51 -0800 Still present in HEAD @ 2010-11-27. Another test case: 'mdconfig -l'. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: kern/140313: [xen] [panic] FreeBSD8 RC2 as PV domU crashes during configure of security/libgcrypt
The following reply was made to PR kern/140313; it has been noted by GNATS. From: Colin Percival cperc...@freebsd.org To: bug-follo...@freebsd.org Cc: Subject: Re: kern/140313: [xen] [panic] FreeBSD8 RC2 as PV domU crashes during configure of security/libgcrypt Date: Sun, 28 Nov 2010 07:48:31 -0800 Note for the audit trail: Also reported as kern/145664. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: system stalls when wallclock/TOD nudged
Hi all, On 11/19/10 23:38, Colin Percival wrote: It seems that this bug is related somehow to the idle thread sleeping -- when I comment out the call to idle_block from cpu_idle_hlt in i386/machdep.c the stalling goes away. Ok, it turns out that the problem was a bit more complicated than this. There were two bugs, actually: First, Xen's timecounter was being rounded from ns precision to 1/HZ precision for no apparent reason; and second, the periodic clock interrupt isn't being delivered. I've fixed the first of these, and that is enough to prevent the stalling; I need to learn more about Xen interrupts, event channels, etc. to track down why we're not getting a clock interrupt. (The timer set by set_timer_op before we call SCHEDOP_block is waking us up -- but the missing clock interrupt means that we aren't preempting userland code, which is a rather Bad Thing.) -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
DomU panic during boot with recent timer changes (+ workaround)
Hi all, Following mav's timer refactoring (r212541) I'm seeing a panic during boot: Fatal trap 1: privileged instruction fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x21:0xc031ac5c stack pointer = 0x29:0xc2719c84 frame pointer = 0x29:0xc2719c88 code segment= base 0x0, limit 0xf9800, type 0x1b = DPL 1, pres 1, def32 1, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 11 (idle: cpu0) trap number = 1 panic: privileged instruction fault cpuid = 0 KDB: stack backtrace: #0 0xc0106057 at kdb_backtrace+0x47 #1 0xc00d3c47 at panic+0x117 #2 0xc032efa3 at dblfault_handler+0x3c3 #3 0xc032f8ad at trap+0x5ed #4 0xc03150bb at alltraps+0x1b #5 0xc031acb0 at cpu_idle+0x30 #6 0xc00f9133 at sched_idletd+0x263 #7 0xc00aa7a8 at fork_exit+0xb8 #8 0xc0315134 at fork_trampoline+0x8 Uptime: 1s It looks like the privileged instruction fault is being triggered by mwait. Setting machdep.idle_mwait=0 prevents this panic; I'm not sure if Xen is supposed to handle mwait or not, so I don't know if mwait should be disabled unconditionally for Xen/PV kernels, but maybe someone here knows more than I do. :-) -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
Re: FreeBSD on EC2
On 10/31/10 02:52, Zladivliba Voskuy wrote: - I just saw 2 netbst images on EC2 named fbsdtest/netbsd-image-5.0.2-i386.manifest.xml and fbsdtest/netbsd-image-5.0.2-amd64.manifest.xml *cough* Please ignore those. They were an experiment which doesn't currently work. Those are both public images. I haven't tried them yet but if someone managed to run netbst I think we're quiet close to have some fun on FreeBSD now. Not necessarily. NetBSD's Xen code is rather better than FreeBSD's. -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to freebsd-xen-unsubscr...@freebsd.org
