Re: FreeBSD PVHVM call for testing
Hi, After some more testing I thought it would be good to put this into production for my personal server. I've used pvhvm_v19 and built it without debugging options and installed it on a FreeBSD 9.1 system. I've run into some hiccups with 9.1 user land and a 10-CURRENT kernel, but that's all solvable[0]. My VPS has some very limited memory (256M), but I've compensated with swap space (1G) Now anytime I'm putting the system under stress, by building ports or by running a git clone on the kernel repository here, I'm seeing a lot of messages about swap_pager: swap_pager: indefinite wait buffer: bufobj: 0, blkno: 132545, size: 4096 The system also becomes very sluggish and sometimes unresponsive. The weird thing was that one of these messages happened right after a reboot when I rebuilt an outdated port and on the main console was checking the swap memory: jeroen:~/ $ swapinfo [8:13:29] Device 1K-blocks UsedAvail Capacity /dev/ada0p2524288 2484 521804 0% /dev/md0 1048576 2364 1046212 0% Total 1572864 4848 1568016 0% swap_pager: indefinite wait buffer: bufobj: 0, blkno: 131424, size: 4096 Is anyone else seeing something similar? I certainly did not experience something like this on 9.0 with a XENHVM kernel. If necessary I can rebuild a kernel with debugging support and do some more recording of what is actually going on. Jeroen. [0]: I have edited bsd.port.mk to always apply the FBSD10_FIX, and for version checking I am running pkg version with UNAME_r=9.1-RELEASE. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
Hi, I have this running for a day or so now, but I'm noticing that the load averages seem a bit off: $ uptime 11:17AM up 17:14, 1 user, load averages: 0.31, 0.27, 0.21 This is for a clean install, with just enough installed to compile this kernel. In top I'm seeing that the machine is idling 98% of the time. But this does not correlate to the load displayed above. Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
On 19 Jun 2013, at 13:34, Roger Pau Monné roger@citrix.com wrote: Could you provide the boot log of the DomU, backtrace, Xen version and Dom0 kernel version? I did not have a console attached when it rebooted, so I did not have a log of the initial boot. Now that I did, I see that it fails to mount its root volume. It had been running previously on pvhvm_v10 for about two weeks without problems. I updated my local git, and recompiled the kernel and rebooted. Then this happened. In order: Booting... GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #2 r+6ff8d00-dirty: Tue Jun 18 12:55:16 CEST 2013 root@image01:/usr/obj/root/freebsd/sys/XENHVM amd64 FreeBSD clang version 3.3 (trunk 178860) 20130405 WARNING: WITNESS option enabled, expect reduced performance. XEN: Hypervisor version 4.0 detected. CPU: Quad-Core AMD Opteron(tm) Processor 2374 HE (2200.07-MHz K8-class CPU) Origin = AuthenticAMD Id = 0x100f42 Family = 0x10 Model = 0x4 Stepping = 2 Features=0x1781fbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x80802001SSE3,CX16,POPCNT,HV AMD Features=0xe2500800SYSCALL,NX,MMX+,FFXSR,LM,3DNow!+,3DNow! AMD Features2=0x1f3LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch real memory = 536870912 (512 MB) avail memory = 472260608 (450 MB) Event timer LAPIC quality 400 ACPI APIC Table: Xen HVM FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 random device not loaded; using insecure entropy ioapic0: Changing APIC ID to 1 MADT: Forcing active-low polarity and level trigger for SCI ioapic0 Version 1.1 irqs 0-47 on motherboard kbd1 at kbdmux0 xen_et0: Xen PV Clock on motherboard Event timer XENTIMER frequency 10 Hz quality 950 Timecounter XENTIMER frequency 10 Hz quality 950 acpi0: Xen on motherboard acpi0: Power Button (fixed) acpi0: Sleep Button (fixed) acpi0: reservation of 0, a (3) failed cpu0: ACPI CPU on acpi0 cpu1: ACPI CPU on acpi0 attimer0: AT timer port 0x40-0x43 irq 0 on acpi0 Timecounter i8254 frequency 1193182 Hz quality 0 Event timer i8254 frequency 1193182 Hz quality 100 atrtc0: AT realtime clock port 0x70-0x71 irq 8 on acpi0 Event timer RTC frequency 32768 Hz quality 0 Timecounter ACPI-fast frequency 3579545 Hz quality 900 acpi_timer0: 32-bit timer at 3.579545MHz port 0x1f48-0x1f4b on acpi0 pcib0: ACPI Host-PCI bridge port 0xcf8-0xcff on acpi0 pci0: ACPI PCI bus on pcib0 isab0: PCI-ISA bridge at device 1.0 on pci0 isa0: ISA bus on isab0 atapci0: Intel PIIX3 WDMA2 controller port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc300-0xc30f at device 1.1 on pci0 ata0: ATA channel at channel 0 on atapci0 ata1: ATA channel at channel 1 on atapci0 pci0: bridge at device 1.3 (no driver attached) vgapci0: VGA-compatible display mem 0xf000-0xf1ff,0xf300-0xf3000fff at device 2.0 on pci0 xenpci0: Xen Platform Device port 0xc000-0xc0ff mem 0xf200-0xf2ff irq 28 at device 3.0 on pci0 xenstore0: XenStore on xenpci0 atkbdc0: Keyboard controller (i8042) port 0x60,0x64 irq 1 on acpi0 atkbd0: AT Keyboard irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: PS/2 Mouse irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4 fdc0: floppy drive controller port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 uart0: 16550 or compatible port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (9600,n,8,1) ppc0: Parallel port port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: Parallel port bus on ppc0 lpt0: Printer on ppbus0 lpt0: Interrupt-driven port ppi0: Parallel I/O on ppbus0 orm0: ISA Option ROM at iomem 0xc9000-0xc97ff on isa0 sc0: System console at flags 0x100 on isa0 sc0: VGA 16 virtual consoles, flags=0x300 vga0: Generic ISA VGA at port 0x3c0-0x3df iomem 0xa-0xb on isa0 fdc0: No FDOUT register! Timecounters tick every 10.000 msec xenbusb_front0: Xen Frontend Devices on xenstore0 cd0 at ata1 bus 0 scbus1 target 0 lun 0 cd0: QEMU QEMU DVD-ROM 0.10 Removable CD-ROM SCSI-0 device cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: cd present [360385 x 2048 byte records] xn0: Virtual Network Interface at device/vif/0 on xenbusb_front0 xn0: Ethernet address: 00:16:3e:2f:b7:22 xn1: Virtual Network Interface at device/vif/1 on xenbusb_front0 xn1: Ethernet address: 00:16:3e:3e:64:c5 xenbusb_back0: Xen Backend Devices on xenstore0 xctrl0: Xen Control Device on xenstore0 xn0: backend features: feature-sg feature-gso-tcp4 xn1: backend features: feature-sg feature-gso-tcp4 xbd0:
Re: FreeBSD PVHVM call for testing
On 19 Jun 2013, at 14:20, Roger Pau Monné roger@citrix.com wrote: That's because Justin recently pushed a commit that changed the ad translation to ada, you should change your /etc/fstab to ada0p2. It's commit 526f3ad11acb296481215d7c2915b3f30f1844f6. Ah, you may want to update the wiki page also to warn for that. :) Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
Hi, On 19 Jun 2013, at 18:15, Justin T. Gibbs gi...@freebsd.org wrote: I've never seen a kernel build take 2 hours, much less 2 hours *longer*. Are you talking about buildworld? It would be interesting to know your results building stable/9 sources in your 10 environment to see if this is just due to build bloat or a true performance regression. I copy/pasted the command from the wiki: # make kernel-toolchain make buildkernel KERNCONF=XENHVM make installkernel KERNCONF=XENHVM On the stable/9 I only did make buildkernel KERNCONF=XENHVM make installkernel KERNCONF=XENHVM I guess the kernel-toolchain takes a long time to build…and from what I can see it does a clean before rebuilding also. I'm doing the kernel-toolchain step only now and will report how long it took. Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
Hi, On 23 May 2013, at 19:41, Roger Pau Monné roger@citrix.com wrote: Hello, I've pushed a new branch, pvhvm_v10 that contains a PV IPI implementation for both amd64 and i386. I've also updated the wiki to point to the pvhvm_v10 branch: I've been running a VM with this kernel for about a week now. It ran fine, until about 3:30 in the morning. The only thing I can see is the following cryptic messages in /var/log/messages, followed by a reboot of the system. May 29 23:42:30 image01 sshd[31227]: error: Received disconnect from 150.165.15.175: 11: Bye Bye [preauth] May 30 03:30:57 image01 kernel: . May 30 03:30:57 image01 ntpd[4436]: ntpd exiting on signal 15 May 30 03:30:57 image01 kernel: . May 30 03:30:58 image01 kernel: . May 30 03:31:00 image01 syslogd: exiting on signal 15 May 30 03:32:52 image01 syslogd: kernel boot file is /boot/kernel/kernel May 30 03:32:52 image01 kernel: Copyright (c) 1992-2013 The FreeBSD Project. May 30 03:32:52 image01 kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 May 30 03:32:52 image01 kernel: The Regents of the University of California. All rights reserved. May 30 03:32:52 image01 kernel: FreeBSD is a registered trademark of The FreeBSD Foundation. I'm happy to help to gather more information, just tell me what you need. Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
Hi, On 30 May 2013, at 11:04, Roger Pau Monné roger@citrix.com wrote: So it looks like the system rebooted (but it was not a crash or a sporadic reboot? the kernel seems to be aware of the reboot request). It would be interesting if you could provide the output of the serial console when this happens, that might be helpful. Did you enable xenconsoled logging? Unfortunately I did not. Also, could you provide more info about your system, Xen version, what workload was the DomU running, Dom0 kernel version? There was no one logged in at the time of the reboot according to the last log. I did do some sysbench tests during the day, but that was way before it rebooted. The only thing that could be running during that time was daily periodic. $ sudo xm info host : soleus01.soleus.nu release: 2.6.32-5-xen-amd64 version: #1 SMP Mon Oct 3 07:53:54 UTC 2011 machine: x86_64 nr_cpus: 8 nr_nodes : 2 cores_per_socket : 4 threads_per_core : 1 cpu_mhz: 2200 hw_caps: 178bf3ff:efd3fbff::1310:00802001::37ff: virt_caps : hvm total_memory : 65534 free_memory: 6866 node_to_cpu: node0:0-3 node1:4-7 node_to_memory : node0:3128 node1:3737 node_to_dma32_mem : node0:3128 node1:0 max_node_id: 1 xen_major : 4 xen_minor : 0 xen_extra : .1 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler : credit xen_pagesize : 4096 platform_params: virt_start=0x8000 xen_changeset : unavailable xen_commandline: placeholder dom0_mem=1852M cc_compiler: gcc version 4.4.5 (Debian 4.4.5-10) cc_compile_by : waldi cc_compile_domain : debian.org cc_compile_date: Wed Jan 12 14:04:06 UTC 2011 xend_config_format : 4 $ uname -a Linux soleus01.soleus.nu 2.6.32-5-xen-amd64 #1 SMP Mon Oct 3 07:53:54 UTC 2011 x86_64 GNU/Linux Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
Hi, On 30 May 2013, at 16:56, Outback Dingo outbackdi...@gmail.com wrote: first is this a public vm ? and if so who is?? May 29 23:42:30 image01 sshd[31227]: error: Received disconnect from 150.165.15.175: 11: Bye Bye [preauth] because it is after this potential ssh login attempt, so is this you, has there been a breach ? only thing i noticed, but it might be nothing. This VM is on a public IP indeed, and SSH connectivity is enabled. As with any publicly accessible host this then becomes the target of ssh scans. I included the message just to show that between it and the reboot nothing had been logged. AFAICT there has not been a breach, and I have not seen any indications at all that there may be one. Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: FreeBSD PVHVM call for testing
Hi, Just remove this line (or pointing to a similar file from the template: (It's part of the disks definition: 'file:/root/freebsd-10.iso,hdc:cdrom,r', Jeroen. On 23 May 2013, at 19:02, Outback Dingo outbackdi...@gmail.com wrote: On Thu, May 23, 2013 at 12:48 PM, Roger Pau Monné roger@citrix.comwrote: On 23/05/13 18:30, Outback Dingo wrote: On Thu, May 23, 2013 at 9:33 AM, Roger Pau Monné roger@citrix.com mailto:roger@citrix.com wrote: On 23/05/13 14:57, Jeroen van der Ham wrote: Hi, On 13 May 2013, at 20:32, Roger Pau Monné roger@citrix.com mailto:roger@citrix.com wrote: Right now the code is in a state where it can be tested by users, so we would like to encourage FreeBSD and Xen users to test it and provide feedback. I've just been able to install it on a VPS using the latest pvhvm_v9 branch. The branch pvhvm_v9 contains an initial implementation of PV IPIs for amd64. I've now finished it and I'm going to port it to i386 also, and push a new branch to the repository. This is good news, because the system I had before actually had trouble with the HVM kernel from 9.1 [0]. I'm going to leave this running for a while and do some more tests on it. Jeroen. [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822 I built the rev_9 branch on a XCP host and rebooted, however I am seeing on boot after ugen0.2: QEMU 0.10.2 at usbus0 run_interrupt_driven_hooks: still waiting after 60 seconds for xenbus_nop_confighook_cb run_interrupt_driven_hooks: still waiting after 120 seconds for xenbus_nop_confighook_cb run_interrupt_driven_hooks: still waiting after 180 seconds for xenbus_nop_confighook_cb run_interrupt_driven_hooks: still waiting after 240 seconds for xenbus_nop_confighook_cb run_interrupt_driven_hooks: still waiting after 300 seconds for xenbus_nop_confighook_cb panic: run_interrupt_driven_confighooks: waited too long cpuid = 0 KDB: enter: panic [ thread pid 0 tid 10 ] Stropped at kdb_enter +0x3b: movq $0,0xad6522(%rip) db From what I've read on the list, it seems like you cannot boot the PVHVM kernel if you have a cdrom attached to the guest, could you try disabling the cdrom and booting again? great how does one go about disabling the cdrom, i get some disk parameters needs to be removed from the vm template before boot ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
