Virtio network: poor performance with KVM hypervisor (latest Proxmox)
Hi all!
I am using the latest Proxmox 4.1 with all updates installed.
I have several VM's with FreeBSD guests and 1 VM with Ubuntu 14 (all KVM).
Host system file download speed: 60 MBps.
FreeBSD guest download speed: 2 MBps on virtio network with TSO enabled, 5-9
MBps with TSO disabled; 12 MBps on e1000 network.
Ubuntu guest: 60 MBps with virtio.
I've tried the following:
1) Different FreeBSD versions: 9.3, 10.2, 10.3-BETA3.
2) Different TSO settings, enabling/disabling RXCSUM.
3) Different TSO settings on host system.
The best results I got described above :(
Does anyone have any ideas how to get full network performance inside FreeBSD
guests?
--
Alexey Tarasov
(\__/)
(='.'=)
E[: | | | | :]З
(")_(")
smime.p7s
Description: S/MIME cryptographic signature
Re: [clang] (gpt)zfsboot is broken: zfs_alloc()/zfs_free() mismatch
On Aug 1, 2011, at 4:24 PM, Test Rat wrote: Anyone else? I can still reproduce with trunk r136607. boot and gptboot seem to be unaffected. IIRC, with previous clang import it just stuck during boot without any error messages. I have the same behavior. GCC build of gptzfsboot works fine. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [clang] (gpt)zfsboot is broken: zfs_alloc()/zfs_free() mismatch
On Aug 1, 2011, at 4:24 PM, Test Rat wrote: Anyone else? I can still reproduce with trunk r136607. boot and gptboot seem to be unaffected. IIRC, with previous clang import it just stuck during boot without any error messages. I have the same behavior. GCC build of gptzfsboot works fine. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: STABLE kernel panic: privileged instruction fault
Hello Andriy! On Aug 18, 2010, at 7:30 PM, Andriy Gapon wrote: on 13/08/2010 00:45 Alexey Tarasov said the following: Fatal trap 1: privileged instruction fault while in kernel mode cpuid = 1; apic id = 01 instruction pointer = 0x20:0xff8040d2cc83 stack pointer = 0x28:0xff8040d2ca80 frame pointer = 0x28:0xff0060c0b740 I suspect that either stack is corrupted or non-code is executed (or both). Stack pointer seems to be too close to instruction pointer and too far from frame pointer. Can you try to use kgdb and disassemble code (or examine data) near instruction pointer address and also near frame pointer address? Also, you might want to rebuild kgdb with a recent change from head, so that it better maps symbols and addresses in kernel modules. We have the similar discussion with Kostik Belousov here: http://lists.freebsd.org/pipermail/freebsd-stable/2010-August/058287.html I'm installing new kernel with DDB on the servers now and waiting for the panic. Thank you for your reply! -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
STABLE kernel panic: privileged instruction fault
Hello. I have a couple of Supermicro servers which got the similar kernel panic with all FreeBSD versions I tried since 6.4. Now I want to investigate into the problem. The servers get into panic with similar workload: file server with a lot of files and connections. Web server software is nginx. File system is UFS+GJOURNAL. Outgoing traffic on each server is ~10 MB/s. I think it is not software problem, because when I've installed Linux with such configuration there were no kernel panics. Here is the short overview of the hardware: CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (2992.51-MHz K8-class CPU) Origin = GenuineIntel Id = 0xf65 Family = f Model = 6 Stepping = 5 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0xe59dSSE3,DTES64,MON,DS_CPL,EST,TM2,CNXT-ID,CX16,xTPR,PDCM AMD Features=0x20100800SYSCALL,NX,LM AMD Features2=0x1LAHF TSC: P-state invariant real memory = 2147483648 (2048 MB) avail memory = 2054619136 (1959 MB) DMESG: http://lexasoft.ru/m/dmesg.txt CORE: http://lexasoft.ru/m/core.txt Fatal trap 1: privileged instruction fault while in kernel mode cpuid = 1; apic id = 01 instruction pointer = 0x20:0xff8040d2cc83 stack pointer = 0x28:0xff8040d2ca80 frame pointer = 0x28:0xff0060c0b740 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 9388 (nginx) trap number = 1 panic: privileged instruction fault cpuid = 1 Uptime: 17d15h48m49s Physical memory: 2032 MB Dumping 1485 MB: 1470 1454 1438 1422 1406 1390 1374 1358 1342 1326 1310 1294 1278 1262 1246 1230 1214 1198 1182 1166 1150 1134 1118 1102 1086 1070 1054 1038 1022 1006 990 974 958 942 926 910 894 878 862 846 830 814 798 782 766 750 734 718 702 686 670 654 638 622 606 590 574 558 542 526 510 494 478 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14 (kgdb) #0 doadump () at pcpu.h:223 #1 0x80590c59 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:416 #2 0x8059108c in panic (fmt=0x80951fc4 %s) at /usr/src/sys/kern/kern_shutdown.c:579 #3 0x80878fd8 in trap_fatal (frame=0xff0060c0b740, eva=Variable eva is not available. ) at /usr/src/sys/amd64/amd64/trap.c:857 #4 0x808799ea in trap (frame=0xff8040d2c9d0) at /usr/src/sys/amd64/amd64/trap.c:644 #5 0x8085f983 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:224 #6 0xff8040d2cc83 in ?? () #7 0xff8040d2cb50 in ?? () #8 0xff8040d2caf0 in ?? () #9 0xff8040d2cbf0 in ?? () #10 0xff0060c0b740 in ?? () #11 0x80b83c60 in sysent () #12 0xff8040d2cc80 in ?? () #13 0xff8040d2cae0 in ?? () #14 0x8059c431 in bintime (bt=0x80ad3140) at /usr/src/sys/kern/kern_tc.c:200 Previous frame inner to this frame (corrupt stack?) (kgdb) -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
I've booted from dvd to fixit mode and got the following: FreeBSD 8.0-STABLE-201002 FreeBSD 8.0-STABLE-201002 #0: Tue Feb 16 21:05:59 UTC 2010 r...@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 ATA channel 0: Master: ad0 TRANSCEND/20091215 ATA/ATAPI revision 0 Slave: no device present ATA channel 2: Master: ad4 WDC WD15EARS-00Z5B1/80.00A80 SATA revision 2.x Slave: no device present ATA channel 3: Master: ad6 WDC WD15EARS-00Z5B1/80.00A80 SATA revision 2.x Slave: no device present ATA channel 4: Master: ad8 WDC WD15EARS-00Z5B1/80.00A80 SATA revision 2.x Slave: no device present ATA channel 5: Master: ad10 WDC WD15EARS-00Z5B1/80.00A80 SATA revision 2.x Slave: no device present /dev/ad4 512 # sectorsize 1500301910016 # mediasize in bytes (1.4T) 2930277168 # mediasize in sectors 0 # stripesize 0 # stripeoffset 2907021 # Cylinders according to firmware. 16 # Heads according to firmware. 63 # Sectors according to firmware. WD-WMAVU1512579 # Disk ident. Seems that mav@ commit doesn't work? o_O On 09.04.2010, at 0:44, Dimitry Andric wrote: That said, if the physical sector size is larger than the logical sector size, the d_stripesize field is initialized with it. So if you run diskinfo -v on the disk, what is the output for stripesize? -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: ZFS raidz and 4k sector disks
Hello. I see considerably increased performance when creating over gnop -S 4096 virtual disk. Even when I create zpool over raw disks the performance is very bad and concurent writes stalls. When using gnop, zfs works VERY fast! Btw, here is another discussion, may be there is a bug in a mav@ commit, because he has added support for 512 sector size: http://lists.freebsd.org/pipermail/freebsd-current/2010-April/016495.html Превед Украине! =) On 09.04.2010, at 12:28, Andriy Gapon wrote: on 08/04/2010 16:55 Alexey Tarasov said the following: Hello. I've tried all methods and realized that unfortunately the only working method is gnop. So you can't use these disks for ZFS at all now. Why? And what are you actually trying to do? My understanding was that even with 512-byte sectors ZFS still aligns its on-disk data with 4K alignment. Do you see otherwise? What problem do you have? -- Andriy Gapon -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
Or the disk doesn't actually report 4096 anywhere anyhow... Have you considered that? If yes, can you verify using any tools of any OS that the disk reports 4K in any way? In the previous discussion we found that the disk reports 512 sector size, but there are additional ATA commands to determine if it has real sector size larger than 4k. I will try to confirm this. P.S. DES's name looks strange in headers :-) Really. :-) P.P.S. A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? Sorry, to few experience. =) -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
On 09.04.2010, at 15:32, Andriy Gapon wrote: on 09/04/2010 14:27 Alexey Tarasov said the following: Or the disk doesn't actually report 4096 anywhere anyhow... Have you considered that? If yes, can you verify using any tools of any OS that the disk reports 4K in any way? In the previous discussion we found that the disk reports 512 sector size, but there are additional ATA commands to determine if it has real sector size larger than 4k. I will try to confirm this. Thank you. I think that this would be an interesting detail. Here is the reference: http://www.wdc.com/wdproducts/library/WhitePapers/ENG/2579-771430.pdf References The ATA8-ACS and SBC-3 standards have provisions for a disk drive to report Advanced Format sector sizes and other performance optimization information. These standards are used for SATA, SAS, USB, and IEEE 1394 based interface technologies. -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
I saw it, but I want to see what's reported in reality. Installing Windows 7 now. How can OS installation be so long? :-) -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
No, the problem is that you must use the ada(4) driver instead of ad(4). The new physical and logical sector support has only been implemented for the newer AHCI-over-CAM stack. pass0: Raw identify data: 0: 427a 3fff c837 0010 003f 8: 2020 2020 2057 442d 574d 4156 16: 5531 3531 3235 3739 0032 3830 24: 2e30 3041 3830 5744 4320 5744 3135 4541 32: 5253 2d30 305a 3542 3120 2020 2020 2020 40: 2020 2020 2020 2020 2020 2020 2020 8010 48: 2f00 4001 0007 3fff 0010 56: 003f fc10 00fb 0110 0fff 0007 64: 0003 0078 0078 0078 0078 72: 001f 1706 0044 0040 80: 01fe 746b 7f61 4123 7469 bc41 4123 88: 407f 00ab 00ab fffe 80fe 96: 7b30 aea8 104: 5001 4ee0 5743 482f 112: 4018 120: 4018 128: 0021 16db 136: 0004 144: 152: 160: 168: 176: 184: 192: 200: 3031 208: 216: 101e 224: 232: 0001 1000 240: 248: 3aa5 pass0: WDC WD15EARS-00Z5B1 80.00A80 ATA-8 SATA 2.x device pass0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) protocol ATA/ATAPI-8 SATA 2.x device model WDC WD15EARS-00Z5B1 firmware revision 80.00A80 serial number WD-WMAVU1512579 WWN 50014ee05743482f cylinders 16383 heads 16 sectors/track 63 sector size logical 512, physical 512, offset 0 LBA supported 268435455 sectors LBA48 supported 2930277168 sectors PIO supported PIO4 DMA supported WDMA2 UDMA6 Feature Support EnableValue Vendor read ahead yes yes write cacheyes yes flush cacheyes yes overlapno Tagged Command Queuing (TCQ) no no Native Command Queuing (NCQ) yes 32 tags SMART yes yes microcode download yes yes security yes no power management yes yes advanced power management no no 0/0x00 automatic acoustic management yes no 254/0xFE128/0x80 media status notification no no power-up in Standbyyes no write-read-verify no no 0/0x0 unload no no free-fall no no data set management (TRIM) no Seems that the only possible way to use this disks is adding permanent (not like gnop) sector size emulation to some part of GEOM. -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
gpart and sector size
Hello. There is only one possibility to change sector size of physical disk (gnop -S 4096 ...). May be it is possible to add such possibility to gpart? e.g. gpart create -S 4096 -t gpt ad0? It will help all unlucky WD Advanced Format disks users. :-D -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
No, no. I mean that gpart should act like gnop presenting another sector size to user. I that possible at all? On 08.04.2010, at 17:36, Dag-Erling Smørgrav wrote: I don't quite see how that would work - do you mean gpart should configure a gnop? AFAIK there is no gnop label, so you can't set up a persistent gnop; you have to set it up manually at boot time every time, and there's a risk that the fs (or other layers higher up) will taste the underlying device instead of the gnop. -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
Ok, in case of GPT? :-) GPT implementation can be the simplest solution to this problem compared to implementing additional ATA commands to determine if disk is in Advanced Format. On 08.04.2010, at 18:09, Dag-Erling Smørgrav wrote: Alexey Tarasov m...@lexasoft.ru writes: I mean that gpart should act like gnop presenting another sector size to user. I that possible at all? That depends on the underlying partition scheme. My guess is no. (it all boils down to whether the desired logical sector size can somehow be recorded on-disk) -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
1) There is already an ATA command to report both physical and logical sector sizes, but the disk lies - it always reports 512/512. Advanced Format disks reports 512, but there is another command in ATA standard which can tell us if it uses 4k sector. 2) The disk may have already been formatted on a system that doesn't support 4k sectors, and may contain unaligned partitions and file systems, which won't be visible if we forcibly and unconditionally use 4k sectors. I mean that when I create *NEW* GPT scheme I can set up sector size emulation. It will never touch existing unaligned partitions. -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
http://www.wdc.com/wdproducts/library/WhitePapers/ENG/2579-771430.pdf References The ATA8-ACS and SBC-3 standards have provisions for a disk drive to report Advanced Format sector sizes and other performance optimization information. These standards are used for SATA, SAS, USB, and IEEE 1394 based interface technologies. On 08.04.2010, at 18:35, Dag-Erling Smørgrav wrote: Alexey Tarasov m...@lexasoft.ru writes: Advanced Format disks reports 512, but there is another command in ATA standard which can tell us if it uses 4k sector. Send me one and I'll look into it :) DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
Hello. Thank you for the information. In 8-STABLE snapshot 201002 diskinfo shows 512k sector size yet. I will try CURRENT tomorrow. On 08.04.2010, at 19:35, Dimitry Andric wrote: On 2010-04-08 17:24, Gary Jennejohn wrote: References The ATA8-ACS and SBC-3 standards have provisions for a disk drive to report Advanced Format sector sizes and other performance optimization information. These standards are used for SATA, SAS, USB, and IEEE 1394 based interface technologies. This is apparently the Long Physical Sector features set. The question is whether it's been implemented. Isn't this already done? At least it looks like it: http://svn.freebsd.org/viewvc/base?view=revisionrevision=198897 It might even have been MFC'd... :) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: gpart and sector size
I agree with you completely. Seems that support of this disks is already commited in CURRENT, will try it tomorrow. A better approach is to have tunables for geom_disk to do this. This should absolutely not be part of a partitioning tool. It violates everything there is to violate AFAICT. FYI, -- Alexey Tarasov (\__/) (='.'=) E[: | | | | :]З ()_() ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
