Re: Addition of lzma/xz compression to HEAD
Thanks, this is very useful. Fabien Le 11 mai 2010 à 21:31, Alexey Shuvaev a écrit : Hello! Just FYI: noticed addition of lzma directory to BSD.include.dist mtree file. Well, now it seems to work! /* Test file size 264 MiB */ [wep4035] ~ ll /usr/local/tinderbox/jails/9-amd64/9-amd64.tar -rw-r--r-- 1 root wheel 277209600 Apr 20 20:58 /usr/local/tinderbox/jails/9-amd64/9-amd64.tar /* Cache file in memory */ [wep4035] ~ cat /usr/local/tinderbox/jails/9-amd64/9-amd64.tar /dev/null /* 30 seconds to gzip it */ [wep4035] ~ time tar -cvzf 9-amd64.tar.tar.gz /usr/local/tinderbox/jails/9-amd64/9-amd64.tar tar: Removing leading '/' from member names a usr/local/tinderbox/jails/9-amd64/9-amd64.tar 30.043u 0.541s 0:15.32 199.6% 37+2093k 0+747io 0pf+0w /* 64 seconds to bzip2 it */ [wep4035] ~ time tar -cvjf 9-amd64.tar.tar.bz2 /usr/local/tinderbox/jails/9-amd64/9-amd64.tar tar: Removing leading '/' from member names a usr/local/tinderbox/jails/9-amd64/9-amd64.tar 63.454u 0.686s 0:32.09 199.8% 37+2108k 0+650io 1pf+0w /* And 140 seconds to xz it */ [wep4035] ~ time tar -cvJf 9-amd64.tar.tar.xz /usr/local/tinderbox/jails/9-amd64/9-amd64.tar tar: Removing leading '/' from member names a usr/local/tinderbox/jails/9-amd64/9-amd64.tar 277.625u 0.857s 2:19.26 199.9% 37+2092k 0+432io 0pf+0w /* Resulting sizes : */ [wep4035] ~ ll 9-amd64.tar.tar.* -rw-r--r-- 1 lexx lexx 84830128 May 11 21:07 9-amd64.tar.tar.bz2 -rw-r--r-- 1 lexx lexx 97667581 May 11 21:07 9-amd64.tar.tar.gz -rw-r--r-- 1 lexx lexx 56366908 May 11 21:10 9-amd64.tar.tar.xz /* 3.5 seconds to gunzip the file (mostly IO-limited) */ [wep4035] ~ cat 9-amd64.tar.tar.gz /dev/null [wep4035] ~ time tar -xvf 9-amd64.tar.tar.gz x usr/local/tinderbox/jails/9-amd64/9-amd64.tar 2.721u 0.747s 0:03.54 97.7% 42+2365k 3+2116io 0pf+0w [wep4035] ~ rm -R usr/ /* 18 seconds to bunzip2 it */ [wep4035] ~ cat 9-amd64.tar.tar.bz2 /dev/null [wep4035] ~ time tar -xvf 9-amd64.tar.tar.bz2 x usr/local/tinderbox/jails/9-amd64/9-amd64.tar 18.136u 0.999s 0:09.59 199.3% 37+2110k 1+2116io 0pf+0w [wep4035] ~ rm -R usr/ /* And only 10 seconds to xzdec it */ [wep4035] ~ cat 9-amd64.tar.tar.xz /dev/null [wep4035] ~ time tar -xvf 9-amd64.tar.tar.xz x usr/local/tinderbox/jails/9-amd64/9-amd64.tar 10.304u 0.771s 0:05.59 198.0% 38+2164k 3+2116io 0pf+0w [wep4035] ~ rm -R usr/ Thanks to all involved in bringing it to HEAD! Alexey. P.S. I'm not claiming any statistical validity of provided timings nor that the testing procedure is correct. It is just to show that tar in HEAD now works with lzma/xz compression. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: fsck unable to read disk sectors
On Wed, May 12, 2010 at 3:46 AM, Bernd Walter ti...@cicely7.cicely.de wrote: On Tue, May 11, 2010 at 10:15:13PM +0200, Alexander Best wrote: i've posted a log here which is pretty self explanatory: http://pastebin.com/tn3NiDDW On Tue, May 11, 2010 at 10:13 PM, Alexander Best alexbes...@uni-muenster.de wrote: the problem is getting more awkward. if i do `fsck /dev/label/rootfs` fsck complains that it cannot read a specific sector of my hdd as i mentioned before. if i run fsck on the device node directly using `fsck /dev/ada0p3` however, fsck succeeds. So this is not hardware it is bad partitioning. puh. that's a relief. but since smartd didn't complain about anything and dd if=/dev/ada0 of=/dev/null bs=1m reported no errors i kinda thought that my hdd wasn't the cause for this. what i did was to boot into single user mode with / being mounted read only. for some reason however fsck will check /dev/label/rootfs in write mode, but if i want fsck to check ada0p3 it will only do so in read mode. this looks like something is really broken. right now the only way to get the clean flag set on my hdd is to boot from a livefs cd and then run `fsck /dev/ada0p3` (again: `fsck /dev/label/rootfs` will NOT succeed). One of the typical problems users have is that they forget that adding a label takes one sector, so the labeled device is smaller. This is no problem if you create the filesystem on the labeled drive, but often enough people add the label after creating the filesystem. Everything seems to work fine until the FS decides to use that special sector. I wouldn't add a label for ufs anyway, since UFS has labeling itself, which is also handled by glabel module and doesn't require extra space. Just setup the ufs label with tunefs -L and use the resulting /dev/ufs/... device. You only need extra label for swap, but this is not problem, since it has no persistent ondisk structures. thanks a lot for the explanation. i never would have thought about that. since / already has a ufs label i'll simply change fstab to use /dev/ufs/rootfs as /, then boot into single user mode and remove the glabel for ada0p3. i followed the steps described in the gpart(8) manual to create my partition layout. maybe the manual should state that if one wants to create a glabel it should happen before creating a filesystem? this is the output of `glabel status` btw: Name Status Components label/boot N/A ada0p1 gptid/e52df583-e446-11de-bb92-000fb58207c8 N/A ada0p1 label/swap N/A ada0p2 label/rootfs N/A ada0p3 cheers. On Tue, Mar 30, 2010 at 2:08 AM, Paul B Mahol one...@gmail.com wrote: On 3/29/10, Alexander Best alexbes...@wwu.de wrote: hi there, when doing fsck on my / fs i get this error: Cannot Read BLK. 471617640 and The Following Disk Sectors could not be read: 471617643. after this message the partition gets marked dirty. i performed the following steps to verify the problem: 1) dd if=/dev/ada0 of=/dev/null bs=1m 2) fsck / under freebsd 7 3) mount -u -o snapshot /.snap/snapshot1 / fsck_ffs /.snap/snapshot1 all three steps showed no problem with that harddrive whatsoever. also smartd doesn't complain about anything. i'm running HEAD (r205860) on amd64. this is the output of `dmesg -a|grep ada0`: ada0 at ahcich2 bus 0 scbus3 target 0 lun 0 ada0: SAMSUNG SP2504C VT100-50 ATA-7 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 238474MB (488395055 512 byte sectors: 16H 63S/T 16383C) Last time I tried ahci on dead disk it did not complained at all (usually I get dead LBA listed on console). -- B.Walter be...@bwct.de http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. -- Alexander Best ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [PATCH:] psm(4) IntelliMouse Explorer KVM hack breaks my mouse
On Tue, 11 May 2010 20:41:41 +0200 Juergen Lock n...@jelal.kn-bremen.de wrote: So now I made a patch that allows disabling that KVM hack via device hints, appended below. (hint.psm.0.flags=0x1 - or do you guys think the hack should be disabled by default instead?) Don't change the behavior, leave it enabled by default. For POLA. -- Gary Jennejohn ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: HEADS UP: 64-bit quotas going in to head today
Kostik Belousov kostik...@gmail.com writes: Dag-Erling Smørgrav d...@des.no writes: It adds quite a bit of code to pretty much every UFS VOP. No, it does not. Essentially, it adds one or two function calls per vop that allocate or deallocate blocks or inodes, and the function bodies verify two array members and return if those are NULL. Read ufs_vnops.c, count the number of #ifdef QUOTA. There's quite a bit more than just one or two function calls per vop. Quite a bit of locking going on, too, e.g. in ufs_accessx(). DES -- Dag-Erling Smørgrav - d...@des.no ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: HEADS UP: 64-bit quotas going in to head today
On Wed, May 12, 2010 at 02:05:06PM +0200, Dag-Erling Sm??rgrav wrote: Kostik Belousov kostik...@gmail.com writes: Dag-Erling Sm??rgrav d...@des.no writes: It adds quite a bit of code to pretty much every UFS VOP. No, it does not. Essentially, it adds one or two function calls per vop that allocate or deallocate blocks or inodes, and the function bodies verify two array members and return if those are NULL. Read ufs_vnops.c, count the number of #ifdef QUOTA. There's quite a bit more than just one or two function calls per vop. Quite a bit of locking going on, too, e.g. in ufs_accessx(). I did read the code when I fine-locked quotas. I stand on my position that it is one or two accounting calls per vop that do nothing in case of disabled quotas. ufs_accessx() path is seldomly executed. You mostly have to open file read-write if you are going to modify inode, and vn_open_cred() already uses exclusive locking there. It is calls like chmod(2) that are catched at this point, performing the operation by path instead of fd. pgp7syKzzvjFV.pgp Description: PGP signature
Re: LOR: ufs vs bufwait
On Mon, 10.05.2010 at 22:53:32 +0200, Attilio Rao wrote: 2010/5/10 Peter Jeremy peterjer...@acm.org: On 2010-May-08 12:20:05 +0200, Ulrich Spörlein u...@spoerlein.net wrote: This LOR also is not yet listed on the LOR page, so I guess it's rather new. I do use SUJ. lock order reversal: 1st 0xc48388d8 ufs (ufs) @ /usr/src/sys/kern/vfs_lookup.c:502 2nd 0xec0fe304 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_softdep.c:11363 3rd 0xc49e56b8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2091 I'm seeing exactly the same LOR (and subsequent deadlock) on a recent -current without SUJ. I think this LOR was reported since a long time. The deadlock may be new and someway related to the vm_page_lock work (if not SUJ). I was not able to reproduce this with a kernel prior to SUJ, a kernel just after SUJ went it shows this deadlock or infinite loop ... Now it might be that the SUJ kernel only increases the pressure so it happens during a systems uptime. It does not seem directly related to actually using SUJ on a volume, as I could reproduce it with SU only, too. I will try to get a hang not involving GELI and also re-do my tests when the volumes have neither SUJ nor SU enabled, which led to 10-20s hangs of the system IIRC. It seems SU/SUJ then only prolongs these hangs ad infinitum. I'll be back next week with new results here Uli ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
Check to make sure the links are all full-duplex. We started seeing bad performance with the em(4) driver on our HP Proliant 360DL G5's using 1000Mbits. It turned out that switch was setting it's port to half-duplex and the emX interface was following suit. HTH, Patrick joe wrote: On 05/08/2010 01:31 PM, Jack Vogel wrote: Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did pull the server and brought it home so that i can test more things quicker. I am going to try using a raid card instead of the onboard sata ports and see if i still encounter the same problem. I would love any suggestions you may have on where to go from here to figure out where the problem might be. joe ___ freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org I think it might have something to so with the nics / switch, and their features. I brought the box home, plugged into my gb switch, and i am able to FTP data to the server at around 35MB/sec. I dont know what would cause this other than some sort of issue with the the 3 different types of nics and the switch i am using. Any suggestions? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
dc(1) -e 6 2 / p is still broken as of r207919
Hello!
Just to remind that still:
~ dc -e 6 2 / p
Segmentation fault (core dumped)
This was already mentioned on this list:
http://lists.freebsd.org/pipermail/freebsd-current/2010-April/016560.html
and there is a patch proposed in the same thread:
http://lists.freebsd.org/pipermail/freebsd-current/2010-April/016603.html
Note, however, that reverting r203438 also fixes the problem (gabor@ CC-ed),
so I'm not sure what is the right way to fix it.
Attached is slightly modified reverse patch to revert 203438.
Thanks,
Alexey.
--- dc.c2010/01/20 21:30:52 202719
+++ dc.c2010/02/03 19:13:41 203438
@@ -82,15 +82,7 @@
{
int ch;
bool extended_regs = false, preproc_done = false;
- char*buf;
- if ((buf = strdup()) == NULL)
- err(1, NULL);
-
- init_bmachine(extended_regs);
- setlinebuf(stdout);
- setlinebuf(stderr);
-
/* accept and ignore a single dash to be 4.4BSD dc(1) compatible */
while ((ch = getopt_long(argc, argv, e:f:Vx, long_options, NULL)) !=
-1) {
switch (ch) {
@@ -123,6 +115,10 @@
argc -= optind;
argv += optind;
+ init_bmachine(extended_regs);
+ setlinebuf(stdout);
+ setlinebuf(stderr);
+
if (argc 1)
usage();
if (argc == 1) {
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: LOR: ufs vs bufwait
On Wed, 12 May 2010, Ulrich Sp?rlein wrote: On Mon, 10.05.2010 at 22:53:32 +0200, Attilio Rao wrote: 2010/5/10 Peter Jeremy peterjer...@acm.org: On 2010-May-08 12:20:05 +0200, Ulrich Sp?rlein u...@spoerlein.net wrote: This LOR also is not yet listed on the LOR page, so I guess it's rather new. I do use SUJ. lock order reversal: 1st 0xc48388d8 ufs (ufs) @ /usr/src/sys/kern/vfs_lookup.c:502 2nd 0xec0fe304 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_softdep.c:11363 3rd 0xc49e56b8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2091 I'm seeing exactly the same LOR (and subsequent deadlock) on a recent -current without SUJ. I think this LOR was reported since a long time. The deadlock may be new and someway related to the vm_page_lock work (if not SUJ). I was not able to reproduce this with a kernel prior to SUJ, a kernel just after SUJ went it shows this deadlock or infinite loop ... Now it might be that the SUJ kernel only increases the pressure so it happens during a systems uptime. It does not seem directly related to actually using SUJ on a volume, as I could reproduce it with SU only, too. I will try to get a hang not involving GELI and also re-do my tests when the volumes have neither SUJ nor SU enabled, which led to 10-20s hangs of the system IIRC. It seems SU/SUJ then only prolongs these hangs ad infinitum. I think Peter Holm also saw this once while we were testing SUJ and reproduced ~30 second hangs with stock sources. At this point we need to brainstorm ideas for adding debugging instrumentation and come up with the quickest possible repro. It would probably be good to add some KTR tracing and log that when it wedges. The core I looked at was hung in bufwait. Is there any cpu activity or io activity when things hang? You'll prboably have to keep iostat/vmstat in memory to find out so they don't try to fault in pages once things are hung. Thanks, Jeff I'll be back next week with new results here Uli ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: LOR: ufs vs bufwait
2010/5/12 Jeff Roberson jrober...@jroberson.net: On Wed, 12 May 2010, Ulrich Sp?rlein wrote: On Mon, 10.05.2010 at 22:53:32 +0200, Attilio Rao wrote: 2010/5/10 Peter Jeremy peterjer...@acm.org: On 2010-May-08 12:20:05 +0200, Ulrich Sp?rlein u...@spoerlein.net wrote: This LOR also is not yet listed on the LOR page, so I guess it's rather new. I do use SUJ. lock order reversal: 1st 0xc48388d8 ufs (ufs) @ /usr/src/sys/kern/vfs_lookup.c:502 2nd 0xec0fe304 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_softdep.c:11363 3rd 0xc49e56b8 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2091 I'm seeing exactly the same LOR (and subsequent deadlock) on a recent -current without SUJ. I think this LOR was reported since a long time. The deadlock may be new and someway related to the vm_page_lock work (if not SUJ). I was not able to reproduce this with a kernel prior to SUJ, a kernel just after SUJ went it shows this deadlock or infinite loop ... Now it might be that the SUJ kernel only increases the pressure so it happens during a systems uptime. It does not seem directly related to actually using SUJ on a volume, as I could reproduce it with SU only, too. I will try to get a hang not involving GELI and also re-do my tests when the volumes have neither SUJ nor SU enabled, which led to 10-20s hangs of the system IIRC. It seems SU/SUJ then only prolongs these hangs ad infinitum. I think Peter Holm also saw this once while we were testing SUJ and reproduced ~30 second hangs with stock sources. At this point we need to brainstorm ideas for adding debugging instrumentation and come up with the quickest possible repro. It would probably be good to add some KTR tracing and log that when it wedges. The core I looked at was hung in bufwait. Is there any cpu activity or io activity when things hang? You'll prboably have to keep iostat/vmstat in memory to find out so they don't try to fault in pages once things are hung. I think I also have some reports about deadlock on unmount -f (not specific to UFS) that seems to me still the same buffer cache async deadlock. I will forward you the traces in a separate e-mail (Peter got to reproduce it with KTR on). Attilio -- Peace can only be achieved by understanding - A. Einstein ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
