Re: 6TB partition, Data only 2TB - aka When you haven't hit the "usual" problem
here is the info requested, if that helps anyone. # uname -a Linux SX20S 4.3.0-040300rc7-generic #201510260712 SMP Mon Oct 26 11:27:59 UTC 2015 i686 i686 i686 GNU/Linux # aptitude show btrfs-tools Package: btrfs-tools State: installed Automatically installed: no Version: 4.2.1+ppa1-1~ubuntu15.10.1 # btrfs --version btrfs-progs v4.2.1 # btrfs fi show Media Label: 'Media' uuid: b397b7ef-6754-4ba4-8b1a-fbf235aa1cf8 Total devices 1 FS bytes used 1.92TiB devid1 size 5.46TiB used 1.93TiB path /dev/sdd1 btrfs-progs v4.2.1 # btrfs fi usage Media Overall: Device size: 5.46TiB Device allocated: 1.93TiB Device unallocated: 3.52TiB Device missing: 0.00B Used: 1.93TiB Free (estimated): 3.53TiB (min: 1.76TiB) Data ratio: 1.00 Metadata ratio: 2.00 Global reserve: 512.00MiB (used: 0.00B) Data,single: Size:1.92TiB, Used:1.92TiB /dev/sdd1 1.92TiB Metadata,single: Size:8.00MiB, Used:0.00B /dev/sdd1 8.00MiB Metadata,DUP: Size:5.00GiB, Used:3.32GiB /dev/sdd1 10.00GiB System,single: Size:4.00MiB, Used:0.00B /dev/sdd1 4.00MiB System,DUP: Size:8.00MiB, Used:224.00KiB /dev/sdd1 16.00MiB Unallocated: /dev/sdd1 3.52TiB # btrfs-show-super /dev/sdd1 superblock: bytenr=65536, device=/dev/sdd1 - csum 0xae174f16 [match] bytenr 65536 flags 0x1 ( WRITTEN ) magic _BHRfS_M [match] fsid b397b7ef-6754-4ba4-8b1a-fbf235aa1cf8 label Media generation 11983 root 34340864 sys_array_size 226 chunk_root_generation 11982 root_level 1 chunk_root 21135360 chunk_root_level 1 log_root 0 log_root_transid 0 log_root_level 0 total_bytes 6001173463040 bytes_used 2115339448320 sectorsize 4096 nodesize 16384 leafsize 16384 stripesize 4096 root_dir 6 num_devices 1 compat_flags 0x0 compat_ro_flags 0x0 incompat_flags 0x61 ( MIXED_BACKREF | BIG_METADATA | EXTENDED_IREF ) csum_type 0 csum_size 4 cache_generation 11983 uuid_tree_generation 11983 dev_item.uuid 819e1c8a-5e55-4992-81d3-f22fdd088dc9 dev_item.fsid b397b7ef-6754-4ba4-8b1a-fbf235aa1cf8 [match] dev_item.type 0 dev_item.total_bytes 6001173463040 dev_item.bytes_used 2124972818432 dev_item.io_align 4096 dev_item.io_width 4096 dev_item.sector_size 4096 dev_item.devid 1 dev_item.dev_group 0 dev_item.seek_speed 0 dev_item.bandwidth 0 dev_item.generation 0 I did mount Media -o enospc_debug and now mount shows: /dev/sdd1 on /media/cheater/Media type btrfs (rw,nosuid,nodev,enospc_debug,_netdev) On Wed, Dec 30, 2015 at 11:13 PM, Chris Murphy <li...@colorremedies.com> wrote: > kernel and btrfs-progs versions > and output from: > 'btrfs fi show ' > 'btrfs fi usage ' > 'btrfs-show-super ' > 'df -h' > > Then umount the volume, and mount with option enospc_debug, and try to > reproduce the problem, then include everything from dmesg from the > time the volume was mounted. > > -- > Chris Murphy On Sat, Jan 2, 2016 at 3:09 AM, cheater00 . <cheate...@gmail.com> wrote: > I have been unable to reproduce so far. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 6TB partition, Data only 2TB - aka When you haven't hit the "usual" problem
I have been unable to reproduce so far. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
6TB partition, Data only 2TB - aka When you haven't hit the "usual" problem
Hi, I have a 6TB partition here, it filled up while still just under 2TB were on it. btrfs fi df showed that Data is 1.92TB: Data, single: total=1.92TiB, used=1.92TiB System, DUP: total=8.00MiB, used=224.00KiB System, single: total=4.00MiB, used=0.00B Metadata, DUP: total=5.00GiB, used=3.32GiB Metadata, single: total=8.00MiB, used=0.00B GlobalReserve, single: total=512.00MiB, used=0.00B btrfs fs resize max . did nothing, I also tried resize -1T and resize +1T and that did nothing as well. On IRC I was directed to this: https://btrfs.wiki.kernel.org/index.php/FAQ#if_your_device_is_large_.28.3E16GiB.29 "When you haven't hit the "usual" problem If the conditions above aren't true (i.e. there's plenty of unallocated space, or there's lots of unused metadata allocation), then you may have hit a known but unresolved bug. If this is the case, please report it to either the mailing list, or IRC. In some cases, it has been possible to deal with the problem, but the approach is new, and we would like more direct contact with people experiencing this particular bug." What do I do now? It's kind of important to me to get that free space. I'm really jonesing for that free space. Thanks. (btw, so far I haven't been able to follow up on that unrelated thread from a while back. But I hope to be able to do that sometime in January.) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
I am getting a sata dock for my laptop next week. Until then, is it possible to perform an action in btrfs (like rm which seems to trigger the issue) and make it log what exactly it's doing? On Thu, Oct 29, 2015 at 9:01 PM, Austin S Hemmelgarn <ahferro...@gmail.com> wrote: > On 2015-10-29 11:49, cheater00 . wrote: >> >> Hi Austin, >> seek times are fine, but this literally freezes my computer for a >> split second. I've had to re-type this email twice because the freezes >> meant letters I typed would not arrive on the screen. >> USB disks are so common they should not be having issues. > > That's debatable. USB is commonly used because it's almost impossible to > find a system that doesn't have it, not because it's reliable. The original > intent was for it to be used for stuff like mice and keyboards, so it was > designed with low-latency and fair scheduling in mind, both of which really > hurt performance of bulk data storage devices. >> >> I have 4.3.0-040300rc7-generic #201510260712 which is just three days old. > > That should be perfectly recent enough, although FWIW, the official version > of 4.3 should be out this Sunday. >> >> >> Please advise. Isn't it better to *not* use a vm to debug this? > > That depends. For something like this, it could go either way. I just use > a VM because that's what I always use, because it's nice not crashing your > system when trying to debug a kernel panic. >> >> BTW, if we are talking about slow speed making things worse, I could >> try downgrading the cable to usb2. >> Is there a standard virtualbox VM that I could use? > > In general, it's pretty easy to set something like Ubuntu up in VirtualBox, > the install is essentially identical to regular hardware aside from the > initial setup of the VM itself. The documentation for VirtualBox is really > good, if you've never used virtualization before, it's definitely worth > reading. >> >> I'll download Gentoo in the meantime. I have never used it. I'm >> getting the "minimal installation cd" from 29th september. >> >> http://distfiles.gentoo.org/releases/x86/autobuilds/20150929/install-x86-minimal-20150929.iso > > I meant by no means that you needed to use Gentoo, I only mentioned it > because it's what I use (which in turn is because that's what I use on just > about everything except stuff like the Raspberry Pi or the BeagleBoard). If > you just want to debug this and then be done with it, I would actually > advise against using Gentoo, it takes a lot of effort to get a system up and > running with it, and it's very involved to maintain compared to Ubuntu. On > the other hand though, if you are willing to learn to use it, it's one of > the most highly customizable Linux distros out there, and can have > noticeably better performance than more generic distros (FWIW, it's also one > of the last big distros that doesn't force systemd on it's users by > default). > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
Hi Liu, after talking with Holger I believe turning off COW on this FS will work to alleviate this issue. However, even with COW on, btrfs shouldn't be making my computer freeze every 5 seconds... especially while the disk is written to at mere tens of kilobytes per second. It's not even the disk holding the system. I consider this a pretty bad bug... should we go on with trying to reproduce a minimum case? How would I go about this? Thanks On Tue, Oct 27, 2015 at 4:26 PM, Austin S Hemmelgarn <ahferro...@gmail.com> wrote: > On 2015-10-27 10:43, cheater00 . wrote: >> >> I have remounted without autodefrag and the issue keeps on happening. > > OK, that at least narrows things down further. My guess is the spikes are > utorrent getting a bunch of blocks at once from one place, and then trying > to write all of them at the same time, which could theoretically cause a > latency spike on any filesystem, and BTRFS may just be making it worse. >> >> >> On Tue, Oct 27, 2015 at 3:30 PM, cheater00 . <cheate...@gmail.com> wrote: >>> >>> Feel free to suggest a good 1.5m USB3 cable, too. Let's get rid of all >>> the unknowns. > > When it comes to external cables, I've had really good success with Amazon's > 'Amazon Basics' branded stuff. It's usually some of the best quality you > can find for the price. The 'Cable Matters' and 'Pluggable' brands also > tend to be really good quality for the price. >>> >>> On Tue, Oct 27, 2015 at 3:26 PM, cheater00 . <cheate...@gmail.com> wrote: >>>> >>>> If you can suggest a dual (or better yet quad) USB3 bay that can be >>>> bought on Amazon, I'll buy it now, and once that arrives, we can be >>>> sure it's not the JMicron chipset. > > I don't really have any suggestions here. Usually when I hook up an > external drive, it's to recover data from a friends computer, so I don't > typically use a enclosure, but just use a simple adapter cable. I would > suggest looking for one advertising 'UAS' or 'UASP' support, as that's a > relatively new standard for USB storage devices, and newer hardware should > be more reliable. It's also notoriously hard to determine what chipset a > given model of external drive bay has (there are people I know who bought > multiples of the same model and each one had a different chipset > internally), and to complicate matters, quite often the exact same hardware > gets marketed under half a dozen different names. JMicron is popular > because their chips are comparatively inexpensive, and while I've not had > good results with them, that doesn't mean that they are all bad (especially > considering that they are highly configurable based on how they are wired > into the device, and not everyone who designs hardware around them properly > understands the implications of some of the features). >>>> >>>> >>>> On Tue, Oct 27, 2015 at 3:22 PM, cheater00 . <cheate...@gmail.com> >>>> wrote: >>>>> >>>>> The (dual) HDD bay and the chipset are, according to lsusb: >>>>> Bus 002 Device 005: ID 152d:0551 JMicron Technology Corp. / JMicron >>>>> USA Technology Corp. >>>>> Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub >>>>> >>>>> Not sure how to find out specific model numbers? I could open up the >>>>> bay. OK I'll open up the bay. >>>>> Good thing I have just the right screwdriver. It's a JMS551, and just >>>>> for records sake, here's the manufacture info: >>>>> >>>>> JMS551 >>>>> 1120 LGAA2 A >>>>> 572QV0024 >>>>> >>>>> The laptop manual says it's either "Intel HM65 Express chipset with >>>>> NEC USB 3.0 (select models only)" or "Intel HM65 Express chipset". >>>>> Here are technical documents for my model: >>>>> Manual: http://docdro.id/hG627JM >>>>> "Intel chipset datasheet": http://docdro.id/yKRupYO >>>>> Service guide: http://docdro.id/AuDgUdE >>>>> Service guide, alt. ver.: http://docdro.id/WwQRpsH > > From what I can tell, you've got the one with the NEC USB 3.0 chip, I'm > pretty cure that the HM65 doesn't have USB 3.0 itself. FWIW, I've never > personally had issues with NEC's USB 3.0 chips, but I've not had much > experience using systems with them either. >>>>> >>>>> >>>>> FWIW I'm using one of the USB3 ports on the left. The ones on the >>>>> right are USB2. >>>>> >>>>> I've never used docdro.id so if
Re: Bad fs performance, IO freezes
Hi Austin, seek times are fine, but this literally freezes my computer for a split second. I've had to re-type this email twice because the freezes meant letters I typed would not arrive on the screen. USB disks are so common they should not be having issues. I have 4.3.0-040300rc7-generic #201510260712 which is just three days old. Please advise. Isn't it better to *not* use a vm to debug this? BTW, if we are talking about slow speed making things worse, I could try downgrading the cable to usb2. Is there a standard virtualbox VM that I could use? I'll download Gentoo in the meantime. I have never used it. I'm getting the "minimal installation cd" from 29th september. http://distfiles.gentoo.org/releases/x86/autobuilds/20150929/install-x86-minimal-20150929.iso On Thu, Oct 29, 2015 at 3:00 PM, Austin S Hemmelgarn <ahferro...@gmail.com> wrote: > On 2015-10-29 09:03, cheater00 . wrote: >> >> Hi Liu, >> after talking with Holger I believe turning off COW on this FS will >> work to alleviate this issue. However, even with COW on, btrfs >> shouldn't be making my computer freeze every 5 seconds... especially >> while the disk is written to at mere tens of kilobytes per second. >> It's not even the disk holding the system. I consider this a pretty >> bad bug... should we go on with trying to reproduce a minimum case? >> How would I go about this? > > > Well, COW can cause some pretty unexpected behavior for some use cases. If > you have a big disk (I think I remember you saying it was larger than 1TB), > then COW can cause some pretty significant seek times because of how it > works. With the current state of BTRFS, I wouldn't personally consider > running BTRFS on anything bigger than 256G with a non-zero seek time with > COW turned on, because large rewrites would have the potential to cause > horrifically long seek times just for a RMW cycle on a single block, and > this is in turn part of why database files and virtual-machine images tend > to be pathological use cases for BTRFS. > > I do agree that this kind of thing is a bug, but it's not something that > causes data corruption, which means that it is slightly lower priority as > far as most people are concerned. Reproducing it might be tricky also, > because I'd be willing to bet that things get better to the point of it > being almost unnoticeable with an internal disk (USB is horrible when it > comes to block storage performance, and has all kinds of potential > reliability issues). > > Normally, when I try to go about reproducing something like this, I use a > virtual machine running the most recent stable version of the Linux kernel, > usually with a minimalistic Gentoo installation (although a clean install of > pretty much any distro works fine). There are a couple of reasons I use > such a setup: > 1. Using a clean install provides a well defined initial state, making it > easier for other people to reproduce any results. > 2. Using the most recent stable kernel available (usually) eliminates the > chances of old bugs causing issues. > 3. Using a VM means that your disk access will be slower, which will visibly > accentuate any kind of performance issues. > 4. Using a VM also means that it is very easy to safely generate crash dumps > and simulate data corruption for testing purposes, and makes it easier to > experiment with different parameters (for example, UP versus SMP, or > different amounts of RAM). > > If you do decide to go this route, my suggestion would be to use VirtualBox > unless you have significant experience with some other hypervisor, as it's > one of the easiest to learn to use (I usually use Xen or QEMU, but both > require significant effort to set up initially, and are decidedly > non-trivial to learn), and learning to debug stuff like this is itself not > an easy task. > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
No, sadly the intervals don't seem to be regular like that. On Tue, Oct 27, 2015 at 7:39 AM, Duncan <1i5t5.dun...@cox.net> wrote: > cheater00 . posted on Tue, 27 Oct 2015 03:00:05 +0100 as excerpted: > >> currently my computer freezes every several seconds for half a second or >> so. Using it feels like I'm playing musical chairs with the kernel. >> I have just one download happening on utorrent right now - this is what >> the graph looks like: >> http://i.imgur.com/LqhMtrJ.png and every time a new spike happens, a >> freeze happens just before that... that's the only time those freezes >> happen, too. > > Is it perchance every 30 seconds? (The graph seems to indicate more like > 50 second cycles, but...) > > Because that's btrfs' normal commit time. If it is, there's a mount > option to change the commit time (the wiki says commit=N, N=30 by > default, since kernel 3.12). You could try fiddling with that, say > setting it to 15 or 60, not necessarily to fix the problem (tho 60 > seconds might increase the problem while making it less frequent, while > 15 might make it more frequent but less of a problem), but to see if the > cycle changes with the commit time option, or stays @ 30 seconds. > > -- > Duncan - List replies preferred. No HTML msgs. > "Every nonfree program has a lord, a master -- > and if you use the program, he is your master." Richard Stallman > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
I'll just add nodatacow to the fstab, because the whole disk is for downloads. There actually is compression happening? On Tue, Oct 27, 2015 at 4:01 PM, Holger Hoffstätte <holger.hoffstae...@googlemail.com> wrote: > On 10/27/15 15:43, cheater00 . wrote: >> I have remounted without autodefrag and the issue keeps on happening. > > At the risk of stating the obvious, the simplest thing is to stop using > COW with compression for torrents. It's fundamentally not useful to have > many small in-place writes, irregular compression and not encounter insane > levels of fragmentation, which - at least in the current state of btrfs - > is known to cause long stalls. Some fixes for that will be in 4.4. > > Until then just do yourself a favor, stop doing the wrong thing and put > torrent downloads into a directory where they won't get COWed. > > See the wiki at: > https://btrfs.wiki.kernel.org/index.php/FAQ#Can_copy-on-write_be_turned_off_for_data_blocks.3F > for more. > > -h > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
Can I have nodatacow but still have checksumming? On Tue, Oct 27, 2015 at 4:05 PM, cheater00 . <cheate...@gmail.com> wrote: > I'll just add nodatacow to the fstab, because the whole disk is for downloads. > There actually is compression happening? > > On Tue, Oct 27, 2015 at 4:01 PM, Holger Hoffstätte > <holger.hoffstae...@googlemail.com> wrote: >> On 10/27/15 15:43, cheater00 . wrote: >>> I have remounted without autodefrag and the issue keeps on happening. >> >> At the risk of stating the obvious, the simplest thing is to stop using >> COW with compression for torrents. It's fundamentally not useful to have >> many small in-place writes, irregular compression and not encounter insane >> levels of fragmentation, which - at least in the current state of btrfs - >> is known to cause long stalls. Some fixes for that will be in 4.4. >> >> Until then just do yourself a favor, stop doing the wrong thing and put >> torrent downloads into a directory where they won't get COWed. >> >> See the wiki at: >> https://btrfs.wiki.kernel.org/index.php/FAQ#Can_copy-on-write_be_turned_off_for_data_blocks.3F >> for more. >> >> -h >> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
I have remounted without autodefrag and the issue keeps on happening. On Tue, Oct 27, 2015 at 3:30 PM, cheater00 . <cheate...@gmail.com> wrote: > Feel free to suggest a good 1.5m USB3 cable, too. Let's get rid of all > the unknowns. > > On Tue, Oct 27, 2015 at 3:26 PM, cheater00 . <cheate...@gmail.com> wrote: >> If you can suggest a dual (or better yet quad) USB3 bay that can be >> bought on Amazon, I'll buy it now, and once that arrives, we can be >> sure it's not the JMicron chipset. >> >> On Tue, Oct 27, 2015 at 3:22 PM, cheater00 . <cheate...@gmail.com> wrote: >>> The (dual) HDD bay and the chipset are, according to lsusb: >>> Bus 002 Device 005: ID 152d:0551 JMicron Technology Corp. / JMicron >>> USA Technology Corp. >>> Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub >>> >>> Not sure how to find out specific model numbers? I could open up the >>> bay. OK I'll open up the bay. >>> Good thing I have just the right screwdriver. It's a JMS551, and just >>> for records sake, here's the manufacture info: >>> >>> JMS551 >>> 1120 LGAA2 A >>> 572QV0024 >>> >>> The laptop manual says it's either "Intel HM65 Express chipset with >>> NEC USB 3.0 (select models only)" or "Intel HM65 Express chipset". >>> Here are technical documents for my model: >>> Manual: http://docdro.id/hG627JM >>> "Intel chipset datasheet": http://docdro.id/yKRupYO >>> Service guide: http://docdro.id/AuDgUdE >>> Service guide, alt. ver.: http://docdro.id/WwQRpsH >>> >>> FWIW I'm using one of the USB3 ports on the left. The ones on the >>> right are USB2. >>> >>> I've never used docdro.id so if it's not good let me know where to >>> upload the PDFs to. >>> >>> autodefrag is on, yes. But I have been having issues before turning it >>> on - I turned it on as a measure towards fixing the issues. I will >>> turn it off and remount, then report. But I don't think that should be >>> it. As you see the transfer speeds are minimal. They're *all* that's >>> happening on the disk. Right now that's under 100 KB/sec and I'm still >>> getting freezes albeit less. Also why would I be getting freezes when >>> the transfer speeds jump up - just for them to drop again? Hmm, maybe >>> utorrent has some sort of scheduler that gets preempted while the >>> spike is happening, and some algorithm in it gets the wrong idea and >>> turns some sort of flow control on, because it thinks it's hit some >>> sort of physical transfer speed barrier. Also notice the upload and >>> download both scale together, but that just might be how torrent >>> works, maybe it just tries to be fair i.e. only uploads as much as it >>> downloaded (scaled by a constant). >>> >>> The system is 32 bit because I installed ubuntu 6 one day and just >>> kept on upgrading. I keep on telling myself I'll update to 64 bits, >>> one of these days. But this laptop only has 8 gigs of ram, so no real >>> reason to upgrade to 64 bit anyways. It's not like I need firefox to >>> be able to eat 8 gb of ram whereas right now it can only eat 4. There >>> is no simple upgrade path that I know of so it's either a fresh >>> install or doing something like this: http://www.ewan.cc/?q=node/132 >>> -- I keep telling myself /one of these days/... >>> >>> On Tue, Oct 27, 2015 at 2:30 PM, Austin S Hemmelgarn >>> <ahferro...@gmail.com> wrote: >>>> On 2015-10-27 09:00, Henk Slager wrote: >>>>> >>>>> I don't have a lot experience with autodefrag, but as indicated by >>>>> Austin, expect a lot of full rewrites of files that are relatively >>>>> slowly filled up by a torrent client, starting with a sparse file. So >>>>> 1st advice would be to remove this option and run it as crontask at >>>>> particular times. >>>>> >>>>> What SATA-USB bridge is between the harddisk and the PC motherboard ? >>>> >>>> I hadn't thought of this, but the specific adapter being used for the disk >>>> can have a lot of impact on how it preforms. I've personally had lots of >>>> issues with JMicron chipsets (ranging from latency issues like what you are >>>> seeing to sever data corruption), but have found that ASMedia ones tend to >>>> be pretty much rock solid reliable and have good performance (although I >>>> think they only do USB 3.0 ad
Re: Bad fs performance, IO freezes
The (dual) HDD bay and the chipset are, according to lsusb: Bus 002 Device 005: ID 152d:0551 JMicron Technology Corp. / JMicron USA Technology Corp. Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Not sure how to find out specific model numbers? I could open up the bay. OK I'll open up the bay. Good thing I have just the right screwdriver. It's a JMS551, and just for records sake, here's the manufacture info: JMS551 1120 LGAA2 A 572QV0024 The laptop manual says it's either "Intel HM65 Express chipset with NEC USB 3.0 (select models only)" or "Intel HM65 Express chipset". Here are technical documents for my model: Manual: http://docdro.id/hG627JM "Intel chipset datasheet": http://docdro.id/yKRupYO Service guide: http://docdro.id/AuDgUdE Service guide, alt. ver.: http://docdro.id/WwQRpsH FWIW I'm using one of the USB3 ports on the left. The ones on the right are USB2. I've never used docdro.id so if it's not good let me know where to upload the PDFs to. autodefrag is on, yes. But I have been having issues before turning it on - I turned it on as a measure towards fixing the issues. I will turn it off and remount, then report. But I don't think that should be it. As you see the transfer speeds are minimal. They're *all* that's happening on the disk. Right now that's under 100 KB/sec and I'm still getting freezes albeit less. Also why would I be getting freezes when the transfer speeds jump up - just for them to drop again? Hmm, maybe utorrent has some sort of scheduler that gets preempted while the spike is happening, and some algorithm in it gets the wrong idea and turns some sort of flow control on, because it thinks it's hit some sort of physical transfer speed barrier. Also notice the upload and download both scale together, but that just might be how torrent works, maybe it just tries to be fair i.e. only uploads as much as it downloaded (scaled by a constant). The system is 32 bit because I installed ubuntu 6 one day and just kept on upgrading. I keep on telling myself I'll update to 64 bits, one of these days. But this laptop only has 8 gigs of ram, so no real reason to upgrade to 64 bit anyways. It's not like I need firefox to be able to eat 8 gb of ram whereas right now it can only eat 4. There is no simple upgrade path that I know of so it's either a fresh install or doing something like this: http://www.ewan.cc/?q=node/132 -- I keep telling myself /one of these days/... On Tue, Oct 27, 2015 at 2:30 PM, Austin S Hemmelgarn <ahferro...@gmail.com> wrote: > On 2015-10-27 09:00, Henk Slager wrote: >> >> I don't have a lot experience with autodefrag, but as indicated by >> Austin, expect a lot of full rewrites of files that are relatively >> slowly filled up by a torrent client, starting with a sparse file. So >> 1st advice would be to remove this option and run it as crontask at >> particular times. >> >> What SATA-USB bridge is between the harddisk and the PC motherboard ? > > I hadn't thought of this, but the specific adapter being used for the disk > can have a lot of impact on how it preforms. I've personally had lots of > issues with JMicron chipsets (ranging from latency issues like what you are > seeing to sever data corruption), but have found that ASMedia ones tend to > be pretty much rock solid reliable and have good performance (although I > think they only do USB 3.0 adapters). >> >> Also what USB host chipset is on the PC motherboard ? > > If it's a Intel motherboard, the USB 2.0 ports are probably routed through > on-board hubs to the ports provided by whatever Intel calls their equivalent > of the south bridge these days, and the USB 3.0 ports are probably a mix of > Intel and ASMedia XHCI controllers (ASMedia was one of the first companies > to do an inexpensive standalone XHCI chip, so they're relatively popular for > extra USB 3.0 ports). FWIW, the first generation of Intel XHCI chips had > some issues with older Linux kernels, although IIRC those issues were along > the lines of a port just disappearing after disconnecting whatever was > connected to it. >> >> Why don't you run 64-bit Ubuntu on this core i7 ? > > 64 versus 32 bit shouldn't cause anything like this to happen (although, if > it can be proven that it does, then that is a serious bug that needs to be > fixed). That said, unless you have some desperate need to be running 32-bit > only, you should seriously look into updating to a 64-bit version, your > whole system should run faster, and Ubuntu has really good 32-bit > compatibility in the 64-bit version (which is part of why it's popular as a > support target for third party software like Steam). > >> >> >> On Tue, Oct 27, 2015 at 12:44 PM, Austin S Hemmelgarn >> <ahferro...@gmail.com> wrote: >>> >>> On 20
Re: Bad fs performance, IO freezes
If you can suggest a dual (or better yet quad) USB3 bay that can be bought on Amazon, I'll buy it now, and once that arrives, we can be sure it's not the JMicron chipset. On Tue, Oct 27, 2015 at 3:22 PM, cheater00 . <cheate...@gmail.com> wrote: > The (dual) HDD bay and the chipset are, according to lsusb: > Bus 002 Device 005: ID 152d:0551 JMicron Technology Corp. / JMicron > USA Technology Corp. > Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub > > Not sure how to find out specific model numbers? I could open up the > bay. OK I'll open up the bay. > Good thing I have just the right screwdriver. It's a JMS551, and just > for records sake, here's the manufacture info: > > JMS551 > 1120 LGAA2 A > 572QV0024 > > The laptop manual says it's either "Intel HM65 Express chipset with > NEC USB 3.0 (select models only)" or "Intel HM65 Express chipset". > Here are technical documents for my model: > Manual: http://docdro.id/hG627JM > "Intel chipset datasheet": http://docdro.id/yKRupYO > Service guide: http://docdro.id/AuDgUdE > Service guide, alt. ver.: http://docdro.id/WwQRpsH > > FWIW I'm using one of the USB3 ports on the left. The ones on the > right are USB2. > > I've never used docdro.id so if it's not good let me know where to > upload the PDFs to. > > autodefrag is on, yes. But I have been having issues before turning it > on - I turned it on as a measure towards fixing the issues. I will > turn it off and remount, then report. But I don't think that should be > it. As you see the transfer speeds are minimal. They're *all* that's > happening on the disk. Right now that's under 100 KB/sec and I'm still > getting freezes albeit less. Also why would I be getting freezes when > the transfer speeds jump up - just for them to drop again? Hmm, maybe > utorrent has some sort of scheduler that gets preempted while the > spike is happening, and some algorithm in it gets the wrong idea and > turns some sort of flow control on, because it thinks it's hit some > sort of physical transfer speed barrier. Also notice the upload and > download both scale together, but that just might be how torrent > works, maybe it just tries to be fair i.e. only uploads as much as it > downloaded (scaled by a constant). > > The system is 32 bit because I installed ubuntu 6 one day and just > kept on upgrading. I keep on telling myself I'll update to 64 bits, > one of these days. But this laptop only has 8 gigs of ram, so no real > reason to upgrade to 64 bit anyways. It's not like I need firefox to > be able to eat 8 gb of ram whereas right now it can only eat 4. There > is no simple upgrade path that I know of so it's either a fresh > install or doing something like this: http://www.ewan.cc/?q=node/132 > -- I keep telling myself /one of these days/... > > On Tue, Oct 27, 2015 at 2:30 PM, Austin S Hemmelgarn > <ahferro...@gmail.com> wrote: >> On 2015-10-27 09:00, Henk Slager wrote: >>> >>> I don't have a lot experience with autodefrag, but as indicated by >>> Austin, expect a lot of full rewrites of files that are relatively >>> slowly filled up by a torrent client, starting with a sparse file. So >>> 1st advice would be to remove this option and run it as crontask at >>> particular times. >>> >>> What SATA-USB bridge is between the harddisk and the PC motherboard ? >> >> I hadn't thought of this, but the specific adapter being used for the disk >> can have a lot of impact on how it preforms. I've personally had lots of >> issues with JMicron chipsets (ranging from latency issues like what you are >> seeing to sever data corruption), but have found that ASMedia ones tend to >> be pretty much rock solid reliable and have good performance (although I >> think they only do USB 3.0 adapters). >>> >>> Also what USB host chipset is on the PC motherboard ? >> >> If it's a Intel motherboard, the USB 2.0 ports are probably routed through >> on-board hubs to the ports provided by whatever Intel calls their equivalent >> of the south bridge these days, and the USB 3.0 ports are probably a mix of >> Intel and ASMedia XHCI controllers (ASMedia was one of the first companies >> to do an inexpensive standalone XHCI chip, so they're relatively popular for >> extra USB 3.0 ports). FWIW, the first generation of Intel XHCI chips had >> some issues with older Linux kernels, although IIRC those issues were along >> the lines of a port just disappearing after disconnecting whatever was >> connected to it. >>> >>> Why don't you run 64-bit Ubuntu on this core i7 ? >> >> 64 versus 32 bit shouldn't cause anything like this
Re: Bad fs performance, IO freezes
Feel free to suggest a good 1.5m USB3 cable, too. Let's get rid of all the unknowns. On Tue, Oct 27, 2015 at 3:26 PM, cheater00 . <cheate...@gmail.com> wrote: > If you can suggest a dual (or better yet quad) USB3 bay that can be > bought on Amazon, I'll buy it now, and once that arrives, we can be > sure it's not the JMicron chipset. > > On Tue, Oct 27, 2015 at 3:22 PM, cheater00 . <cheate...@gmail.com> wrote: >> The (dual) HDD bay and the chipset are, according to lsusb: >> Bus 002 Device 005: ID 152d:0551 JMicron Technology Corp. / JMicron >> USA Technology Corp. >> Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub >> >> Not sure how to find out specific model numbers? I could open up the >> bay. OK I'll open up the bay. >> Good thing I have just the right screwdriver. It's a JMS551, and just >> for records sake, here's the manufacture info: >> >> JMS551 >> 1120 LGAA2 A >> 572QV0024 >> >> The laptop manual says it's either "Intel HM65 Express chipset with >> NEC USB 3.0 (select models only)" or "Intel HM65 Express chipset". >> Here are technical documents for my model: >> Manual: http://docdro.id/hG627JM >> "Intel chipset datasheet": http://docdro.id/yKRupYO >> Service guide: http://docdro.id/AuDgUdE >> Service guide, alt. ver.: http://docdro.id/WwQRpsH >> >> FWIW I'm using one of the USB3 ports on the left. The ones on the >> right are USB2. >> >> I've never used docdro.id so if it's not good let me know where to >> upload the PDFs to. >> >> autodefrag is on, yes. But I have been having issues before turning it >> on - I turned it on as a measure towards fixing the issues. I will >> turn it off and remount, then report. But I don't think that should be >> it. As you see the transfer speeds are minimal. They're *all* that's >> happening on the disk. Right now that's under 100 KB/sec and I'm still >> getting freezes albeit less. Also why would I be getting freezes when >> the transfer speeds jump up - just for them to drop again? Hmm, maybe >> utorrent has some sort of scheduler that gets preempted while the >> spike is happening, and some algorithm in it gets the wrong idea and >> turns some sort of flow control on, because it thinks it's hit some >> sort of physical transfer speed barrier. Also notice the upload and >> download both scale together, but that just might be how torrent >> works, maybe it just tries to be fair i.e. only uploads as much as it >> downloaded (scaled by a constant). >> >> The system is 32 bit because I installed ubuntu 6 one day and just >> kept on upgrading. I keep on telling myself I'll update to 64 bits, >> one of these days. But this laptop only has 8 gigs of ram, so no real >> reason to upgrade to 64 bit anyways. It's not like I need firefox to >> be able to eat 8 gb of ram whereas right now it can only eat 4. There >> is no simple upgrade path that I know of so it's either a fresh >> install or doing something like this: http://www.ewan.cc/?q=node/132 >> -- I keep telling myself /one of these days/... >> >> On Tue, Oct 27, 2015 at 2:30 PM, Austin S Hemmelgarn >> <ahferro...@gmail.com> wrote: >>> On 2015-10-27 09:00, Henk Slager wrote: >>>> >>>> I don't have a lot experience with autodefrag, but as indicated by >>>> Austin, expect a lot of full rewrites of files that are relatively >>>> slowly filled up by a torrent client, starting with a sparse file. So >>>> 1st advice would be to remove this option and run it as crontask at >>>> particular times. >>>> >>>> What SATA-USB bridge is between the harddisk and the PC motherboard ? >>> >>> I hadn't thought of this, but the specific adapter being used for the disk >>> can have a lot of impact on how it preforms. I've personally had lots of >>> issues with JMicron chipsets (ranging from latency issues like what you are >>> seeing to sever data corruption), but have found that ASMedia ones tend to >>> be pretty much rock solid reliable and have good performance (although I >>> think they only do USB 3.0 adapters). >>>> >>>> Also what USB host chipset is on the PC motherboard ? >>> >>> If it's a Intel motherboard, the USB 2.0 ports are probably routed through >>> on-board hubs to the ports provided by whatever Intel calls their equivalent >>> of the south bridge these days, and the USB 3.0 ports are probably a mix of >>> Intel and ASMedia XHCI controllers (ASMedia was one of the first companies >>
Re: Bad fs performance, IO freezes
Hello, currently my computer freezes every several seconds for half a second or so. Using it feels like I'm playing musical chairs with the kernel. I have just one download happening on utorrent right now - this is what the graph looks like: http://i.imgur.com/LqhMtrJ.png and every time a new spike happens, a freeze happens just before that... that's the only time those freezes happen, too. Please advise. On Mon, Oct 26, 2015 at 7:31 PM, cheater00 . <cheate...@gmail.com> wrote: > I do not experience btrfs-transacti going up to 100% for minutes at a > time now (not reproduced yet) but I have it spiking up to say 30% for > a short while and everything jags during that time. So, say, if I am > watching youtube, the sound cuts out and the video drops out for a > bit. And if I'm typing, then what I typed during that time gets lost, > like if I never typed that. > > I have also connected the same HDD bay with a USB3 cable instead of > USB2. It's on an USB3 port. So it's running via USB3 now. > > > On Mon, Oct 26, 2015 at 6:43 PM, cheater00 . <cheate...@gmail.com> wrote: >> So far I cannot reproduce. If I don't post again this means the issue >> has been fixed by updating the kernel. >> >> On Mon, Oct 26, 2015 at 4:40 PM, cheater00 . <cheate...@gmail.com> wrote: >>> I have located 4.3.0-rc7 binaries which I will now try. >>> >>> On Mon, Oct 26, 2015 at 3:38 PM, cheater00 . <cheate...@gmail.com> wrote: >>>> Thanks for the reply. What version did this go into? I'll try getting >>>> a prebuilt backport of the kernel, building source could slow things >>>> down considerably, but debs will not be available for the latest few >>>> minor versions I guess. So if you can tell me a min version, I'll try >>>> to find the latest deb newer than that, or I'll build if that's not >>>> available. >>>> >>>> On Mon, Oct 26, 2015 at 3:25 PM, Liu Bo <bo.li@oracle.com> wrote: >>>>> On 10/26/2015 08:16 PM, cheater00 . wrote: >>>>>> >>>>>> Hi guys, >>>>>> I am running into really bad performance. Here's my setup: >>>>>> >>>>>> WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu >>>>>> 32-bit with kernel 4.0.4-040004-generic #201505171336. >>>>>> >>>>>> Single btrfs partition covering whole disk. >>>>>> >>>>>> Autodefrag is on. >>>>>> >>>>>> fstab line: >>>>>> UUID=... /media/X btrfs rw,nosuid,nodev,autodefrag 0 0 >>>>>> >>>>>> Sometimes when files are being modified or removed, I see >>>>>> btrfs-transacti eat 100% cpu; during this time no io operations >>>>>> succeed, that is, they're all stalled. You can't even ls on that fs. >>>>>> This happens for several minutes then normal operation resumes. There >>>>>> doesn't seem to be a rule to what will trigger this, other than >>>>>> opening a single file and reading usually works quite well. (say, >>>>>> watching a movie while all other programs are closed). But even moving >>>>>> files off the disks triggers some sort of bug. Just now I am moving a >>>>>> few files (just 30gb worth) onto another disk, and the bug triggers. >>>>>> So btrfs-transacti was eating my cpu for over 5 minutes and according >>>>>> to mv's output after this was done and cpu usage went back to normal >>>>>> what I was waiting for was for a tiny png file to be removed. This is >>>>>> pretty bad. >>>>>> >>>>>> I have tried defragmenting directories where files are being accessed >>>>>> and moved. This hasn't helped. >>>>>> >>>>>> This happens whether the FS is near full or not. It currently is near >>>>>> full but it wasn't before and it still did that. It still has about ~ >>>>>> 100GB free space now. >>>>>> >>>>>> The more things are happening the more often this bug gets triggered. >>>>>> So if I have utorrent running and its temporary downloads directory is >>>>>> there, its download speed graph will be a few spikes of running at >>>>>> several MB/sec separated by durations of 0 download speed. >>>>>> >>>>>> Nothing seems to show up in dmesg or syslog. >>>>>> >>>>>> I have asked in #btrfs but the suggestions ended up not fixing the >>>>>> issue (autodefrag, defrag dirs). >>>>>> >>>>>> Please advise what I should do with this issue. >>>>> >>>>> >>>>> It might be related to delayed ref rework, the last time I saw this kind >>>>> of >>>>> hanging problem about btrfs-transaction eating cpu is that because btrfs >>>>> doesn't merge delayed refs, it'd be best to try the lastest kernel and if >>>>> the issue is not resolved, then we can work out a reproducer and provide >>>>> debugging. >>>>> >>>>> Thanks, >>>>> >>>>> Liubo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Bad fs performance, IO freezes
Hi guys, I am running into really bad performance. Here's my setup: WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu 32-bit with kernel 4.0.4-040004-generic #201505171336. Single btrfs partition covering whole disk. Autodefrag is on. fstab line: UUID=... /media/X btrfs rw,nosuid,nodev,autodefrag 0 0 Sometimes when files are being modified or removed, I see btrfs-transacti eat 100% cpu; during this time no io operations succeed, that is, they're all stalled. You can't even ls on that fs. This happens for several minutes then normal operation resumes. There doesn't seem to be a rule to what will trigger this, other than opening a single file and reading usually works quite well. (say, watching a movie while all other programs are closed). But even moving files off the disks triggers some sort of bug. Just now I am moving a few files (just 30gb worth) onto another disk, and the bug triggers. So btrfs-transacti was eating my cpu for over 5 minutes and according to mv's output after this was done and cpu usage went back to normal what I was waiting for was for a tiny png file to be removed. This is pretty bad. I have tried defragmenting directories where files are being accessed and moved. This hasn't helped. This happens whether the FS is near full or not. It currently is near full but it wasn't before and it still did that. It still has about ~ 100GB free space now. The more things are happening the more often this bug gets triggered. So if I have utorrent running and its temporary downloads directory is there, its download speed graph will be a few spikes of running at several MB/sec separated by durations of 0 download speed. Nothing seems to show up in dmesg or syslog. I have asked in #btrfs but the suggestions ended up not fixing the issue (autodefrag, defrag dirs). Please advise what I should do with this issue. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
Thanks for the reply. What version did this go into? I'll try getting a prebuilt backport of the kernel, building source could slow things down considerably, but debs will not be available for the latest few minor versions I guess. So if you can tell me a min version, I'll try to find the latest deb newer than that, or I'll build if that's not available. On Mon, Oct 26, 2015 at 3:25 PM, Liu Bo <bo.li@oracle.com> wrote: > On 10/26/2015 08:16 PM, cheater00 . wrote: >> >> Hi guys, >> I am running into really bad performance. Here's my setup: >> >> WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu >> 32-bit with kernel 4.0.4-040004-generic #201505171336. >> >> Single btrfs partition covering whole disk. >> >> Autodefrag is on. >> >> fstab line: >> UUID=... /media/X btrfs rw,nosuid,nodev,autodefrag 0 0 >> >> Sometimes when files are being modified or removed, I see >> btrfs-transacti eat 100% cpu; during this time no io operations >> succeed, that is, they're all stalled. You can't even ls on that fs. >> This happens for several minutes then normal operation resumes. There >> doesn't seem to be a rule to what will trigger this, other than >> opening a single file and reading usually works quite well. (say, >> watching a movie while all other programs are closed). But even moving >> files off the disks triggers some sort of bug. Just now I am moving a >> few files (just 30gb worth) onto another disk, and the bug triggers. >> So btrfs-transacti was eating my cpu for over 5 minutes and according >> to mv's output after this was done and cpu usage went back to normal >> what I was waiting for was for a tiny png file to be removed. This is >> pretty bad. >> >> I have tried defragmenting directories where files are being accessed >> and moved. This hasn't helped. >> >> This happens whether the FS is near full or not. It currently is near >> full but it wasn't before and it still did that. It still has about ~ >> 100GB free space now. >> >> The more things are happening the more often this bug gets triggered. >> So if I have utorrent running and its temporary downloads directory is >> there, its download speed graph will be a few spikes of running at >> several MB/sec separated by durations of 0 download speed. >> >> Nothing seems to show up in dmesg or syslog. >> >> I have asked in #btrfs but the suggestions ended up not fixing the >> issue (autodefrag, defrag dirs). >> >> Please advise what I should do with this issue. > > > It might be related to delayed ref rework, the last time I saw this kind of > hanging problem about btrfs-transaction eating cpu is that because btrfs > doesn't merge delayed refs, it'd be best to try the lastest kernel and if > the issue is not resolved, then we can work out a reproducer and provide > debugging. > > Thanks, > > Liubo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
I don't remember doing that, but just to exclude everything, how do I check? On Mon, Oct 26, 2015 at 2:45 PM, Donald Pearson <donaldwhpear...@gmail.com> wrote: > AFAIK quotas aren't a mount option, but if you never enabled them and > created the qgroups by hand that's your answer and the issue must be > something else. > > On Mon, Oct 26, 2015 at 8:36 AM, cheater00 . <cheate...@gmail.com> wrote: >> There are no quotas. I haven't enabled them. I believe the fstab says >> that - could they be enabled in another way? How do I check for sure? >> The man page doesn't say how to check the status: >> https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-quota >> >> On Mon, Oct 26, 2015 at 2:32 PM, Donald Pearson >> <donaldwhpear...@gmail.com> wrote: >>> Accidentally didn't reply to the list the 1st time. >>> >>> I see the same issue when I have quotas enabled. If you have quotas >>> on, see if turning them off helps. >>> >>> On Mon, Oct 26, 2015 at 7:16 AM, cheater00 . <cheate...@gmail.com> wrote: >>>> Hi guys, >>>> I am running into really bad performance. Here's my setup: >>>> >>>> WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu >>>> 32-bit with kernel 4.0.4-040004-generic #201505171336. >>>> >>>> Single btrfs partition covering whole disk. >>>> >>>> Autodefrag is on. >>>> >>>> fstab line: >>>> UUID=... /media/X btrfs rw,nosuid,nodev,autodefrag 0 0 >>>> >>>> Sometimes when files are being modified or removed, I see >>>> btrfs-transacti eat 100% cpu; during this time no io operations >>>> succeed, that is, they're all stalled. You can't even ls on that fs. >>>> This happens for several minutes then normal operation resumes. There >>>> doesn't seem to be a rule to what will trigger this, other than >>>> opening a single file and reading usually works quite well. (say, >>>> watching a movie while all other programs are closed). But even moving >>>> files off the disks triggers some sort of bug. Just now I am moving a >>>> few files (just 30gb worth) onto another disk, and the bug triggers. >>>> So btrfs-transacti was eating my cpu for over 5 minutes and according >>>> to mv's output after this was done and cpu usage went back to normal >>>> what I was waiting for was for a tiny png file to be removed. This is >>>> pretty bad. >>>> >>>> I have tried defragmenting directories where files are being accessed >>>> and moved. This hasn't helped. >>>> >>>> This happens whether the FS is near full or not. It currently is near >>>> full but it wasn't before and it still did that. It still has about ~ >>>> 100GB free space now. >>>> >>>> The more things are happening the more often this bug gets triggered. >>>> So if I have utorrent running and its temporary downloads directory is >>>> there, its download speed graph will be a few spikes of running at >>>> several MB/sec separated by durations of 0 download speed. >>>> >>>> Nothing seems to show up in dmesg or syslog. >>>> >>>> I have asked in #btrfs but the suggestions ended up not fixing the >>>> issue (autodefrag, defrag dirs). >>>> >>>> Please advise what I should do with this issue. >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >>>> the body of a message to majord...@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
fwiw, I did this: sudo btrfs qgroup show /media/X ERROR: can't perform the search - No such file or directory ERROR: can't list qgroups: No such file or directory I assume this means no qgroups present, which means no quotas present. Please correct me if I'm wrong. So yes, the issue must lie elsewhere. On Mon, Oct 26, 2015 at 2:46 PM, cheater00 . <cheate...@gmail.com> wrote: > I don't remember doing that, but just to exclude everything, how do I check? > > On Mon, Oct 26, 2015 at 2:45 PM, Donald Pearson > <donaldwhpear...@gmail.com> wrote: >> AFAIK quotas aren't a mount option, but if you never enabled them and >> created the qgroups by hand that's your answer and the issue must be >> something else. >> >> On Mon, Oct 26, 2015 at 8:36 AM, cheater00 . <cheate...@gmail.com> wrote: >>> There are no quotas. I haven't enabled them. I believe the fstab says >>> that - could they be enabled in another way? How do I check for sure? >>> The man page doesn't say how to check the status: >>> https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-quota >>> >>> On Mon, Oct 26, 2015 at 2:32 PM, Donald Pearson >>> <donaldwhpear...@gmail.com> wrote: >>>> Accidentally didn't reply to the list the 1st time. >>>> >>>> I see the same issue when I have quotas enabled. If you have quotas >>>> on, see if turning them off helps. >>>> >>>> On Mon, Oct 26, 2015 at 7:16 AM, cheater00 . <cheate...@gmail.com> wrote: >>>>> Hi guys, >>>>> I am running into really bad performance. Here's my setup: >>>>> >>>>> WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu >>>>> 32-bit with kernel 4.0.4-040004-generic #201505171336. >>>>> >>>>> Single btrfs partition covering whole disk. >>>>> >>>>> Autodefrag is on. >>>>> >>>>> fstab line: >>>>> UUID=... /media/X btrfs rw,nosuid,nodev,autodefrag 0 0 >>>>> >>>>> Sometimes when files are being modified or removed, I see >>>>> btrfs-transacti eat 100% cpu; during this time no io operations >>>>> succeed, that is, they're all stalled. You can't even ls on that fs. >>>>> This happens for several minutes then normal operation resumes. There >>>>> doesn't seem to be a rule to what will trigger this, other than >>>>> opening a single file and reading usually works quite well. (say, >>>>> watching a movie while all other programs are closed). But even moving >>>>> files off the disks triggers some sort of bug. Just now I am moving a >>>>> few files (just 30gb worth) onto another disk, and the bug triggers. >>>>> So btrfs-transacti was eating my cpu for over 5 minutes and according >>>>> to mv's output after this was done and cpu usage went back to normal >>>>> what I was waiting for was for a tiny png file to be removed. This is >>>>> pretty bad. >>>>> >>>>> I have tried defragmenting directories where files are being accessed >>>>> and moved. This hasn't helped. >>>>> >>>>> This happens whether the FS is near full or not. It currently is near >>>>> full but it wasn't before and it still did that. It still has about ~ >>>>> 100GB free space now. >>>>> >>>>> The more things are happening the more often this bug gets triggered. >>>>> So if I have utorrent running and its temporary downloads directory is >>>>> there, its download speed graph will be a few spikes of running at >>>>> several MB/sec separated by durations of 0 download speed. >>>>> >>>>> Nothing seems to show up in dmesg or syslog. >>>>> >>>>> I have asked in #btrfs but the suggestions ended up not fixing the >>>>> issue (autodefrag, defrag dirs). >>>>> >>>>> Please advise what I should do with this issue. >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >>>>> the body of a message to majord...@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
There are no quotas. I haven't enabled them. I believe the fstab says that - could they be enabled in another way? How do I check for sure? The man page doesn't say how to check the status: https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-quota On Mon, Oct 26, 2015 at 2:32 PM, Donald Pearson <donaldwhpear...@gmail.com> wrote: > Accidentally didn't reply to the list the 1st time. > > I see the same issue when I have quotas enabled. If you have quotas > on, see if turning them off helps. > > On Mon, Oct 26, 2015 at 7:16 AM, cheater00 . <cheate...@gmail.com> wrote: >> Hi guys, >> I am running into really bad performance. Here's my setup: >> >> WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu >> 32-bit with kernel 4.0.4-040004-generic #201505171336. >> >> Single btrfs partition covering whole disk. >> >> Autodefrag is on. >> >> fstab line: >> UUID=... /media/X btrfs rw,nosuid,nodev,autodefrag 0 0 >> >> Sometimes when files are being modified or removed, I see >> btrfs-transacti eat 100% cpu; during this time no io operations >> succeed, that is, they're all stalled. You can't even ls on that fs. >> This happens for several minutes then normal operation resumes. There >> doesn't seem to be a rule to what will trigger this, other than >> opening a single file and reading usually works quite well. (say, >> watching a movie while all other programs are closed). But even moving >> files off the disks triggers some sort of bug. Just now I am moving a >> few files (just 30gb worth) onto another disk, and the bug triggers. >> So btrfs-transacti was eating my cpu for over 5 minutes and according >> to mv's output after this was done and cpu usage went back to normal >> what I was waiting for was for a tiny png file to be removed. This is >> pretty bad. >> >> I have tried defragmenting directories where files are being accessed >> and moved. This hasn't helped. >> >> This happens whether the FS is near full or not. It currently is near >> full but it wasn't before and it still did that. It still has about ~ >> 100GB free space now. >> >> The more things are happening the more often this bug gets triggered. >> So if I have utorrent running and its temporary downloads directory is >> there, its download speed graph will be a few spikes of running at >> several MB/sec separated by durations of 0 download speed. >> >> Nothing seems to show up in dmesg or syslog. >> >> I have asked in #btrfs but the suggestions ended up not fixing the >> issue (autodefrag, defrag dirs). >> >> Please advise what I should do with this issue. >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
I have located 4.3.0-rc7 binaries which I will now try. On Mon, Oct 26, 2015 at 3:38 PM, cheater00 . <cheate...@gmail.com> wrote: > Thanks for the reply. What version did this go into? I'll try getting > a prebuilt backport of the kernel, building source could slow things > down considerably, but debs will not be available for the latest few > minor versions I guess. So if you can tell me a min version, I'll try > to find the latest deb newer than that, or I'll build if that's not > available. > > On Mon, Oct 26, 2015 at 3:25 PM, Liu Bo <bo.li@oracle.com> wrote: >> On 10/26/2015 08:16 PM, cheater00 . wrote: >>> >>> Hi guys, >>> I am running into really bad performance. Here's my setup: >>> >>> WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu >>> 32-bit with kernel 4.0.4-040004-generic #201505171336. >>> >>> Single btrfs partition covering whole disk. >>> >>> Autodefrag is on. >>> >>> fstab line: >>> UUID=... /media/X btrfs rw,nosuid,nodev,autodefrag 0 0 >>> >>> Sometimes when files are being modified or removed, I see >>> btrfs-transacti eat 100% cpu; during this time no io operations >>> succeed, that is, they're all stalled. You can't even ls on that fs. >>> This happens for several minutes then normal operation resumes. There >>> doesn't seem to be a rule to what will trigger this, other than >>> opening a single file and reading usually works quite well. (say, >>> watching a movie while all other programs are closed). But even moving >>> files off the disks triggers some sort of bug. Just now I am moving a >>> few files (just 30gb worth) onto another disk, and the bug triggers. >>> So btrfs-transacti was eating my cpu for over 5 minutes and according >>> to mv's output after this was done and cpu usage went back to normal >>> what I was waiting for was for a tiny png file to be removed. This is >>> pretty bad. >>> >>> I have tried defragmenting directories where files are being accessed >>> and moved. This hasn't helped. >>> >>> This happens whether the FS is near full or not. It currently is near >>> full but it wasn't before and it still did that. It still has about ~ >>> 100GB free space now. >>> >>> The more things are happening the more often this bug gets triggered. >>> So if I have utorrent running and its temporary downloads directory is >>> there, its download speed graph will be a few spikes of running at >>> several MB/sec separated by durations of 0 download speed. >>> >>> Nothing seems to show up in dmesg or syslog. >>> >>> I have asked in #btrfs but the suggestions ended up not fixing the >>> issue (autodefrag, defrag dirs). >>> >>> Please advise what I should do with this issue. >> >> >> It might be related to delayed ref rework, the last time I saw this kind of >> hanging problem about btrfs-transaction eating cpu is that because btrfs >> doesn't merge delayed refs, it'd be best to try the lastest kernel and if >> the issue is not resolved, then we can work out a reproducer and provide >> debugging. >> >> Thanks, >> >> Liubo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
So far I cannot reproduce. If I don't post again this means the issue has been fixed by updating the kernel. On Mon, Oct 26, 2015 at 4:40 PM, cheater00 . <cheate...@gmail.com> wrote: > I have located 4.3.0-rc7 binaries which I will now try. > > On Mon, Oct 26, 2015 at 3:38 PM, cheater00 . <cheate...@gmail.com> wrote: >> Thanks for the reply. What version did this go into? I'll try getting >> a prebuilt backport of the kernel, building source could slow things >> down considerably, but debs will not be available for the latest few >> minor versions I guess. So if you can tell me a min version, I'll try >> to find the latest deb newer than that, or I'll build if that's not >> available. >> >> On Mon, Oct 26, 2015 at 3:25 PM, Liu Bo <bo.li@oracle.com> wrote: >>> On 10/26/2015 08:16 PM, cheater00 . wrote: >>>> >>>> Hi guys, >>>> I am running into really bad performance. Here's my setup: >>>> >>>> WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu >>>> 32-bit with kernel 4.0.4-040004-generic #201505171336. >>>> >>>> Single btrfs partition covering whole disk. >>>> >>>> Autodefrag is on. >>>> >>>> fstab line: >>>> UUID=... /media/X btrfs rw,nosuid,nodev,autodefrag 0 0 >>>> >>>> Sometimes when files are being modified or removed, I see >>>> btrfs-transacti eat 100% cpu; during this time no io operations >>>> succeed, that is, they're all stalled. You can't even ls on that fs. >>>> This happens for several minutes then normal operation resumes. There >>>> doesn't seem to be a rule to what will trigger this, other than >>>> opening a single file and reading usually works quite well. (say, >>>> watching a movie while all other programs are closed). But even moving >>>> files off the disks triggers some sort of bug. Just now I am moving a >>>> few files (just 30gb worth) onto another disk, and the bug triggers. >>>> So btrfs-transacti was eating my cpu for over 5 minutes and according >>>> to mv's output after this was done and cpu usage went back to normal >>>> what I was waiting for was for a tiny png file to be removed. This is >>>> pretty bad. >>>> >>>> I have tried defragmenting directories where files are being accessed >>>> and moved. This hasn't helped. >>>> >>>> This happens whether the FS is near full or not. It currently is near >>>> full but it wasn't before and it still did that. It still has about ~ >>>> 100GB free space now. >>>> >>>> The more things are happening the more often this bug gets triggered. >>>> So if I have utorrent running and its temporary downloads directory is >>>> there, its download speed graph will be a few spikes of running at >>>> several MB/sec separated by durations of 0 download speed. >>>> >>>> Nothing seems to show up in dmesg or syslog. >>>> >>>> I have asked in #btrfs but the suggestions ended up not fixing the >>>> issue (autodefrag, defrag dirs). >>>> >>>> Please advise what I should do with this issue. >>> >>> >>> It might be related to delayed ref rework, the last time I saw this kind of >>> hanging problem about btrfs-transaction eating cpu is that because btrfs >>> doesn't merge delayed refs, it'd be best to try the lastest kernel and if >>> the issue is not resolved, then we can work out a reproducer and provide >>> debugging. >>> >>> Thanks, >>> >>> Liubo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Bad fs performance, IO freezes
I do not experience btrfs-transacti going up to 100% for minutes at a time now (not reproduced yet) but I have it spiking up to say 30% for a short while and everything jags during that time. So, say, if I am watching youtube, the sound cuts out and the video drops out for a bit. And if I'm typing, then what I typed during that time gets lost, like if I never typed that. I have also connected the same HDD bay with a USB3 cable instead of USB2. It's on an USB3 port. So it's running via USB3 now. On Mon, Oct 26, 2015 at 6:43 PM, cheater00 . <cheate...@gmail.com> wrote: > So far I cannot reproduce. If I don't post again this means the issue > has been fixed by updating the kernel. > > On Mon, Oct 26, 2015 at 4:40 PM, cheater00 . <cheate...@gmail.com> wrote: >> I have located 4.3.0-rc7 binaries which I will now try. >> >> On Mon, Oct 26, 2015 at 3:38 PM, cheater00 . <cheate...@gmail.com> wrote: >>> Thanks for the reply. What version did this go into? I'll try getting >>> a prebuilt backport of the kernel, building source could slow things >>> down considerably, but debs will not be available for the latest few >>> minor versions I guess. So if you can tell me a min version, I'll try >>> to find the latest deb newer than that, or I'll build if that's not >>> available. >>> >>> On Mon, Oct 26, 2015 at 3:25 PM, Liu Bo <bo.li@oracle.com> wrote: >>>> On 10/26/2015 08:16 PM, cheater00 . wrote: >>>>> >>>>> Hi guys, >>>>> I am running into really bad performance. Here's my setup: >>>>> >>>>> WD Red 6 TB connected over USB2 to my core i7 laptop, running Ubuntu >>>>> 32-bit with kernel 4.0.4-040004-generic #201505171336. >>>>> >>>>> Single btrfs partition covering whole disk. >>>>> >>>>> Autodefrag is on. >>>>> >>>>> fstab line: >>>>> UUID=... /media/X btrfs rw,nosuid,nodev,autodefrag 0 0 >>>>> >>>>> Sometimes when files are being modified or removed, I see >>>>> btrfs-transacti eat 100% cpu; during this time no io operations >>>>> succeed, that is, they're all stalled. You can't even ls on that fs. >>>>> This happens for several minutes then normal operation resumes. There >>>>> doesn't seem to be a rule to what will trigger this, other than >>>>> opening a single file and reading usually works quite well. (say, >>>>> watching a movie while all other programs are closed). But even moving >>>>> files off the disks triggers some sort of bug. Just now I am moving a >>>>> few files (just 30gb worth) onto another disk, and the bug triggers. >>>>> So btrfs-transacti was eating my cpu for over 5 minutes and according >>>>> to mv's output after this was done and cpu usage went back to normal >>>>> what I was waiting for was for a tiny png file to be removed. This is >>>>> pretty bad. >>>>> >>>>> I have tried defragmenting directories where files are being accessed >>>>> and moved. This hasn't helped. >>>>> >>>>> This happens whether the FS is near full or not. It currently is near >>>>> full but it wasn't before and it still did that. It still has about ~ >>>>> 100GB free space now. >>>>> >>>>> The more things are happening the more often this bug gets triggered. >>>>> So if I have utorrent running and its temporary downloads directory is >>>>> there, its download speed graph will be a few spikes of running at >>>>> several MB/sec separated by durations of 0 download speed. >>>>> >>>>> Nothing seems to show up in dmesg or syslog. >>>>> >>>>> I have asked in #btrfs but the suggestions ended up not fixing the >>>>> issue (autodefrag, defrag dirs). >>>>> >>>>> Please advise what I should do with this issue. >>>> >>>> >>>> It might be related to delayed ref rework, the last time I saw this kind of >>>> hanging problem about btrfs-transaction eating cpu is that because btrfs >>>> doesn't merge delayed refs, it'd be best to try the lastest kernel and if >>>> the issue is not resolved, then we can work out a reproducer and provide >>>> debugging. >>>> >>>> Thanks, >>>> >>>> Liubo -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
