Re: frustrations with handling of crash reports

2014-06-19 Thread Marc MERLIN
On Wed, Jun 18, 2014 at 09:22:50PM +, Duncan wrote: > Tho at least in Marc's case, he's running kernels a couple back in some > cases and they may still have BUGONs already replaced in the most current > kernel. The machine I originally has that one last bug on (balance crash) was an ubuntu

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

2014-06-19 Thread Marc MERLIN
On Thu, Jun 19, 2014 at 09:12:13AM -0700, Josef Bacik wrote: > Ok undo what you did and apply this and re-run. It is going spit out a > metric > shittone of data, but all I want is the last chunk of stuff between > > running build_backref_tree > > block wasn't checked > done building backref t

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

2014-06-19 Thread Marc MERLIN
On Thu, Jun 19, 2014 at 03:50:16PM -0700, Josef Bacik wrote: > Ok same drill as before, reset and apply this, hopefully no panic this time > > > diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c > index 65245a0..bca5240 100644 Here's the output BTRFS info (device sdb1): disk space cachi

Re: btrfs balance crash BUG ON fs/btrfs/relocation.c:1062 or RIP build_backref_tree+0x9fc/0xcc4

2014-06-25 Thread Marc MERLIN
On Fri, Jun 20, 2014 at 08:40:49AM -0700, Josef Bacik wrote: > On 06/19/2014 05:53 PM, Marc MERLIN wrote: > >On Thu, Jun 19, 2014 at 03:50:16PM -0700, Josef Bacik wrote: > >>Ok same drill as before, reset and apply this, hopefully no panic this > >>time > >

Also seeing full deadlocks with 3.15.1

2014-06-27 Thread Marc MERLIN
My laptop deadlocked some more times (everything works until it needs to touch the filesystem, and then it's deadlocked). Unfortunately, I can trigger sysrq, but it doesn't get committed to disk and netconsole eats half of it because it goes too fast for UDP apparently Now, I just captured that on

Re: Also seeing full deadlocks with 3.15.1

2014-06-27 Thread Marc MERLIN
On Fri, Jun 27, 2014 at 11:50:09AM -0700, Marc MERLIN wrote: > My laptop deadlocked some more times (everything works until it needs to > touch the filesystem, and then it's deadlocked). > Unfortunately, I can trigger sysrq, but it doesn't get committed to disk and > netc

Re: Also seeing full deadlocks with 3.15.1

2014-06-27 Thread Marc MERLIN
On Fri, Jun 27, 2014 at 02:50:10PM -0700, ronnie sahlberg wrote: > > If I don't hear anything by the end of today, I'll just delete the > > filesystem and start over. > > At some stage it would be nice to see not only fixes but also changes > to fsck to make it able to repair these problems. > Blo

Re: Also seeing full deadlocks with 3.15.1

2014-06-27 Thread Marc MERLIN
On Fri, Jun 27, 2014 at 03:36:08PM -0700, Josef Bacik wrote: > On 06/27/2014 11:50 AM, Marc MERLIN wrote: > >My laptop deadlocked some more times (everything works until it needs to > >touch the filesystem, and then it's deadlocked). > >Unfortunately, I can trigger

3.15.1: kernel BUG at fs/btrfs/locking.c:269

2014-07-02 Thread Marc MERLIN
This got triggered by an rsync I think. I'm not sure which of my btrfs FS has the issue yet since BUG_ON isn't very helpful as discussed earlier. [160562.925463] parent transid verify failed on 2776298520576 wanted 41015 found 18120 [160562.950297] [ cut here ] [160562.965

Re: 3.15.1: kernel BUG at fs/btrfs/locking.c:269

2014-07-03 Thread Marc MERLIN
Thanks for the patch. Hopefully this will make it to the next 3.15.x kernel. I also went back to 3.14 anyway since the 'blocked for 120 seconds' look like another instance of deadlocks we've been discussing here. But just curious: > >>[160562.925463] parent transid verify failed on 2776298520576

Is btrfs related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-03 Thread Marc MERLIN
I upgraded my server from 3.14 to 3.15.1 last week, and since then it's been running out of memory and deadlocking (panic= doesn't even work). I downgraded back to 3.14, but I already had the problem once since then. OOM comes in, even though I have 0 swap used and AFAIK all my RAM isn't gone, it

Re: 3.15.1: kernel BUG at fs/btrfs/locking.c:269

2014-07-03 Thread Marc MERLIN
On Fri, Jul 04, 2014 at 11:07:22AM +0800, Liu Bo wrote: > > > >>[160562.925463] parent transid verify failed on 2776298520576 wanted > > > >>41015 found 18120 > > > > What should I be doing about this? > > Does it mean that I do have some kind of corruption/damage on my > > filesystem? > > > If

Re: 3.15.1: kernel BUG at fs/btrfs/locking.c:269

2014-07-03 Thread Marc MERLIN
On Fri, Jul 04, 2014 at 01:29:29PM +0800, Wang Shilong wrote: > >Well, I explained the problem, ext4 and others of course tell me which > >devid > >an error is on, hopefully btrfs will able to do so in the near future. > > So it is ok for you to print one of btrfs filesystem device(for example > d

Re: Is btrfs related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-03 Thread Marc MERLIN
On Fri, Jul 04, 2014 at 02:33:06PM +1000, Russell Coker wrote: > On Thu, 3 Jul 2014 18:19:38 Marc MERLIN wrote: > > I upgraded my server from 3.14 to 3.15.1 last week, and since then it's been > > running out of memory and deadlocking (panic= doesn't even work). > >

Re: 3.15.1: kernel BUG at fs/btrfs/locking.c:269

2014-07-04 Thread Marc MERLIN
generation verification failure. > > Since @extent_buffer structure has a member @fs_info, let's output > btrfs device info. I don't know the btrfs code much, but looking at the patch, it looks good, thank you for that. Marc > Reported-by: Marc MERLIN > Signed-off-by:

Re: Is btrfs related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-04 Thread Marc MERLIN
Thank you for your answer. I'll put the conclusion and question at the top for easier reading: So, should I understand that 1) I have enough RAM in my system but all of it disappears, apparently claimed by the kernel and not released 2) this could be a kernel memory leak in btrfs or somewhere

Re: Is btrfs related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-04 Thread Marc MERLIN
On Sat, Jul 05, 2014 at 12:45:55AM +1000, Russell Coker wrote: > > But the last times I had this OOM problem with 3.15.1 it was happening > > within 6 hours sometimes, and I was not starting scrub every time the > > system booted, so scrub may be partially responsible but it's not the > > core prob

Re: Is btrfs related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-05 Thread Marc MERLIN
On Sat, Jul 05, 2014 at 09:47:09AM -0400, Andrew E. Mileski wrote: > On 2014-07-03 9:19 PM, Marc MERLIN wrote: > >I upgraded my server from 3.14 to 3.15.1 last week, and since then it's been > >running out of memory and deadlocking (panic= doesn't even work). > >I

Re: Is btrfs related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-06 Thread Marc MERLIN
On Sat, Jul 05, 2014 at 07:43:18AM -0700, Marc MERLIN wrote: > On Sat, Jul 05, 2014 at 09:47:09AM -0400, Andrew E. Mileski wrote: > > On 2014-07-03 9:19 PM, Marc MERLIN wrote: > > >I upgraded my server from 3.14 to 3.15.1 last week, and since then it's > > >bee

Re: btrfs is related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-13 Thread Marc MERLIN
On Sun, Jul 06, 2014 at 07:58:15AM -0700, Marc MERLIN wrote: > > As an update, after 1.7 days of scrubbing, the system has started > > getting sluggish, I'm getting synchronization problems/crashes in some of > > my tools that talk to serial ports (likely due to mini dead

Re: btrfs partition remounted read-only

2014-07-13 Thread Marc MERLIN
On Fri, Jul 04, 2014 at 10:00:29AM -0700, Konstantin Svist wrote: > I have an overnight cron job with > > /sbin/fstrim -v / > /bin/bedup dedup --defrag > > Every once in a while, it causes the FS to be remounted read-only. > Problem is pretty intermittent so far (aside from a few kernel revisions

Re: Fwd: btrfs Wiki account request

2014-07-13 Thread Marc MERLIN
On Tue, Jul 08, 2014 at 10:19:08PM +0200, Sjon Hortensius wrote: > I wrote a simple alternative for a script that is currently on the > btrfs wiki (for showing quotas) in bash instead of Python. I have > attempted to add this to the page but after entering the captcha 15 > times my account was reje

Re: btrfs is related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-13 Thread Marc MERLIN
On Sun, Jul 13, 2014 at 07:29:18AM -0700, Marc MERLIN wrote: > Is /proc/slabinfo supposed to show anything useful? > > This is the filesystem in question: > gargamel:~# btrfs fi df /mnt/btrfs_pool2/ > Data, single: total=3.34TiB, used=3.32TiB > System, DUP: total=8.00MiB, used=

Re: btrfs quotas related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-13 Thread Marc MERLIN
On Sun, Jul 13, 2014 at 08:37:34AM -0700, Marc MERLIN wrote: > On Sun, Jul 13, 2014 at 07:29:18AM -0700, Marc MERLIN wrote: > > Is /proc/slabinfo supposed to show anything useful? > > > > This is the filesystem in question: > > gargamel:~# btrfs fi df /mnt/btrfs_poo

Re: btrfs quotas related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-13 Thread Marc MERLIN
On Mon, Jul 14, 2014 at 09:36:28AM +0800, Qu Wenruo wrote: > When you enable quota and create a subvolume, a qgroup(0/ id>) will be created and bind to the newly created subvolume. > But on the other hand, when you delete the subvolume, the qgroup > will *not* be deleted automatically. > So you nee

Re: btrfs is related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-16 Thread Marc MERLIN
On Mon, Jul 14, 2014 at 09:24:11AM +0800, Qu Wenruo wrote: > >But generally, is there a tool to locate which kernel function allocated > >all that RAM that seems to get allocated and forgotten? > This can be done by kernel memleak detection. > Location: > -> Kernel hacking > -> Memory Debugging

Re: btrfs is related to OOM death problems on my 8GB server with both 3.15.1 and 3.14?

2014-07-16 Thread Marc MERLIN
On Wed, Jul 16, 2014 at 08:55:32AM -0700, Marc MERLIN wrote: > On Mon, Jul 14, 2014 at 09:24:11AM +0800, Qu Wenruo wrote: > > >But generally, is there a tool to locate which kernel function allocated > > >all that RAM that seems to get allocated and forgotten? > >

Re: Blocked tasks on 3.15.1, raid1 btrfs is no ends of trouble for me

2014-07-18 Thread Marc MERLIN
On Thu, Jul 17, 2014 at 09:18:07AM -0400, Chris Mason wrote: > > [ deadlocks during rsync in 3.15 with compression enabled ] > > Hi everyone, > > I still haven't been able to reproduce this one here, but I'm going > through a series of tests with lzo compression foraced and every > operation for

Re: Blocked tasks on 3.15.1, raid1 btrfs is no ends of trouble for me

2014-07-18 Thread Marc MERLIN
On Fri, Jul 18, 2014 at 05:33:45PM -0700, Marc MERLIN wrote: > Howver, I have found that btrfs raid 1 on top of dmcrypt has given me no ends > of trouble. > I lost that filesystem twice due to corruption, and now it hangs my machine > (strace finds > that df is hanging on

Re: Blocked tasks on 3.15.1, raid1 btrfs is no ends of trouble for me

2014-07-18 Thread Marc MERLIN
works well enough. On Fri, Jul 18, 2014 at 05:44:57PM -0700, Marc MERLIN wrote: > On Fri, Jul 18, 2014 at 05:33:45PM -0700, Marc MERLIN wrote: > > Howver, I have found that btrfs raid 1 on top of dmcrypt has given me no > > ends of trouble. > > I lost that filesystem twice due t

Re: Blocked tasks on 3.15.1, raid1 btrfs is no ends of trouble for me

2014-07-18 Thread Marc MERLIN
On Sat, Jul 19, 2014 at 11:59:24AM +1000, Chris Samuel wrote: > On Fri, 18 Jul 2014 05:44:57 PM Marc MERLIN wrote: > > > Sorry, this may be a bit misleading. I actually lost 2 filesystems that > > were raid0 on top of dmcrypt. > > Stupid question I know, but does thi

Re: Blocked tasks on 3.15.1

2014-07-22 Thread Marc MERLIN
On Tue, Jul 22, 2014 at 10:53:03AM -0400, Chris Mason wrote: > Thanks for the help in tracking this down everyone. We'll get there! > Are you all running multi-disk systems (from a btrfs POV, more than one > device?) I don't care how many physical drives this maps to, just does > btrfs think ther

Re: 40TB volume taking over 16 hours to mount, any ideas?

2014-08-09 Thread Marc MERLIN
On Sat, Aug 09, 2014 at 05:01:24PM +, Duncan wrote: > Jose Ildefonso Camargo Tolosa posted on Sat, 09 Aug 2014 11:06:37 -0500 as > excerpted: > > > 3.16 (still in development) > > ?? > > 3.16 has been out for nearly a week now and we're nearing half-way thru > the 3.17 commit-window. Based

Re: [PATCH] Btrfs: fix csum tree corruption, duplicate and outdated checksums

2014-08-09 Thread Marc MERLIN
On Sat, Aug 09, 2014 at 09:22:27PM +0100, Filipe Manana wrote: (100 lines of detailled explanations snipped) > - slot = 0; > + slot = path->slots[0]; And this is why, trying to rank kernel contributions by number of lines or characters is a very poor guide

Re: Fixing the btrfs deadlocks

2014-08-10 Thread Marc MERLIN
On Sun, Aug 10, 2014 at 12:43:31PM +, Holger Hoffstätte wrote: > On Sat, 09 Aug 2014 11:21:13 -0700, Marc MERLIN wrote: > > > I sure hope that deadlock patch is going to be added to the 3.16.x tree, > > I'm not super stocked with being stuck at 3.14. > >

Re: Fixing the btrfs deadlocks

2014-08-10 Thread Marc MERLIN
On Sun, Aug 10, 2014 at 03:42:09PM +, Holger Hoffstätte wrote: > On Sun, 10 Aug 2014 07:39:00 -0700, Marc MERLIN wrote: > > > My apologies if I missed some Emails, but I'm a bit confused. > > The deadlocks happen reliably with 3.15+, but those patches are marked as >

Re: File system stuck in scrub

2014-08-11 Thread Marc MERLIN
On Mon, Aug 11, 2014 at 11:45:45AM -0400, Calvin Walton wrote: > > $ sudo btrfs scrub start /home/nikratio/ > > ERROR: scrub is already running. > > To cancel use 'btrfs scrub cancel /home/nikratio/'. > > To see the status use 'btrfs scrub status [-d] /home/nikratio/'. > My guess is that this is a

Re: Blocked tasks on 3.15.1

2014-08-11 Thread Marc MERLIN
On Mon, Aug 11, 2014 at 08:55:21PM -0600, Charles Cazabon wrote: > The blocked tasks issue that got significantly worse in 3.15 -- did anything > go into 3.16 related to this? I didn't see a single "btrfs" in Linus' 3.16 > announcement, so I don't know whether it should be better, the same, or wor

Re: 40TB volume taking over 16 hours to mount, any ideas?

2014-08-12 Thread Marc MERLIN
On Tue, Aug 12, 2014 at 04:15:26AM +, Duncan wrote: > Jose Ildefonso Camargo Tolosa posted on Mon, 11 Aug 2014 16:33:36 -0500 as > excerpted: > > > As I hate when a thread is left "hanging", you deserve to know what > > happened in the end, you likely already guessed, but anyway: I nuked the >

Re: delete missing /dev/sdd which is now added as /dev/sdd1

2014-08-13 Thread Marc MERLIN
On Wed, Aug 13, 2014 at 07:35:49AM +0200, Rasmus Abrahamsen wrote: > I added the /dev/sdd1 to my raid and deleted the missing. Now the > /dev/sdd does not show up anymore and I have a /dev/sdd1. But I still > have the Some devices missing and the command btrfs delete missing > /mnt does not ac

btrfs-zero-log fails, can't mount FS

2014-08-14 Thread Marc MERLIN
Running 3.15.5, laptop hung overnight, I was forced to reboot with sysrq. After that, it wouldn't mount anymore: [ 689.366125] BTRFS: device label btrfs_pool1 devid 1 transid 237214 /dev/dm-1 [ 716.384377] BTRFS info (device dm-1): disk space caching is enabled [ 716.566974] BTRFS: detected SSD

Re: btrfs-zero-log fails, can't mount FS

2014-08-14 Thread Marc MERLIN
On Thu, Aug 14, 2014 at 12:52:35PM -0400, Austin S Hemmelgarn wrote: > I don't think it is likely that the Samsung SSD is to blame, in my > experience Samsung's SSD's are better than almost every other brand > except Intel, and I know that they honor write-barriers correctly. > The likely issue is

Re: btrfs-zero-log fails, can't mount FS

2014-08-14 Thread Marc MERLIN
On Thu, Aug 14, 2014 at 01:10:05PM -0600, Chris Murphy wrote: > > On Aug 14, 2014, at 11:27 AM, Marc MERLIN wrote: > > > On Thu, Aug 14, 2014 at 12:52:35PM -0400, Austin S Hemmelgarn wrote: > >> I don't think it is likely that the Samsung SSD is to blame, in my &g

Re: btrfs-zero-log fails, can't mount FS

2014-08-14 Thread Marc MERLIN
On Thu, Aug 14, 2014 at 06:03:09PM -0400, Chris Mason wrote: > At least I'll get to buy you a beer this time. Haha, no worries :) > Lets just see if the log root is the only problem. This will get you > through btrfs-zero-log It sure did, thanks much for the patch. It output absolutely nothing

Re: btrfs-zero-log fails, can't mount FS

2014-08-14 Thread Marc MERLIN
On Thu, Aug 14, 2014 at 08:17:02PM -0400, Chris Mason wrote: > Yes, btrfs-zero log doesn't need that root to be read. I'll fix it up, Cool, thanks for fixing that, this one was easy considering :) > really glad it worked for you. You and me both :) Your timely reply today was very much appreci

Re: Questions on using BtrFS for fileserver

2014-08-19 Thread Marc MERLIN
On Tue, Aug 19, 2014 at 06:21:52PM +0200, M G Berberich wrote: > · incremental send/receive works. Yes. > · There is no support for hotspares (spare disks that automatically > replaces faulty disk). Correct > · BtrFS with RAID1 is fairly stable. >From what I know. > · RAID 5/6 spreads all

Re: btrfs restore

2014-08-21 Thread Marc MERLIN
On Thu, Aug 21, 2014 at 05:52:01AM +, Mihail Zaporozhets wrote: > # btrfs-zero-log /dev/sda1 > warning devid 5 not found already > Check tree block failed, want=16845270495232, have=0 > read block failed check_tree_block > Couldn't read tree root You may be hitting the

Re: btrfs restore

2014-08-21 Thread Marc MERLIN
uffer_uptodate(root->node)) { fprintf(stderr, "Couldn't read tree root\n"); return -EIO; } This looks more serious, and I'm not sure if btrfs-zero-log can help with that. I'll let someone else answer. Marc On Thu, Aug 21, 2014 at

Re: Significance of high number of mails on this list?

2014-08-21 Thread Marc MERLIN
On Fri, Aug 22, 2014 at 09:10:55AM +0530, Shriramana Sharma wrote: > Hello people. Thank you for your detailed replies, esp Duncan. > > In essence, I plan on using BTRFS for my production data -- mainly > programs/documents I write in connection with my academic research. > I'm not a professional

Re: Significance of high number of mails on this list?

2014-08-22 Thread Marc MERLIN
On Fri, Aug 22, 2014 at 09:56:55AM +0300, Konstantinos Skarlatos wrote: > I would stay with rsync for a while, because there is always the > possibility of a bug that corrupts both your primary filesystem and > your backup one, or send propagating corruption from one filesystem > to another (Or may

Re: Distro vs latest kernel for BTRFS?

2014-08-22 Thread Marc MERLIN
On Fri, Aug 22, 2014 at 05:29:29PM +0530, Shriramana Sharma wrote: > Hello. I've seen repeated advices to use the latest kernel. While > hearing of the recent compression bug affecting recent kernels does > somewhat warn one off the previous advice, I would like to know what > people who are runnin

Unclean shutdowns cause google-chrome profile to be corrupted in various ways

2014-08-22 Thread Marc MERLIN
Someone just told me yesterday they had the same problem, so I filed a bug: https://bugzilla.kernel.org/show_bug.cgi?id=83041 Fairly often (over 20 times for me so far with various kernel versions), when I reboot after a crash, my google-chrome profile is damaged in one of 2 ways: 1) open tabs don

Re: Unclean shutdowns cause google-chrome profile to be corrupted in various ways

2014-08-22 Thread Marc MERLIN
On Fri, Aug 22, 2014 at 12:32:27PM -0500, Eric Sandeen wrote: > On 8/22/14, 10:50 AM, Marc MERLIN wrote: > > > But if my kernel hangs due to a bug that isn't btrfs' fault and I need > > to power off and back on, after reboot my google-chrome profile is > >

Re: Unclean shutdowns cause google-chrome profile to be corrupted in various ways

2014-08-22 Thread Marc MERLIN
On Fri, Aug 22, 2014 at 06:17:38PM +, Duncan wrote: > Marc MERLIN posted on Fri, 22 Aug 2014 08:50:40 -0700 as excerpted: > > > Fairly often (over 20 times for me so far with various kernel versions), > > when I reboot after a crash, my google-chrome profile is damaged in

Re: Unclean shutdowns cause google-chrome profile to be corrupted in various ways

2014-08-22 Thread Marc MERLIN
On Sat, Aug 23, 2014 at 02:52:16AM +, Duncan wrote: > > For mysql, I got: > > InnoDB: Page directory corruption: > > infimum not pointed to 140708 11:53:58 > > InnoDB: Page dump in ascii and hex (16384 bytes): > > len 16384; hex (16KB of 0's). > > Is that on ssd or spinning rust, and

Re: Unclean shutdowns cause google-chrome profile to be corrupted in various ways

2014-08-23 Thread Marc MERLIN
On Sat, Aug 23, 2014 at 02:45:25PM +0900, Naohiro Aota wrote: > On Sat, Aug 23, 2014 at 12:10 PM, Marc MERLIN wrote: > > On Sat, Aug 23, 2014 at 02:52:16AM +, Duncan wrote: > >> > For mysql, I got: > >> > InnoDB: Page directory corruption: > >>

Re: Unclean shutdowns cause google-chrome profile to be corrupted in various ways

2014-08-23 Thread Marc MERLIN
On Sat, Aug 23, 2014 at 05:56:28AM +, Duncan wrote: > Of course that begs the question of whether it was a normal COW file or > if you had it NOCOW. Setting it NOCOW (of course doing the correct set I had it at the default of COW, both chrome and mysql. Marc -- "A mouse is a device used t

Re: btrfs unmountable, any btrfs tool segfaults

2014-08-23 Thread Marc MERLIN
On Sun, Aug 24, 2014 at 01:21:17AM +, Duncan wrote: > It used to be common courtesy to read a couple weeks of the the backgroup/ > backlist before posting questions as they might be answered already. I > guess it isn't so these days... This is not scalable. There shouldn't be any recovery wi

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-03 Thread Marc MERLIN
First, a big thank you for taking the time to post this very informative message. On Wed, Jan 01, 2014 at 12:37:42PM +, Duncan wrote: > Apparently the way some distribution installation scripts work results in > even a brand new installation being highly fragmented. =:^( If in > addition th

Re: coredump in btrfsck

2014-01-03 Thread Marc MERLIN
On Thu, Jan 02, 2014 at 10:37:28AM -0700, Chris Murphy wrote: > > On Jan 1, 2014, at 3:35 PM, Oliver Mangold wrote: > > > On 01.01.2014 22:58, Chris Murphy wrote: > >> On Jan 1, 2014, at 2:27 PM, Oliver Mangold wrote: > >> > >>> I fear, I broke my FS by running btrfsck. I tried 'btrfsck --repa

Re: Is anyone using btrfs send/receive for backups instead of rsync?

2014-01-03 Thread Marc MERLIN
On Mon, Dec 30, 2013 at 09:57:40AM -0800, Marc MERLIN wrote: > On Mon, Dec 30, 2013 at 10:48:10AM -0700, Chris Murphy wrote: > > > > On Dec 30, 2013, at 10:10 AM, Marc MERLIN wrote: > > > > > > If one day, it could at least work on a subvolume level (only sync

Re: coredump in btrfsck

2014-01-05 Thread Marc MERLIN
On Fri, Jan 03, 2014 at 05:14:56PM -0700, Chris Murphy wrote: > > On Jan 3, 2014, at 5:33 AM, Marc MERLIN wrote: > > > > Would it be possible for whoever maintains btrfs-tools to change both > > the man page and the help included in the tool to clearly state that >

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Marc MERLIN
On Fri, Jan 03, 2014 at 09:34:10PM +, Duncan wrote: > > Thank you for that tip, I had been unaware of it 'till now. > > This will make my virtualbox image directory much happier :) > > I think I said it, but it bears repeating. Once you set that attribute > on the dir, you may want to move t

Re: Is anyone using btrfs send/receive howto?

2014-01-07 Thread Marc MERLIN
I read different howtos on the wiki and oracle docs, but I can't get it to work: legolas:/mnt/btrfs_pool1# btrfs subvolume snapshot -r tmp tmp_read_only_new Create a readonly snapshot of 'tmp' in './tmp_read_only_new' legolas:/mnt/btrfs_pool1# sync legolas:/mnt/btrfs_pool1# btrfs send tmp_read_onl

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-07 Thread Marc MERLIN
On Fri, Jan 03, 2014 at 09:34:10PM +, Duncan wrote: > IIRC someone also mentioned problems with autodefrag and an about 3/4 gig > systemd journal. My gut feeling (IOW, *NOT* benchmarked!) is that double- > digit MiB files should /normally/ be fine, but somewhere in the lower > triple digits,

Re: Is anyone using btrfs send/receive howto?

2014-01-08 Thread Marc MERLIN
On Tue, Jan 07, 2014 at 10:53:29AM +, Hugo Mills wrote: >You need to move /mnt/btrfs_pool2/tmp_read_only_new to a different > name as well. The send stream contains the name of the subvolume it > wants to create, so it's trying to create a subvolume called > "tmp_read_only_new" in /mnt/btrf

Re: Fwd: btrfsck failes

2014-01-15 Thread Marc MERLIN
On Wed, Jan 15, 2014 at 08:53:55AM +0100, Holger Brandsmeier wrote: > # btrfs-zero-log /dev/sda5 > parent transid verify failed on 602529792 wanted 23460 found 23463 > parent transid verify failed on 602529792 wanted 23460 found 23463 > parent transid verify failed on 602529792 wanted 23460 found 2

Re: btrfsck failes

2014-01-15 Thread Marc MERLIN
On Mon, Jan 13, 2014 at 05:37:31PM -0700, Chris Murphy wrote: > > On Jan 13, 2014, at 3:58 PM, Holger Brandsmeier wrote: > > > > Currently btrfsck failes to repair my partition, I get the output: > > > > [root@ho-think bholger]# btrfsck --repair /dev/sda5 > > This is almost the last resort and

Re: btrfsck failes

2014-01-16 Thread Marc MERLIN
On Wed, Jan 15, 2014 at 10:16:13AM -0700, Chris Murphy wrote: > > On Jan 15, 2014, at 9:15 AM, Mitch Harder > wrote: > > > On Mon, Jan 13, 2014 at 6:37 PM, Chris Murphy > > wrote: > >> > >> On Jan 13, 2014, at 3:58 PM, Holger Brandsmeier > >> wrote: > >>> > >>> Currently btrfsck failes to

Re: btrfs-convert destroyed my system

2014-01-18 Thread Marc MERLIN
On Fri, Jan 17, 2014 at 07:30:49PM -0800, Sir Civit wrote: > > > To start off, I have an encrypted LVM setup with a root logical volume and a > home > logical volume. Today decided to upgrade my home LV to btrfs for > compression. I installed btrfs-progs, unmounted /home, and ran > btrfs-conve

Is there a lightweight copy from one subvolume to another?

2014-01-24 Thread Marc MERLIN
If I have /mnt/btrfs1/subvol1 /mnt/btrfs1/subvol2 Can I copy/move data from subvol1 to subvol2 without having to physically copy all the blocks as if it were an entirely different filesystem? reflink doesn't work for this: gargamel [mc]# cp -av --reflink=always misc/olympic Video/misc/ `misc/oly

Re: Fwd: Is there a lightweight copy from one subvolume to another?

2014-01-25 Thread Marc MERLIN
On Sat, Jan 25, 2014 at 09:50:48PM +1300, Maxdamantus wrote: > Due to what I understand is some VFS limitation, you can only > reflink-copy between subvolumes under one Linux mountpoint. If > /mnt/btrfs1 provides access to all subvolumes, you should be able to > copy between them through that path,

Can you keep reflink relationship during a copy/backup to another filesystem?

2014-01-28 Thread Marc MERLIN
So I used to use hardlinks to do historical backups of the same filesystem but I know it's preferable to use refllink with btrfs to avoid having too many hardlinks. But if I need to backup this filesystem to another one some other way than btrfs send/receive (let's say cp -a, tar, or rsync), is it

btrfs snapshot is killing IO and hanging my device with delayed writes for 10mn+

2014-02-07 Thread Marc MERLIN
On my workstation, which unfortunately I can't easily upgrade the kernel on, so it's running 3.8.0 for now, I've had pretty perpelexing hangs from btrfs snapshot when I make a new snapshot every hour. I see the btrfs snapshot command in iotop for over a minute, and it completes eventually. Other

Re: Can you keep reflink relationship during a copy/backup to another filesystem?

2014-02-07 Thread Marc MERLIN
On Wed, Jan 29, 2014 at 08:05:14AM +, Hugo Mills wrote: > On Tue, Jan 28, 2014 at 11:50:25PM -0800, Marc MERLIN wrote: > > So I used to use hardlinks to do historical backups of the same filesystem > > but I know it's preferable to use refllink with btrfs to avoid having to

Bedup bug report

2014-02-08 Thread Marc MERLIN
kernel 3.12.7, python 2.7.6-5, debian testing/unstable, bedup installed as per pip install --user bedup I tried installing the git version, but the error is the same: Anyway, with the other bedup, I get: gargamel:/mnt/dshelf2/backup# bedup show Traceback (most recent call last): File "/usr/loca

Re: Bedup bug report

2014-02-12 Thread Marc MERLIN
Does anyone know who the maintainer to send bug reports to, is? On Sat, Feb 08, 2014 at 09:19:36PM -0800, Marc MERLIN wrote: > kernel 3.12.7, python 2.7.6-5, debian testing/unstable, bedup installed as per > pip install --user bedup > > I tried installing the git version, but the

How to recover from failing btrffs send | btrfs receive?

2014-02-12 Thread Marc MERLIN
So, I've veen running this for a few weeks, and soon should have something half decent to share for others to use. Unfortunately, one of my backups is now failing like so: btrfs send -p "$src_snap" "$src_newsnap" | btrfs receive "$dest_pool/" + btrfs send -p /mnt/btrfs_pool1/home_ro.20140209_12:0

Re: How to recover from failing btrffs send | btrfs receive?

2014-02-13 Thread Marc MERLIN
Ok, let me try something else :) Of those who are using btrfs send/receive, has anyone gotten in a state where incrementals will not apply anymore? Thanks, Marc On Wed, Feb 12, 2014 at 06:22:07AM -0800, Marc MERLIN wrote: > So, I've veen running this for a few weeks, and soon sho

Re: How to recover from failing btrfs send | btrfs receive?

2014-02-16 Thread Marc MERLIN
2, 2014 at 06:22:07AM -0800, Marc MERLIN wrote: > So, I've veen running this for a few weeks, and soon should have > something half decent to share for others to use. > > Unfortunately, one of my backups is now failing like so: > > btrfs send -p "$src_snap" "$s

Re: How to recover from failing btrfs send | btrfs receive?

2014-02-16 Thread Marc MERLIN
On Sun, Feb 16, 2014 at 03:38:18PM +, Filipe David Manana wrote: > On Sun, Feb 16, 2014 at 2:23 PM, Marc MERLIN wrote: > > Hi Fillipe, I see you have another fix for btrfs send (attached below), > > as ell as your other patch on Jan 21st (neither are in my 3.12.7). >

Re: How to recover from failing btrfs send | btrfs receive?

2014-02-16 Thread Marc MERLIN
On Sun, Feb 16, 2014 at 09:08:57PM +, Filipe David Manana wrote: > I'll see if I come up with other ways of getting into that issue. If you're collecting them, I found another bug, although it might not matter to most: if I put my laptop in S3 sleep during a send/receive, it reliably breaks th

btrfs send ioctl failed with -25: Inappropriate ioctl for device

2014-02-22 Thread Marc MERLIN
On Sun, Feb 16, 2014 at 09:32:32PM -0800, Marc MERLIN wrote: > On Sun, Feb 16, 2014 at 09:08:57PM +, Filipe David Manana wrote: > > I'll see if I come up with other ways of getting into that issue. > > If you're collecting them, I found another bug, although it might

btrfs userland interface isn't 32/64bit clean (breaks lsattr and btrfs send)

2014-02-23 Thread Marc MERLIN
I was trying to make sense out of this: gargamel:~# lsattr lsattr: Inappropriate ioctl for device While reading flags on ./satapmtool lsattr: Inappropriate ioctl for device While reading flags on ./usbreset As well as the btrfs send issue I reported: gargamel:/mnt/btrfs_pool1# btrfs send media_ro

3.13.5 kernel hangs some processes with btrfs

2014-02-23 Thread Marc MERLIN
Does someone know how I can debug further why this is hanging? It seems that accessing a certain directory on one of my btrfs filesystems causes this. The rest of my system seems ok, as long as I'm not touching this filesystem. Is this a bug, or a performance problem? [ 1930.287192] INFO: task

Re: 3.13.5 kernel hangs some processes with btrfs

2014-02-23 Thread Marc MERLIN
On Sun, Feb 23, 2014 at 10:14:26PM -0800, Marc MERLIN wrote: > Does someone know how I can debug further why this is hanging? > > It seems that accessing a certain directory on one of my btrfs filesystems > causes this. > > The rest of my system seems ok, as long as I&#

Re: 3.13.5 kernel hangs some processes with btrfs

2014-02-23 Thread Marc MERLIN
On Mon, Feb 24, 2014 at 02:27:46PM +0800, Wang Shilong wrote: > >Note that it says running for 5 seconds, but it started 4H ago. > > > >Any idea what's going on here? > What is dmesg output? > > Did it output something like "Skip abort transaction"? Also > what is your mount option? did you enable

Re: 3.13.5 kernel hangs some processes with btrfs

2014-02-23 Thread Marc MERLIN
On Mon, Feb 24, 2014 at 06:42:30AM +, Duncan wrote: > I believe there's a fix coming (a cancel that blows away the tracking > file if it finds it and no actual running scrub is the most obvious fix), > but meanwhile, see the /var/lib/btrfs/scrub.status.* files. That's where > scrub state is

Re: 3.13.5 kernel hangs some processes with btrfs

2014-02-24 Thread Marc MERLIN
On Mon, Feb 24, 2014 at 07:29:58AM +, Duncan wrote: > > But I'm still seeing these, albeit less often. > > Any idea what they could be linked to? > > (I have a btrs send/receive going right now, it could hanging > > /mnt/btrfs_pool1 in a way that affects smbd, but the array feels ok > > otherwi

Re: 3.14rc3 kernel also hangs some processes with btrfs

2014-02-24 Thread Marc MERLIN
On Mon, Feb 24, 2014 at 09:35:19AM -0800, Marc MERLIN wrote: > On Mon, Feb 24, 2014 at 07:29:58AM +, Duncan wrote: > > > But I'm still seeing these, albeit less often. > > > Any idea what they could be linked to? > > > (I have a btrs send/receive going rig

3.14.0rc3: did not find backref in send_root

2014-02-24 Thread Marc MERLIN
I got this during a btrfs send: BTRFS error (device dm-2): did not find backref in send_root. inode=22672, offset=524288, disk_byte=1490517954560 found extent=1490517954560 I'll try a scrub when I've finished my backup, but is there anything I can run on the file I've found from the inode? garga

Re: btrfs userland interface isn't 32/64bit clean (breaks lsattr and btrfs send)

2014-02-24 Thread Marc MERLIN
On Mon, Feb 24, 2014 at 08:43:44AM +, Duncan wrote: > Hugo Mills posted on Mon, 24 Feb 2014 08:29:38 + as excerpted: > > > On Mon, Feb 24, 2014 at 06:32:14AM +, Duncan wrote: > >> > >> This is a known issue. There's patches in the pipeline for 32-bit > >> userspace on a 64-bit kernel

Re: 3.14.0rc3: btrfs send ioctl failed with -5: Input/output error

2014-02-25 Thread Marc MERLIN
On Tue, Feb 25, 2014 at 03:50:15PM +0800, Wang Shilong wrote: > Hi Marc, > > This seems a regression which has been fixed by the following > commit(only pushed into btrfs-next): > > https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/?id=1334bebe71bebbca47b3b92f25511ea980fdea

Re: 3.14.0rc3: did not find backref in send_root

2014-02-25 Thread Marc MERLIN
On Wed, Feb 26, 2014 at 11:38:30AM +0800, Wang Shilong wrote: > Hi Marc, > > On 02/26/2014 01:30 AM, Marc MERLIN wrote: > >On Tue, Feb 25, 2014 at 03:50:15PM +0800, Wang Shilong wrote: > >>Hi Marc, > >> > >>This seems a regression which has been fixed by

Re: 3.14.0rc3: did not find backref in send_root

2014-02-26 Thread Marc MERLIN
On Wed, Feb 26, 2014 at 03:51:37PM +0800, Wang Shilong wrote: > >I've applied your patch from > >https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/?id=1334bebe71bebbca47b3b92f25511ea980fdeab8 I can confirm this fixed the btrfs send error on my server, thank you. > >At snaps

3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-02-27 Thread Marc MERLIN
This does not happen consistently, but sometimes: PM: Preparing system for mem sleep Freezing user space processes ... (...) Freezing of tasks failed after 20.002 seconds (1 tasks refusing to freeze, wq_busy=0): btrfs D 88017639c800 0 12239 12224 0x0084 880165ec196

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-02-27 Thread Marc MERLIN
On Thu, Feb 27, 2014 at 11:06:56AM -0800, Marc MERLIN wrote: > This does not happen consistently, but sometimes: > > PM: Preparing system for mem sleep > Freezing user space processes ... > (...) > Freezing of tasks failed after 20.002 seconds (1 tasks refusing to fr

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-03-01 Thread Marc MERLIN
On Fri, Feb 28, 2014 at 09:09:37PM -0800, Marc MERLIN wrote: > On Fri, Feb 28, 2014 at 09:18:06AM +0800, Wang Shilong wrote: > > Could you run the following command when scrub is blocked, we can know more > > why scrub is blocked here. > > > > # echo w > /proc/sy

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-03-02 Thread Marc MERLIN
On Mon, Mar 03, 2014 at 11:17:51AM +0800, Wang Shilong wrote: > Hi Marc, > > On 03/01/2014 11:22 PM, Marc MERLIN wrote: > >On Fri, Feb 28, 2014 at 09:09:37PM -0800, Marc MERLIN wrote: > >>On Fri, Feb 28, 2014 at 09:18:06AM +0800, Wang Shilong wrote: > >>>Coul

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-03-03 Thread Marc MERLIN
On Mon, Mar 03, 2014 at 02:50:33PM +0800, Wang Shilong wrote: > >Here's the log of failure: > >http://marc.merlins.org/tmp/btrfs_nofreeze2.txt > > Unfortunately, i could not reproduce this problem here. > > It should not be the problem that i addressed before, there is not > deadlock here. > try

<    1   2   3   4   5   6   7   8   >