Re: severe hardlink bug
Jan Schmidt list.btrfs at jan-o-sch.net writes: Please give the patch set btrfs: extended inode refs by Mark Fasheh a try (http://lwn.net/Articles/498226/). It eliminates the hard links per directory limit (introducing a rather random, artificial limit of 64k instead). Hi, Jan! I'm happy to see that there is something done on fixing that issue. Unfortunately I cannot afford to have unstable patched kernel on the production server. Probably I'll give another try to btrfs in 2013. ^__^ K. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
On 08/08/2012 04:37 PM, Konstantin Dmitriev wrote: Unfortunately I cannot afford to have unstable patched kernel on the production server. I'd suggest that using btrfs for production servers is probably a bit brave (no matter what Oracle say), and what you've found illustrates the point rather well I'm afraid.. cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
Am 29.07.2012 21:13, schrieb C Anthony Risinger: On Sun, Jul 29, 2012 at 2:02 PM, Konstantin Dmitriev ksee.zelga...@gmail.com wrote: Dipl.-Ing. Michael Niederle mniederle at gmx.at writes: I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction. I have hit the same issue - tried to run BackupPC with a pool on btrfs filesystem. After some time the error of too many links (31) appeared to me. Now I'm forced to migrate to some other filesystem... btrfs only fails when you have hundreds of hardlinks to the same file in the *same* directory ... certainly not a standard use case. Actually, hundreds of hardlinks is certainly over optimistic. In my testing 15 links in the same directory were enough to get the Too many links error. It depends on the length of the file name of the hardlinks. Best regards Arnd -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
On Mon, Jul 30, 2012 at 4:36 AM, Arnd Hannemann a...@arndnet.de wrote: Am 29.07.2012 21:13, schrieb C Anthony Risinger: On Sun, Jul 29, 2012 at 2:02 PM, Konstantin Dmitriev ksee.zelga...@gmail.com wrote: Dipl.-Ing. Michael Niederle mniederle at gmx.at writes: I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction. I have hit the same issue - tried to run BackupPC with a pool on btrfs filesystem. After some time the error of too many links (31) appeared to me. Now I'm forced to migrate to some other filesystem... btrfs only fails when you have hundreds of hardlinks to the same file in the *same* directory ... certainly not a standard use case. Actually, hundreds of hardlinks is certainly over optimistic. In my testing 15 links in the same directory were enough to get the Too many links error. It depends on the length of the file name of the hardlinks. Yes, per the linked patch it states 4k as the limit ... I thought I recalled a limit of 256 but it seems I may have been mistaken. The purpose of my initial response was to suggest an alternative strategy -- one complementing btrfs's strengths -- a simple rsync + snapshot is much more effective than BackupPC IMO ... but then again, I'm bias, because I generally think BackupPC is junk. -- C Anthony -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
Hallo, Arnd, Du meintest am 30.07.12: btrfs only fails when you have hundreds of hardlinks to the same file in the *same* directory ... certainly not a standard use case. Actually, hundreds of hardlinks is certainly over optimistic. In my testing 15 links in the same directory were enough to get the Too many links error. It depends on the length of the file name of the hardlinks. And then I may get problems with my favourite backup program rsnapshot; it uses hard links, and my backups often have more than 20 hard links to 1 file. Sometimes about 80 characters long. Viele Gruesse! Helmut -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
Am Sonntag, 24. Januar 2010 schrieb Michael Niederle: I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of a Gentoo Linux installation. Upgrade your kernel! This kernel is wy to old for any production use of BTRFS. Heck, upstream still did not mark BTRFS as stable although Oracle and SUSE do support it. Current kernel have a many fixes. Use at least 3.2. And have backups anyway. The hard-link issue is known since ages and being worked on (see link from Jan in this thread). If you want to help, test the patch. So, folks, can we get over it? Thanks, -- Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
Dipl.-Ing. Michael Niederle mniederle at gmx.at writes: I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction. I have hit the same issue - tried to run BackupPC with a pool on btrfs filesystem. After some time the error of too many links (31) appeared to me. Now I'm forced to migrate to some other filesystem... -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
On Sun, Jul 29, 2012 at 2:02 PM, Konstantin Dmitriev
ksee.zelga...@gmail.com wrote:
Dipl.-Ing. Michael Niederle mniederle at gmx.at writes:
I reinstalled over 700 packages - plt-scheme beeing the only one failing due
to
the btrfs link restriction.
I have hit the same issue - tried to run BackupPC with a pool on btrfs
filesystem. After some time the error of too many links (31) appeared to me.
Now I'm forced to migrate to some other filesystem...
btrfs only fails when you have hundreds of hardlinks to the same file
in the *same* directory ... certainly not a standard use case.
use snapshots to your advantage:
- snap source
- rsync --inplace source to target (with some other opts that have
been discussed on list)
- snap target
- {rinse-and-repeat-in-24-hrs}
--
C Anthony
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
C Anthony Risinger anthony at xtfx.me writes:
btrfs only fails when you have hundreds of hardlinks to the same file
in the *same* directory ... certainly not a standard use case.
use snapshots to your advantage:
- snap source
- rsync --inplace source to target (with some other opts that have
been discussed on list)
- snap target
- {rinse-and-repeat-in-24-hrs}
I understand that the case is only for *same* directory.
You can claim that it's not a standard use case, but first Michael hit that,
now me. There's at least one more case -
https://lists.samba.org/archive/rsync/2011-December/027117.html
The count of such cases will be increasing and the sooner it will be fixed -
the less pain it will bring to the users. I know fixing that is a big
structural change, but it will become worse with time.
If it's not going to be fixed - I don't care. Right now I'm forced
to migrate to old mdadm raid-1 or ZFS. The sad thing is that
I really LOVED btrfs. Only that. ^__^
K.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
On Sun, Jan 24, 2010 at 12:04:32PM +0100, Stephan von Krawczynski wrote: On Sun, 24 Jan 2010 09:09:44 +0100 Goffredo Baroncelli kreij...@gmail.com wrote: On Sunday 24 January 2010, Michael Niederle wrote: I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of a Gentoo Linux installation. While attempting to install the plt-scheme package a strange error about link counts occurred ([Error 31] Too many Links). See this thread: Mass-Hardlinking Oops - http://thread.gmane.org/gmane.comp.file- systems.btrfs/3427 There is a limit of the number of hardlink for a file. The maximum number of the link depends by name length. Honestly, this dependency is braindead. How do the fs authors think an application programmer should judge how many hardlinks are possible for a certain fs ? This is a really bad design issue. Are we really in year 2010 ? Yes, this was a design trade off to maintain the back links for the file. Keep in mind the limit is only per-directory, as long as you're working between different directories you can have any many back links as you want. But, it's on the list to get fixed once raid5/6 are out of the way. -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
On Sunday 24 January 2010, Michael Niederle wrote: I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of a Gentoo Linux installation. While attempting to install the plt-scheme package a strange error about link counts occurred ([Error 31] Too many Links). See this thread: Mass-Hardlinking Oops - http://thread.gmane.org/gmane.comp.file- systems.btrfs/3427 There is a limit of the number of hardlink for a file. The maximum number of the link depends by name length. BR Goffredo -- gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) kreijackATinwind.it Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
On Sun, 24 Jan 2010 09:09:44 +0100 Goffredo Baroncelli kreij...@gmail.com wrote: On Sunday 24 January 2010, Michael Niederle wrote: I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of a Gentoo Linux installation. While attempting to install the plt-scheme package a strange error about link counts occurred ([Error 31] Too many Links). See this thread: Mass-Hardlinking Oops - http://thread.gmane.org/gmane.comp.file- systems.btrfs/3427 There is a limit of the number of hardlink for a file. The maximum number of the link depends by name length. Honestly, this dependency is braindead. How do the fs authors think an application programmer should judge how many hardlinks are possible for a certain fs ? This is a really bad design issue. Are we really in year 2010 ? BR Goffredo -- Regards, Stephan -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
Hi, Goffredo! See this thread: Mass-Hardlinking Oops - http://thread.gmane.org/gmane.comp.file- systems.btrfs/3427 Sorry, I didn't find the thread myself - despite of massive googeling. :-( The gentoo scenario however is a real world application - no artificial situation, so it would be really nice to fix the hard link problem somewhere in the future! The problem will happen in all busybox-like scenarios - and long filenames are not so rarely used. Greetings, Michael -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
On Sunday 24 January 2010, you (Dipl.-Ing. Michael Niederle) wrote: Hi, Goffredo! See this thread: Mass-Hardlinking Oops - May be that the link was broken in two line in the previous mail. I copied again the ling. please give a try to see the thread. http://thread.gmane.org/gmane.comp.file-systems.btrfs/3427 Sorry, I didn't find the thread myself - despite of massive googeling. :-( The gentoo scenario however is a real world application - no artificial situation, so it would be really nice to fix the hard link problem somewhere in the future! At the time of the thread, some peoples raised concerns about this limit. But at the time nobody highlighted a real case which required a lot of hard links. Pay attention that in order to reach the limits the hard links have to be in the same directory. If there is a real case in gentoo/bosybox, please send all the info. But in order solve this problem, it was told that a disk format change is required. BR Goffredo The problem will happen in all busybox-like scenarios - and long filenames are not so rarely used. Greetings, Michael -- gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) kreijackATinwind.it Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: severe hardlink bug
Hi, Goffredo! I understood that a disk layout change will be necessary to accomodate for a bigger total filename-space per file. So it is best done at once when more issues arise that will need layout changes. To recreate the error situation you need a Gentoo Linux with a btrfs root file system and then try to install the plt-scheme package: emerge plt-scheme Compilation works quite fine, but the installation aborts with a Too many links error. I reinstalled over 700 packages - plt-scheme beeing the only one failing due to the btrfs link restriction. Greetings, Michael -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
severe hardlink bug
I'm using btrfs with a kernel 2.6.32.2 (builtin) as the root file system of a Gentoo Linux installation. While attempting to install the plt-scheme package a strange error about link counts occurred ([Error 31] Too many Links). I was able to create a simple scenario to reproduce the error: Create an empty directory and run the following script within the directory: #!/bin/bash echo abc .tab-snip-class_scrbl.dep._portage_merge_.9766 for x in $(seq 80) do echo $x; ln .tab-snip-class_scrbl.dep._portage_merge_.9766 \ .tab-snip-class_scrbl.dep._portage_merge_.9766.$x; done (The names are taken from the original Gentoo package.) When I run this script the first 66 links are created properly. All further link creations fail: ... 63 64 65 66 67 ln: creating hard link to `.tab-snip-class_scrbl.dep._portage_merge_.9766': Too many links 68 ln: creating hard link to `.tab-snip-class_scrbl.dep._portage_merge_.9766': Too many links 69 ln: creating hard link to `.tab-snip-class_scrbl.dep._portage_merge_.9766': Too many links 70 ln: creating hard link to `.tab-snip-class_scrbl.dep._portage_merge_.9766': Too many links ... I was able to reproduce the bug independently on two different machines. During earlier tests I determined that filename length has an effect as well as the link count. Greetings, Michael -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
