Re: [gentoo-user] ext4 inode limit reached
On 13.12.2011 01:44, Frank Steinmetzger wrote: On my system, /usr/portage currently contains 127000 files. But for reason of increased performance I put it into a squashfs file. (There was a nice howto on this ML some months ago). You could try that, which will free those inodes up and ideally leaves you with one used inode for the squashfs image. Plus, if you have enough RAM, you could put /var/tmp/portage into tmpfs. I have 3GB, and this is fairly enough. For other hogs like firefox, LO and java, I use binary packages though. For comparision, I too have one (seldom two) kernel source trees and everything else on / except /home. And while of the 17GB capacity barely 1GB is left free, I still have 480k inodes free of the 1M in total. (I figured that I may have more space for content if I reserved less for inodes). I had portage in a squashfs before too - that was nice :) That's also the reason I had /usr/portage /var/cache/edb and /var/db/pkg on one filesystem - all together in the squashfs :) Because one day my / became full I moved /usr/src onto that partition too (it was now on a reiser3fs). All fine, and other partitions less fragmented... until I moved to ext4. Now I have a SSD, and it's simpler than squashfs'ing and still fast. distfiles is on a HDD (Thinkpad notebook with ultrabay - love it) and compilation on tmpfs (8GB RAM, so no problems). I always have like 8 kernel trees lying around, so there are already like 400k files... For various reasons I like to keep my stuff on separate partitions - my system is distributed over 6 partitions and my personal data over 3 partitions :) That inode-trouble was actually quiet interesting ;) Daniel -- PGP key @ http://pgpkeys.pca.dfn.de/pks/lookup?search=0xBB9D4887op=get # gpg --recv-keys --keyserver hkp://subkeys.pgp.net 0xBB9D4887 signature.asc Description: OpenPGP digital signature
[gentoo-user] ext4 inode limit reached
Hello :) I have an ext4-filesystem that contains /usr/src, the /usr/portage and /var/cache/edb. It previously also contained /var/db/pkg, but I had to move that some weeks ago, because the fs was full. Now it's full again, though it has free blocks. But no inodes are left: $ fsck -vf /dev/sda5 [..] 655360 inodes used (100.00%) [..] $ find /gentoo -xdev | wc -l 655338 That's really disappointing. I was using reiser3fs and XFS before, and they didn't have that kind of limitation... Uhm... not meant as a rant - I like ext4 - that's why I'm moving (almost?) everything to it... Is there any way to raise the number of inodes without using $ mkfs.ext4 -N BIGNUM Thank you, Daniel -- PGP key @ http://pgpkeys.pca.dfn.de/pks/lookup?search=0xBB9D4887op=get # gpg --recv-keys --keyserver hkp://subkeys.pgp.net 0xBB9D4887 signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] ext4 inode limit reached
Daniel Troeder writes: I have an ext4-filesystem that contains /usr/src, the /usr/portage and /var/cache/edb. It previously also contained /var/db/pkg, but I had to move that some weeks ago, because the fs was full. Now it's full again, though it has free blocks. But no inodes are left: $ fsck -vf /dev/sda5 [..] 655360 inodes used (100.00%) [..] $ find /gentoo -xdev | wc -l 655338 That's really disappointing. I was using reiser3fs and XFS before, and they didn't have that kind of limitation... Uhm... not meant as a rant - I like ext4 - that's why I'm moving (almost?) everything to it... Is there any way to raise the number of inodes without using $ mkfs.ext4 -N BIGNUM Not really I think. You can enlarge the file system with resize2fs, this will also increase the number of inodes, but that's probably hard when not using LVM, and it's not really what you want, as the file system will be larger than it needs to be. Wonko
Re: [gentoo-user] ext4 inode limit reached
On 12/12/11 12:15, Daniel Troeder wrote: Hello :) I have an ext4-filesystem that contains /usr/src, the /usr/portage and /var/cache/edb. It previously also contained /var/db/pkg, but I had to move that some weeks ago, because the fs was full. Now it's full again, though it has free blocks. But no inodes are left: $ fsck -vf /dev/sda5 [..] 655360 inodes used (100.00%) [..] $ find /gentoo -xdev | wc -l 655338 That's really disappointing. I was using reiser3fs and XFS before, and they didn't have that kind of limitation... Uhm... not meant as a rant - I like ext4 - that's why I'm moving (almost?) everything to it... Is there any way to raise the number of inodes without using $ mkfs.ext4 -N BIGNUM Thank you, Daniel That is scary. I just install new HD with 2TB capacity and ext4 that is 2% full and: $ find /home/joseph/ -xdev | wc -l shows: 169977 that is 26% full. So will run out of inodes before I run out of hard disk space :-/ that is not good. If other filesystems don't have these kind of limitation I'll be switching. -- Joseph
Re: [gentoo-user] ext4 inode limit reached
Quick googling around indicates that JFS, or XFS don't have these limitation. -quote Many computer programs used by system administrators in UNIX operating systems often designate files with inode numbers. Examples include popular disk integrity checking utilities such as the fsck or pfiles. Thus, the need naturally arises to translate inode numbers to file pathnames and vice versa. This can be accomplished using the file finding utility find with the -inum option, or the ls command with the proper option (-i on POSIX compliant platforms). It is possible to use up a device's set of inodes. When this happens, new files cannot be created on the device, even though there may be free space available. For example, a mail server may have many small files that don't fill up the disk, but use many inodes to point to the numerous files. Filesystems (such as JFS, or XFS) escape this limitation with extents and/or dynamic inode allocation, which can 'grow' the filesystem and/or increase the number of inodes. end quote-- -- Joseph
Re: [gentoo-user] ext4 inode limit reached
On Dec 12, 2011 9:39 PM, Joseph syscon...@gmail.com wrote: Quick googling around indicates that JFS, or XFS don't have these limitation. -quote Many computer programs used by system administrators in UNIX operating systems often designate files with inode numbers. Examples include popular disk integrity checking utilities such as the fsck or pfiles. Thus, the need naturally arises to translate inode numbers to file pathnames and vice versa. This can be accomplished using the file finding utility find with the -inum option, or the ls command with the proper option (-i on POSIX compliant platforms). It is possible to use up a device's set of inodes. When this happens, new files cannot be created on the device, even though there may be free space available. For example, a mail server may have many small files that don't fill up the disk, but use many inodes to point to the numerous files. Filesystems (such as JFS, or XFS) escape this limitation with extents and/or dynamic inode allocation, which can 'grow' the filesystem and/or increase the number of inodes. end quote-- ReiserFS also doesn't have problems with inodes because everything are kept in b*trees that can keep growing indefinitely. In fact, I think I read somewhere that ReiserFS is perfect for /var/tmp and /usr/src due to the amount of small files in those directories. Rgds,
Re: [gentoo-user] ext4 inode limit reached
Joseph writes: On 12/12/11 12:15, Daniel Troeder wrote: Hello :) I have an ext4-filesystem that contains /usr/src, the /usr/portage and /var/cache/edb. It previously also contained /var/db/pkg, but I had to move that some weeks ago, because the fs was full. Now it's full again, though it has free blocks. But no inodes are left: $ fsck -vf /dev/sda5 [..] 655360 inodes used (100.00%) [..] $ find /gentoo -xdev | wc -l 655338 That's really disappointing. I was using reiser3fs and XFS before, and they didn't have that kind of limitation... Uhm... not meant as a rant - I like ext4 - that's why I'm moving (almost?) everything to it... Is there any way to raise the number of inodes without using $ mkfs.ext4 -N BIGNUM Thank you, Daniel That is scary. I just install new HD with 2TB capacity and ext4 that is 2% full and: $ find /home/joseph/ -xdev | wc -l shows: 169977 that is 26% full. No, that is 26% of the number of total inodes _Daniel_ has on his small partition. Yours is bigger, so you have more inodes. My largest partition has 724G, and 46 million inodes. Use df -i to see how many you have. So will run out of inodes before I run out of hard disk space :-/ that is not good. If other filesystems don't have these kind of limitation I'll be switching. Having too few inodes has been a problem for me in the past. But that was either a tiny partition for the portage tree, which has so many small files. Or the partition where I back up my /var partition with rdiff-backup, so it also has lots of files, and with each backup the same amount gets added. If you intend to place unusually many files on a partition, check how many inodes mkfs has created, and re-create the file system using the -N option, giving it a somewhat larger number. Wonko
Re: [gentoo-user] ext4 inode limit reached
On 12.12.2011 15:54, Alex Schuster wrote: Joseph writes: That is scary. I just install new HD with 2TB capacity and ext4 that is 2% full and: $ find /home/joseph/ -xdev | wc -l shows: 169977 that is 26% full. No, that is 26% of the number of total inodes _Daniel_ has on his small partition. Yours is bigger, so you have more inodes. My largest partition has 724G, and 46 million inodes. Use df -i to see how many you have. Ah yes... My partition is only 10GB, and mkfs.ext4 sais in its man page (at -N option) it uses a calculation [..] based on the number of blocks and the bytes-per-inode ratio. So a small partition will have fewer inodes than a big partition. mkfs.extX uses settings for the inode-block-ratio from /etc/mke2fs.conf. The -T option configures which one to use, my partition falls into category 512MB default 4TB, which makes it use inode_ratio=16384. I think I should use the news type. It has inode_ratio=4096, which should give me 4 times the inodes testing... yes: 2621440 inodes instead of 655360. Previously I used reiser3fs for this kind of filesystem usage, but it's not faster than extX anymore. Moreover it's running on a SSD now, and afaik reiser3fs doesn't support TRIM :( OK - thank you all. It seems I'll have to reformat. Bye, Daniel -- PGP key @ http://pgpkeys.pca.dfn.de/pks/lookup?search=0xBB9D4887op=get # gpg --recv-keys --keyserver hkp://subkeys.pgp.net 0xBB9D4887 signature.asc Description: OpenPGP digital signature
Re: [gentoo-user] ext4 inode limit reached
On Mon, Dec 12, 2011 at 12:15:52PM +0100, Daniel Troeder wrote: I have an ext4-filesystem that contains /usr/src, the /usr/portage and /var/cache/edb. It previously also contained /var/db/pkg, but I had to move that some weeks ago, because the fs was full. Now it's full again, though it has free blocks. But no inodes are left: $ fsck -vf /dev/sda5 [..] 655360 inodes used (100.00%) [..] $ find /gentoo -xdev | wc -l 655338 Is there any way to raise the number of inodes without using $ mkfs.ext4 -N BIGNUM You can’t increase the number in an existing FS, you’ll have to reformat it. If you have a backup, as you should anyway ;-) , you could boot a live system and reformat. On my system, /usr/portage currently contains 127000 files. But for reason of increased performance I put it into a squashfs file. (There was a nice howto on this ML some months ago). You could try that, which will free those inodes up and ideally leaves you with one used inode for the squashfs image. Plus, if you have enough RAM, you could put /var/tmp/portage into tmpfs. I have 3GB, and this is fairly enough. For other hogs like firefox, LO and java, I use binary packages though. For comparision, I too have one (seldom two) kernel source trees and everything else on / except /home. And while of the 17GB capacity barely 1GB is left free, I still have 480k inodes free of the 1M in total. (I figured that I may have more space for content if I reserved less for inodes). -- Gruß | Greetings | Qapla' I forbid any use of my email addresses with Facebook services. A hammer is a wonderful tool, but it is plain unsuitable for cleaning windows. (SelfHTML forum) pgpyh5Ew7w2Gy.pgp Description: PGP signature
Re: [gentoo-user] ext4 inode limit reached
On 12/12/2011 06:15 AM, Daniel Troeder wrote: $ fsck -vf /dev/sda5 [..] 655360 inodes used (100.00%) [..] $ find /gentoo -xdev | wc -l 655338 That's really disappointing. I was using reiser3fs and XFS before, and they didn't have that kind of limitation... Uhm... not meant as a rant - I like ext4 - that's why I'm moving (almost?) everything to it... Is there any way to raise the number of inodes without using $ mkfs.ext4 -N BIGNUM 640k should be enough for anybody.
