Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On Sunday 21 Aug 2011 05:47:16 Hilco Wijbenga wrote: On 20 August 2011 21:21, Nilesh Govindarajan cont...@nileshgr.com wrote: On 08/21/2011 09:00 AM, Hilco Wijbenga wrote: Yes, df -i says /portage is out of inodes. I've never run into that before. I reran mke2fs to increase the inode count and that fixed things. Sorry for the drop in, but I never knew that mke2fs can increase the number of inodes! I think I'll now place the portage tree on an ext2 disk image to speed up things, / has got fragmented badly due to portage tree :-\ Well, for the record, I'm not using ext2 but ext3 (mke2fs -j). Although, now that I think about it, I suppose there's not much point in having the Portage tree on a journaled FS. If you run man mke2fs, you should check out -N and -i. It was trial-and-error (for me, anyway) to find the right number. Presumably, -I fits in there somewhere as well. Do note that it only works when creating the FS, you can't change the inode count dynamically. I've never run out of inodes, even on small partitions. I just let ext4 make a fs with its default settings. Is there a magic formula to determine how many inodes are optimal? -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
creating the FS, you can't change the inode count dynamically. I've never run out of inodes, even on small partitions. I just let ext4 make a fs with its default settings. Is there a magic formula to determine how many inodes are optimal? Some FSes allocate inodes as required. I know btrfs does this and i think reiser does it too.
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
If you run man mke2fs, you should check out -N and -i. It was trial-and-error (for me, anyway) to find the right number. Consider using reiserfs for /usr/portage. No real performance advantage over ext[234], but works well with lots of small files and there's no inode count to worry about. In my experience the main downside of reiserfs is that fsck.reiserfs is almost never able to recover cleanly if the filesystem metadata does get corrupted in a non-trivial way. But for the portage snapshot this isn't really a problem... andrea
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On Sun 21 August 2011 11:13:53 Mick did opine thusly: On Sunday 21 Aug 2011 05:47:16 Hilco Wijbenga wrote: On 20 August 2011 21:21, Nilesh Govindarajan cont...@nileshgr.com wrote: On 08/21/2011 09:00 AM, Hilco Wijbenga wrote: Yes, df -i says /portage is out of inodes. I've never run into that before. I reran mke2fs to increase the inode count and that fixed things. Sorry for the drop in, but I never knew that mke2fs can increase the number of inodes! I think I'll now place the portage tree on an ext2 disk image to speed up things, / has got fragmented badly due to portage tree :-\ Well, for the record, I'm not using ext2 but ext3 (mke2fs -j). Although, now that I think about it, I suppose there's not much point in having the Portage tree on a journaled FS. If you run man mke2fs, you should check out -N and -i. It was trial-and-error (for me, anyway) to find the right number. Presumably, -I fits in there somewhere as well. Do note that it only works when creating the FS, you can't change the inode count dynamically. I've never run out of inodes, even on small partitions. I just let ext4 make a fs with its default settings. Is there a magic formula to determine how many inodes are optimal? No, there's no such formula. The answer to How many inodes do I need? is always How many do you need? -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
Hilco Wijbenga writes: Yes, df -i says /portage is out of inodes. I've never run into that before. I reran mke2fs to increase the inode count and that fixed things. Would LVM somehow prevent these sort of things from happening? LVM doesn't affect inode usage, does it? AFAIK you will gain more inodes when you increase the size. What exactly are the advantages of LVM? Is it just that it's easier to resize LVM partitions after the fact? (That would, of course, already be very useful.) Mainly. But it also allows you to create snapshots, I use this often to make backups without needing to boot from a live-cd - I can even start emerges meanwhile, but the backup will have the state the file system was in when the snapshot was taken. And you can create logical volumes that reside on partitions on different physical drives. Or you could move logical volumes from one drive or location to another one, while being in use all the time, without the need to unmount the file system. I also like the naming scheme (/dev/volume group/logical volume/ instead of /dev/sdXN), although you can also use file system labels so this is not a big problem. Wonko
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
Would LVM somehow prevent these sort of things from happening? LVM doesn't affect inode usage, does it? LVM has nothing to do with inodes. Inodes are a filesystem concept, and filesystems do not really care about the kind of block device they reside on. Well, generally. AFAIK you will gain more inodes when you increase the size. Only because by unless you specify a value mke2fs allocates a number of inodes proportional to the size of the filesystem, with the default being 1 inode every 16kB (see /etc/mke2fs.conf). But for ext[234] the number of inodes is fixed at filesystem creation, so even if you use LVM you can't increase it by -- say -- growing the underlying LV and then using resize2fs. andrea
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
Andrea Conti writes: AFAIK you will gain more inodes when you increase the size. Only because by unless you specify a value mke2fs allocates a number of inodes proportional to the size of the filesystem, with the default being 1 inode every 16kB (see /etc/mke2fs.conf). But for ext[234] the number of inodes is fixed at filesystem creation, so even if you use LVM you can't increase it by -- say -- growing the underlying LV and then using resize2fs. So I just tried that, create a small fs, filled it until no inodes were left. Resized, and gained more inodes: weird ~ # lvcreate -L 4M -n inodetest weird Logical volume inodetest created weird ~ # mke2fs -j /dev/weird/inodetest mke2fs 1.41.14 (22-Dec-2010) Filesystem label= OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) Stride=0 blocks, Stripe width=0 blocks 1024 inodes, 4096 blocks [...] weird ~ # mount /dev/weird/inodetest /mnt/ weird ~ # for (( i=1; ; i++ )) do touch /mnt/$( printf file %06d $i ) || break done touch: cannot touch `/mnt/file 001014': No space left on device weird ~ # df -i /mnt/ Filesystem Inodes IUsed IFree IUse% Mounted on /dev/mapper/weird-inodetest 1024 1024 0 100% /mnt weird ~ # lvresize -L 16M /dev/weird/inodetest Extending logical volume inodetest to 16,00 MiB Logical volume inodetest successfully resized weird ~ # resize2fs /dev/weird/inodetest resize2fs 1.41.14 (22-Dec-2010) Filesystem at /dev/weird/inodetest is mounted on /mnt; on-line resizing required old desc_blocks = 1, new_desc_blocks = 1 Performing an on-line resize of /dev/weird/inodetest to 16384 (1k) blocks. The filesystem on /dev/weird/inodetest is now 16384 blocks long. weird ~ # df -i /mnt/ Filesystem Inodes IUsed IFree IUse% Mounted on /dev/mapper/weird-inodetest 2048 1024 1024 50% /mnt Wonko
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On 21 August 2011 03:46, Andrea Conti a...@alyf.net wrote: If you run man mke2fs, you should check out -N and -i. It was trial-and-error (for me, anyway) to find the right number. Consider using reiserfs for /usr/portage. No real performance advantage over ext[234], but works well with lots of small files and there's no inode count to worry about. In my experience the main downside of reiserfs is that fsck.reiserfs is almost never able to recover cleanly if the filesystem metadata does get corrupted in a non-trivial way. But for the portage snapshot this isn't really a problem... I have always used ReiserFS for everything but /boot. That explains why I never ran into the inode issue, I guess. I'm trying to install AMD64 and the handbook says that ([1]) JFS and ReiserFS may work but need more testing. If you're really adventurous you can try the other filesystems.. That didn't sound too promising so I went with ext3. :-) The X86 handbook doesn't have this text. Is ReiserFS on AMD64 really only for the adventurous? Or should this warning be removed? [1] http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=1chap=4#filesystemsdesc
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
Filesystem Inodes IUsed IFree IUse% Mounted on /dev/mapper/weird-inodetest 1024 1024 0 100% /mnt /dev/mapper/weird-inodetest 2048 1024 1024 50% /mnt Then I stand corrected. I guess that the man page for mke2fs saying that the inode count of a filesystem cannot be changed does not take resizing into account. I also thought that if resize2fs had the ability to extend the inode table, then it would have options to give the user some degree of control over the process. Apparently that's not the case. andrea
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On Sunday 21 August 2011 19:14:53 Hilco Wijbenga wrote: The X86 handbook doesn't have this text. Is ReiserFS on AMD64 really only for the adventurous? Certainly not. It's 100% stable as far as I know. Or should this warning be removed? ASAP -- Rgds Peter Linux Counter 5290, 1994-04-23
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On Sun 21 August 2011 21:23:15 Andrea Conti did opine thusly: Filesystem Inodes IUsed IFree IUse% Mounted on /dev/mapper/weird-inodetest 1024 1024 0 100% /mnt /dev/mapper/weird-inodetest 2048 1024 1024 50% /mnt Then I stand corrected. I guess that the man page for mke2fs saying that the inode count of a filesystem cannot be changed does not take resizing into account. Correct. A resized fs is technically a re-formatted fs. So to the ultra-pedantic the man page is actually still correct. I also thought that if resize2fs had the ability to extend the inode table, then it would have options to give the user some degree of control over the process. Apparently that's not the case. inodes are not dynamic, they are laid down at exact points on the disk and the info about their location is in the superblock(s). The simplistic explanation is something like this: You have X number of inodes, spaced Y blocks apart on a disk of size Z bytes. A directory listing declares a file is at inode #M therefore it's physical position on the disk is guaranteed to be at M * (block size) and the filesystem driver can seek directly to that spot. Two things are immediately self-evident: 1. Changing the density of inodes is not realistic (unless you want to invest the same effort that went into Window's defrag) 2. Fixed inodes are an ancient concept that provoke an Eh? Say what? You still doing that response. ReiserFS was developed in part to address this and make an -- alan dot mckinnon at gmail dot com
[gentoo-user] [Gentoo install] Disk full at 35%?
Hi all, It's been quite a few years but I decided to try another Gentoo install (on a VirtualBox instance). I wanted to try out some new things... I created a ton of partitions including /usr (I want to see if I can get that to work), /portage, and /distfiles. The idea was to mount /portage on top of /usr and /distfiles on top of /portage. This all works fine. However, when I try to extract the Portage snapshot, I get No space left on device a long way into the untar process. According to df /portage (i.e. /mnt/gentoo/usr/portage) is only 35% full. In fact, not a single partition or mount is even close to full (except for /mnt/static, the DVD). If I untar directly to /usr (after unmounting /portage), everything works fine. If I then try to copy or move to /portage, I get the No space left on device again. And at the same place. Does anyone know what's going on here? I didn't realize I was doing such strange things. At least not this early on. :-) Cheers, Hilco
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
Hilco Wijbenga hilco.wijbe...@gmail.com wrote: Hi all, It's been quite a few years but I decided to try another Gentoo install (on a VirtualBox instance). I wanted to try out some new things... I created a ton of partitions including /usr (I want to see if I can get that to work), /portage, and /distfiles. The idea was to mount /portage on top of /usr and /distfiles on top of /portage. This all works fine. However, when I try to extract the Portage snapshot, I get No space left on device a long way into the untar process. According to df /portage (i.e. /mnt/gentoo/usr/portage) is only 35% full. In fact, not a single partition or mount is even close to full (except for /mnt/static, the DVD). If I untar directly to /usr (after unmounting /portage), everything works fine. If I then try to copy or move to /portage, I get the No space left on device again. And at the same place. Does anyone know what's going on here? I didn't realize I was doing such strange things. At least not this early on. :-) See if you are out of inodes. The only way to get the inodes that I am aware of is to debugfs to the partition and do stat from within -- if there is a better way please let me know. But why not use lvm? -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici cov...@ccs.covici.com
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On Sat, Aug 20, 2011 at 8:41 PM, Hilco Wijbenga hilco.wijbe...@gmail.com wrote: However, when I try to extract the Portage snapshot, I get No space left on device a long way into the untar process. According to df /portage (i.e. /mnt/gentoo/usr/portage) is only 35% full. In fact, not a single partition or mount is even close to full (except for /mnt/static, the DVD). Try df -i to check your inode usage.
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On 20 August 2011 18:52, cov...@ccs.covici.com wrote: Hilco Wijbenga hilco.wijbe...@gmail.com wrote: Hi all, It's been quite a few years but I decided to try another Gentoo install (on a VirtualBox instance). I wanted to try out some new things... I created a ton of partitions including /usr (I want to see if I can get that to work), /portage, and /distfiles. The idea was to mount /portage on top of /usr and /distfiles on top of /portage. This all works fine. However, when I try to extract the Portage snapshot, I get No space left on device a long way into the untar process. According to df /portage (i.e. /mnt/gentoo/usr/portage) is only 35% full. In fact, not a single partition or mount is even close to full (except for /mnt/static, the DVD). If I untar directly to /usr (after unmounting /portage), everything works fine. If I then try to copy or move to /portage, I get the No space left on device again. And at the same place. Does anyone know what's going on here? I didn't realize I was doing such strange things. At least not this early on. :-) See if you are out of inodes. The only way to get the inodes that I am aware of is to debugfs to the partition and do stat from within -- if there is a better way please let me know. But why not use lvm? Yes, df -i says /portage is out of inodes. I've never run into that before. I reran mke2fs to increase the inode count and that fixed things. Would LVM somehow prevent these sort of things from happening? LVM doesn't affect inode usage, does it? What exactly are the advantages of LVM? Is it just that it's easier to resize LVM partitions after the fact? (That would, of course, already be very useful.)
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On 20 August 2011 20:05, Paul Hartman paul.hartman+gen...@gmail.com wrote: On Sat, Aug 20, 2011 at 8:41 PM, Hilco Wijbenga hilco.wijbe...@gmail.com wrote: However, when I try to extract the Portage snapshot, I get No space left on device a long way into the untar process. According to df /portage (i.e. /mnt/gentoo/usr/portage) is only 35% full. In fact, not a single partition or mount is even close to full (except for /mnt/static, the DVD). Try df -i to check your inode usage. Yes, thanks, I had just found out about df -i myself. :-)
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On 08/21/2011 09:00 AM, Hilco Wijbenga wrote: Yes, df -i says /portage is out of inodes. I've never run into that before. I reran mke2fs to increase the inode count and that fixed things. Sorry for the drop in, but I never knew that mke2fs can increase the number of inodes! I think I'll now place the portage tree on an ext2 disk image to speed up things, / has got fragmented badly due to portage tree :-\ Thanks man! -- Nilesh Govindarajan http://nileshgr.com
Re: [gentoo-user] [Gentoo install] Disk full at 35%?
On 20 August 2011 21:21, Nilesh Govindarajan cont...@nileshgr.com wrote: On 08/21/2011 09:00 AM, Hilco Wijbenga wrote: Yes, df -i says /portage is out of inodes. I've never run into that before. I reran mke2fs to increase the inode count and that fixed things. Sorry for the drop in, but I never knew that mke2fs can increase the number of inodes! I think I'll now place the portage tree on an ext2 disk image to speed up things, / has got fragmented badly due to portage tree :-\ Well, for the record, I'm not using ext2 but ext3 (mke2fs -j). Although, now that I think about it, I suppose there's not much point in having the Portage tree on a journaled FS. If you run man mke2fs, you should check out -N and -i. It was trial-and-error (for me, anyway) to find the right number. Presumably, -I fits in there somewhere as well. Do note that it only works when creating the FS, you can't change the inode count dynamically.
