Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Dale
Mark Knecht wrote:
>
>
> On Fri, Aug 26, 2022 at 7:25 AM Rich Freeman  > wrote:
> >
> > On Fri, Aug 26, 2022 at 10:09 AM Mark Knecht  > wrote:
> > >
> > > Obviously you can do what you are most comfortable with but to me
> a NAS machine with a bunch of external drives does not sound very
> reliable.
> > >
> >
> > I would have thought the same, but messing around with LizardFS I've
> > found that the USB3 hard drives never disconnect from their Pi4 hosts.
> > I've had more issues with LSI HBAs dying.  Of course I have host-level
> > redundancy so if one Pi4 flakes out I can just reboot it with zero
> > downtime - the master server is on an amd64 container.  I only have
> > about 2 drives per Pi right now as well - at this point I'd probably
> > add more drives per host but I wanted to get out to 5-6 hosts first so
> > that I get better performance especially during rebuilds.  Gigabit
> > networking is definitely a bottleneck, but with all the chunkservers
> > on one switch they each get gigabit full duplex to all the others so
> > rebuilds are still reasonably fast.  To go with 10GbE you'd need
> > hardware with better IO than a Pi4 I'd think, but the main bottleneck
> > on the Pi4 I'm having is with encryption which hits the CPU.  I am
> > using dm-crypt for this which I think is hardware-optimized.  I will
> > say that zfs encryption is definitely not hardware-optimized and
> > really gets CPU-bound, so I'm running zfs on top of dm-crypt.  I
> > should probably consider if dm-integrity makes more sense than zfs in
> > this application.
> >
> > --
> > Rich
>
> Quite interesting Rich. Thanks!
>
> My needs may be too 'simple'. I'm not overly worried about the government 
> or foreign actors invading my world. (Even though I'm sure they could.) I
> just have a router-based firewall. My backup machines are powered down
> unless they are being used and they don't respond to wake-up over the
> network so they are safe enough for me. The one in my office backs up
> my two machines (desktop and video file server) and the second
> NAS backs up the first. They are both ZFS RAID1 using TrueNAS. I
> don't use encryption at all. A real dummy...
>
> But again, I'm not even a Gentoo user any more. I'm a KDE user
> and I could see no performance improvement using Gentoo over
> Kubuntu. My updates happen once a week, roughly, and never
> take more than 5 minutes. In 4 years I've never had an update
> fail. Kubuntu just works for me - but I'll be the first to admit I don't
> know what's running on my machine anymore so I'm not much better
> than being a Windows user in terms of control. 
>
> In the old days (2001) I was a computer OS enthusiast. Today 
> I play guitar, bake bread and drink a little wine. Life and focus
> changed. For a guy at home life is ok and I have backups to boot.


I have looked into OpenNAS and other NAS OS stuff.  Some are on USB
sticks and basically, you shut it down, upgrade the USB stick, insert it
back into NAS and boot up.  Then again, if it doesn't have a GUI type
stuff, I could run Gentoo or something and updates wouldn't be to large
since it is a base system mostly.  I read once where a guy set up a NAS
and it ran for years without ever even being rebooted.  I think his
uptime was like 5 or 6 years.  It was one of those 'out of sight, out of
mind' type things.  He actively used it but never updated it or even
blew the dust out of it.  Then one day it hit him, I better check that
thing.  LOL 

I do plan to use encryption and they will be locked when not in use.  I
use cryptsetup commands to do all that.  I think it is dmcrypt on the
low level stuff.  It's one reason I wanted to stay away from the
Raspberry.  It is low power which is great but not so much when using
encrypted files.  Then there is the USB to SATA thing that I've had bad
experiences with.  It's not like hal but still, I've had hard drives in
USB enclosures turn into door stops.  I just don't trust it.  It would
make me worry, a lot.

This certainly something I need to deal with tho.  This fast internet is
like poking a hornets nest.  It's causing all kinds of problems.  ROFL 

Dale

:-)  :-) 


Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Dale
Rich Freeman wrote:
> On Fri, Aug 26, 2022 at 7:26 AM Dale  wrote:
>> I looked into the Raspberry and the newest version, about $150 now, doesn't 
>> even have SATA ports.
> The Pi4 is definitely a step up from the previous versions in terms of
> IO, but it is still pretty limited.  It has USB3 and gigabit, and they
> don't share a USB host or anything like that, so you should get close
> to full performance out of both.  The CPU is of course pretty limited,
> as is RAM.  Biggest benefit is the super-low power consumption, and
> that is something I take seriously as for a lot of cheap hardware that
> runs 24x7 the power cost rapidly exceeds the purchase price.  I see
> people buying old servers for $100 or whatever and those things will
> often go through $100 worth of electricity in a few months.
>
> How many hard drives are you talking about?  There are two general
> routes to go for something like this.  The simplest and most
> traditional way is a NAS box of some kind, with RAID.  The issue with
> these approaches is that you're limited by the number of hard drives
> you can run off of one host, and of course if anything other than a
> drive fails you're offline.  The other approach is a distributed
> filesystem.  That ramps up the learning curve quite a bit, but for
> something like media where IOPS doesn't matter it eliminates the need
> to try to cram a dozen hard drives into one host.  Ceph can also do
> IOPS but you're talking 10GbE + NVMe and big bucks, and that is how
> modern server farms would do it.
>
> I'll describe the traditional route since I suspect that is where
> you're going to end up.  If you only had 2-4 drives total you could
> probably get away with a Pi4 and USB3 drives, but if you want
> encryption or anything CPU-intensive you're probably going to
> bottleneck on the CPU.  It would be fine if you're more concerned with
> capacity than storage.
>
> For more drives than that, or just to be more robust, then any
> standard amd64 build will be fine.  Obviously a motherboard with lots
> of SATA ports will help here.  However, that almost always is a
> bottleneck on consumer gear, and the typical solution to that for SATA
> is a host bus adapter.  They're expensive new, but cheap on ebay (I've
> had them fail though, which is probably why companies tend to sell
> them while they're still working).  They also use a ton of power -
> I've measured them using upwards of 60W - they're designed for servers
> where nobody seems to care.  A typical HBA can provide 8-32 SATA
> ports, via mini-SAS breakout cables (one mini-SAS port can provide 4
> SATA ports).  HBAs tend to use a lot of PCIe lanes - you don't
> necessarily need all of them if you only have a few drives and they're
> spinning disks, but it is probably easiest if you get a CPU with
> integrated graphics and use the 16x slot for the HBA.  That or get a
> motherboard with two large slots (they usually aren't 16x, but getting
> 4-8x slots on a consumer motherboard isn't super-common).
>
> For software I'd use mdadm plus LVM.  ZFS or btrfs are your other
> options, and those can run on bare metal, but btrfs is immature and
> ZFS cannot be reshaped the way mdadm can, so there are tradeoffs.  If
> you want to use your existing drives and don't have a backup to
> restore or want to do it live, then the easiest option there is to add
> one drive to the system to expand capacity.  Put mdadm on that drive
> as a degraded raid1 or whatever, then put LVM on top, and migrate data
> from an existing disk live over to the new one, freeing up one or more
> existing drives.  Then put mdadm on those and LVM and migrate more
> data onto them, and so on, until everything is running on top of
> mdadm.  Of course you need to plan how you want the array to look and
> have enough drives that you get the desired level of redundancy.  You
> can start with degraded arrays (which is no worse than what you have
> now), then when enough drives are freed up they can be added as pairs
> to fill it out.
>
> If you want to go the distributed storage route then CephFS is the
> canonical solution at this point but it is RAM-hungry so it tends to
> be expensive.  It is also complex, but there are ansible playbooks and
> so on to manage that (though playbooks with 100+ plays in them make me
> nervous).  For something simpler MooseFS or LizardFS are probably
> where I'd start.  I'm running LizardFS but they've been on the edge of
> death for years upstream and MooseFS licensing is apparently better
> now, so I'd probably look at that first.  I did a talk on lizardfs
> recently: https://www.youtube.com/watch?v=dbMRcVrdsQs
>


This is some good info.  It will likely start off with just a few hard
drives but that will grow over time.  I also plan to have a large drive
as a spare as well, in case one starts having issues and needs replacing
quick.  I'd really like to be using RAID at least the two copies one but
may take time, plus I got to learn how to do the thing. 

Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Mark Knecht
On Fri, Aug 26, 2022 at 7:25 AM Rich Freeman  wrote:
>
> On Fri, Aug 26, 2022 at 10:09 AM Mark Knecht  wrote:
> >
> > Obviously you can do what you are most comfortable with but to me a NAS
machine with a bunch of external drives does not sound very reliable.
> >
>
> I would have thought the same, but messing around with LizardFS I've
> found that the USB3 hard drives never disconnect from their Pi4 hosts.
> I've had more issues with LSI HBAs dying.  Of course I have host-level
> redundancy so if one Pi4 flakes out I can just reboot it with zero
> downtime - the master server is on an amd64 container.  I only have
> about 2 drives per Pi right now as well - at this point I'd probably
> add more drives per host but I wanted to get out to 5-6 hosts first so
> that I get better performance especially during rebuilds.  Gigabit
> networking is definitely a bottleneck, but with all the chunkservers
> on one switch they each get gigabit full duplex to all the others so
> rebuilds are still reasonably fast.  To go with 10GbE you'd need
> hardware with better IO than a Pi4 I'd think, but the main bottleneck
> on the Pi4 I'm having is with encryption which hits the CPU.  I am
> using dm-crypt for this which I think is hardware-optimized.  I will
> say that zfs encryption is definitely not hardware-optimized and
> really gets CPU-bound, so I'm running zfs on top of dm-crypt.  I
> should probably consider if dm-integrity makes more sense than zfs in
> this application.
>
> --
> Rich

Quite interesting Rich. Thanks!

My needs may be too 'simple'. I'm not overly worried about the government
or foreign actors invading my world. (Even though I'm sure they could.) I
just have a router-based firewall. My backup machines are powered down
unless they are being used and they don't respond to wake-up over the
network so they are safe enough for me. The one in my office backs up
my two machines (desktop and video file server) and the second
NAS backs up the first. They are both ZFS RAID1 using TrueNAS. I
don't use encryption at all. A real dummy...

But again, I'm not even a Gentoo user any more. I'm a KDE user
and I could see no performance improvement using Gentoo over
Kubuntu. My updates happen once a week, roughly, and never
take more than 5 minutes. In 4 years I've never had an update
fail. Kubuntu just works for me - but I'll be the first to admit I don't
know what's running on my machine anymore so I'm not much better
than being a Windows user in terms of control.

In the old days (2001) I was a computer OS enthusiast. Today
I play guitar, bake bread and drink a little wine. Life and focus
changed. For a guy at home life is ok and I have backups to boot.


Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Rich Freeman
On Fri, Aug 26, 2022 at 10:09 AM Mark Knecht  wrote:
>
> Obviously you can do what you are most comfortable with but to me a NAS 
> machine with a bunch of external drives does not sound very reliable.
>

I would have thought the same, but messing around with LizardFS I've
found that the USB3 hard drives never disconnect from their Pi4 hosts.
I've had more issues with LSI HBAs dying.  Of course I have host-level
redundancy so if one Pi4 flakes out I can just reboot it with zero
downtime - the master server is on an amd64 container.  I only have
about 2 drives per Pi right now as well - at this point I'd probably
add more drives per host but I wanted to get out to 5-6 hosts first so
that I get better performance especially during rebuilds.  Gigabit
networking is definitely a bottleneck, but with all the chunkservers
on one switch they each get gigabit full duplex to all the others so
rebuilds are still reasonably fast.  To go with 10GbE you'd need
hardware with better IO than a Pi4 I'd think, but the main bottleneck
on the Pi4 I'm having is with encryption which hits the CPU.  I am
using dm-crypt for this which I think is hardware-optimized.  I will
say that zfs encryption is definitely not hardware-optimized and
really gets CPU-bound, so I'm running zfs on top of dm-crypt.  I
should probably consider if dm-integrity makes more sense than zfs in
this application.

-- 
Rich



Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Mark Knecht
On Fri, Aug 26, 2022 at 4:27 AM Dale  wrote:

>
> I looked into the Raspberry and the newest version, about $150 now,
doesn't even have SATA ports.  I can add a thing called a "hat" I think
that adds a couple but thing is, that costs more and still isn't enough.  I
really don't like USB and hard drive mixing.  Every time I do that, the
hard drive turns into a door stop.  Currently, I have three Rosewill
external enclosures and they have USB and eSATA ports.  I use the eSATA
connections and no problems.  It's also really fast.  So, I plan to stick
with SATA connections.

You do NOT want  the Rasp Pi for this. You would have to compile and
maintain the OS yourself just adding work and the disk interfaces aren't
high performance enough.

Obviously you can do what you are most comfortable with but to me a NAS
machine with a bunch of external drives does not sound very reliable.

>
> I have a old computer that I might could use.  It is 4 core something and
I think it has 4GBs of memory, maxed out.  I think it will perform well
enough but wish it had a little more horses in it.

That's more than enough horsepower for TrueNAS Core. If the box will hold 3
drives then you have 1 system drive and 2 data drives for a ZFS RAID1.
That's how both of my NAS boxes are set up.

You can buy more memory at lots of places inexpensively but you don't need
it to start. 4GB will work with TrueNAS Core. My machines have 8 & 12GB. I
never use it all.

https://www.truenas.com/truenas-core/

Even if your old box has only 2 drives, download TrueNAS and just set it up
on one systemdrive. It's not Gentoo difficult. It's a fully formed install
system which will probably be running in an hour. You can use 1 drive in
your data tank and add additional drives later.

The speed of a NAS is _mostly_ a balance between network speed and disk
speed. Processor usage for me is generally about 20%. If your network is
GigaBit then you can sustain somewhere about 850Mb/S on the cables which
translates nicely to about 100 MegaByte/S on your disk drives. There isn't
that much CPU usage as it's mostly compression when backing up.

Unless you use the box as a file server getting data back off is a once in
a while event where you don't care too much about speed, or at least I
don't.

Just do it. Download the install disc and give it a try. Nothing much to
lose.

Good luck.
Mark


Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Wols Lists

On 26/08/2022 12:27, Dale wrote:

I think it's saved a lot of bacon over the years:-)  Even if I've
mostly edited it. I haven't written much of it from scratch.

Cheers,
Wol

I see typos.  Do they matter to you?


Apart from the one intentional one, I'd like to fix any others :-)

Cheers,
Wol



Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Rich Freeman
On Fri, Aug 26, 2022 at 7:26 AM Dale  wrote:
>
> I looked into the Raspberry and the newest version, about $150 now, doesn't 
> even have SATA ports.

The Pi4 is definitely a step up from the previous versions in terms of
IO, but it is still pretty limited.  It has USB3 and gigabit, and they
don't share a USB host or anything like that, so you should get close
to full performance out of both.  The CPU is of course pretty limited,
as is RAM.  Biggest benefit is the super-low power consumption, and
that is something I take seriously as for a lot of cheap hardware that
runs 24x7 the power cost rapidly exceeds the purchase price.  I see
people buying old servers for $100 or whatever and those things will
often go through $100 worth of electricity in a few months.

How many hard drives are you talking about?  There are two general
routes to go for something like this.  The simplest and most
traditional way is a NAS box of some kind, with RAID.  The issue with
these approaches is that you're limited by the number of hard drives
you can run off of one host, and of course if anything other than a
drive fails you're offline.  The other approach is a distributed
filesystem.  That ramps up the learning curve quite a bit, but for
something like media where IOPS doesn't matter it eliminates the need
to try to cram a dozen hard drives into one host.  Ceph can also do
IOPS but you're talking 10GbE + NVMe and big bucks, and that is how
modern server farms would do it.

I'll describe the traditional route since I suspect that is where
you're going to end up.  If you only had 2-4 drives total you could
probably get away with a Pi4 and USB3 drives, but if you want
encryption or anything CPU-intensive you're probably going to
bottleneck on the CPU.  It would be fine if you're more concerned with
capacity than storage.

For more drives than that, or just to be more robust, then any
standard amd64 build will be fine.  Obviously a motherboard with lots
of SATA ports will help here.  However, that almost always is a
bottleneck on consumer gear, and the typical solution to that for SATA
is a host bus adapter.  They're expensive new, but cheap on ebay (I've
had them fail though, which is probably why companies tend to sell
them while they're still working).  They also use a ton of power -
I've measured them using upwards of 60W - they're designed for servers
where nobody seems to care.  A typical HBA can provide 8-32 SATA
ports, via mini-SAS breakout cables (one mini-SAS port can provide 4
SATA ports).  HBAs tend to use a lot of PCIe lanes - you don't
necessarily need all of them if you only have a few drives and they're
spinning disks, but it is probably easiest if you get a CPU with
integrated graphics and use the 16x slot for the HBA.  That or get a
motherboard with two large slots (they usually aren't 16x, but getting
4-8x slots on a consumer motherboard isn't super-common).

For software I'd use mdadm plus LVM.  ZFS or btrfs are your other
options, and those can run on bare metal, but btrfs is immature and
ZFS cannot be reshaped the way mdadm can, so there are tradeoffs.  If
you want to use your existing drives and don't have a backup to
restore or want to do it live, then the easiest option there is to add
one drive to the system to expand capacity.  Put mdadm on that drive
as a degraded raid1 or whatever, then put LVM on top, and migrate data
from an existing disk live over to the new one, freeing up one or more
existing drives.  Then put mdadm on those and LVM and migrate more
data onto them, and so on, until everything is running on top of
mdadm.  Of course you need to plan how you want the array to look and
have enough drives that you get the desired level of redundancy.  You
can start with degraded arrays (which is no worse than what you have
now), then when enough drives are freed up they can be added as pairs
to fill it out.

If you want to go the distributed storage route then CephFS is the
canonical solution at this point but it is RAM-hungry so it tends to
be expensive.  It is also complex, but there are ansible playbooks and
so on to manage that (though playbooks with 100+ plays in them make me
nervous).  For something simpler MooseFS or LizardFS are probably
where I'd start.  I'm running LizardFS but they've been on the edge of
death for years upstream and MooseFS licensing is apparently better
now, so I'd probably look at that first.  I did a talk on lizardfs
recently: https://www.youtube.com/watch?v=dbMRcVrdsQs

-- 
Rich



Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Gerrit Kuehn


On Fri, 26 Aug 2022 06:26:39 -0500
Dale  wrote:

> I looked at something called ITX but they have only one PCIe slot
> usually.  That's not enough.  I'd like to have two 6 or 8 port SATA
> cards.  Then balance the drives on each.  I think some of the through
> put is shared so the more drives on it, the slower it can be.  I'd
> like to have two such cards. 12 or 16 drives should be enough to last
> a while.  Part of me wants to do RAID but not sure about that.  Yet.
> I think I'm just going to go with ATX since it has several PCIe
> slots. 

Usually, an ITX mainboard will feature a PCIe slot /and/ additional
onboard SATA connectors. So you might be fine with an 8port controller
card and the onboard connections.
However, even if you want 16 SATA connections on one PCIe card, you can
buy that. Broadcom SAS 9201-16i is one example. If that is enough
bandwidth-wise will depend on your PCIe slot and the drives you're
going to attach.
I don't see any reason to do hardware raid these days, just a HBA and
software raid (zfs or other solutions) should be fine.
Everything just my 2¢ here, of course...


cu
  Gerrit



Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Dale
Wols Lists wrote:
> On 26/08/2022 00:56, Dale wrote:
>> Wols Lists wrote:
>>> On 25/08/2022 19:59, Dale wrote:
 While at it, can I move the drives on LVM to another system without
 having to copy anything?  Just physically move the drives and LVM see
 them correctly on the new system?  I may try to build a small computer
 for a NAS soon.  I'm not sure what is the least I can buy that will
 perform well.  I need to look into small mobos to see what options I
 have.  I mostly need a CPU to handle moving files, memory to pass it
 through and lots of SATA ports.  I figure a fast card for most SATA
 ports.
>>>
>>> https://raid.wiki.kernel.org/index.php/Linux_Raid
>>>
>>> That might be a good read ... I know I push it a bit, but it does go
>>> into disk management a decent bit.
>>>
>>> If you can think of any improvements, they'll be welcome! :-)
>>>
>> It seems I've been to that link before, may even have it bookmarked,
>> somewhere.  I'll give it another read tho.  After all, it has to be good
>> or you wouldn't share it.  ;-)
>>
> I think it's saved a lot of bacon over the years :-) Even if I've
> mostly edited it. I haven't written much of it from scratch.
>
> Cheers,
> Wol

I see typos.  Do they matter to you? 

Dale

:-)  :-) 



Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Dale
Mark Knecht wrote:
>
>
> On Thu, Aug 25, 2022 at 4:59 PM Dale  > wrote:
> 
> > I may do some mobo hunting shortly.  See what little thing I can buy
> > that is powerful enough.  I don't think a Raspberry Pi is enough.  It
> > gets close tho.  Biggest thing, I'd need a lot of SATA ports.  LOTS of
> > them.
>
> Granted, I had a couple of old cases which lowered the cost but
> I went to a local computer store and bought used motherboards
> that came with processors and memory. They were both Core i7
> but I paid only about $75 each. I needed power supplies and hard drives
> so each machine ended up around $350 or so by the time I was done.
> Each has 2 4TB drives for storage and a 1TB drive for the OS. A lot
> of used motherboards have on-board VGA and Gb/S networking.
>
> These are TrueNAS machines, FreeBSD not Linux, but they have
> a Linux version now if that makes you more comfortable. 
>
> I'd stick with AMD64 as it's better tested and I don't think you'll
> get the network throughput you need to be fast with a Raspberry Pi


I looked into the Raspberry and the newest version, about $150 now,
doesn't even have SATA ports.  I can add a thing called a "hat" I think
that adds a couple but thing is, that costs more and still isn't
enough.  I really don't like USB and hard drive mixing.  Every time I do
that, the hard drive turns into a door stop.  Currently, I have three
Rosewill external enclosures and they have USB and eSATA ports.  I use
the eSATA connections and no problems.  It's also really fast.  So, I
plan to stick with SATA connections.

I have a old computer that I might could use.  It is 4 core something
and I think it has 4GBs of memory, maxed out.  I think it will perform
well enough but wish it had a little more horses in it.

I looked at something called ITX but they have only one PCIe slot
usually.  That's not enough.  I'd like to have two 6 or 8 port SATA
cards.  Then balance the drives on each.  I think some of the through
put is shared so the more drives on it, the slower it can be.  I'd like
to have two such cards. 12 or 16 drives should be enough to last a
while.  Part of me wants to do RAID but not sure about that.  Yet.  I
think I'm just going to go with ATX since it has several PCIe slots. 

While I don't think I need a super powerful machine, I do want enough
that it will perform well.  I may use actual NAS software too.  I'm sure
Gentoo would work to with proper tweaking but then I need to deal with
compiling things.  Of course, no libreoffice or anything big so it may
not be to bad.  Thing is, the NAS software will likely be more efficient
since it is designed for the purpose. 

I just know I need a proper machine for the task.  I'm getting lots of
data fast now.  I hit the 80% mark overnight.  At 90%, I consider it
critical.  Something must be done soon. 

Keep those ideas coming.  I'll put them in the blender and see what
comes out.  ;-)

Dale

:-)  :-) 


Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread David Haller
Hello,

On Thu, 25 Aug 2022, Dale wrote:
>Jack wrote:
[..]
>> Related question - how much space would you actually save by
>> decreasing the number of inodes by 90%?  Enough for one or two more
>> videos?
>
>Now I have to admit, that is a question I have too.

From my tests with a swapfile (which matches what I remember from real
FSen), I think '-T largefile' vs. default frees up around 1.6% of the
capacity, so for 9.1T it'd be around 150G which might be worthwhile
_iff_ you are sure about what kind of files will go on that FS.

FWIW, in a pinch if you run out of Inodes, you can create an image
file on that fs, taking just 1 Inode, format that image differently,
loop-mount it and put ton's of files inside the image. It'll eat a bit
of performance though.

And about the average filesize in a dir: Just find out the size (e.g.: 
 du -msx /foo
) and the number of used inodes (e.g.:
find /foo -xdev | wc -l
[1]) and then just divide:
summed_size_in_unit / number_of_files = avg_size_in_unit
for FSen, just divide used space by used inodes.

HTH,
-dnh

[1] assuming you have no files with '\n' in the filename

-- 
printk (KERN_ERR "%s: Oops - your private data area is hosed!\n", ...)
linux-2.6.6/drivers/net/ewrk3.c



Re: [gentoo-user] Getting maximum space out of a hard drive

2022-08-26 Thread Wols Lists

On 26/08/2022 00:56, Dale wrote:

Wols Lists wrote:

On 25/08/2022 19:59, Dale wrote:

While at it, can I move the drives on LVM to another system without
having to copy anything?  Just physically move the drives and LVM see
them correctly on the new system?  I may try to build a small computer
for a NAS soon.  I'm not sure what is the least I can buy that will
perform well.  I need to look into small mobos to see what options I
have.  I mostly need a CPU to handle moving files, memory to pass it
through and lots of SATA ports.  I figure a fast card for most SATA
ports.


https://raid.wiki.kernel.org/index.php/Linux_Raid

That might be a good read ... I know I push it a bit, but it does go
into disk management a decent bit.

If you can think of any improvements, they'll be welcome! :-)


It seems I've been to that link before, may even have it bookmarked,
somewhere.  I'll give it another read tho.  After all, it has to be good
or you wouldn't share it.  ;-)

I think it's saved a lot of bacon over the years :-) Even if I've mostly 
edited it. I haven't written much of it from scratch.


Cheers,
Wol