Re: [BackupPC-users] considerations on i-nodes

[G.W. Haywood via BackupPC-users]

Hello again,

On Fri, 20 Apr 2018, Michael Stowe wrote:


... I don't think we're disagreeing ...


Agreed. :)


"I want to futz around with inode allocations" is rarely at the top
of people's to do lists.


Perhaps not, but

"What do I need to ask of a filesystem for it to be able to support my
planned backup regime?"

perhaps ought to be.

--

73,
Ged.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] considerations on i-nodes

[Bzzzz]

On Thu, 19 Apr 2018 16:55:59 + (UTC)
Michael Stowe  wrote:

> While there's nothing inherently wrong with selecting an older 
> filesystem, ext4's design decision of backward compatibility has 
> essentially set some of its limitations in stone.  (Your article below 
> elaborates on this point; it's not a next generation filesystem, it's 
> just something that works.)

IIRC, EXT4 was launched almost only to counter ReiserFS that was raising
hard at this time and had the favour of people, opposing to what kernel
people were thinking was the best for others (as you see, development
democrature isn't really new and take it's roots at the source;)

Jean-Yves

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] considerations on i-nodes

[Michael Stowe]

On 2018-04-19 06:31, G.W. Haywood via BackupPC-users wrote:

Hi there,

On Thu, 19 Apr 2018, Michael Stowe wrote:


... Those who do reach back a decade or more to select a filesystem,


That's like saying the Linux kernel is a hangover from the end of the
20th century.  It's misleading, and more than a bit unfair especially
considering the numbers of posts to the ext4 mailing list:

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7RuEFgozAWvnlNmELy4oKuiA3Uh2zVaVksANvOXpuehpkneX0E-2FfKBOjzEo25YDMGw-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CasKnxVRTUBW0lnnPUiBFDbnzrPzFGfYmk0Iwn1duJneKWemz0bfm83-2Fl8P8pIa0YGeA8QAnhxRLigz6DuEWH0WyS6jHH3rHg5QJhsnUJVs1DRyBbhwrZ-2FLQ9SNo7ZD6ANz0-2BbsyrFGuo-2FUHmLHgg9NSil8n8wpRXy0GiOoHi4bIQ3Lhx4MOCnpVaarfVSQNm


While there's nothing inherently wrong with selecting an older 
filesystem, ext4's design decision of backward compatibility has 
essentially set some of its limitations in stone.  (Your article below 
elaborates on this point; it's not a next generation filesystem, it's 
just something that works.)



I generally expect to have good reason ...


Like, er, it works, it's currently under active development, and it's
supported by every Linux distribution you're likely to meet? :)

Yes, even the developers will agree that it's a stop-gap measure.

But it's a big gap, and the rest of them haven't quite plugged it yet:

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7UY4nHmLotbnt-2B5EKj0ng5UcSekH-2BHZpU1dS98SsvUShSx4-2BUbCV8Vb8B6dBgin8IA-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CasKnxVRTUBW0lnnPUiBFDZN1h9k4ItyZ8gGNYlU-2B0dRBvelHnc-2BeUkG1G6dU7PDf-2FElfMd2R-2FZAdFHXec7SgsItHvPTxrnMLjr3JiKAKszqOf3OTw4zwTUilcgLC8kMmaSdKlN7-2BtdgplPjuEfgnssjx7gIlgRgctYxuXZesbWmzum-2FxNj24SzlNV-2FxfC2p7


I don't think we're disagreeing, but I note that I'm specifically not 
talking about what filesystem one might select for a disposable Linux 
system and whatever came with the distro, but presumably a backup system 
from which one wants to store and recover files, into which one 
presumably would wish to select something most appropriate to the task, 
and "I want to futz around with inode allocations" is rarely at the top 
of people's to do lists.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] considerations on i-nodes

[Michael Stowe]

On 2018-04-19 07:37, Gerald Brandt wrote:

I've had data loss on XFS, so I'm very wary of it. It used to be my go
to filesystem.

Lately, I've been looking at bcachefs2. Anyone experience using it?

Gerald


I've also had data loss on XFS, for which I have no satisfactory 
explanation, but I'm equally wary.  Somewhere around here there's a 
limited comparison I did where I recommend btrfs and JFS.  bcachefs 
seems intriguing, but I haven't used it in anger and I haven't seen 
anything practical to distinguish it from btrfs, but I haven't dived 
into it in much depth, either.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] considerations on i-nodes

[G.W. Haywood via BackupPC-users]

Hi there,

On Thu, 19 Apr 2018, Michael Stowe wrote:


... Those who do reach back a decade or more to select a filesystem,


That's like saying the Linux kernel is a hangover from the end of the
20th century.  It's misleading, and more than a bit unfair especially
considering the numbers of posts to the ext4 mailing list:

https://marc.info/?l=linux-ext4=1=201804=2


I generally expect to have good reason ...


Like, er, it works, it's currently under active development, and it's
supported by every Linux distribution you're likely to meet? :)

Yes, even the developers will agree that it's a stop-gap measure.

But it's a big gap, and the rest of them haven't quite plugged it yet:

https://opensource.com/article/18/4/ext4-filesystem

--

73,
Ged.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] considerations on i-nodes

[frush]

Torsten-
Nice writeup of a way to help manage the backupPC file system.   
On Wed, 2018-04-18 at 13:25 +0200, f...@igh.de wrote:
> Dear List, 
> 
> running BackupPC v4 I sometimes ran out of i-nodes...
> 
-- 
Ray Frush "Either you are part of the solution
T:970.491.5527 or part of the precipitate."
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
Colorado State University | IS | System Administrator--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] considerations on i-nodes

[fi]

Dear List, 

running BackupPC v4 I sometimes ran out of i-nodes. 

I have about twelve servers to backup with a total amount of about 2
TB and aboz 20 million files. I keep about 50 backups.  

On an ext4 file system it is likely to run out of i-nodes rather than
to run out of disk space (this holds if the file system was created
using the default parameters). In my situation I have only used half
the space but almost all i-nodes available. 

Because I prefer ext4 for it's robustness and reliability I have to
consider i-nodes. 

We have the pool (or cpool) storing the actual files. And we have the
pc directory storing the structure. The most files in pc are small or
even empty, but they are many (in my case ten times more than the
actual files in pool). They waste an enourmous amount of disk
space. 

So I created a separate (logical) partition of only 200 GB in size and
formatted with 1024 bytes per inode and per block (mkfs.ext4 -b 1024
-i 1024 ...). I later use it for pc.

Then I simply copied the pc entries (note: only the entries, not the
complete directory) to the new partition using cp -a. This took about
50 hours but worked correctly (this also shows, that BackupPC v4 can be
replicated). Afterwards I removed the entries from the original pc
directory and mounted the new partition there. 

I started BackupPC once again and it works like a charm. 

In this constellation remarkable space for the future backup space was
released. If I ever would run out of space or i-nodes again I will
simply increase either of the partitions.

As a suggestion I would recommend an information about i-nodes
consumption in the server status page. 


Finally I am looking for a formula that could be used to estimate the
required disk space and i-nodes depending on the number  and size of
files and the number of backups kept. 


Best regards


Torsten



-- 

Torsten Finke
f...@igh.de
 
Ingenieurgemeinschaft IgH
Gesellschaft für Ingenieurleistungen mbH
Heinz-Bäcker-Str. 34
D-45356 Essen



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/