Re: [BackupPC-users] Disk space used by single host

[backuppc]

Kenneth Porter wrote at about 13:37:02 -0700 on Monday, May 10, 2021:
 > --On Monday, May 10, 2021 5:25 PM -0400 backu...@kosowsky.org wrote:
 > 
 > > Even if you were content with seeing a standalone heat map for one
 > > backup (as unhelpful as it may be), creating it would be quite slow
 > > since you would need to access, unpack/uncompress, and read each attrib
 > > file to calculate the file sizes.
 > 
 > Indeed. If you've used the treemap utilities, you know you leave them 
 > running in the background for an hour or more and then view the result. 
 > kdirmap has the feature that you can load a previously-saved map. Hence, 
 > I'd think such a plugin would run in the background or load a kdirmap file. 
 > Hmm, perhaps this might be better as a standalone web page rather than a 
 > BackupPC plugin.

Sounds like a standalone routine would be simpler and more in keeping
with the structure of BackupPC.

> 
 > Presentation would be interesting. Most useful would be a treemap of the 
 > pool, with tooltips showing what each file was known as on clients, and how 
 > many clients shared it.
Again not easy, as you would have to recurse through all the backups
to determine which source file name is represented by the pool
file. This does not scale well and is computationally expensive as you
would need to recurse and read through all attrib files.

 > One could optionally weight a file's size by how 
 > many clients shared it.

This is doable as the pool refcnt will say how many backups share it
-- though it won't distinguish between multiple backups on one machine
vs. multiple machines. To distinguish, you would again need to recurse
through all the machines...

 > (Is there a way to add links to external facilities in BackupPC? I 
 > currently have a separate page with the link to the cygwin rsyncd package 
 > to use when setting up a new PC.)

You just run a separate program and optionally query the BackupPC
daemon if you need to know status of the current BackupPC processes.

> 
 > > Of course, if you "welcome" such a plugin, you are more than
 > > "welcome" to write one as this is an open source project...
 > >
 > > I just don't see it happening by itself as the dev team is small (mostly
 > > Craig) and focused on core stability and functionality.
 > 
 > Oh, I'm quite aware it would be SMOP, hence the suggestion that it be a 3rd 
 > party plugin. ;) But if someone's looking for a school project that might 
 > be a nice one to share.
 > 
Unlikely, but good luck if you go on the journey of writing this yourself...


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Disk space used by single host

[Kenneth Porter]

--On Monday, May 10, 2021 5:25 PM -0400 backu...@kosowsky.org wrote:


Even if you were content with seeing a standalone heat map for one
backup (as unhelpful as it may be), creating it would be quite slow
since you would need to access, unpack/uncompress, and read each attrib
file to calculate the file sizes.


Indeed. If you've used the treemap utilities, you know you leave them 
running in the background for an hour or more and then view the result. 
kdirmap has the feature that you can load a previously-saved map. Hence, 
I'd think such a plugin would run in the background or load a kdirmap file. 
Hmm, perhaps this might be better as a standalone web page rather than a 
BackupPC plugin.


Presentation would be interesting. Most useful would be a treemap of the 
pool, with tooltips showing what each file was known as on clients, and how 
many clients shared it. One could optionally weight a file's size by how 
many clients shared it.


(Is there a way to add links to external facilities in BackupPC? I 
currently have a separate page with the link to the cygwin rsyncd package 
to use when setting up a new PC.)



Of course, if you "welcome" such a plugin, you are more than
"welcome" to write one as this is an open source project...

I just don't see it happening by itself as the dev team is small (mostly
Craig) and focused on core stability and functionality.


Oh, I'm quite aware it would be SMOP, hence the suggestion that it be a 3rd 
party plugin. ;) But if someone's looking for a school project that might 
be a nice one to share.





___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Disk space used by single host

[backuppc]

Kenneth Porter wrote at about 10:36:44 -0700 on Monday, May 10, 2021:
 > --On Monday, May 10, 2021 1:16 PM -0500 Gerald Brandt  
 > wrote:
 > 
 > > I'm looking for actual physical disk usage of the backup. My disk is 96%
 > > full and I want to know which host is using most of the space.
 > 
 > Coincidentally, I was recently thinking that it would be nifty to replace 
 > the current tree display with the treemap-based one from kdirstat and 
 > WinDirStat. The treemap makes it really easy to see where the disk hogs 
 > are, and those programs are open source, so a plugin for BackupPC would be 
 > welcome.
 > 
 > Check out the screenshots of the two program:
 > 
 > 
 > 
 > 

As others have pointed out, finding the disk hogs is not a simple
problem since files are shared across multiple backups (for a single
machine) and across multiple machines for common files.

Hence, a single treemap wouldn't necessarily make a lot of sense since
there is no easy way to distinguish a directory that consumes a lot of
backup space since it's branches contain unique files that combined
add up to a lot of space from a directory that seems similarly large
but actually adds little incremental space since its files are shared
across multiple backups and machines.

One would have to walk the full directory tree to calculate how much of each
directory is unique vs. shared, let alone knowing how broadly it is
shared (eg., across how many backups for a given machine? across how
many different machines).

Even if you were content with seeing a standalone heat map for one
backup (as unhelpful as it may be), creating it would be quite slow
since you would need to access, unpack/uncompress, and read each attrib file
to calculate the file sizes.

Try to do a 'du -s' on a top level directory under backupcfs and you
will see how slow this is.

Of course, if you "welcome" such a plugin, you are more than
"welcome" to write one as this is an open source project...

I just don't see it happening by itself as the dev team is small (mostly Craig)
and focused on core stability and functionality.


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Disk space used by single host

[Kenneth Porter]

--On Monday, May 10, 2021 1:16 PM -0500 Gerald Brandt  
wrote:



I'm looking for actual physical disk usage of the backup. My disk is 96%
full and I want to know which host is using most of the space.


Coincidentally, I was recently thinking that it would be nifty to replace 
the current tree display with the treemap-based one from kdirstat and 
WinDirStat. The treemap makes it really easy to see where the disk hogs 
are, and those programs are open source, so a plugin for BackupPC would be 
welcome.


Check out the screenshots of the two program:






___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Disk space used by single host

[Kenneth Porter]

--On Monday, May 10, 2021 1:16 PM -0500 Gerald Brandt  
wrote:



I'm looking for actual physical disk usage of the backup. My disk is 96%
full and I want to know which host is using most of the space.


That's only useful if that host is mostly unique files not present on other 
hosts. Perhaps that's true in your installation. My site has a lot of 
Windows workstations and the common operating system files dominate each 
backup.




___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Disk space used by single host

[Gerald Brandt]

I'm looking for actual physical disk usage of the backup. My disk is 96% 
full and I want to know which host is using most of the space.



Gerald


On 2021-05-07 1:49 a.m., Dave Sherohman wrote:
I think you'd first have to define what you mean by "how much disk 
space is used by a single host's backups".  Because of BPC's 
deduplication functions, the answer will be very different if you mean 
"how much space would I need to make a full copy of this host's data" 
vs. if you mean "how much space would be freed on the backup server if 
I deleted all of this host's backups".


On 5/6/21 6:18 PM, Gerald Brandt wrote:

Hi,

Is there an easy way to find out how much disk space is used by a 
single hosts backups?



Gerald




___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Disk space used by single host

[Dave Sherohman]

I think you'd first have to define what you mean by "how much disk space 
is used by a single host's backups".  Because of BPC's deduplication 
functions, the answer will be very different if you mean "how much space 
would I need to make a full copy of this host's data" vs. if you mean 
"how much space would be freed on the backup server if I deleted all of 
this host's backups".


On 5/6/21 6:18 PM, Gerald Brandt wrote:

Hi,

Is there an easy way to find out how much disk space is used by a 
single hosts backups?



Gerald




___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

[BackupPC-users] Disk space used by single host

[Gerald Brandt]

Hi,

Is there an easy way to find out how much disk space is used by a single 
hosts backups?



Gerald




___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/