Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Ray Frush]

Les-

Putting the Mac vs (anything else) argument aside, as it detracts from the
discussion,  I would like to add that even low end NAS devices like the
QNAP offer filesystem snapshots that are, for the purposes of a home office
or small business seamless.   At $WORK, I've used NetApp, EMC Isilon, and
IBM's Storewise V7000 Unified storage, which all do a nice job of
snapshots, and we've never experienced issues with overhead during the
snapshot process.   Point is, there are a lot of options out there,
including using modern file system features.

--
Ray Frush
Colorado State University.


On Fri, Jul 21, 2017 at 8:27 AM, Les Mikesell  wrote:

> On Fri, Jul 21, 2017 at 8:59 AM, B  wrote:
> >
> > @GW Haywood: this would be limited to executive people that usually know
> > what they're doing and are the only ones that are working on not-to-lose
> > docs ie: big spreadsheets  - the idea is to entirely pull off any admin
> > from the restoration process, which isn't the case w/ snapshots.
>
> The quick fix here is to use a Mac with an external or network drive
> for time machine.  If you aren't familiar with it, it does exactly
> what you suggested with easy access for the user and filesystem tricks
> for efficiency.  For a more enterprise flavor, NetApp fileservers have
> done filesystem snapshots at configurable intervals into hidden but
> accessible directories for decades now with next to no overhead, again
> with filesystem tricks.  (But their patents are probably what has kept
> everyone else from doing it well for so long...).
>
> --
>Les Mikesell
>  lesmikes...@gmail.com
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>



-- 
Time flies like an arrow, but fruit flies like a banana.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Les Mikesell]

On Fri, Jul 21, 2017 at 8:59 AM, B  wrote:
>
> @GW Haywood: this would be limited to executive people that usually know
> what they're doing and are the only ones that are working on not-to-lose
> docs ie: big spreadsheets  - the idea is to entirely pull off any admin
> from the restoration process, which isn't the case w/ snapshots.

The quick fix here is to use a Mac with an external or network drive
for time machine.  If you aren't familiar with it, it does exactly
what you suggested with easy access for the user and filesystem tricks
for efficiency.  For a more enterprise flavor, NetApp fileservers have
done filesystem snapshots at configurable intervals into hidden but
accessible directories for decades now with next to no overhead, again
with filesystem tricks.  (But their patents are probably what has kept
everyone else from doing it well for so long...).

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Bzzzz]

On Fri, 21 Jul 2017 06:29:46 -0700
Kenneth Porter  wrote:

> A better solution would be a change to rsyncd to monitor its
> filesystem and remember which files were touched since the last
> backup.

Yep, this is much closer to what I imagined.

> Some filesystems have a "backdoor" like inotify that lets you
> get notified of files getting touched. rsyncd could log these and only
> consult this list, not the whole filesystem. You'd need some way to
> reset the list and force a full filesystem check.

Not necessarily, as other files than work aren't important in this
matter; the idea (I forgot to allude to formerly:/) is to confine
the modified version of BPC to ~/WORK or ~/Documents + ~/Pictures
in order to lower ressources consumption hourly (and because that's
all that's needed:)

In fact, I just realize that, given what I wrote above, the only things
needed would be: a secondary list of directories to backup and 
secondaries schedule parms to achieve that.

So, may be there's no need to modify anything, just launch (and stop)
another instance of BPC by a crontab while feeding it with different
configuration files (IF it doesn't check itself for an already running
instance before starting ?)

@GW Haywood: this would be limited to executive people that usually know
what they're doing and are the only ones that are working on not-to-lose
docs ie: big spreadsheets  - the idea is to entirely pull off any admin
from the restoration process, which isn't the case w/ snapshots.

JY

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Kenneth Porter]

--On Thursday, July 20, 2017 3:54 AM +0200 B  wrote:


An addition to BPC could do the trick, preferably saving the result in
another directory than the main one, by checking which files have been
touched the present day and save them automatically; it may be triggered
from a crontab. And before each complete backup, BPC would empty this
daily directory for the next day.


A better solution would be a change to rsyncd to monitor its filesystem and 
remember which files were touched since the last backup. Some filesystems 
have a "backdoor" like inotify that lets you get notified of files getting 
touched. rsyncd could log these and only consult this list, not the whole 
filesystem. You'd need some way to reset the list and force a full 
filesystem check.


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[G.W. Haywood]

Hi there,

On Thu, 20 Jul 2017, B wrote:


... clumsy users would be able to recover their work very rapidly
with at most one hour loss ...

How about that ?


You want to apply a technical answer to a non-technical question.

Bad idea.

It is vastly more expensive to employ highly trained people to run
around cleaning up after badly trained incompetents [*] than it is to
train people not to be clumsy in the first place.

[*] You can't even let them do it themselves, they'll just recover the
wrong backup and make the whole thing twenty times worse - believe me,
I've see it happen.

We used to say 'GIGO'.  If the users are so clumsy, there is no point
in backing up their work because it is very probably garbage anyway.

So train clumsy users not to be so clumsy.

One way of doing that is, every time they have done something clumsy,
make them do their work all over again.  They will probably learn to
be careful very quickly.  If they cannot be trained to be careful and
accurate they should be redeployed, or unemployed.  The management's
job is to define limits on care and accuracy and enforce them.  If the
management can't do that then it is the fault of the management, not
of the people being (mis)managed.

--

73,
Ged.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Bzzzz]

On Thu, 20 Jul 2017 17:27:30 +0200
Daniel Berteaud  wrote:

> Because what you want to achieve is not really backups, but some kind
> of rotative snapshots. There are lots of different ways to do this
> (LVM, LVM-thin, btrfs, zfs etc..), and this is very dependant on the
> system hosting your data, which is not controlled by BackupPC.
> BackupPC is just a backup tool. You could configure more frequent incr
> (every hour), but the performance impact won't be the same as
> snapshots. This can be a solution depending on the amount of data you
> have to manage, but it's already possible, without any modification to
> BackupPC

Fair enough, I'm gonna watch this closely - thanks.

JY

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Daniel Berteaud]

Le 20/07/2017 à 15:54, B a écrit :

On Thu, 20 Jul 2017 10:26:18 +0200
Daniel Berteaud  wrote:


I think this is out of BackupPC's scope

Please develop, don't drop me dry, why is that?
Why adding a kinda-Xtiple-fugitive-daily-snapshots of only touched files
is out of the BPC's scope ? On the other hand, I see this as the missing
complement to get a professional ubiquitous backup system.


Because what you want to achieve is not really backups, but some kind of 
rotative snapshots. There are lots of different ways to do this (LVM, 
LVM-thin, btrfs, zfs etc..), and this is very dependant on the system 
hosting your data, which is not controlled by BackupPC. BackupPC is just 
a backup tool. You could configure more frequent incr (every hour), but 
the performance impact won't be the same as snapshots. This can be a 
solution depending on the amount of data you have to manage, but it's 
already possible, without any modification to BackupPC


++

--

Logo FWS

*Daniel Berteaud*

FIREWALL-SERVICES SAS.
Société de Services en Logiciels Libres
Tel : 05 56 64 15 32 
Visio : http://vroom.fws.fr/dani
/www.firewall-services.com/

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Bzzzz]

On Thu, 20 Jul 2017 08:18:44 -0600
Ray Frush  wrote:

> I believe you could do something like you propose with the current
> BackupPC by setting the "IncrPeriod' to 0.04 (1/24 of a day).   You'd
> have to make some interesting settings for "FillCycle" and
> "FullKeepCnt" to make it keep a usable schedule, but you could then
> have  'hourly' incrementals.

"interesting settings" is the corner stone of this.

> As Les Mikesell just pointed out, the downside would be that for large
> instances, you'd be doing a lot of fairly expensive (compute time)
> operations every hour to scan the file system for changes.
> 
> I believe that FS snapshots are faster, and more efficient than
> BackupPC could ever be for this.

OK, fair explanation this time; too bad.
I'll dive into the snapshot code to see if a timestamp check can be
easily implemented - thanks.

JY

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Bzzzz]

On Thu, 20 Jul 2017 09:17:29 -0500
Les Mikesell  wrote:

> You can set the schedule to run as often as you like, but the
> underlying tools are going to have to traverse the whole directory
> tree to find the touched files, which you probably don't want to
> happen while you are working.

This stage just slow down machines for a bout a minute, which is
acceptable.

>   The easy way to get this facility is to do your work on a
> Mac.

If I didn't want serious security, I'd use w$, which is about the same
level…

JY

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Ray Frush]

I believe you could do something like you propose with the current BackupPC
by setting the "IncrPeriod' to 0.04 (1/24 of a day).   You'd have to make
some interesting settings for "FillCycle" and "FullKeepCnt" to make it keep
a usable schedule, but you could then have  'hourly' incrementals.

As Les Mikesell just pointed out, the downside would be that for large
instances, you'd be doing a lot of fairly expensive (compute time)
operations every hour to scan the file system for changes.

I believe that FS snapshots are faster, and more efficient than BackupPC
could ever be for this.

--
Ray Frush

On Thu, Jul 20, 2017 at 7:54 AM, B  wrote:

> On Thu, 20 Jul 2017 10:26:18 +0200
> Daniel Berteaud  wrote:
>
> > I think this is out of BackupPC's scope
>
> Please develop, don't drop me dry, why is that?
> Why adding a kinda-Xtiple-fugitive-daily-snapshots of only touched files
> is out of the BPC's scope ? On the other hand, I see this as the missing
> complement to get a professional ubiquitous backup system.
>
> JY
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>



-- 
Time flies like an arrow, but fruit flies like a banana.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Les Mikesell]

On Thu, Jul 20, 2017 at 8:54 AM, B  wrote:
> On Thu, 20 Jul 2017 10:26:18 +0200
> Daniel Berteaud  wrote:
>
>> I think this is out of BackupPC's scope
>
> Please develop, don't drop me dry, why is that?
> Why adding a kinda-Xtiple-fugitive-daily-snapshots of only touched files
> is out of the BPC's scope ? On the other hand, I see this as the missing
> complement to get a professional ubiquitous backup system.

You can set the schedule to run as often as you like, but the
underlying tools are going to have to traverse the whole directory
tree to find the touched files, which you probably don't want to
happen while you are working.   Apple makes their time machine work
efficiently by using their own file system that propagates timestamp
changes up the directory tree so it can ignore whole inactive
branches.  The easy way to get this facility is to do your work on a
Mac.

-- 
  Les Mikesell
 lesmikes...@gmail.com

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Bzzzz]

On Thu, 20 Jul 2017 10:26:18 +0200
Daniel Berteaud  wrote:

> I think this is out of BackupPC's scope

Please develop, don't drop me dry, why is that?
Why adding a kinda-Xtiple-fugitive-daily-snapshots of only touched files
is out of the BPC's scope ? On the other hand, I see this as the missing
complement to get a professional ubiquitous backup system.

JY

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] I'd like to make a suggestion to the BPC devs

[Daniel Berteaud]

Le 20/07/2017 à 02:54, B a écrit :

Hi Bacukppcers,


My suggestion is to avoid using such things as FS snapshots during the
day to avoid work losses.

An addition to BPC could do the trick, preferably saving the result in
another directory than the main one, by checking which files have been
touched the present day and save them automatically; it may be triggered
from a crontab. And before each complete backup, BPC would empty this
daily directory for the next day.

This way, if we take an hypothesis of an hourly crontab, clumsy users
would be able to recover their work very rapidly with at most one hour
loss - and the hourly backups being confined to only touched files
should be quite transparent/invisible/lightweight for them.

How about that ?


I think this is out of BackupPC's scope

++

--

Logo FWS

*Daniel Berteaud*

FIREWALL-SERVICES SAS.
Société de Services en Logiciels Libres
Tel : 05 56 64 15 32 
Visio : http://vroom.fws.fr/dani
/www.firewall-services.com/

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] I'd like to make a suggestion to the BPC devs

[Bzzzz]

Hi Bacukppcers,


My suggestion is to avoid using such things as FS snapshots during the
day to avoid work losses.

An addition to BPC could do the trick, preferably saving the result in
another directory than the main one, by checking which files have been
touched the present day and save them automatically; it may be triggered
from a crontab. And before each complete backup, BPC would empty this
daily directory for the next day.

This way, if we take an hypothesis of an hourly crontab, clumsy users
would be able to recover their work very rapidly with at most one hour
loss - and the hourly backups being confined to only touched files
should be quite transparent/invisible/lightweight for them.

How about that ?

Jean-Yves

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/