Re: Live Fille System Backup

thank you for mentioning bup. That's a great option I am considering. On Fri, May 12, 2017 at 5:52 AM, Stefan Monnier wrote: > > I would like a backup tool that does not bring a million dependencies > with > > MBs of files. Something that works on server without X

Re: Live Fille System Backup

> I would like a backup tool that does not bring a million dependencies with > MBs of files. Something that works on server without X Windows and can > send backup to an externally attached USB drive. Nothing fancy. No > network infrastructure. Incremental backups would be greatly

Re: Live Fille System Backup

On 2017-05-09, Greg Wooledge wrote: > On Tue, May 09, 2017 at 12:03:14PM +, Curt wrote: >> Because someone mentioned Btrfs and its backup "magic," I'm looking at >> the wiki, which says (amongst other things): >> >>

Re: Live Fille System Backup

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, May 09, 2017 at 08:08:48AM -0400, Greg Wooledge wrote: [...] > So, basically the same as LVM snapshots, which are independent of the > type of file system that sits on the LVM logical volume. To be fair, btrfs knows about the file system,

Re: Live Fille System Backup

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, May 09, 2017 at 12:03:14PM +, Curt wrote: [...] > Because someone mentioned Btrfs and its backup "magic," I'm looking at > the wiki, which says (amongst other things): > > https://btrfs.wiki.kernel.org/index.php/Incremental_Backup > >

Re: Live Fille System Backup

On Tue, May 09, 2017 at 12:03:14PM +, Curt wrote: > Because someone mentioned Btrfs and its backup "magic," I'm looking at > the wiki, which says (amongst other things): > > https://btrfs.wiki.kernel.org/index.php/Incremental_Backup > > Instant, Atomic COW Snapshots > Since the snapshots

Re: Live Fille System Backup

On 2017-05-09, to...@tuxteam.de wrote: > > On Mon, May 08, 2017 at 04:03:54PM -0700, Sergei G wrote: >> And if you think application consistency is nothing to worry about know >> that Git on Ext4 has failed to recover. >> >> My current conclusion: >> >> 1. If I want

Re: Live Fille System Backup

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, May 08, 2017 at 04:03:54PM -0700, Sergei G wrote: > And if you think application consistency is nothing to worry about know > that Git on Ext4 has failed to recover. > > My current conclusion: > > 1. If I want application consistency I have

Re: Live Fille System Backup

And if you think application consistency is nothing to worry about know that Git on Ext4 has failed to recover. My current conclusion: 1. If I want application consistency I have to take down any application that has an internal database or backup those applications prior to FS backup. 2.

Re: Live Fille System Backup

We have to remember that things are actually worse at the application consistency level. An application may think that it has committed its write, but the file system has not written all the bytes to the disk yet. File system crash consistency is a sibling of the backup topic. Here is relevant

Re: Live Fille System Backup

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, May 07, 2017 at 08:42:36AM -0400, rhkra...@gmail.com wrote: > On Sunday, May 07, 2017 05:47:20 AM to...@tuxteam.de wrote: > > > > > Otherwise, "on line" backup is simply not an option. [...] > I guess the way that comes to mind, maybe

Re: Live Fille System Backup

On Sunday, May 07, 2017 05:47:20 AM to...@tuxteam.de wrote: > Otherwise, "on line" backup is simply not an option. Thanks! I've been paying at least peripheral attention to this thread (as I do to many), and got to thinking / wondering about how to deal with backup in an "enterprise" type

Re: Live Fille System Backup

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, May 07, 2017 at 10:53:51AM +1200, Ben Caradoc-Davies wrote: [also a reply to Henrique, elsewhere in this thread] > If a file is updated while it is being copied, it may contain only > half a change set and be in an internally inconsistent

Re: Live Fille System Backup

On 07/05/17 01:56, to...@tuxteam.de wrote: On Sat, May 06, 2017 at 06:42:46AM -0400, Dan Ritter wrote: On Sat, May 06, 2017 at 07:52:25AM +0200, to...@tuxteam.de wrote: On Fri, May 05, 2017 at 01:35:35PM -0700, Sergei G wrote: it would be naive at best to think that busy files [...] What is

Re: Live Fille System Backup

On Sat, 06 May 2017, to...@tuxteam.de wrote: > > 2. one or more components of a set of several files change *during* > >backup (files may be internally consistent, but the set of files > >itself isn't). > > Only the application can know that. But this is a realm where snapshots > won't

Re: Live Fille System Backup

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, May 06, 2017 at 02:18:47PM -0300, Henrique de Moraes Holschuh wrote: > On Sat, 06 May 2017, to...@tuxteam.de wrote: > > Hm. That just means that the file will change after being copied. > > There can also exist busy file *sets*, which need to

Re: Live Fille System Backup

On Sat, 06 May 2017, to...@tuxteam.de wrote: > Hm. That just means that the file will change after being copied. There can also exist busy file *sets*, which need to be atomically snapshotted (this is, of course, application specific). "busy", as far as backups go, means: 1. a file changes

Re: Live Fille System Backup

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, May 06, 2017 at 06:42:46AM -0400, Dan Ritter wrote: > On Sat, May 06, 2017 at 07:52:25AM +0200, to...@tuxteam.de wrote: > > On Fri, May 05, 2017 at 01:35:35PM -0700, Sergei G wrote: > > > it would be naive at best to think that busy files

Re: Live Fille System Backup

On Sat, May 06, 2017 at 07:52:25AM +0200, to...@tuxteam.de wrote: > On Fri, May 05, 2017 at 01:35:35PM -0700, Sergei G wrote: > > it would be naive at best to think that busy files [...] > > What is a "busy" file? Open for writing. -dsr-

Re: Live Fille System Backup

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, May 05, 2017 at 01:35:35PM -0700, Sergei G wrote: > it would be naive at best to think that busy files [...] What is a "busy" file? regards - -- t -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux)

Re: Live Fille System Backup

On Fri, 05 May 2017, John Conover wrote: > Also be advised if "dest/" is a FAT formatted SD card, there is a file > size limit of 4GB, (and the -aAXv argument to rsync(1) may not be > viable to restore the permission/ACL of files/directories.) Using rsync on non-POSIX filesystems is a Bad Idea,

Re: Live Fille System Backup

Henrique de Moraes Holschuh writes: > On Fri, 05 May 2017, Sergei G wrote: > > I am not convinced that rsync can handle all device type files, all types > > of links and other attributes I don't even know about. I am not convinced > > that anyone actually restored file system using rsync. Does

Re: Live Fille System Backup

On 5/4/17 7:17 PM, Sergei G wrote: > I am running Raspberry PI and I would like to dump full file system without > shutting down the system. One machine runs nginx and another runs > PostgreSQL. I have had a good success with FreeBSD and dump software, because > it is part of the OS and core

Re: Live Fille System Backup

On Fri, 05 May 2017, Sergei G wrote: > I am not convinced that rsync can handle all device type files, all types > of links and other attributes I don't even know about. I am not convinced > that anyone actually restored file system using rsync. Does it really work? Yes, it does. Been there,

Re: Live Fille System Backup

On Fri, 05 May 2017, Sergei G wrote: > it would be naive at best to think that busy files can be handled at > application level. No, rsync cannot handle the problem of entire file > system dump. Neither it is safe to handle all special file cases. I am > thinking about sparse files (if Linux

Re: Live Fille System Backup

I just realized that busy files are pretty much impossible to deal with no matter what. If DB is in the middle of the transaction, then a "snapshot" based backup will work (we simply loose that transaction data as it is equivalent to crash). Other busy files might be corrupted if those files

Re: Live Fille System Backup

it would be naive at best to think that busy files can be handled at application level. No, rsync cannot handle the problem of entire file system dump. Neither it is safe to handle all special file cases. I am thinking about sparse files (if Linux has those). On Thu, May 4, 2017 at 11:52 PM,

Re: Live Fille System Backup

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, May 04, 2017 at 04:17:46PM -0700, Sergei G wrote: > I am running Raspberry PI and I would like to dump full file system without > shutting down the system. One machine runs nginx and another runs > PostgreSQL. I have had a good success with

Re: Live Fille System Backup

On 04-05-17, Sergei G wrote: > That's good to know. > > > Thank you > > > On 5/4/17 6:46 PM, Anders Andersson wrote: > > On Fri, May 5, 2017 at 1:17 AM, Sergei G wrote: > > > I would like a backup tool that does not bring a million dependencies with > > > MBs of

Re: Live Fille System Backup

That's good to know. Thank you On 5/4/17 6:46 PM, Anders Andersson wrote: On Fri, May 5, 2017 at 1:17 AM, Sergei G wrote: I would like a backup tool that does not bring a million dependencies with MBs of files. Something that works on server without X Windows and

Re: Live Fille System Backup

On Fri, May 5, 2017 at 1:17 AM, Sergei G wrote: > I would like a backup tool that does not bring a million dependencies with > MBs of files. Something that works on server without X Windows and can send > backup to an externally attached USB drive. Nothing fancy. No

Live Fille System Backup

I am running Raspberry PI and I would like to dump full file system without shutting down the system. One machine runs nginx and another runs PostgreSQL. I have had a good success with FreeBSD and dump software, because it is part of the OS and core team maintains it. However, dump utility is