Re: [BackupPC-users] BackupPC Archive Failing
Hi, backu...@kosowsky.org wrote on 2015-06-02 20:03:04 -0400 [Re: [BackupPC-users] BackupPC Archive Failing]: Mark Campbell wrote at about 10:53:04 -0700 on Tuesday, June 2, 2015: [...] Firstly, what is the difference between running nightly manually vs it being run by the daemon? It performs the same tasks either way, no? So, shouldn't the inverse be true as well? In other words, if it's not safe to run nightly manually, wouldn't it also be unsafe being run by the daemon? What makes it safe for the daemon to run it? Because there is a (very small) chance of collisions if BackupPC_link and BackupPC_nightly run concurrently... I don't want to imagine what could happen if BackupPC decided to start a second instance of BackupPC_nightly (for the same part of the pool). BackupPC is beautifully constructed so that almost all operations are atomic and non-interfering. However, an error could occur if BackupPC_nightly deletes or chain renumbers a pool file that BackupPC_link is simultaneously trying to link to. The daemon makes sure that BackupPC_nightly won't run if BackupPC_link is running. Conversely, the daemon holds off starting new BackupPC_link processes while BackupPC_nightly is running. I would just like to add that this is dependent on the version of BackupPC. In earlier versions, the conflict used to be between BackupPC_nightly and BackupPC_dump (and, possibly, BackupPC_link). In future versions it might be yet different. The point being that the daemon is programmed to do the right thing for the version of BackupPC it belongs to. As Jeffrey said: BackupPC_nightly was never meant to be run by users. So it may not always be clearly documented when it would be safe to do so, just like it is not clearly documented how you should manipulate attrib files or the on-disk metadata of your favorite file system. You just shouldn't. Secondly, if it's always been a terrible idea to do so, how did the idea get started to run it manually in the first place, and why is it so pervasive? People do a lot of stupid things pervasively. Like pulling the power cord on a PC before shutting it down gracefully... You mean, they shut it down gracefully *after* having pulled the plug? :-) Regards, Holger -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC Archive Failing
Apologies if I'm hijacking this thread, but there's two things I don't understand... Firstly, what is the difference between running nightly manually vs it being run by the daemon? It performs the same tasks either way, no? So, shouldn't the inverse be true as well? In other words, if it's not safe to run nightly manually, wouldn't it also be unsafe being run by the daemon? What makes it safe for the daemon to run it? Secondly, if it's always been a terrible idea to do so, how did the idea get started to run it manually in the first place, and why is it so pervasive? Thanks, --Mark -Original Message- From: backuppc-users-boun...@lists.sourceforge.net [mailto:backuppc-users-boun...@lists.sourceforge.net] On Behalf Of Holger Parplies Sent: Tuesday, June 02, 2015 1:31 PM To: Random; kmwatt...@proobject.com Subject: Re: [BackupPC-users] BackupPC Archive Failing Hi, Random wrote on 2015-06-02 05:13:36 -0700 [[BackupPC-users] BackupPC Archive Failing]: [...] The previous issue turned out to be the main filesystem filling up. I thought I was joking when I suggested that ... [...] The cpool is filling up and the nightly clean hasn't been clearing up any space.. for the last couple of weeks at least. How do I get the nightly process to start cleaning the cpool again? Well, reduce the amount of data stored in your backups (meaning all of them, not just the new ones; how to do that is a completely different topic). Due to BackupPC's pooling mechanism and typical data usage patterns, removing old backups (e.g. by automatic expiration done by BackupPC) tends to free far less space than you might expect. New backups only take as much space as you have new content not already found in the pool (i.e. copying or renaming a file won't take any new space, changing one byte of an existing file will create a new copy in the pool, if you don't happen to have an identical file somewhere else already). Conversely, deleting old backups only gives you back space for content you no longer have *anywhere*. After deleting a file (or changing one), you will have to wait until *all backups* that include the old version of the file have expired before you get anything [much] back. There is a *slight* amount of metadata per backup, but even that is pooled, so you won't get anything back e.g. for attrib files of unchanged directories. As I've written before, if you *think* you need BackupPC_nightly to run (or do a better job), you really either need more disk space, or need to rethink what data you want to keep backups of. I've ran it manually a few times and there's been no difference. That is one of my favorite topics. NEVER RUN BackupPC_nightly MANUALLY! NEVER! There are better ways to void your warranty. One of these days, I'll write a patch that makes BackupPC_nightly refuse to run if not invoked by the daemon. Why does everyone assume that if something doesn't happen as they imagine, the software must be faulty, and that by doing something arbitrary *with this supposedly faulty software*, they will fix things? Because that is the way Windoze works? BackupPC is not powered by millions of dollars worth of paid developer time. It's free. It's open source. It doesn't need the incentive of tons of bugs to get people to spend money on ever the newest versions of new bugs. It just works, it's stable, and it's versatile. It's not as if you couldn't still run a 2.x version and get perfectly usable backups. There are bugs left (e.g. you can run BackupPC_nightly manually), but *most of the time* misbehaviour turns out to be due to usage errors, configuration errors, or hardware errors. Or misunderstanding the concepts. Just to prove my point, you went to the trouble of running it *a few times*!? What is that supposed to achieve? Do you want BackupPC to free a certain amount of space each time? If I bug it long enough, it will delete something, regardless of what data is still needed and what data isn't? |This was sent by kmwatt...@proobject.com via Backup Central. [...] Regards, Holger -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC Archive Failing
Mark Campbell wrote at about 10:53:04 -0700 on Tuesday, June 2, 2015: Apologies if I'm hijacking this thread, but there's two things I don't understand... Firstly, what is the difference between running nightly manually vs it being run by the daemon? It performs the same tasks either way, no? So, shouldn't the inverse be true as well? In other words, if it's not safe to run nightly manually, wouldn't it also be unsafe being run by the daemon? What makes it safe for the daemon to run it? Because there is a (very small) chance of collisions if BackupPC_link and BackupPC_nightly run concurrently... BackupPC is beautifully constructed so that almost all operations are atomic and non-interfering. However, an error could occur if BackupPC_nightly deletes or chain renumbers a pool file that BackupPC_link is simultaneously trying to link to. The daemon makes sure that BackupPC_nightly won't run if BackupPC_link is running. Conversely, the daemon holds off starting new BackupPC_link processes while BackupPC_nightly is running. The beauty of the architecture is that BackupPC_backup does not interfere and can continue to create backups. Only the final linking to pool entries needs to be deferred. Thus, BackupPC_nightly is only safe to run manually if you have shut down the BackupPC service (daemon). Secondly, if it's always been a terrible idea to do so, how did the idea get started to run it manually in the first place, and why is it so pervasive? People do a lot of stupid things pervasively. Like pulling the power cord on a PC before shutting it down gracefully... BackupPC_nightly was never meant to be run by users. It's merely an executable sitting inside the BackupPC bin directory. Sometimes users think they are smarter than they really are and just start running programs that they think will solve their problems... only to risk creating worse problems... as evidenced by this user who figured that if BackupPC_nightly did not automatically free up the amount of disk space he wanted, then perhaps running it manually multiple times would convince the program to try harder and find heretofore unfreeable space. Sort of liking praying repeatedly if your initial prayer is not answered... -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC Archive Failing
Hi, Random wrote on 2015-05-20 05:55:49 -0700 [[BackupPC-users] BackupPC Archive Failing]: [...] an archive started failing. (I run it manually twice a month). I've tried to run it twice and it's failed each time with the same error. [...] Archive failed (Error: /usr/share/backuppc/bin/BackupPC_tarCreate, compress or split failed) [...] The archive runs part way, before failing, so I can see two backup files show up on the drive, but not the other 4. Individually, the all 6 machines are backing up as usual, except a 2 millisecond timeout issue that's actually 2 seconds, right? that happens periodically. (An issue I recently discovered but don't believe is related to my current problem). It might be. Did the failure occur in the last backup of one of the machines you are trying to create an archive for? Is it always one machine, or does the timeout occur for all of the machines? Periodically means what exactly? Once per backup? Once per month? Occasionally? Does the *archive* always fail on the same machine? Probably so, since you get archives for two machines. Have you tried to start the archive a third time, preferrably for different backups of the target machines? Just above the error message, there should be a line reading Executing: [details of the command]. Try running the command quoted manually *as the backuppc user* to see if you get a more verbose diagnostic. What could be going wrong? Destination file system full? Hardware error? Corruption of metadata within the BackupPC pool (as in: attrib file corrupt or data file corrupt in a way that it will fail to decompress)? Something else less obvious? Random wrote on 2015-05-22 05:18:23 -0700 [[BackupPC-users] BackupPC Archive Failing]: Anyone? Your messages didn't reach the mailing list until 2015-06-01, so it was hard for anyone to answer before that. Probably a problem with Backup Central. Regards, Holger -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC Archive Failing
Check dmesg and see if you are having hardware errors on the drive. Make sure it is not failing Henry Henry B. Burroughs, III On Mon, Jun 1, 2015 at 5:08 PM, Random backuppc-fo...@backupcentral.com wrote: My company runs BackupPC on an Ubuntu machine. Everything typically run without any problems until the other day when, out the blue, an archive started failing. (I run it manually twice a month). I've tried to run it twice and it's failed each time with the same error. (Will paste below) I'm using the same backup drive that I've been using, so the permissions haven't change. The archive runs part way, before failing, so I can see two backup files show up on the drive, but not the other 4. Individually, the all 6 machines are backing up as usual, except a 2 millisecond timeout issue that happens periodically. (An issue I recently discovered but don't believe is related to my current problem). Here's the error that I get: Archive failed (Error: /usr/share/backuppc/bin/BackupPC_tarCreate, compress or split failed) I've checked both the archive log and backupPC's log via the web interface and they both list the same error. What could be going wrong? Thanks. +-- |This was sent by kmwatt...@proobject.com via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/-- ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
