Re: [BackupPC-users] Missing backup files
On 11/14/22 12:57, backu...@kosowsky.org wrote: > Can you define how this situation is a "phantom bug". I pretty > clearly explained the lack of feedback from the UI when this condition > occurs and you get sent a zero byte zip file with no explanation. The logs were clear -- this is Enterprise software and when something doesn't work, one is expected to check the logs (as well as doing other basic troubleshooting homework). If backups or restores don't work -- for whatever reason -- the first thing to do is check the logs!!! (how many times do I have to say that???) Correct, the logs were clear. Honestly I didn't expect to see anything in the logs. You don't have to say it anymore as someone else already commented. When you're sounding like a broken record, the first thing to do is switch to a new record. If you feel like you're owed an apology, then sure. I made a bad assumption here about where the details about the backup download would appear. This is not child-proof Windows consumer software where the UI is expected to do everything and spoon-feed all the answers. The GUI is only there to help with basic tasks -- indeed the GUI itself is mostly focused on making it easier for users to change config parameters and launch backups/restores -- it is not meant to be idiot-proof or a substitute for logs. Speaking of assumptions. I never said it was child-proof windows consumer software, but now that you explain the ideology behind what gui is expected to do, this makes more sense now. Isn't this better? Now we're getting some formal discourse and you're not just wasting energy attacking someone you don't know on a list that in theory, is designed to facilitate discussion about backuppc. Indeed, many users (myself included) never even use the GUI since the CLI is much more powerful and accessible. As such, there is not a huge ongoing investment being made in adding more bells and whistles (unless an individual user -- hint you -- contributes code). Thanks for explaining the normal expected use. The fact that the UI doesn't pop up a nice shiny, themed dialog box explaining why your restore failed is NOT a BUG. At most, it's a feature request to include more debugging info directly in the GUI -- and you are welcome to submit code that provides such messaging. Sounds like we have a different definition of bug. But sure, as a perl developer I would be interested in addressing this. That being said, the GUI did undoubtedly signal an error -- just not in a pop-up. Had you looked under "Restore Summary", you would have seen presumably that your restore had no files or no data. The top status text may also have noted something. The gui did not actually signal an error. Try a reproduction of this on your own system. Do a zip file download from a system that has a current backup job running. You'll notice that you get a zero byte zip file. "Restore Summary" does not log this. There are no new entries in the restore summary page to show there was any kind of restore issue. The only place this is logged is under 'Log file' when doing a "Direct Restore". The web page generated makes no mention of any issues preparing the zip file. Here's what happens. Main 'LOG file '2022-11-14 12:22:48 User backuppc downloaded zip archive for comm-nyc1, backup 52; share /mnt/volume_nyc1_01; files were: /debian-c-2-nyc1-01/var/log/apache2/access.log.169.gz 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump is running on host xyz I don't get any new detail in 'LOG file' at the per-server level. So it looks like it creates the zip file first, and then logs the error... which to me is quite strange. Other than the global log, there's no indication to the user that there's an issue. But.. now that I know a zip file of size zero is indicative of an error occurring, I'll be aware that this is the case going forward. I can assure you that there are many, many even more egregious or alternatively subtle ways for backups (or restores) to fail in part or in full and there is no nice shiny GUI that signals many such errors. That's why you need to check and test backups as well as restores -- if there are issues, then... drumbeat... check the logs! > Again... unneeded hostility. Not hostility -- just reaction to: 1. Laziness -- you made all types of claims and request for help before doing even minimal troubleshooting yourself I already explained the reason behind the original posting. Obviously you missed the part where I wasn't being lazy. It was more of a "omg it's broken -- panic-mode" which is not my typical modus. 2. Calling "BUG" multiple times just because you didn't get the notification in the way you wanted. At most this is a "Feature Request" for more direct error messages for those who don't want to trouble themselves to look in the logs or even on the status page. Personally, though, I think adding
Re: [BackupPC-users] Missing backup files
Mark Murawski wrote at about 11:46:24 -0500 on Monday, November 14, 2022: > > On 11/3/22 14:41, backu...@kosowsky.org wrote: > > Mark Murawski wrote at about 14:29:44 -0400 on Thursday, November 3, 2022: > > > Hi Christopher, > > > > > > Thanks for the kind response. > > > > > > I found the possibly culprit log entry that I attached to this thread > > > just earlier. > > > 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump > > > is running on host xyz > > > > You think Log seems pretty clear. > > Might be a good idea to look at the logs *before* bothering us with > > likely phantom bugs. > > Can you define how this situation is a "phantom bug". I pretty > clearly explained the lack of feedback from the UI when this condition > occurs and you get sent a zero byte zip file with no explanation. The logs were clear -- this is Enterprise software and when something doesn't work, one is expected to check the logs (as well as doing other basic troubleshooting homework). If backups or restores don't work -- for whatever reason -- the first thing to do is check the logs!!! (how many times do I have to say that???) This is not child-proof Windows consumer software where the UI is expected to do everything and spoon-feed all the answers. The GUI is only there to help with basic tasks -- indeed the GUI itself is mostly focused on making it easier for users to change config parameters and launch backups/restores -- it is not meant to be idiot-proof or a substitute for logs. Indeed, many users (myself included) never even use the GUI since the CLI is much more powerful and accessible. As such, there is not a huge ongoing investment being made in adding more bells and whistles (unless an individual user -- hint you -- contributes code). The fact that the UI doesn't pop up a nice shiny, themed dialog box explaining why your restore failed is NOT a BUG. At most, it's a feature request to include more debugging info directly in the GUI -- and you are welcome to submit code that provides such messaging. That being said, the GUI did undoubtedly signal an error -- just not in a pop-up. Had you looked under "Restore Summary", you would have seen presumably that your restore had no files or no data. The top status text may also have noted something. I can assure you that there are many, many even more egregious or alternatively subtle ways for backups (or restores) to fail in part or in full and there is no nice shiny GUI that signals many such errors. That's why you need to check and test backups as well as restores -- if there are issues, then... drumbeat... check the logs! > Again... unneeded hostility. Not hostility -- just reaction to: 1. Laziness -- you made all types of claims and request for help before doing even minimal troubleshooting yourself 2. Calling "BUG" multiple times just because you didn't get the notification in the way you wanted. At most this is a "Feature Request" for more direct error messages for those who don't want to trouble themselves to look in the logs or even on the status page. Personally, though, I think adding more error messages will only give a false sense of security leading users to think that their backups and restores are intact just so long as they don't have error messages. Given the large number of ways in which backups and restores can fail subtly, this would only give a false sense of security. > Check your attitude at home. Pot-kettle. If you want help: (1) Be polite (2) Do your homework (3) Don't expect free tech support to compensate for your laziness (4) Don't call things bugs until you have verified that they are indeed bugs (5) Drop the entitled attitude and implication that others should be writing better error notification code to suit your specific needs > Why are you doing your best to push new people away? Because you still don't seem to get that your approach has been rude and lazy from the get-go. I am happy to encourage users who are respectful of other peoples time, who are willing to learn a little and at least read the FAQ before asking for help, and who are potentially willing to even give back. BTW, you clearly don't seem to use much FOSS, especially Enterprise-level. I wish I had a nickel for every time I encountered so-called "attitude" when I would jump in as a newbie asking on some Forum, for help without doing any prep work and without giving even the most major of relevant background info. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
On 11/3/22 14:41, backu...@kosowsky.org wrote: Mark Murawski wrote at about 14:29:44 -0400 on Thursday, November 3, 2022: > Hi Christopher, > > Thanks for the kind response. > > I found the possibly culprit log entry that I attached to this thread > just earlier. > 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump > is running on host xyz You think Log seems pretty clear. Might be a good idea to look at the logs *before* bothering us with likely phantom bugs. Can you define how this situation is a "phantom bug". I pretty clearly explained the lack of feedback from the UI when this condition occurs and you get sent a zero byte zip file with no explanation. Again... unneeded hostility. Check your attitude at home. Everyone else has been really nice here. Why are you doing your best to push new people away? > > Thanks for the note on file storage. The empty paths got me really > worked up and panicked that we had no backups. Perhaps I remembered > incorrectly from the last time I touched the filesystem about where this > stuff is stored. It was a really scary and strange behavior to download > an empty zip file as well as an empty tar file when I tried to restore. > > It could be an artifact of the version I'm running (4.4.0), or maybe > it's a lack of error checking or a bug that allows a download of an > empty zip with no error or warnings about the cause to the user. Or more likely it could be an "artifact" of you not taking the minimum effort of reading the documentation and looking at the log file. This seems much more likely than there being a "bug" in the program. > I'm going to monitor this and collect some more details and see if this > issue is still happening after all backup jobs are complete. > Wise words to collect some more details before claiming likely spurious bugs. Again, not spurious and definitely is a bug. I'll post to the appropriate channels about this to see about a nice fix for this. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Adam Goryachev via BackupPC-users wrote at about 11:03:20 +1100 on Saturday, November 5, 2022: > I just had to comment here > > I don't understand why you would NOT want backuppc to have at least read > access to ALL data, including data only accessible to root. I assume you > would not be suggesting that you run a separate backup system for each > user, so why would you want to either: > > 1) Not backup root data > 2) Run a separate backup solution just for root data > > I guess this will go back to how you setup your data security etc, but > regardless of what you do, I would strongly suggest you ensure ALL data > is backed up (because it is always the unimportant file that needs to be > restored most urgently and is critical). > > So, for my, I use SSH + rsync to backup ALL target systems, and do that > using the root user on the destination, and I simply use the same method > for localhost. For some additional security and peace-of-mind, I create a new user on my remote machines that you could call say backuppc_client that I then access using sudo with very restricted sudoer privileges based on the specific rsync command used to backup. Now I know that this isn't perfect, but it feels safer than just ssh'ing as root... ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Hi there, On Sat, 5 Nov 2022, Adam Goryachev wrote: I don't understand why you would NOT want backuppc to have at least read access to ALL data, including data only accessible to root. Because I wouldn't want BackupPC to be a vector for compromise. The whole point of giving read access only to root is so that *only* root has access. :) It's a security posture. I assume you would not be suggesting that you run a separate backup system for each user Correct. so why would you want to either: 1) Not backup root data ??!! 2) Run a separate backup solution just for root data I guess this will go back to how you setup your data security etc, Exactly. The BackupPC system is rather complex. As commonly used it involves, for example, a Web server and a browser. These things are notoriously insecure. Browsers are amongst the most complex bits of software on the planet and amongst the most frequently compromised. You could even have pathological cases like a well-secured Linux box running BackupPC accessed by Internet Explorer running under XP... Rather than let a random browser have access to things which properly only root should be able to access, if I were being cautious I'd back up the private things separately. It might be as simple as creating a cron job or something like that to zip (with a passphrase known only to me) the root-only files to an archive, and then copy that archive to a place accessible to the backup run by BackupPC. Clearly there's a bit of work to be done there, keeping things current. but regardless of what you do, I would strongly suggest you ensure ALL data is backed up... Well at least all important data, agreed, obviously. :) -- 73, Ged. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
On 5/11/2022 01:04, G.W. Haywood via BackupPC-users wrote: Hi there, On Fri, 4 Nov 2022, Mark Murawski wrote: ... This is the most recently finished full backup [51] for /etc/ssl/private ... There's no files in there!! Just directories!? Everything is missing And it looks like the *entire* backup system looks like this.? I didn't even know that my backups are completely broken and missing all files. Incidentally I'm not sure that I'd want the 'backuppc' user to be able to read private data normally only readable by root, but it's your call and it might even be that you have it set up that way - I don't know. FTAOD I'm just trying to help. I just had to comment here I don't understand why you would NOT want backuppc to have at least read access to ALL data, including data only accessible to root. I assume you would not be suggesting that you run a separate backup system for each user, so why would you want to either: 1) Not backup root data 2) Run a separate backup solution just for root data I guess this will go back to how you setup your data security etc, but regardless of what you do, I would strongly suggest you ensure ALL data is backed up (because it is always the unimportant file that needs to be restored most urgently and is critical). So, for my, I use SSH + rsync to backup ALL target systems, and do that using the root user on the destination, and I simply use the same method for localhost. As for advice, definitely, test your backups, make sure they work, verify by restoring some large enough sample of files and comparing the actual content matches what you would expect. One neat "feature request" would be to have BPC perform a "verify" where it would simply show all files that have changed since the last backup, ie, it does everything except adding changed/new files to the pool. So, while I haven't followed the whole thread, consider posting your log and/or config for the host in question, along with output such as: $ sudo ls -ld /etc /etc/ssl /etc/ssl/private /etc/ssl/private/* Then we could provide additional guidance/suggestions. Regards, Adam ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Hi there, On Fri, 4 Nov 2022, Mark Murawski wrote: ... This is the most recently finished full backup [51] for /etc/ssl/private ... There's no files in there!! Just directories!? Everything is missing And it looks like the *entire* backup system looks like this.? I didn't even know that my backups are completely broken and missing all files. Trying to sut through all the emotion, if I'm reading your posts correctly your backup system has, since the time when you started using BackupPC, performed 51 full backups plus I'd guess quite a lot more incrementals. Obviously it all depends on your configuration, but you didn't show us any of that. Guessing that your full backups aren't being run more often than about daily - and a lot of mine only run monthly - it seems to me likely that (1) your backup system has been running for at least several weeks, and (2) this is the first time you've tested a backup. If my deductions are correct, I'd be inclined to say you should take your backups more seriously. And please don't be so defensive when people try to help, even if they're perhaps understandably impatient with posters who haven't done their homework. Humility works a lot better than escalation. Bear in mind that some people will probably just ignore argumentative posters, and you could be cutting yourself off from a great deal of valuable experience that way. Incidentally I'm not sure that I'd want the 'backuppc' user to be able to read private data normally only readable by root, but it's your call and it might even be that you have it set up that way - I don't know. FTAOD I'm just trying to help. -- 73, Ged. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
backu...@kosowsky.org wrote at about 15:04:42 -0400 on Thursday, November 3, 2022: > Les Mikesell wrote at about 13:58:26 -0500 on Thursday, November 3, 2022: > > On Thu, Nov 3, 2022 at 1:39 PM wrote: > > > > > > Also, /var/log/BackupPC is a weird location unless it is a link to > > > something like TopDir/log in which case it should contain files of > > > form LOG.z and LOG.N.z -- but again these are not the relevant log > > > files. > > > > > > Reading the documentation would be a helpful thing to do -- it was > > > written for a purpose! > > > > The log location is almost certain a distribution-packaging choice, so > > along with recommending reading documentation you might also point out > > that the source documentation may not match what is actually installed > > and running. > > > > I think it must be a link to TopDir/log since in my understanding of > the BackupPC codebase, references are relative to TopDir -- unless the distro > decided to essentially rewrite tons of core code. My guess is the > distro wanted to put a link under /var/log so that newbie users could > find the log files even if they are not aware of the backuppc dir structure. > I was wrong here. LogDir is configurable in config.pl which then overrides the defaults in Lib.pm (as well as in config.p)where LogDir is defined as TopDir/LogDir. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Kim Scarborough wrote at about 14:12:32 -0500 on Thursday, November 3, 2022: > FWIW, the log directory in Arch Linux is /var/log/backuppc, and it's not > a symlink. Ahhh.. you are right... LogDir is separately configurable from TopDir. > > On 11/3/22 14:04, backu...@kosowsky.org wrote: > > Les Mikesell wrote at about 13:58:26 -0500 on Thursday, November 3, 2022: > > > On Thu, Nov 3, 2022 at 1:39 PM wrote: > > > > > > > > Also, /var/log/BackupPC is a weird location unless it is a link to > > > > something like TopDir/log in which case it should contain files of > > > > form LOG.z and LOG.N.z -- but again these are not the relevant log > > > > files. > > > > > > > > Reading the documentation would be a helpful thing to do -- it was > > > > written for a purpose! > > > > > > The log location is almost certain a distribution-packaging choice, so > > > along with recommending reading documentation you might also point out > > > that the source documentation may not match what is actually installed > > > and running. > > > > > > > I think it must be a link to TopDir/log since in my understanding of > > the BackupPC codebase, references are relative to TopDir -- unless the > > distro > > decided to essentially rewrite tons of core code. My guess is the > > distro wanted to put a link under /var/log so that newbie users could > > find the log files even if they are not aware of the backuppc dir > > structure. > > > > > > ___ > > BackupPC-users mailing list > > BackupPC-users@lists.sourceforge.net > > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > Wiki:https://github.com/backuppc/backuppc/wiki > > Project:https://backuppc.github.io/backuppc/ > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
backu...@kosowsky.org wrote: > READ THE DOCUMENTATION (I am sounding like a broken record). Indeed, you are. =-- paul fox, p...@foxharp.boston.ma.us (arlington, ma, where it's 59.5 degrees) ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
FWIW, the log directory in Arch Linux is /var/log/backuppc, and it's not a symlink. On 11/3/22 14:04, backu...@kosowsky.org wrote: Les Mikesell wrote at about 13:58:26 -0500 on Thursday, November 3, 2022: > On Thu, Nov 3, 2022 at 1:39 PM wrote: > > > > Also, /var/log/BackupPC is a weird location unless it is a link to > > something like TopDir/log in which case it should contain files of > > form LOG.z and LOG.N.z -- but again these are not the relevant log > > files. > > > > Reading the documentation would be a helpful thing to do -- it was > > written for a purpose! > > The log location is almost certain a distribution-packaging choice, so > along with recommending reading documentation you might also point out > that the source documentation may not match what is actually installed > and running. > I think it must be a link to TopDir/log since in my understanding of the BackupPC codebase, references are relative to TopDir -- unless the distro decided to essentially rewrite tons of core code. My guess is the distro wanted to put a link under /var/log so that newbie users could find the log files even if they are not aware of the backuppc dir structure. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project:https://backuppc.github.io/backuppc/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Les Mikesell wrote at about 13:58:26 -0500 on Thursday, November 3, 2022: > On Thu, Nov 3, 2022 at 1:39 PM wrote: > > > > Also, /var/log/BackupPC is a weird location unless it is a link to > > something like TopDir/log in which case it should contain files of > > form LOG.z and LOG.N.z -- but again these are not the relevant log > > files. > > > > Reading the documentation would be a helpful thing to do -- it was > > written for a purpose! > > The log location is almost certain a distribution-packaging choice, so > along with recommending reading documentation you might also point out > that the source documentation may not match what is actually installed > and running. > I think it must be a link to TopDir/log since in my understanding of the BackupPC codebase, references are relative to TopDir -- unless the distro decided to essentially rewrite tons of core code. My guess is the distro wanted to put a link under /var/log so that newbie users could find the log files even if they are not aware of the backuppc dir structure. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
On 11/3/22 14:58, Les Mikesell wrote: ...snip The log location is almost certain a distribution-packaging choice, so along with recommending reading documentation you might also point out that the source documentation may not match what is actually installed and running. Thanks! ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
On Thu, Nov 3, 2022 at 1:39 PM wrote: > > Also, /var/log/BackupPC is a weird location unless it is a link to > something like TopDir/log in which case it should contain files of > form LOG.z and LOG.N.z -- but again these are not the relevant log > files. > > Reading the documentation would be a helpful thing to do -- it was > written for a purpose! The log location is almost certain a distribution-packaging choice, so along with recommending reading documentation you might also point out that the source documentation may not match what is actually installed and running. -- Les Mikesell lesmikes...@gmail.com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Mark Murawski wrote at about 14:45:41 -0400 on Thursday, November 3, 2022: > Dear kosowsky, > > Did you not see my note on unnecessary hostility? Did you not see my multiple suggestions to read the documentation and not assume user error or lack of knowledge = bug? > > You provided some good detail in your last message, but I could deal > without the attitude. I might just have to block you if this keeps > up. Feel free... You still have failed to share even the most basic info necessary for helpful assistance. I'm sorry if you interpret aggravation as attitude. > > And this does seem like a bug. Why would you present an empty zip file > to the user without any associated information as to why this is the > case? This is a glaring indication of some error handling issues, not > PEBCAK. It might help if you told us *what* you did to trigger the backup? - Did you use the web gui? - Did you use the CLI command? - Were you trying to backup an 'active' backup or a previous backup? And the log shows a clear error. And if you used the web gui, then I'm pretty sure that somewhere an error was noted or listed. And even if there is not an error message in the explicit form that you seem to expect, that is not a "bug"... though it could be an opportunity to improve clarity. Feel free to submit an enhancement patch if you think that would be helpful. > > > > On 11/3/22 14:41, backu...@kosowsky.org wrote: > > Mark Murawski wrote at about 14:29:44 -0400 on Thursday, November 3, 2022: > > > Hi Christopher, > > > > > > Thanks for the kind response. > > > > > > I found the possibly culprit log entry that I attached to this thread > > > just earlier. > > > 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump > > > is running on host xyz > > > > You think Log seems pretty clear. > > Might be a good idea to look at the logs *before* bothering us with > > likely phantom bugs. > > > > > > Thanks for the note on file storage. The empty paths got me really > > > worked up and panicked that we had no backups. Perhaps I remembered > > > incorrectly from the last time I touched the filesystem about where > > this > > > stuff is stored. It was a really scary and strange behavior to > > download > > > an empty zip file as well as an empty tar file when I tried to restore. > > > > > > It could be an artifact of the version I'm running (4.4.0), or maybe > > > it's a lack of error checking or a bug that allows a download of an > > > empty zip with no error or warnings about the cause to the user. > > > > Or more likely it could be an "artifact" of you not taking the minimum > > effort of reading the documentation and looking at the log file. > > This seems much more likely than there being a "bug" in the program. > > > > > I'm going to monitor this and collect some more details and see if this > > > issue is still happening after all backup jobs are complete. > > > > > > > Wise words to collect some more details before claiming likely > > spurious bugs. > > > > > > On 11/3/22 13:26, Christopher Diekkamp wrote: > > > > Hi Mark, > > > > > > > > It is normal that there are no files. The files are in the pool or > > > > cpool, not the host folder. > > > > > > > > But why you are getting an empty file from the restore is not clear. > > > > Please help us helping you and provide logs from the backup you try > > to > > > > restore and some general infos about your setup. > > > > > > > > Cheers > > > > Christopher > > > > > > > > > > > > > > > > Am 3. November 2022 16:22:00 UTC schrieb Mark Murawski > > > > : > > > > > > > > Hello! > > > > > > > > > > > > This is the most recently finished full backup [51] for > > > > /etc/ssl/private > > > > > > > > > > > > > > > > > > > > When I restore or download this file it's 0 bytes! Not the > > > > reported 1428 > > > > > > > > > > > > And this is my backup path on the backuppc > > > > > > > > > > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate > > > > total 24K > > > > drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . > > > > drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. > > > > -rw-r- 1 backuppc backuppc 0 Oct 1 03:24 > > > > attrib_ee15c72043d4a708b79599530373a35b > > > > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d > > > > drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold > > > > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned > > > > [root@BackupPC > > > > > > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# > > > > > > > > > > > > There's no files in there!! Just directories! Everything is > > missing > > > > > > > > And it looks
Re: [BackupPC-users] Missing backup files
Dear kosowsky, Did you not see my note on unnecessary hostility? You provided some good detail in your last message, but I could deal without the attitude. I might just have to block you if this keeps up. And this does seem like a bug. Why would you present an empty zip file to the user without any associated information as to why this is the case? This is a glaring indication of some error handling issues, not PEBCAK. On 11/3/22 14:41, backu...@kosowsky.org wrote: Mark Murawski wrote at about 14:29:44 -0400 on Thursday, November 3, 2022: > Hi Christopher, > > Thanks for the kind response. > > I found the possibly culprit log entry that I attached to this thread > just earlier. > 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump > is running on host xyz You think Log seems pretty clear. Might be a good idea to look at the logs *before* bothering us with likely phantom bugs. > > Thanks for the note on file storage. The empty paths got me really > worked up and panicked that we had no backups. Perhaps I remembered > incorrectly from the last time I touched the filesystem about where this > stuff is stored. It was a really scary and strange behavior to download > an empty zip file as well as an empty tar file when I tried to restore. > > It could be an artifact of the version I'm running (4.4.0), or maybe > it's a lack of error checking or a bug that allows a download of an > empty zip with no error or warnings about the cause to the user. Or more likely it could be an "artifact" of you not taking the minimum effort of reading the documentation and looking at the log file. This seems much more likely than there being a "bug" in the program. > I'm going to monitor this and collect some more details and see if this > issue is still happening after all backup jobs are complete. > Wise words to collect some more details before claiming likely spurious bugs. > > On 11/3/22 13:26, Christopher Diekkamp wrote: > > Hi Mark, > > > > It is normal that there are no files. The files are in the pool or > > cpool, not the host folder. > > > > But why you are getting an empty file from the restore is not clear. > > Please help us helping you and provide logs from the backup you try to > > restore and some general infos about your setup. > > > > Cheers > > Christopher > > > > > > > > Am 3. November 2022 16:22:00 UTC schrieb Mark Murawski > > : > > > > Hello! > > > > > > This is the most recently finished full backup [51] for > > /etc/ssl/private > > > > > > > > > > When I restore or download this file it's 0 bytes! Not the > > reported 1428 > > > > > > And this is my backup path on the backuppc > > > > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate > > total 24K > > drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . > > drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. > > -rw-r- 1 backuppc backuppc 0 Oct 1 03:24 > > attrib_ee15c72043d4a708b79599530373a35b > > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d > > drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold > > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned > > [root@BackupPC > > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# > > > > > > There's no files in there!! Just directories! Everything is missing > > > > And it looks like the *entire* backup system looks like this. I > > didn't even know that my backups are completely broken and missing > > all files. > > > > > > > > > > > > ___ > > BackupPC-users mailing list > > BackupPC-users@lists.sourceforge.net > > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > Wiki:https://github.com/backuppc/backuppc/wiki > > Project:https://backuppc.github.io/backuppc/ > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Mark Murawski wrote at about 14:35:45 -0400 on Thursday, November 3, 2022: > Some info about my setup: > > * The servers PID is 1865, on host backuppc, version 4.4.0, started at > 4/20 18:08. > * This status was generated at 11/3 11:31. > * The configuration was last loaded at 11/3 09:22. > * PCs will be next queued at 11/3 12:00. > * Other info: > o 2 pending backup requests from last scheduled wakeup, > o 0 pending user backup requests, > o 0 pending command requests, > o Pool is 1404.19+0.00GiB comprising 8463345+0 files and 16512+10 > directories (as of 10/25 01:33), > o Pool hashing gives 0+0 repeated files with longest chain 0+0, > o Nightly cleanup removed 302434+0 files of size 74.66+0.00GiB > (around 10/25 01:33), > o Pool file system was recently at 8% (11/3 11:25), today's max is > 8% (11/3 01:00) and yesterday's max was 8%. > o Pool file system inode usage was recently at 8% (11/3 11:25), > today's max is 8% (11/3 01:00) and yesterday's max was 8%. > > What might be the reason that the deduplication stats are zero? > > Note: > > Pool hashing gives 0+0 repeated files with longest chain 0+0, READ THE DOCUMENTATION (I am sounding like a broken record). Hint: pool hashing != de-duplication Always amazes me how some people are unwilling to put in even the most minimal amount of effort to understand before posting... > > > Obviously deduplication is working because: > > * 69 full backups of total size 6081.09GiB (prior to pooling and > compression), > * 123 incr backups of total size 10142.87GiB (prior to pooling and > compression). > > And my total pool is 1.4T. So deduplication is definitely happening. > > > > On 11/3/22 14:29, Mark Murawski wrote: > > Hi Christopher, > > > > Thanks for the kind response. > > > > I found the possibly culprit log entry that I attached to this thread > > just earlier. > > 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump > > is running on host xyz > > > > Thanks for the note on file storage. The empty paths got me really > > worked up and panicked that we had no backups. Perhaps I remembered > > incorrectly from the last time I touched the filesystem about where > > this stuff is stored. It was a really scary and strange behavior to > > download an empty zip file as well as an empty tar file when I tried > > to restore. > > > > It could be an artifact of the version I'm running (4.4.0), or maybe > > it's a lack of error checking or a bug that allows a download of an > > empty zip with no error or warnings about the cause to the user. > > > > I'm going to monitor this and collect some more details and see if > > this issue is still happening after all backup jobs are complete. > > > > > > > > On 11/3/22 13:26, Christopher Diekkamp wrote: > >> Hi Mark, > >> > >> It is normal that there are no files. The files are in the pool or > >> cpool, not the host folder. > >> > >> But why you are getting an empty file from the restore is not clear. > >> Please help us helping you and provide logs from the backup you try > >> to restore and some general infos about your setup. > >> > >> Cheers > >> Christopher > >> > >> > >> > >> Am 3. November 2022 16:22:00 UTC schrieb Mark Murawski > >> : > >> > >> Hello! > >> > >> > >> This is the most recently finished full backup [51] for > >> /etc/ssl/private > >> > >> > >> > >> > >> When I restore or download this file it's 0 bytes! Not the > >> reported 1428 > >> > >> > >> And this is my backup path on the backuppc > >> > >> > >> /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate > >> total 24K > >> drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . > >> drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. > >> -rw-r- 1 backuppc backuppc 0 Oct 1 03:24 > >> attrib_ee15c72043d4a708b79599530373a35b > >> drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d > >> drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold > >> drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned > >> [root@BackupPC > >> > >> /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# > >> > >> > >> There's no files in there!! Just directories! Everything is missing > >> > >> And it looks like the *entire* backup system looks like this. I > >> didn't even know that my backups are completely broken and > >> missing all files. > >> > >> > >> > >> > >> > >> ___ > >> BackupPC-users mailing list > >> BackupPC-users@lists.sourceforge.net > >> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > >> Wiki:https://github.com/backuppc/backuppc/wiki > >>
Re: [BackupPC-users] Missing backup files
Mark Murawski wrote at about 14:29:44 -0400 on Thursday, November 3, 2022: > Hi Christopher, > > Thanks for the kind response. > > I found the possibly culprit log entry that I attached to this thread > just earlier. > 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump > is running on host xyz You think Log seems pretty clear. Might be a good idea to look at the logs *before* bothering us with likely phantom bugs. > > Thanks for the note on file storage. The empty paths got me really > worked up and panicked that we had no backups. Perhaps I remembered > incorrectly from the last time I touched the filesystem about where this > stuff is stored. It was a really scary and strange behavior to download > an empty zip file as well as an empty tar file when I tried to restore. > > It could be an artifact of the version I'm running (4.4.0), or maybe > it's a lack of error checking or a bug that allows a download of an > empty zip with no error or warnings about the cause to the user. Or more likely it could be an "artifact" of you not taking the minimum effort of reading the documentation and looking at the log file. This seems much more likely than there being a "bug" in the program. > > I'm going to monitor this and collect some more details and see if this > issue is still happening after all backup jobs are complete. > Wise words to collect some more details before claiming likely spurious bugs. > > > On 11/3/22 13:26, Christopher Diekkamp wrote: > > Hi Mark, > > > > It is normal that there are no files. The files are in the pool or > > cpool, not the host folder. > > > > But why you are getting an empty file from the restore is not clear. > > Please help us helping you and provide logs from the backup you try to > > restore and some general infos about your setup. > > > > Cheers > > Christopher > > > > > > > > Am 3. November 2022 16:22:00 UTC schrieb Mark Murawski > > : > > > > Hello! > > > > > > This is the most recently finished full backup [51] for > > /etc/ssl/private > > > > > > > > > > When I restore or download this file it's 0 bytes! Not the > > reported 1428 > > > > > > And this is my backup path on the backuppc > > > > > > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate > > total 24K > > drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . > > drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. > > -rw-r- 1 backuppc backuppc 0 Oct 1 03:24 > > attrib_ee15c72043d4a708b79599530373a35b > > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d > > drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold > > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned > > [root@BackupPC > > > > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# > > > > > > There's no files in there!! Just directories! Everything is missing > > > > And it looks like the *entire* backup system looks like this. I > > didn't even know that my backups are completely broken and missing > > all files. > > > > > > > > > > > > ___ > > BackupPC-users mailing list > > BackupPC-users@lists.sourceforge.net > > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > Wiki:https://github.com/backuppc/backuppc/wiki > > Project:https://backuppc.github.io/backuppc/ > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Mark Murawski wrote at about 14:20:42 -0400 on Thursday, November 3, 2022: > Hi, > > Regards to: > 1) > I know there are no data files stored in the pc tree. I understand that > BackupPC uses hard links for file-level deduplication. In traditional > BackupPC datastores that I've worked on, the hard links on a per-file > basis do exist in the pc directory. Unless this was changed? It's been > a while since I've went into the internals. I was just shocked that > after poking around I don't see any data being backed up despite > successes being reported on the portal. This was a bit worrisome since > there's no backup files present. > AGAIN READ THE DOCUMENTATION v4 DOES NOT USE HARD LINKS and Based on the form of the attrib file you list, you are using v4. > 2) > My post here was a starting point for getting some insight into the > issue. I wasn't expecting a "here's exactly how to fix it instruction > manual" as a first response. I was expecting something like "I see you > have some backup restoration issues... but have you tried X?". To be > fair, there is no guide so far that I see on the proper process for > reporting trouble, other than "provide the most detail possible". Which > I did my initial investigation and pasted in my findings... again a > starting point. You don't have to jump down people's throats who didn't > live up to the undefined expectations. Honestly this kind of hostility > is is why some open source communities get a bad reputation. I'm not > offended.. it's just part of the open source world... Just food for thought. > But we don't have time for vague initial speculation. Do your homework and provide as much background as possible. You just give a list of a single directory in your pc tree. You don't give any background info (such as what version, though I could figure it out), whether this is full or incremental, etc. You don't share any logs. You don't say how you are trying to restore (web gui, command line, looking for files in the pc tree?). You don't say what distro you are using. Every (free) public support forum that I have ever participated in expects that the user make some minimal effort to troubleshoot and provide details. > > There's nothing in /var/log/BackupPC other than LOCK and status.pl These are NOT the relevant (or at least most relevant log files) -- you need the file-specific log files which are in the pc tree for each host. Assuming you are referring to TopDir/log, such log files only show high level status of backup success or failure. Also, /var/log/BackupPC is a weird location unless it is a link to something like TopDir/log in which case it should contain files of form LOG.z and LOG.N.z -- but again these are not the relevant log files. Reading the documentation would be a helpful thing to do -- it was written for a purpose! > > > I did find this in the web portal shortcut for 'LOG' file > > 2022-11-03 08:08:49 User backuppc downloaded zip archive for xyz, backup > 51; share /; files were: /etc/ssl/private/xyz.crt > 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump is > running on host xyz > See above - these are not the detailed log files. If you are using the web gui, then you should see the log files for each backup. Read the documentation and don't just expect us to do all your work for you! > > I'll try downloading it again when there's no jobs running... is this a > setting? Is it not possible to download a zip when a backup job is > running on the same host? > > > > > On 11/3/22 13:24, backu...@kosowsky.org wrote: > > Two problems: > > 1. You need to at least take the effort of reading the basic > > documentation before seeking help here. > > Hint #1: there are no "data files" stored in the pc tree > > Hint #2: Your statement "I didn't even know that my backups are > > completely broken and missing all files." sounds ridiculous > > > > 2. Even if there really is a problem, how in the world can we expect > > to help you if you don't provide any log files or other details > > regarding the backup, including in particular for "this" file? > > > > Please show us the courtesy of taking the effort to ask an > > appropriately-prepared and supported question... > > > > Mark Murawski wrote at about 12:22:00 -0400 on Thursday, November 3, 2022: > > > Hello! > > > > > > > > > This is the most recently finished full backup [51] for > > /etc/ssl/private > > > > > > > > > > > > > > > When I restore or download this file it's 0 bytes! Not the reported > > 1428 > > > > > > > > > And this is my backup path on the backuppc > > > > > > > > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate > > > total 24K > > > drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . > > > drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. > > >
Re: [BackupPC-users] Missing backup files
Some info about my setup: * The servers PID is 1865, on host backuppc, version 4.4.0, started at 4/20 18:08. * This status was generated at 11/3 11:31. * The configuration was last loaded at 11/3 09:22. * PCs will be next queued at 11/3 12:00. * Other info: o 2 pending backup requests from last scheduled wakeup, o 0 pending user backup requests, o 0 pending command requests, o Pool is 1404.19+0.00GiB comprising 8463345+0 files and 16512+10 directories (as of 10/25 01:33), o Pool hashing gives 0+0 repeated files with longest chain 0+0, o Nightly cleanup removed 302434+0 files of size 74.66+0.00GiB (around 10/25 01:33), o Pool file system was recently at 8% (11/3 11:25), today's max is 8% (11/3 01:00) and yesterday's max was 8%. o Pool file system inode usage was recently at 8% (11/3 11:25), today's max is 8% (11/3 01:00) and yesterday's max was 8%. What might be the reason that the deduplication stats are zero? Note: Pool hashing gives 0+0 repeated files with longest chain 0+0, Obviously deduplication is working because: * 69 full backups of total size 6081.09GiB (prior to pooling and compression), * 123 incr backups of total size 10142.87GiB (prior to pooling and compression). And my total pool is 1.4T. So deduplication is definitely happening. On 11/3/22 14:29, Mark Murawski wrote: Hi Christopher, Thanks for the kind response. I found the possibly culprit log entry that I attached to this thread just earlier. 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump is running on host xyz Thanks for the note on file storage. The empty paths got me really worked up and panicked that we had no backups. Perhaps I remembered incorrectly from the last time I touched the filesystem about where this stuff is stored. It was a really scary and strange behavior to download an empty zip file as well as an empty tar file when I tried to restore. It could be an artifact of the version I'm running (4.4.0), or maybe it's a lack of error checking or a bug that allows a download of an empty zip with no error or warnings about the cause to the user. I'm going to monitor this and collect some more details and see if this issue is still happening after all backup jobs are complete. On 11/3/22 13:26, Christopher Diekkamp wrote: Hi Mark, It is normal that there are no files. The files are in the pool or cpool, not the host folder. But why you are getting an empty file from the restore is not clear. Please help us helping you and provide logs from the backup you try to restore and some general infos about your setup. Cheers Christopher Am 3. November 2022 16:22:00 UTC schrieb Mark Murawski : Hello! This is the most recently finished full backup [51] for /etc/ssl/private When I restore or download this file it's 0 bytes! Not the reported 1428 And this is my backup path on the backuppc /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate total 24K drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. -rw-r- 1 backuppc backuppc 0 Oct 1 03:24 attrib_ee15c72043d4a708b79599530373a35b drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned [root@BackupPC /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# There's no files in there!! Just directories! Everything is missing And it looks like the *entire* backup system looks like this. I didn't even know that my backups are completely broken and missing all files. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project:https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project:https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Hi Christopher, Thanks for the kind response. I found the possibly culprit log entry that I attached to this thread just earlier. 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump is running on host xyz Thanks for the note on file storage. The empty paths got me really worked up and panicked that we had no backups. Perhaps I remembered incorrectly from the last time I touched the filesystem about where this stuff is stored. It was a really scary and strange behavior to download an empty zip file as well as an empty tar file when I tried to restore. It could be an artifact of the version I'm running (4.4.0), or maybe it's a lack of error checking or a bug that allows a download of an empty zip with no error or warnings about the cause to the user. I'm going to monitor this and collect some more details and see if this issue is still happening after all backup jobs are complete. On 11/3/22 13:26, Christopher Diekkamp wrote: Hi Mark, It is normal that there are no files. The files are in the pool or cpool, not the host folder. But why you are getting an empty file from the restore is not clear. Please help us helping you and provide logs from the backup you try to restore and some general infos about your setup. Cheers Christopher Am 3. November 2022 16:22:00 UTC schrieb Mark Murawski : Hello! This is the most recently finished full backup [51] for /etc/ssl/private When I restore or download this file it's 0 bytes! Not the reported 1428 And this is my backup path on the backuppc /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate total 24K drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. -rw-r- 1 backuppc backuppc 0 Oct 1 03:24 attrib_ee15c72043d4a708b79599530373a35b drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned [root@BackupPC /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# There's no files in there!! Just directories! Everything is missing And it looks like the *entire* backup system looks like this. I didn't even know that my backups are completely broken and missing all files. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project:https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Hi, Regards to: 1) I know there are no data files stored in the pc tree. I understand that BackupPC uses hard links for file-level deduplication. In traditional BackupPC datastores that I've worked on, the hard links on a per-file basis do exist in the pc directory. Unless this was changed? It's been a while since I've went into the internals. I was just shocked that after poking around I don't see any data being backed up despite successes being reported on the portal. This was a bit worrisome since there's no backup files present. 2) My post here was a starting point for getting some insight into the issue. I wasn't expecting a "here's exactly how to fix it instruction manual" as a first response. I was expecting something like "I see you have some backup restoration issues... but have you tried X?". To be fair, there is no guide so far that I see on the proper process for reporting trouble, other than "provide the most detail possible". Which I did my initial investigation and pasted in my findings... again a starting point. You don't have to jump down people's throats who didn't live up to the undefined expectations. Honestly this kind of hostility is is why some open source communities get a bad reputation. I'm not offended.. it's just part of the open source world... Just food for thought. There's nothing in /var/log/BackupPC other than LOCK and status.pl I did find this in the web portal shortcut for 'LOG' file 2022-11-03 08:08:49 User backuppc downloaded zip archive for xyz, backup 51; share /; files were: /etc/ssl/private/xyz.crt 2022-11-03 08:08:49 Can't run BackupPC_zipCreate while BackupPC_dump is running on host xyz I'll try downloading it again when there's no jobs running... is this a setting? Is it not possible to download a zip when a backup job is running on the same host? On 11/3/22 13:24, backu...@kosowsky.org wrote: Two problems: 1. You need to at least take the effort of reading the basic documentation before seeking help here. Hint #1: there are no "data files" stored in the pc tree Hint #2: Your statement "I didn't even know that my backups are completely broken and missing all files." sounds ridiculous 2. Even if there really is a problem, how in the world can we expect to help you if you don't provide any log files or other details regarding the backup, including in particular for "this" file? Please show us the courtesy of taking the effort to ask an appropriately-prepared and supported question... Mark Murawski wrote at about 12:22:00 -0400 on Thursday, November 3, 2022: > Hello! > > > This is the most recently finished full backup [51] for /etc/ssl/private > > > > > When I restore or download this file it's 0 bytes! Not the reported 1428 > > > And this is my backup path on the backuppc > > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate > total 24K > drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . > drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. > -rw-r- 1 backuppc backuppc 0 Oct 1 03:24 > attrib_ee15c72043d4a708b79599530373a35b > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d > drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned > [root@BackupPC > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# > > > There's no files in there!! Just directories! Everything is missing > > And it looks like the *entire* backup system looks like this. I didn't > even know that my backups are completely broken and missing all files. > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Hi Mark, It is normal that there are no files. The files are in the pool or cpool, not the host folder. But why you are getting an empty file from the restore is not clear. Please help us helping you and provide logs from the backup you try to restore and some general infos about your setup. Cheers Christopher Am 3. November 2022 16:22:00 UTC schrieb Mark Murawski : >Hello! > > >This is the most recently finished full backup [51] for /etc/ssl/private > > > > >When I restore or download this file it's 0 bytes! Not the reported 1428 > > >And this is my backup path on the backuppc > >/home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate >total 24K >drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . >drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. >-rw-r- 1 backuppc backuppc 0 Oct 1 03:24 >attrib_ee15c72043d4a708b79599530373a35b >drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d >drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold >drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned >[root@BackupPC >/home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# > > >There's no files in there!! Just directories! Everything is missing > >And it looks like the *entire* backup system looks like this. I didn't even >know that my backups are completely broken and missing all files. > > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Missing backup files
Two problems: 1. You need to at least take the effort of reading the basic documentation before seeking help here. Hint #1: there are no "data files" stored in the pc tree Hint #2: Your statement "I didn't even know that my backups are completely broken and missing all files." sounds ridiculous 2. Even if there really is a problem, how in the world can we expect to help you if you don't provide any log files or other details regarding the backup, including in particular for "this" file? Please show us the courtesy of taking the effort to ask an appropriately-prepared and supported question... Mark Murawski wrote at about 12:22:00 -0400 on Thursday, November 3, 2022: > Hello! > > > This is the most recently finished full backup [51] for /etc/ssl/private > > > > > When I restore or download this file it's 0 bytes! Not the reported 1428 > > > And this is my backup path on the backuppc > > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate > total 24K > drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . > drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. > -rw-r- 1 backuppc backuppc 0 Oct 1 03:24 > attrib_ee15c72043d4a708b79599530373a35b > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d > drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold > drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned > [root@BackupPC > /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# > > > There's no files in there!! Just directories! Everything is missing > > And it looks like the *entire* backup system looks like this. I didn't > even know that my backups are completely broken and missing all files. > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
[BackupPC-users] Missing backup files
Hello! This is the most recently finished full backup [51] for /etc/ssl/private When I restore or download this file it's 0 bytes! Not the reported 1428 And this is my backup path on the backuppc /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate total 24K drwxr-x--- 5 backuppc backuppc 6 Oct 1 03:24 . drwxr-x--- 4 backuppc backuppc 5 Oct 1 03:24 .. -rw-r- 1 backuppc backuppc 0 Oct 1 03:24 attrib_ee15c72043d4a708b79599530373a35b drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fcertbot-renew.d drwxr-x--- 2 backuppc backuppc 2 Oct 1 03:24 fold drwxr-x--- 2 backuppc backuppc 3 Oct 1 03:24 fselfsigned [root@BackupPC /home/backuppc/pc/comm-nyc1.somethingorother.local/51/f%2f/fetc/fssl/fprivate]# There's no files in there!! Just directories! Everything is missing And it looks like the *entire* backup system looks like this. I didn't even know that my backups are completely broken and missing all files. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
