Re: [BackupPC-users] BackupPC_tarCreate with only certain types of files
Holger Parplies wrote on 05/23/2015 09:29:25 AM: > for the archives: you don't strictly *need* the free space. You can pipe the > output of BackupPC_tarCreate directly into a 'tar x' and tell tar to only > extract files named '*.pdf', something like > >BackupPC_tarCreate -h host -n 123 -s /share /path \ > | tar xf - --wildcards '*.pdf' '*.PDF' Forget the archives: I appreciate the tip. It'd be nice to avoid all the I/O to have BackupPC stream all that data just so it can end up in the bit bucket, but at least it doesn't end up on my disk again, too! :) > This seems to be another good case for using the fuse module. You can navigate > the backup view (or run a 'find', 'rsync', ...) at a relatively low cost and > only need to read/decompress the file data you actually act on - andyou don't > need any intermediate storage space. I completely forgot about that. I've already done most of the data so far, but that's an excellent suggestion, too. once again, thank you. Tim Massey Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmas...@obscorp.com 22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796 -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC_tarCreate with only certain types of files
Hi, Timothy J Massey wrote on 2015-05-23 08:40:21 -0400 [Re: [BackupPC-users] BackupPC_tarCreate with only certain types of?files]: > [...] > In this instance, I was able to come up with enough free space to be able > to do the complete restore and then grab the data I needed from that. for the archives: you don't strictly *need* the free space. You can pipe the output of BackupPC_tarCreate directly into a 'tar x' and tell tar to only extract files named '*.pdf', something like BackupPC_tarCreate -h host -n 123 -s /share /path \ | tar xf - --wildcards '*.pdf' '*.PDF' (just to demonstrate that you can specify more than one glob). This can be problematic, in a way: if you get something wrong about the tar command line, you will likely find out that the file(s) you were expecting were not extracted, and you will need to repeat the whole lengthy process, whereas with a complete restore, you can simply navigate the result and select whatever you need. This seems to be another good case for using the fuse module. You can navigate the backup view (or run a 'find', 'rsync', ...) at a relatively low cost and only need to read/decompress the file data you actually act on - and you don't need any intermediate storage space. Regards, Holger -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC_tarCreate with only certain types of files
Thank you very much for the confirmation that I'm not crazy and you can't do wildcards with BackupPC_tarCreate. In this instance, I was able to come up with enough free space to be able to do the complete restore and then grab the data I needed from that. But thank you very much for your code: I will experiment with it is a need to use it in the future. Timothy J. Massey Sent from my iPad > On May 23, 2015, at 7:43 AM, Holger Parplies wrote: > > Hi, > > Timothy J Massey wrote on 2015-05-22 20:40:52 -0400 [Re: [BackupPC-users] BackupPC_tarCreate with only certain types of?files]: > > Les Mikesell wrote on 05/22/2015 04:24:56 PM: > > > > > > What am I missing? How do I get BackupPC_tarCreate to create a > > > > tar file that contains all PDF's stored in that path? > > > > [...] > > > > > > Can't help with BackupPC_tarCreate's wildcard concepts > > I can: there are none. BackupPC_tarCreate gets a list of path names to include > in the tar. Each of these is looked up verbatim and included (with > substructure if it happens to be a directory). > > > The problem is not that I couldn't figure out how to get the PDF's at all, > > but how I could avoid restoring 500GB of data for the 500*MB* I actually > > need! :) > > It should be *fairly* simple to patch BackupPC_tarCreate for the simple case > you need. A more general case would add an option and include correct handling > of hardlinks and symlinks. In sub TarWriteFile, after the calculation of > $tarPath and before the big file type "if" (line 455 in the 3.3.0 source), > I'd add > > return > if $hdr->{type} != BPC_FTYPE_DIR and $tarPath !~ /\.pdf$/i; > > (omit the "i" modifier if you really only mean lowercase "pdf"). Note that > this is once again completely untested ;-). > > Hope that helps. > > Regards, > Holger -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Issue with multiple restores
Hi, Les Mikesell wrote on 2015-05-22 11:53:10 -0500 [Re: [BackupPC-users] Issue with multiple restores]: > On Fri, May 22, 2015 at 11:17 AM, Andy Taylor wrote: > > Thank you for the advice Les. I compiled rsync 2.6.9 and tried a restore > > with it. It seems to hang now once it's got past the last file of the > > restore - I've run strace against the rsync processes and I can see it close > > the last file and then just hang, with this sort of output: > > > > [pid 15625] select(4, [3], [], NULL, {53, 660732} > > [pid 15624] select(7, [3 6], [], NULL, NULL > > [pid 15662] select(1, [0], [], NULL, {6, 915513}) = 0 (Timeout) > > [pid 15662] select(1, [0], [], NULL, {60, 0} > > [pid 15625] <... select resumed> ) = 0 (Timeout) > > [pid 15625] select(4, [3], [], NULL, {60, 0} > > [pid 15662] <... select resumed> ) = 0 (Timeout) > > > > Do you know what that might indicate about the restore? perhaps a corrupt attrib file, as in: one end of the connection is expecting [much] more data than the other end sends. This would seem to fit in with your error message from newer rsync versions as well as the fact that it only happens for one host. Have you tried forcing a full backup for that host? > [...] > What's the big picture here? Whenever I revisit this thread, I immediately think "BackupPC fuse module?", but I haven't used it myself yet, so I can't give you any detail. But it might well be something to look into - providing it fits into the "big picture" you haven't given ;-). Regards, Holger -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC_tarCreate with only certain types of files
Hi, Timothy J Massey wrote on 2015-05-22 20:40:52 -0400 [Re: [BackupPC-users] BackupPC_tarCreate with only certain types of?files]: > Les Mikesell wrote on 05/22/2015 04:24:56 PM: > > > > What am I missing? How do I get BackupPC_tarCreate to create a > > > tar file that contains all PDF's stored in that path? > > > [...] > > > > Can't help with BackupPC_tarCreate's wildcard concepts I can: there are none. BackupPC_tarCreate gets a list of path names to include in the tar. Each of these is looked up verbatim and included (with substructure if it happens to be a directory). > The problem is not that I couldn't figure out how to get the PDF's at all, > but how I could avoid restoring 500GB of data for the 500*MB* I actually > need! :) It should be *fairly* simple to patch BackupPC_tarCreate for the simple case you need. A more general case would add an option and include correct handling of hardlinks and symlinks. In sub TarWriteFile, after the calculation of $tarPath and before the big file type "if" (line 455 in the 3.3.0 source), I'd add return if $hdr->{type} != BPC_FTYPE_DIR and $tarPath !~ /\.pdf$/i; (omit the "i" modifier if you really only mean lowercase "pdf"). Note that this is once again completely untested ;-). Hope that helps. Regards, Holger -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/