Re: [BackupPC-users] BackupPC_tarCreate with only certain types of files
Holger Parplies wb...@parplies.de wrote on 05/23/2015 09:29:25 AM: for the archives: you don't strictly *need* the free space. You can pipe the output of BackupPC_tarCreate directly into a 'tar x' and tell tar to only extract files named '*.pdf', something like BackupPC_tarCreate -h host -n 123 -s /share /path \ | tar xf - --wildcards '*.pdf' '*.PDF' Forget the archives: I appreciate the tip. It'd be nice to avoid all the I/O to have BackupPC stream all that data just so it can end up in the bit bucket, but at least it doesn't end up on my disk again, too! :) This seems to be another good case for using the fuse module. You can navigate the backup view (or run a 'find', 'rsync', ...) at a relatively low cost and only need to read/decompress the file data you actually act on - andyou don't need any intermediate storage space. I completely forgot about that. I've already done most of the data so far, but that's an excellent suggestion, too. once again, thank you. Tim Massey Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmas...@obscorp.com 22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796 -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC_tarCreate with only certain types of files
Thank you very much for the confirmation that I'm not crazy and you can't
do wildcards with BackupPC_tarCreate.
In this instance, I was able to come up with enough free space to be able
to do the complete restore and then grab the data I needed from that. But
thank you very much for your code: I will experiment with it is a need to
use it in the future.
Timothy J. Massey
Sent from my iPad
On May 23, 2015, at 7:43 AM, Holger Parplies wb...@parplies.de wrote:
Hi,
Timothy J Massey wrote on 2015-05-22 20:40:52 -0400 [Re: [BackupPC-users]
BackupPC_tarCreate with only certain types of?files]:
Les Mikesell lesmikes...@gmail.com wrote on 05/22/2015 04:24:56 PM:
What am I missing? How do I get BackupPC_tarCreate to create a
tar file that contains all PDF's stored in that path?
[...]
Can't help with BackupPC_tarCreate's wildcard concepts
I can: there are none. BackupPC_tarCreate gets a list of path names to
include
in the tar. Each of these is looked up verbatim and included (with
substructure if it happens to be a directory).
The problem is not that I couldn't figure out how to get the PDF's at
all,
but how I could avoid restoring 500GB of data for the 500*MB* I
actually
need! :)
It should be *fairly* simple to patch BackupPC_tarCreate for the simple
case
you need. A more general case would add an option and include correct
handling
of hardlinks and symlinks. In sub TarWriteFile, after the calculation of
$tarPath and before the big file type if (line 455 in the 3.3.0
source),
I'd add
return
if $hdr-{type} != BPC_FTYPE_DIR and $tarPath !~ /\.pdf$/i;
(omit the i modifier if you really only mean lowercase pdf). Note
that
this is once again completely untested ;-).
Hope that helps.
Regards,
Holger
--
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Issue with multiple restores
Hi,
Les Mikesell wrote on 2015-05-22 11:53:10 -0500 [Re: [BackupPC-users] Issue
with multiple restores]:
On Fri, May 22, 2015 at 11:17 AM, Andy Taylor andy.tay...@sift.com wrote:
Thank you for the advice Les. I compiled rsync 2.6.9 and tried a restore
with it. It seems to hang now once it's got past the last file of the
restore - I've run strace against the rsync processes and I can see it close
the last file and then just hang, with this sort of output:
[pid 15625] select(4, [3], [], NULL, {53, 660732} unfinished ...
[pid 15624] select(7, [3 6], [], NULL, NULL unfinished ...
[pid 15662] select(1, [0], [], NULL, {6, 915513}) = 0 (Timeout)
[pid 15662] select(1, [0], [], NULL, {60, 0} unfinished ...
[pid 15625] ... select resumed ) = 0 (Timeout)
[pid 15625] select(4, [3], [], NULL, {60, 0} unfinished ...
[pid 15662] ... select resumed ) = 0 (Timeout)
Do you know what that might indicate about the restore?
perhaps a corrupt attrib file, as in: one end of the connection is expecting
[much] more data than the other end sends. This would seem to fit in with your
error message from newer rsync versions as well as the fact that it only
happens for one host. Have you tried forcing a full backup for that host?
[...]
What's the big picture here?
Whenever I revisit this thread, I immediately think BackupPC fuse module?,
but I haven't used it myself yet, so I can't give you any detail. But it might
well be something to look into - providing it fits into the big picture you
haven't given ;-).
Regards,
Holger
--
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC_tarCreate with only certain types of files
Hi,
Timothy J Massey wrote on 2015-05-22 20:40:52 -0400 [Re: [BackupPC-users]
BackupPC_tarCreate with only certain types of?files]:
Les Mikesell lesmikes...@gmail.com wrote on 05/22/2015 04:24:56 PM:
What am I missing? How do I get BackupPC_tarCreate to create a
tar file that contains all PDF's stored in that path?
[...]
Can't help with BackupPC_tarCreate's wildcard concepts
I can: there are none. BackupPC_tarCreate gets a list of path names to include
in the tar. Each of these is looked up verbatim and included (with
substructure if it happens to be a directory).
The problem is not that I couldn't figure out how to get the PDF's at all,
but how I could avoid restoring 500GB of data for the 500*MB* I actually
need! :)
It should be *fairly* simple to patch BackupPC_tarCreate for the simple case
you need. A more general case would add an option and include correct handling
of hardlinks and symlinks. In sub TarWriteFile, after the calculation of
$tarPath and before the big file type if (line 455 in the 3.3.0 source),
I'd add
return
if $hdr-{type} != BPC_FTYPE_DIR and $tarPath !~ /\.pdf$/i;
(omit the i modifier if you really only mean lowercase pdf). Note that
this is once again completely untested ;-).
Hope that helps.
Regards,
Holger
--
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC_tarCreate with only certain types of files
Hi, Timothy J Massey wrote on 2015-05-23 08:40:21 -0400 [Re: [BackupPC-users] BackupPC_tarCreate with only certain types of?files]: [...] In this instance, I was able to come up with enough free space to be able to do the complete restore and then grab the data I needed from that. for the archives: you don't strictly *need* the free space. You can pipe the output of BackupPC_tarCreate directly into a 'tar x' and tell tar to only extract files named '*.pdf', something like BackupPC_tarCreate -h host -n 123 -s /share /path \ | tar xf - --wildcards '*.pdf' '*.PDF' (just to demonstrate that you can specify more than one glob). This can be problematic, in a way: if you get something wrong about the tar command line, you will likely find out that the file(s) you were expecting were not extracted, and you will need to repeat the whole lengthy process, whereas with a complete restore, you can simply navigate the result and select whatever you need. This seems to be another good case for using the fuse module. You can navigate the backup view (or run a 'find', 'rsync', ...) at a relatively low cost and only need to read/decompress the file data you actually act on - and you don't need any intermediate storage space. Regards, Holger -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
